Airport runway FOD inspection and operation safety guarantee system

Abstract

The invention discloses an airport runway FOD inspection and operation safety guarantee system in the field of airport runway safety detection. A ground control platform receives three-mode image data of an unmanned aerial vehicle platform and sends flight control data; the unmanned aerial vehicle platform receives the flight control data, executes flight actions, collects three-mode image data and sends the three-mode image data to the ground control platform; the ground control platform comprises an FOD detection module, a map module and a task scheduling module; the FOD detection module receives the three-mode image data and performs video image recognition to generate image recognition data; a gridding electronic map is arranged in the map module, and the FOD detection module superposes the image recognition data on the gridding electronic map to generate map obstacle data; and the task scheduling module sends the flight control data to the unmanned aerial vehicle platform. According to the invention, three-mode image recognition is carried out on the airport scene through the unmanned aerial vehicle platform to enhance the FOD inspection detection accuracy, and the unmanned aerial vehicle platform is utilized to increase the monitoring range, so that emergencies can be found in time.

Description

Technical Field

[0001] The invention relates to the field of airport runway safety detection, in particular to an airport runway FOD inspection and operation safety guarantee system. Background Art

[0002] The airport surface is a key area for aircraft takeoff, landing and taxiing. Any foreign objects on the surface may have a serious impact on the safe operation of the aircraft and cause serious consequences. The more typical FOD (foreign dangerous object) detection system abroad uses fixed detection base stations on both sides of the runway, mainly using millimeter-wave radar detection and supplemented by video image recognition; some use multi-point video image recognition technology for detection; some use further upgrades to the detection mode to achieve a vehicle-mounted mobile detection system. Although these systems can realize active detection of FOD, their detection accuracy and range will be affected by the installation method and location of the system. In order to achieve large-scale and high-precision FOD detection, a large number of detection points need to be arranged at the same time, which greatly increases the system cost. In addition, due to the complex electromagnetic environment and surface layout of the airport, the installation and use of detection points have great limitations, which limits the application and promotion of such systems.

[0003] At present, most airports in my country still rely on runway inspectors to conduct manual inspections, which requires the closure of runways for inspections. This is not only inefficient and unreliable, but also takes up valuable runway usage time.

[0004] Therefore, the problems that need to be urgently solved by technical personnel in this field are as follows: airport security personnel can realize remote monitoring of the foreign object safety situation in the field area with large coverage, and mark the detected foreign objects at fixed points, and send the corresponding information to the airport operation monitoring system for recording; they can also plan the best path for fixed-point removal of foreign objects based on the marked information, minimize the workload of airport security personnel, reduce the time and manpower costs spent by the airport on FOD detection and cleaning, and improve the airport operation efficiency; at the same time, improve the accuracy of FOD detection and reduce the safety risks of airport operations. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art in which the detection accuracy of FOD is low due to the simple detection means and tight human resources when performing FOD detection on the airport scene. An airport runway FOD inspection and operation safety assurance system is provided, which enhances the FOD inspection detection accuracy by performing three-mode image recognition on the airport scene through an unmanned aerial vehicle platform, and uses the unmanned aerial vehicle platform to increase the monitoring range, so that emergencies can be discovered in time.

[0006] The purpose of the present invention is mainly achieved through the following technical solutions: An airport runway FOD inspection and operation safety assurance system includes a ground control platform and an unmanned aerial vehicle mission platform, wherein: The ground control platform is used to receive the three-mode image data of the UAV platform and send flight control data to the UAV platform; The UAV platform is used to receive flight control data and execute flight actions, collect three-mode image data and send it to the ground control platform; The ground control platform includes a FOD detection module, a map module and a task scheduling module; The FOD detection module is used to receive the three-mode image data and perform video image recognition to generate image recognition data; The map module is provided with a grid electronic map, and the FOD detection module superimposes the image recognition data on the grid electronic map to generate map obstacle data; The task scheduling module is used to receive the map obstacle data and generate flight control data to send to the UAV platform.

[0007] In the present invention, the three-mode image data includes image data in three modes: RGB, depth, and thermal. The three-mode image data provides a richer and more intuitive spatial information display in a graphical form, making the abstract spatial information easy to understand and operate, which is conducive to finding obstacles on the airport scene. The gridded electronic map divides the map into partitions by grids, and can mark different grid areas, thereby improving the accuracy of positioning when obstacles are found.

[0008] Furthermore, the UAV platform includes a high-definition image acquisition module, a laser radar module, an infrared sensor module, a flight execution module and a wireless transmission module, wherein: The high-definition picture acquisition module is used to acquire video image data; The laser radar module is used to receive the control signal of the flight execution module and collect radar image data according to the control signal, and send the radar image data to the ground control platform; The infrared sensor module is used to receive the control signal of the flight execution module and collect infrared image data according to the control signal, and send the infrared image data to the ground control platform; The flight execution module is used to receive flight control data and execute flight actions. The laser radar module and the infrared sensor module send control signals according to the flight control data items; The wireless transmission module is used for data communication with the ground control platform.

[0009] In the present invention, the high-definition picture acquisition module obtains video image data by shooting with a camera device, the laser radar module obtains radar image data by a laser radar, and the infrared sensor module obtains infrared image data by an infrared camera.

[0010] Furthermore, the map obstacle data includes map information and obstacle information, the map information marks the importance of different locations through the gridded electronic map, and the obstacle information is marked with importance according to the physical properties of the obstacle; The importance mark is assigned a value according to the critical situation, recorded as an emergency value, the emergency value of the map information and the emergency value of the obstacle information are superimposed, the grid electronic map divides the map into a plurality of grid areas, and the FOD inspection emergency parameters of each grid area are obtained; An early warning threshold is set. When the FOD inspection emergency parameter exceeds the early warning threshold, a safety alarm is issued in the corresponding grid area.

[0011] Furthermore, a review threshold is set, and the review threshold is lower than the warning threshold. When the FOD inspection emergency parameter does not reach the review threshold, the laser radar module and the infrared sensor module are in sleep mode. When the FOD inspection emergency parameter reaches the review threshold but does not exceed the warning threshold, the laser radar module and the infrared sensor module are started, and the ground control platform operates the flight execution module to review and execute the flight corresponding to the grid area.

[0012] Furthermore, the video image recognition in the FOD detection module includes the following steps: Acquire video image data and retrieve original images, and compare the video image data with the original images; Record the difference between the video image data and the original image to generate first identification data; Sending the first identification data to a map module, and observing whether there is an obstacle threat in conjunction with the map module; If there is no obstacle threat, the first identification data and the UAV platform flight trajectory data in the flight control data are combined and sent to the map module to generate map obstacle data; If there is an obstacle threat, radar image data and infrared image data are obtained, and deep learning target detection is performed by combining the video image data, radar image data and infrared image data to obtain second recognition data; The second identification data and the UAV platform flight trajectory data in the flight control data are combined and sent to the map module to generate map obstacle data.

[0013] Further, when the task scheduling module obtains the map obstacle data, it determines whether the second identification data is obtained: If the first identification data is obtained, flight control data is generated according to the preset flight trajectory and sent to the UAV platform; If the first identification data is not obtained, the first flight control data is generated for the map position where the obstacle threat exists in the map obstacle data, and sent to the UAV platform.

[0014] Furthermore, the task scheduling module is used to receive flight operation dynamic information and adjust flight control data in real time based on the flight operation dynamic information.

[0015] The inspection route planning of the drone platform in the present invention is optimized to reduce energy consumption and improve operation efficiency. In addition, the inspection flight path of the drone platform of this system is constrained by the dynamic operation of airport flights. While improving the automation of task execution, it ensures the safe interval when the drone aircraft and aircraft are mixed in operation, preventing potential conflicts on the same flight.

[0016] In summary, the present invention has the following beneficial effects compared with the prior art: (1) The use of drone technology combined with three-mode image data processing can realize the detection of foreign objects on the airport surface. The drone can be equipped with a detection device composed of multiple sensors such as high-definition cameras, laser radars and infrared sensors to perform real-time detection of the airport surface in the airport airspace according to a certain flight trajectory. It can realize FOD detection on airport runways or taxiways over large areas and long distances. At the same time, the drone can be maneuvered and flexibly moved to facilitate rapid entry and exit, which avoids the need to close the airport runway for manual inspection under the traditional method, which takes up a lot of time. (2) The electronic map is gridded. After each grid area is numbered, the location information of the airport site can be regarded as the numbered position of different grid areas. At this time, the airport site is inspected in combination with the number of the grid areas. The map information can mark different grid areas with different importance according to their importance, such as ordinary, important, relatively important, extremely important, etc. Different importance marks involve different values. After the obstacle information is collected, since different obstacle information also has different importance marks, when the obstacle information assignment and the map information assignment are superimposed, the FOD inspection urgency of the corresponding location can be effectively displayed. Therefore, the FDP inspection emergency parameter can effectively feedback the FOD inspection urgency; (3) Reduce the use of lidar and infrared sensors by means of inspection and review of flight operations, and reduce the startup time of lidar and infrared sensors to a large extent, thereby achieving the goal of improving the endurance of the UAV platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings: Figure 1 This is a diagram of the execution module of the present invention; Figure 2 This is a diagram of the execution module of the UAV platform of the present invention; Figure 3 This is a flowchart of the video image recognition execution of the present invention. DETAILED DESCRIPTION

[0018] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0019] Example:

[0020] like Figure 1~Figure 2 As shown, this embodiment relates to an airport runway FOD inspection and operation safety assurance system, including a ground control platform and an unmanned aerial vehicle mission platform, wherein: The ground control platform is used to receive the three-mode image data of the UAV platform and send flight control data to the UAV platform; The UAV platform is used to receive flight control data and execute flight actions, collect three-mode image data and send it to the ground control platform; The ground control platform includes a FOD detection module, a map module and a task scheduling module; The FOD detection module is used to receive the three-mode image data and perform video image recognition to generate image recognition data; The map module is provided with a grid electronic map, and the FOD detection module superimposes the image recognition data on the grid electronic map to generate map obstacle data; The task scheduling module is used to receive the map obstacle data and generate flight control data to send to the UAV platform.

[0021] The UAV platform includes a high-definition image acquisition module, a laser radar module, an infrared sensor module, a flight execution module and a wireless transmission module, wherein: The high-definition picture acquisition module is used to acquire video image data; The laser radar module is used to receive the control signal of the flight execution module and collect radar image data according to the control signal, and send the radar image data to the ground control platform; The infrared sensor module is used to receive the control signal of the flight execution module and collect infrared image data according to the control signal, and send the infrared image data to the ground control platform; The flight execution module is used to receive flight control data and execute flight actions. The laser radar module and the infrared sensor module send control signals according to the flight control data items; The wireless transmission module is used for data communication with the ground control platform.

[0022] In this embodiment, drone technology is combined with three-mode image data processing to realize the detection of foreign objects in the airport scene. The drone can be equipped with a detection device composed of multiple sensors such as high-definition cameras, lidar and infrared sensors to perform real-time detection of the airport scene in the airport airspace according to a certain flight trajectory, which can realize FOD detection in large-area and long-distance airport runways or taxiways; at the same time, the drone can be maneuvered and flexibly moved to facilitate rapid entry and exit, which avoids the need to close the airport runway for manual inspection under the traditional method, which takes up a lot of time and the resulting reduction in airport operation efficiency, and also avoids the impact of interrupting flight operations and the resulting economic losses to airlines.

[0023] This embodiment is mainly divided into two parts: the ground control platform and the UAV platform. The UAV platform is responsible for performing scene scanning and data collection, and the ground control platform is responsible for image processing and path planning, and displays the working dynamics in real time. The database of the ground control platform contains basic data such as the electronic map of the mission airport, the airport airspace no-fly zone information, and the working parameters of the UAV platform as the working safety restriction information of this system.

[0024] The task scheduling module is used to receive flight operation dynamic information and adjust flight control data in real time based on the flight operation dynamic information.

[0025] During operation, the ground control platform will access the air traffic control automation system of the mission airport to obtain flight dynamic information during the system operation, and combine the mission time, fleet number, electronic fence and other data set for this mission as the initial setting information for this mission. The work safety restriction information and the initial mission setting information will be used as the constraint conditions for the inspection route planning of the drone mission platform. In addition, the corresponding control instructions can also be issued through the ground control platform during the execution of the inspection mission to interrupt or modify the mission.

[0026] After determining the constraints, the task scheduling module will plan the inspection path of the drone platform for this task according to the corresponding information when planning the flight path. The task scheduling module has five algorithms built-in, including branch cutting algorithm, mileage saving algorithm, artificial bee colony algorithm, continued fraction approximation algorithm and clustering-path algorithm. They generate the FOD inspection flight path of the drone task platform according to the constraints, and then output the optimal path according to the inspection coverage and task completion time. After the spatial positioning conversion and flight control command mapping, the path data is transmitted to the drone platform as the flight control parameters of this FOD inspection task.

[0027] In the process of performing FOD inspection flight missions, the multi-sensor combination detection device carried by the UAV mission platform records the current scene in real time, and transmits the generated video images, laser radar depth images, infrared images and corresponding location information back to the ground control platform in real time through the wireless signal transmission module. The FOD detection module of the ground control platform integrates video object recognition components, laser radar depth image recognition components and infrared image recognition components, which can intelligently analyze different types of image data sent back to determine whether there is FOD on the scene.

[0028] Since this embodiment uses a drone platform to collect data, the drone's battery life becomes an important issue affecting the FOD inspection efficiency. Therefore, on the basis of the constant cruising power of the drone platform, how to improve the drone platform's battery life by reducing power consumption becomes the key to improving the FOD inspection efficiency.

[0029] In this embodiment, the map obstacle data includes map information and obstacle information. The map information marks the importance of different locations through the gridded electronic map, and the obstacle information is marked with importance according to the physical properties of the obstacle. The importance mark is assigned a value according to the critical situation, recorded as an emergency value, the emergency value of the map information and the emergency value of the obstacle information are superimposed, the grid electronic map divides the map into a plurality of grid areas, and the FOD inspection emergency parameters of each grid area are obtained; An early warning threshold is set. When the FOD inspection emergency parameter exceeds the early warning threshold, a safety alarm is issued in the corresponding grid area.

[0030] A review threshold is set, and the review threshold is lower than the warning threshold. When the FOD inspection emergency parameter does not reach the review threshold, the laser radar module and the infrared sensor module are in sleep mode. When the FOD inspection emergency parameter reaches the review threshold but does not exceed the warning threshold, the laser radar module and the infrared sensor module are started, and the ground control platform operates the flight execution module to review and execute the flight in the corresponding grid area.

[0031] During the FOD inspection of airport runways, since the electronic map is gridded, after each grid area is numbered, the location information of the airport site can be regarded as the numbered position of different grid areas. At this time, the airport site is inspected in combination with the numbering of the grid areas, and the map information can mark different grid areas with different importance according to their importance, such as ordinary, important, relatively important, extremely important and other different marks. Different importance marks involve different values. After the obstacle information is collected, since different obstacle information also has different importance marks, when the assignment of the obstacle information and the assignment of the map information are superimposed, the urgency of the FOD inspection at the corresponding location can be effectively displayed, so the FDP inspection emergency parameter can effectively feedback the urgency of the FOD inspection.

[0032] At this time, based on the obtained FOD inspection emergency parameters, the ground control platform can take a variety of different response methods, such as driving the drone platform to review the situation in the grid area where the warning appears, and whether it is necessary to take containment measures to deal with obstacles.

[0033] In this embodiment, the laser radar module and the infrared sensor module are in sleep mode at regular times, thereby ensuring the inspection endurance of the UAV platform at regular times, thereby achieving the purpose of enhancing the application capability of this embodiment.

[0034] The review threshold setting in this embodiment should be relatively low, so as to reduce the probability of misjudgment of this embodiment. When a smaller threat is found, a review inspection will be carried out on the corresponding grid area through review flight, so as to improve the security capability of this embodiment while minimizing the power consumption of the drone platform of this embodiment.

[0035] Since the daily maintenance of airport sites is relatively strict, even if the review threshold is lowered and the obstacle judgment standard of video image recognition is improved, the startup time of lidar and infrared sensors can be reduced to a large extent, thereby achieving the purpose of improving the endurance of the UAV platform.

[0036] like Figure 3 As shown, the video image recognition in the FOD detection module includes the following steps: Acquire video image data and retrieve original images, and compare the video image data with the original images; Record the difference between the video image data and the original image to generate first identification data; Sending the first identification data to a map module, and observing whether there is an obstacle threat in conjunction with the map module; If there is no obstacle threat, the first identification data and the UAV platform flight trajectory data in the flight control data are combined and sent to the map module to generate map obstacle data; If there is an obstacle threat, radar image data and infrared image data are obtained, and deep learning target detection is performed by combining the video image data, radar image data and infrared image data to obtain second recognition data; The second identification data and the UAV platform flight trajectory data in the flight control data are combined and sent to the map module to generate map obstacle data.

[0037] When the task scheduling module obtains the map obstacle data, it determines whether the second identification data is obtained: If the first identification data is obtained, flight control data is generated according to the preset flight trajectory and sent to the UAV platform; If the first identification data is not obtained, the first flight control data is generated for the map position where the obstacle threat exists in the map obstacle data, and sent to the UAV platform.

[0038] In this embodiment, the ground control platform stores the original image, which includes basic data such as the electronic map of the mission airport and the no-fly zone information of the airport airspace. In combination with the working parameters of the UAV platform, the video image data can be effectively superimposed on the gridded electronic map. Therefore, based on the difference between the video image data and the original image, it is possible to find out whether there are obstacles at the corresponding position and other situations that need to be investigated in the FOD inspection. After eliminating misjudgment through image recognition calculation, the first recognition data can be obtained. The image recognition calculation in this embodiment includes image recognition processing technologies such as biometric recognition and deep learning image recognition.

[0039] After the first identification data is obtained, it is determined whether the review threshold is reached through combination with the map module. When the review threshold is reached, the task scheduling module sends a control signal for reviewing the flight to the UAV platform, thereby obtaining radar image data and infrared image data through reviewing the flight. Relying on the combination of radar image data and infrared image data, more accurate video image data can be obtained, thereby updating the second identification data. The obtained second identification data can accurately feedback accurate map obstacle data, thereby obtaining accurate FOD inspection emergency parameters. This embodiment can effectively reduce the use time of lidar and infrared sensors by distinguishing between the first identification data and the second identification data, thereby saving the power consumption of the UAV platform and improving its endurance.

[0040] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An airport runway FOD inspection and operation safety assurance system, characterized in that: It includes ground control platform and UAV mission platform, among which, The ground control platform is used to receive the three-mode image data of the UAV platform and send flight control data to the UAV platform; The UAV platform is used to receive flight control data and execute flight actions, collect three-mode image data and send it to the ground control platform; The ground control platform includes a FOD detection module, a map module and a task scheduling module; The FOD detection module is used to receive the three-mode image data and perform video image recognition to generate image recognition data; The map module is provided with a grid electronic map, and the FOD detection module superimposes the image recognition data on the grid electronic map to generate map obstacle data; The task scheduling module is used to receive the map obstacle data and generate flight control data to send to the UAV platform.

2. The airport runway FOD inspection and operation safety guarantee system according to claim 1 is characterized in that: The UAV platform includes a high-definition image acquisition module, a laser radar module, an infrared sensor module, a flight execution module and a wireless transmission module, wherein: The high-definition picture acquisition module is used to acquire video image data; The laser radar module is used to receive the control signal of the flight execution module and collect radar image data according to the control signal, and send the radar image data to the ground control platform; The infrared sensor module is used to receive the control signal of the flight execution module and collect infrared image data according to the control signal, and send the infrared image data to the ground control platform; The flight execution module is used to receive flight control data and execute flight actions, and send control signals to the laser radar module and the infrared sensor module according to the flight control data; The wireless transmission module is used for data communication with the ground control platform.

3. The airport runway FOD inspection and operation safety guarantee system according to claim 2 is characterized in that: The map obstacle data includes map information and obstacle information, wherein the map information marks the importance of different locations through the gridded electronic map, and the obstacle information is marked with importance according to the physical properties of the obstacle; The importance mark is assigned a value according to the critical situation, recorded as an emergency value, the emergency value of the map information and the emergency value of the obstacle information are superimposed, the grid electronic map divides the map into a plurality of grid areas, and the FOD inspection emergency parameters of each grid area are obtained; An early warning threshold is set. When the FOD inspection emergency parameter exceeds the early warning threshold, a safety alarm is issued in the corresponding grid area.

4. The airport runway FOD inspection and operation safety guarantee system according to claim 3 is characterized in that: A review threshold is set, and the review threshold is lower than the warning threshold. When the FOD inspection emergency parameter does not reach the review threshold, the laser radar module and the infrared sensor module are in sleep mode. When the FOD inspection emergency parameter reaches the review threshold but does not exceed the warning threshold, the laser radar module and the infrared sensor module are started, and the ground control platform operates the flight execution module to review and execute the flight in the corresponding grid area.

5. The airport runway FOD inspection and operation safety guarantee system according to claim 2 is characterized in that: The video image recognition in the FOD detection module includes the following steps: Acquire video image data and retrieve original images, and compare the video image data with the original images; Record the difference between the video image data and the original image to generate first identification data; Sending the first identification data to a map module, and observing whether there is an obstacle threat in conjunction with the map module; If there is no obstacle threat, the first identification data and the UAV platform flight trajectory data in the flight control data are combined and sent to the map module to generate map obstacle data; If there is an obstacle threat, radar image data and infrared image data are obtained, and deep learning target detection is performed by combining the video image data, radar image data and infrared image data to obtain second recognition data; The second identification data and the UAV platform flight trajectory data in the flight control data are combined and sent to the map module to generate map obstacle data.

6. The airport runway FOD inspection and operation safety guarantee system according to claim 5 is characterized in that: When the task scheduling module obtains the map obstacle data, it determines whether the second identification data is obtained: If the first identification data is obtained, flight control data is generated according to the preset flight trajectory and sent to the UAV platform; If the first identification data is not obtained, the first flight control data is generated for the map position where the obstacle threat exists in the map obstacle data, and sent to the UAV platform.

7. The airport runway FOD inspection and operation safety guarantee system according to claim 1 is characterized in that: The task scheduling module is used to receive flight operation dynamic information and adjust flight control data in real time based on the flight operation dynamic information.

Images