Airport scene FOD inspection and operation safety guarantee system based on unmanned system

Abstract

The invention discloses an airport scene FOD inspection and operation safety guarantee system based on an unmanned system. An unmanned aerial vehicle technology and a deep learning target detection algorithm are combined to realize detection of airport scene foreign matters. The unmanned aerial vehicle can carry detection equipment combined by multiple sensors such as a high-definition camera, a laser radar and an infrared sensor, carries out real-time detection on an airport scene according to a certain flight path in an airport airspace, and can realize FOD detection in a large-area and long-distance airport runway or taxiway.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to an airport surface FOD inspection and operation safety guarantee system based on an unmanned system. Background Art

[0002] The airport surface is a key area for aircraft takeoff, landing and taxiing. Any foreign object on the surface may have a serious impact on the safe operation of the aircraft and cause serious consequences. Generally, fixed detection base stations on both sides of the runway are used to detect mainly by millimeter-wave radar and supplemented by video image recognition. Multi-point video image recognition technology can also be used for detection. Although these systems can achieve active detection of FOD, their detection accuracy and range will be affected by the installation method and location of the system. To achieve large-range and high-precision FOD detection, a large number of detection points need to be arranged simultaneously, which greatly increases the system cost. Moreover, limited by the complex electromagnetic environment and surface layout of the airport, the installation and use of detection points have great limitations, restricting the application and promotion of such systems. Currently, most airports still rely on pavement inspection personnel for manual inspections, which requires closing the runway for inspections. This not only has low efficiency and poor reliability, but also occupies valuable runway use time. In addition, the current airport runway foreign object detection system has not been widely applied because the existing system has a high cost and there is no corresponding mature available technology or product.

[0003] Therefore, it is necessary to research and develop a low-cost, large detection range, convenient and practical airport runway FOD inspection and operation safety guarantee system, so that airport security personnel can realize remote monitoring of the foreign object safety situation in the surface area with a large area coverage, and perform fixed-point marking on the detected foreign objects, and send the corresponding information to the airport operation monitoring system for recording; it can also plan the best path for fixed-point removal of foreign objects according to the marked information, minimize the workload of airport security personnel, reduce the time and labor costs spent by the airport on FOD detection and cleaning, and improve the airport operation efficiency; at the same time, improve the FOD detection accuracy rate and reduce the safety risks of airport operation.

[0004] At the same time, from a management perspective, managers can analyze the safety situation of flights and airport operations based on the FOD detection data, provide references for optimizing the corresponding operation rules and regulations, and have important significance for the high-quality development of civil aviation. Summary of the Invention

[0005] The purpose of the present invention is to provide an airport surface FOD inspection and operation safety guarantee system based on an unmanned system to solve the above problems existing in the prior art.

[0006] The airport surface FOD inspection and operation safety guarantee system based on the unmanned system includes a detection device, an FOD detection module, a task scheduling module, and a wireless data transmission module: The detection device includes a high-definition camera, a lidar, and an infrared sensor; the detection device is fixed on the unmanned aerial vehicle; The task scheduling module is used to adjust the shooting position of the detection device to obtain a second detection path; the second detection path consists of multiple shooting positions as the nodes of the path; the second detection path is the shortest path and the images taken by all the nodes can obtain the complete surface of the detection object; the detection object refers to the object that needs to be inspected; The detection device is used to obtain a set of three-mode images corresponding to the nodes on the second detection path; the set of three-mode images includes a laser depth image, an infrared image, and a camera image; The FOD detection module is used to obtain multiple foreign object positions based on the set of three-mode images through an image recognition network; The task scheduling module is used to perform path planning based on multiple foreign object positions to obtain a foreign object detection path; the foreign object detection path represents the path for removing foreign objects.

[0007] Optionally, the task scheduling module includes a constraint condition module, an algorithm module, and a deep learning module; The constraint condition module is used to obtain multiple constraint data; Multiple path detection algorithms are built in the algorithm module; The deep learning module has a built-in path detection network; The path detection network is trained using multiple path detection algorithms.

[0008] Optionally, the adjustment of the shooting position of the detection device to obtain a second detection path includes: According to multiple constraint data, mark the positions where the unmanned aerial vehicle is restricted from flying to obtain a three-dimensional area image; the positions with a gray value of 1 in the three-dimensional area image represent the positions where the unmanned aerial vehicle can fly, and the positions with a gray value of 0 represent the restricted flying positions; Based on the three-dimensional area image, obtain a three-dimensional detection object position image and an initial detection path; the three-dimensional detection object position image represents an image marked with the positions of the detection object and the flight restriction positions; the initial detection path represents an initially set path for obtaining the complete surface of the detection object; Through the trained path detection network, based on the three-dimensional detection object position image, discriminate multiple path detection algorithms to obtain a second detection path; Replace the initial detection path with the second detection path.

[0009] Optionally, the training of the path detection network using multiple path detection algorithms includes: Obtain a training three-dimensional detection object position image; the training three-dimensional detection object position image represents the three-dimensional detection object position image used for training at a historical time point; Based on the training three-dimensional detection object position image, through multiple path detection algorithms, obtain a first detection path; the first detection path represents the shortest path among the multiple path detection algorithms that can obtain the complete surface of the detection object; Take the path detection algorithm corresponding to the first detection path as the determination algorithm; Based on the training three-dimensional detection object position image, through a path detection network, obtain a first detection algorithm; the first detection algorithm represents the predicted algorithm suitable for the three-dimensional detection object position image; Calculate the loss between the determination algorithm and the first detection algorithm to train the path detection network.

[0010] Optionally, the step of obtaining a first detection path based on the training three-dimensional detection object position image through multiple path detection algorithms includes: The input of the path detection algorithm is the position of the detection object in the training three-dimensional detection object position image and multiple constraint data in the corresponding constraint condition module; Multiple path detection algorithms respectively obtain multiple training detection paths; the training detection paths represent the paths that can obtain the complete surface of the detection object corresponding to the path detection algorithms; Among the multiple training detection paths, take the training detection path with the fewest number of nodes as the first detection path.

[0011] Optionally, the step of obtaining a first detection algorithm based on the training three-dimensional detection object position image through a path detection network includes: The input of the path detection network is the training three-dimensional detection object position image; The output of the path detection network is the first detection algorithm.

[0012] Optionally, the step of discriminating multiple path detection algorithms based on the three-dimensional detection object position image through a trained path detection network to obtain a second detection path includes: Input the three-dimensional detection object position image into the trained path detection network to obtain a second detection algorithm; Input multiple constraint data in the constraint condition module and the position of the detection object in the three-dimensional detection object position image into the second detection algorithm to obtain a second detection path.

[0013] Optionally, the number of output neurons of the path detection network is equal to the number of built-in path detection algorithms in the algorithm module.

[0014] Optionally, the step of obtaining multiple foreign object positions based on the three-mode image set through an image recognition network includes: The image recognition network includes a first foreign object detection network, a second foreign object detection network, and a third foreign object detection network; Input the laser depth image, the infrared image, and the camera image into the first foreign object detection network, the second foreign object detection network, and the third foreign object detection network respectively, to obtain a first foreign object feature, a second foreign object feature, and a third foreign object feature; Superimpose the first foreign object feature, the second foreign object feature, and the third foreign object feature to obtain an overlapping feature; Input the overlapping feature into a fully connected neural network to obtain multiple foreign object positions.

[0015] Optionally, obtaining a three-dimensional detection object position image and an initial detection path based on the three-dimensional region image includes: Obtain multiple laser perspective images; the laser perspective images represent images containing detection objects acquired by a lidar at multiple perspectives of the unmanned aerial vehicle; Arrange the positions of the unmanned aerial vehicle corresponding to the multiple laser perspective images in the order of time points from early to late to obtain an initial detection path; Perform three-dimensional reconstruction on the multiple laser perspective images to obtain a detection object position image; the detection object position image represents the position of the detection object in three dimensions; Mark the positions corresponding to the detection objects in the detection object position image in the three-dimensional region image to obtain a three-dimensional detection object position image.

[0016] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects: Through the training of multiple dynamic programming path detection algorithms, a path detection network suitable for the current constraint data and detection objects can be identified, enabling a more accurate and rapid finding of the optimal path.

[0017] The combination of unmanned aerial vehicle technology and deep learning object detection algorithms is used to detect foreign objects on the airport surface. The unmanned aerial vehicle can carry a detection device combined with multiple sensors such as a high-definition camera, a lidar, and an infrared device, and perform real-time detection of the airport surface in the airport airspace according to a certain flight trajectory, and can realize the detection of FOD in large areas and at long distances on airport runways or taxiways; at the same time, the unmanned aerial vehicle can perform flexible displacement, facilitating quick takeoff and landing, which avoids the need to close the airport runway for manual inspection in the traditional method, thus occupying a large amount of time and the resulting reduction in airport operation efficiency, and also avoids the impact of interrupting flight operations and the resulting economic losses to airlines.

[0018] Implement intelligent inspection of runway FOD based on drones and multi-sensors, significantly improving the detection efficiency and accuracy of FOD. The inspection route planning of the drone is optimized, which can reduce energy consumption and improve operation efficiency. In addition, while enhancing the automation of task execution, the inspection flight path of the drone in this system ensures a safe interval when the drone operates in mixed mode with aircraft, preventing potential conflicts with the same flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flowchart of a method executed in an airport surface FOD inspection and operation safety assurance system based on an unmanned system provided by an embodiment of the present invention.

[0020] Figure 2 is a schematic structural diagram of an algorithm module and a deep learning module in an airport surface FOD inspection and operation safety assurance system based on an unmanned system provided by an embodiment of the present invention.

[0021] Figure 3 is a working flowchart of an airport surface FOD inspection and operation safety assurance system based on an unmanned system provided by an embodiment of the present invention.

[0022] Figure 4 is a principle block diagram of an airport surface FOD inspection and operation safety assurance system based on an unmanned system provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The present invention will be described in detail below with reference to the accompanying drawings.

[0024] Embodiment

[0025] An embodiment of the present invention provides an airport surface FOD inspection and operation safety assurance system based on an unmanned system, including a detection device, an FOD detection module, a task scheduling module, and a wireless data transmission module: Among them, the wireless data transmission module is used to transmit the drone detection information to the FOD detection module.

[0026] The detection device includes a high-definition camera, a lidar, and an infrared sensor; the detection device is fixed on the drone.

[0027] The task scheduling module is used for, S101: Adjust the shooting position of the detection device to obtain a second detection path; the second detection path consists of multiple shooting positions as nodes of the path; the second detection path is the shortest path and the images taken by all nodes can obtain the complete surface of the detected object; the detected object refers to the object that needs to be inspected.

[0028] Among them, the shooting position represents the position where the detection device shoots the laser depth image, the infrared image, and the camera image.

[0029] Among them, in this embodiment, the number of nodes in the path is set to n, the side length connecting the nodes is set to m, and the value of 0.7n + 0.5m is used as the shortest path. n and m are positive integers.

[0030] Among them, the reason why the images taken by all the nodes can obtain the complete surface of the detected object is that each node can obtain a part of the detected object when shooting, and the images in all directions taken by all the nodes can obtain all regions of the surface of the detected object.

[0031] Among them, the airport surface FOD inspection and operation safety guarantee system based on the unmanned system further includes a positioning module. The positioning module is used on the UAV platform to control the detection device to go to the shooting position to perform airport surface scanning and data collection work. In this embodiment, the positioning module is a Beidou-inertial navigation positioning module.

[0032] Among them, image processing and path planning are performed on the ground console through the FOD detection module and the task scheduling module. Various types of data such as images and paths are used to display the work dynamics in real time on the data visualization platform.

[0033] Among them, data is transmitted between the ground console and the UAV platform through a wireless data transmission module. In this embodiment, the wireless data transmission module is a 5G / wireless data transmission module.

[0034] Among them, the task scheduling module includes a cleaning path planning component, an airport operation electronic situation component, and a flight path planning component. The cleaning path planning component is used to obtain the path for the staff to clean foreign objects. The flight path planning component is used to obtain the path for the UAV to fly. The airport operation electronic situation component is used to judge whether the detected object in the airport is in a static or moving state.

[0035] Among them, the principle block diagram of the airport surface FOD inspection and operation safety guarantee system based on the unmanned system is as Figure 4 shown.

[0036] Among them, the second detection path is obtained by the flight path planning component.

[0037] Among them, the second detection path is the path optimized according to Figure 3 the strategy of maximum coverage and fastest speed described above. In the strategy optimization, the maximum coverage is represented by obtaining the complete surface of the detected object. Path mapping and instruction generation are used for path generation. The path mapping is used to map to the second detection path, and the instruction generation is used to generate an instruction that has obtained the second detection path.

[0038] The detection device is used for, S102: Obtain the set of three - mode images corresponding to the nodes on the second detection path; the set of three - mode images includes a laser depth image, an infrared image, and a camera image.

[0039] Among them, during the execution of the FOD inspection flight mission, the detection device carried by the UAV platform records the current scene in real - time, and transmits the generated laser depth image, infrared image, and camera image back to the ground console in real - time through a 5G wireless data transmission module.

[0040] Among them, as Figure 3 shown, flight control instructions are used for video image acquisition, infrared image acquisition, and lidar depth image acquisition. The flight control instructions represent the instructions for controlling the UAV to fly to the shooting position for shooting.

[0041] The FOD detection module is used for, S103: Through an image recognition network, based on the set of three - mode images, obtain multiple foreign object positions.

[0042] Among them, the foreign object positions are detected through an image recognition network and displayed on the data visualization platform.

[0043] Among them, as Figure 3 shown, the set of three - mode images is used as image data for object detection based on deep learning. In this embodiment, an image recognition network is used for object detection.

[0044] The task scheduling module is used for, S104: According to multiple foreign object positions, perform path planning to obtain a foreign object detection path; the foreign object detection path represents the path for removing foreign objects.

[0045] Among them, through Figure 3 shown, exact algorithms, heuristic algorithms, multi - stage algorithms, continuous approximation algorithms, and meta - heuristic algorithms in algorithm selection are used to find the path with the shortest distance between foreign object positions for path mapping.

[0046] Among them, as Figure 3 shown, the foreign object detection path obtained by path mapping is output as navigation information.

[0047] Among them, the flowchart of the method executed in the airport scene FOD inspection and operation safety guarantee system based on the unmanned system is as Figure 1 shown.

[0048] Optionally, the task scheduling module includes a constraint condition module, an algorithm module, and a deep learning module.

[0049] Among them, the structural schematic diagrams of the algorithm module and the deep learning module are asFigure 2 as shown Figure 2 In the three-dimensional detected object position image, the position of the detected object and the multiple constraint data in the corresponding constraint condition module represent the data input during detection. Therefore, the corresponding first training detection path, second training detection path, and first detection path represent the paths obtained during detection.

[0050] The constraint condition module is used to obtain multiple constraint data.

[0051] Among them, the constraint data is obtained by the airport operation electronic situation component.

[0052] Among them, in this embodiment, the constraint data includes safety restrictions, mission constraints, and special instructions.

[0053] Among them, the work safety restriction information is stored in the database of the ground console. The constraint data includes an electronic map, no-fly zone, and endurance as safety restrictions. Mission constraints include mission time, fleet size, electronic fence, and flight dynamics. The flight dynamics indicates whether the airport is shut down. The electronic fence represents the area that the drone cannot pass through; special instructions represent the instructions for controlling the drone due to mission interruption or emergency evacuation. The fleet size represents the number of fleets in the airport. The special instructions include instructions for mission interruption and emergency evacuation. When receiving special instructions, the drone is controlled to stop operating.

[0054] A plurality of path detection algorithms are built into the algorithm module.

[0055] Among them, the path detection algorithms include exact algorithms, heuristic algorithms, multi-stage algorithms, continuous approximation algorithms, and meta-heuristic algorithms. In this embodiment, the branch-and-cut algorithm, savings mileage algorithm, artificial bee colony algorithm, continued fraction approximation algorithm, and clustering-path algorithm are adopted.

[0056] The deep learning module has a built-in path detection network.

[0057] A plurality of path detection algorithms are used to train the path detection network.

[0058] Optionally, adjusting the shooting position of the detection device to obtain a second detection path includes: Marking the positions where the drone is restricted from flying according to multiple constraint data to obtain a three-dimensional area image; the positions with a gray value of 1 in the three-dimensional area image represent the positions where the drone can fly, and the positions with a gray value of 0 represent the restricted flying positions.

[0059] Among them, the three-dimensional area image is a gray image.

[0060] Based on the three-dimensional regional image, obtain a three-dimensional detected object position image and an initial detection path; the three-dimensional detected object position image represents an image marked with the positions of the detected object and the no-fly positions; the initial detection path represents a path that is initially set and used to obtain the complete surface of the detected object.

[0061] Through the trained path detection network, based on the three-dimensional detected object position image, discriminate multiple path detection algorithms to obtain a second detection path.

[0062] Replace the initial detection path with the second detection path.

[0063] Optionally, the training of the path detection network using multiple path detection algorithms includes: Obtain a training three-dimensional detected object position image; the training three-dimensional detected object position image represents the three-dimensional detected object position image used for training at a historical time point.

[0064] Based on the training three-dimensional detected object position image, through multiple path detection algorithms, obtain a first detection path. The first detection path represents the shortest path among the multiple path detection algorithms that can obtain the complete surface of the detected object.

[0065] Among them, the first detection path is the path obtained during training.

[0066] Use the path detection algorithm corresponding to the first detection path as the determination algorithm.

[0067] Among them, the determination algorithm is represented by the label of the corresponding path detection network.

[0068] Based on the training three-dimensional detected object position image, through the path detection network, obtain a first detection algorithm; the first detection algorithm represents the algorithm predicted to be suitable for the three-dimensional detected object position image.

[0069] Among them, the first detection algorithm is represented by the label of the corresponding path detection network.

[0070] Calculate the loss between the determination algorithm and the first detection algorithm to train the path detection network.

[0071] Among them, in this embodiment, the cross-entropy loss function is used to calculate the loss value.

[0072] Optionally, the obtaining of the first detection path through multiple path detection algorithms based on the training three-dimensional detected object position image includes: The input of the path detection algorithm is the position of the detected object in the training three-dimensional detected object position image and multiple constraint data in the corresponding constraint condition module.

[0073] Among them, the constraint data in the constraint condition module is also the constraint data for training at a historical time point.

[0074] Multiple path detection algorithms respectively obtain multiple training detection paths; the training detection paths represent the paths corresponding to the path detection algorithms that can obtain the complete surface of the detected object.

[0075] Among the multiple training detection paths, the training detection path with the shortest path is used as the first detection path.

[0076] Among them, in this embodiment, the branch cutting algorithm is used as the first path detection algorithm, and the output optimal path is used as the first training detection path. The saving mileage algorithm is used as the second path detection algorithm, and the output optimal path is used as the second training detection path. The artificial bee colony algorithm is used as the third path detection algorithm, and the output optimal path is used as the third training detection path. The continued fraction approximation algorithm is used as the fourth path detection algorithm, and the output optimal path is used as the fourth training detection path. The clustering-path algorithm is used as the fifth path detection algorithm, and the output optimal path is used as the fifth training detection path. The shortest path among the first training detection path, the second training detection path, the third training detection path, the fourth training detection path, and the fifth training detection path is used as the first detection path.

[0077] Optionally, obtaining the first detection algorithm based on the training three-dimensional detected object position image through a path detection network includes: The input of the path detection network is the three-dimensional detected object position image; The output of the path detection network is the first detection algorithm.

[0078] Among them, in this embodiment, the path detection network is a three-dimensional convolutional neural network (3D Convolutional Neural Networks, 3D CNN).

[0079] Optionally, discriminating multiple path detection algorithms based on the three-dimensional detected object position image through a trained path detection network to obtain a second detection path includes: Inputting the three-dimensional detected object position image into the trained path detection network to obtain a second detection algorithm; Inputting the multiple constraint data in the constraint condition module and the position of the detected object in the three-dimensional detected object position image into the second detection algorithm to obtain the second detection path.

[0080] Optionally, the number of output neurons of the path detection network is equal to the number of built-in path detection algorithms in the algorithm module.

[0081] Optionally, obtaining multiple foreign object positions based on the three-mode image set through an image recognition network includes: The image recognition network includes a first foreign object detection network, a second foreign object detection network, and a third foreign object detection network.

[0082] Input the laser depth image, infrared image, and camera image into the first foreign object detection network, the second foreign object detection network, and the third foreign object detection network respectively to obtain a first foreign object feature, a second foreign object feature, and a third foreign object feature.

[0083] Among them, in this embodiment, the first foreign object detection network, the second foreign object detection network, and the third foreign object detection network are yolov8 networks. Because the input data is different, the parameters of the networks trained by the first foreign object detection network, the second foreign object detection network, and the third foreign object detection network are different.

[0084] Among them, in this embodiment, as Figure 4 shown, the FOD detection module integrates a video image recognition component, a lidar depth image recognition component, and an infrared graphic recognition component, which can intelligently analyze different types of image data transmitted back. Different types of image data are used to determine whether there is foreign object debris (FOD) on the airport surface. If there is a foreign object, the shooting position of the transmitted back image data is extracted. If there is no foreign object, a signal indicating non-existence is sent. The YOLOv8 target detection algorithm is integrated in the video image recognition component of this system.

[0085] Through the above method, because the YOLOv8 target detection algorithm has an efficient detection effect and a very wide application range, it can accurately identify and detect foreign objects existing on the airport surface.

[0086] Overlay the first foreign object feature, the second foreign object feature, and the third foreign object feature to obtain an overlapping feature.

[0087] Input the overlapping feature into a fully connected neural network to obtain multiple foreign object positions.

[0088] Among them, in this embodiment, the fully connected neural network (FCN).

[0089] Optionally, obtaining the three-dimensional detection object position image and the initial detection path based on the three-dimensional region image includes: Obtain multiple laser perspective images; the laser perspective images represent images containing the detection object acquired by the lidar at multiple perspectives of the unmanned aerial vehicle.

[0090] Among them, multiple laser perspective images can obtain the complete surface of the detection object; Arrange multiple laser perspective images corresponding to the position of the drone in the order from early to late time points to obtain an initial detection path.

[0091] Perform three-dimensional reconstruction on multiple laser perspective images to obtain a detected object position image; the detected object position image represents the position of the detected object in three dimensions.

[0092] Among them, in this embodiment, multiple laser perspective images are input into a Neural Radiance Fields (NeRF) for three-dimensional reconstruction.

[0093] Mark the positions corresponding to the detected objects in the detected object position image in the three-dimensional region image to obtain a three-dimensional detected object position image.

[0094] Among them, the three-dimensional region image and the detected object position image have the same size.

[0095] Among them, mark with the value 2, and the positions with a gray value of 2 in the three-dimensional detected object position image represent the positions of the detected objects.

[0096] Among them, the three-dimensional detected object position image is a grayscale image.

[0097] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The structure required to construct such a system will be apparent from the above description. In addition, the present invention is not directed to any particular programming language. It should be understood that the content of the present invention described herein can be implemented using various programming languages, and the description of the specific language above is for disclosing the best mode of the present invention.

[0098] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

Claims

1. An airport surface FOD inspection and operation safety guarantee system based on an unmanned system, characterized in that, It includes a detection device, an FOD detection module, a task scheduling module, and a wireless data transmission module: The detection device includes a high-definition camera, a lidar, and an infrared sensor; the detection device is fixed on a drone; The task scheduling module is used to adjust the shooting position of the detection device to obtain a second detection path; the second detection path consists of multiple shooting positions as the nodes of the path; the second detection path is the shortest path and the images captured by all the nodes can obtain the complete surface of the detected object; The detected object refers to the object that needs to be inspected; The detection device is used to obtain a set of three-mode images corresponding to the nodes on the second detection path; the set of three-mode images includes a laser depth image, an infrared image, and a camera image; The FOD detection module is used to obtain multiple foreign object positions based on the set of three-mode images through an image recognition network; The task scheduling module is used to perform path planning according to multiple foreign object positions to obtain a foreign object detection path; the foreign object detection path represents the path for removing foreign objects.

2. The airport surface FOD inspection and operation safety guarantee system based on an unmanned system according to claim 1, wherein, The task scheduling module includes a constraint condition module, an algorithm module, and a deep learning module; The constraint condition module is used to obtain multiple constraint data; Multiple path detection algorithms are built in the algorithm module; The deep learning module has a built-in path detection network; The path detection network is trained using multiple path detection algorithms.

3. The airport surface FOD inspection and operation safety guarantee system based on an unmanned system according to claim 2, wherein The adjusting the shooting position of the detection device to obtain a second detection path includes: According to multiple constraint data, mark the positions where the drone is restricted from flying to obtain a three-dimensional area image; the positions with a gray value of 1 in the three-dimensional area image represent the positions where the drone can fly, and the positions with a gray value of 0 represent the restricted flying positions; Based on the three-dimensional area image, obtain a three-dimensional detected object position image and an initial detection path; the three-dimensional detected object position image represents an image marked with the position of the detected object and the restricted flying positions; the initial detection path represents an initially set path for obtaining the complete surface of the detected object; Through the trained path detection network, based on the three-dimensional detected object position image, discriminate multiple path detection algorithms to obtain a second detection path; Use the second detection path to replace the initial detection path.

4. The airport surface FOD inspection and operation safety guarantee system based on the unmanned system according to claim 2, characterized in that The training the path detection network using multiple path detection algorithms includes: Obtain a training three-dimensional detected object position image; the training three-dimensional detected object position image represents the three-dimensional detected object position image used for training at a historical time point; Based on the training three-dimensional detected object position image, through multiple path detection algorithms, obtain a first detection path; the first detection path represents the shortest path among multiple path detection algorithms that can obtain the complete surface of the detected object; Take the path detection algorithm corresponding to the first detection path as the determined algorithm; Based on the training three-dimensional detected object position image, through the path detection network, obtain a first detection algorithm; the first detection algorithm represents the algorithm predicted to be suitable for the three-dimensional detected object position image; Calculate the loss between the determined algorithm and the first detection algorithm to train the path detection network.

5. The airport surface FOD inspection and operation safety guarantee system based on the unmanned system according to claim 4, characterized in that, The obtaining the first detection path through multiple path detection algorithms based on the training three-dimensional detected object position image includes: The input of the path detection algorithm is the position of the detected object in the training three-dimensional detected object position image and multiple constraint data in the corresponding constraint condition module; Multiple path detection algorithms respectively obtain multiple training detection paths; the training detection paths represent the paths that can obtain the complete surface of the detected object corresponding to the path detection algorithms; Among the multiple training detection paths, the training detection path with the fewest number of nodes is used as the first detection path.

6. The airport surface FOD inspection and operation safety guarantee system based on the unmanned system according to claim 4, characterized in that Based on the training three-dimensional detected object position image, through the path detection network, to obtain the first detection algorithm, including: The input of the path detection network is the training three-dimensional detected object position image; The output of the path detection network is the first detection algorithm.

7. The airport surface FOD inspection and operation safety guarantee system based on an unmanned system according to claim 3, wherein Through the trained path detection network, based on the three-dimensional detected object position image, to discriminate multiple path detection algorithms, to obtain the second detection path, including: Input the three-dimensional detected object position image into the trained path detection network to obtain the second detection algorithm; Input the multiple constraint data in the constraint condition module and the position of the detected object in the three-dimensional detected object position image into the second detection algorithm to obtain the second detection path.

8. The airport surface FOD inspection and operation safety guarantee system based on an unmanned system according to claim 2, wherein, The number of output neurons of the path detection network is equal to the number of built-in path detection algorithms in the algorithm module.

9. The airport surface FOD inspection and operation safety guarantee system based on an unmanned system according to claim 1, characterized in that Based on the three-mode image set through the image recognition network, to obtain multiple foreign object positions, including: The image recognition network includes a first foreign object detection network, a second foreign object detection network, and a third foreign object detection network; Input the laser depth image, the infrared image, and the camera image into the first foreign object detection network, the second foreign object detection network, and the third foreign object detection network respectively to obtain a first foreign object feature, a second foreign object feature, and a third foreign object feature; Overlay the first foreign object feature, the second foreign object feature, and the third foreign object feature to obtain an overlapping feature; Input the overlapping feature into the fully connected neural network to obtain multiple foreign object positions.

10. The airport surface FOD inspection and operation safety guarantee system based on an unmanned system according to claim 3, wherein Based on the three-dimensional region image, to obtain the three-dimensional detected object position image and the initial detection path, including: Obtain multiple laser perspective images; the laser perspective images represent the images containing the detected object acquired by the lidar at multiple perspectives of the unmanned aerial vehicle; Arrange the positions of the unmanned aerial vehicle corresponding to the multiple laser perspective images in the order from the earliest time point to the latest time point to obtain the initial detection path; Perform three-dimensional reconstruction on the multiple laser perspective images to obtain the detected object position image; the detected object position image represents the position of the detected object in three dimensions; Mark the position corresponding to the detected object in the detected object position image in the three-dimensional region image to obtain the three-dimensional detected object position image.

Images