Airport runway foreign object detection methods, devices and electronic equipment

Abstract

This invention provides a method, apparatus, and electronic device for detecting foreign objects (FOOs) on airport runways. The method involves acquiring first laser point cloud data of the airport runway using each lidar sensor; dividing the first laser point cloud data into ground point cloud data and non-ground point cloud data; if at least one potential FEO is detected based on the non-ground point cloud data, each potential FEO is then screened; if a target FEO is detected from the at least one potential FEO, the target FEO is located to determine its position information. This method divides the first laser point cloud data acquired by multiple lidar sensors into non-ground point cloud data, detects potential FEOs based on the non-ground point cloud data, and then further screens and detects the target FEO, thereby improving the efficiency and accuracy of FEO detection on airport runways.

Description

Technical Field

[0001] This invention relates to the field of foreign object detection technology for airport runways, and in particular to a method, apparatus and electronic equipment for detecting foreign objects on airport runways. Background Technology

[0002] Currently, airports worldwide are increasingly demanding higher standards for runway foreign object detection, especially with the development of drones and electric aircraft, which will exacerbate the impact of foreign objects on aircraft. Common technologies involve firing a linear laser beam at the target area. If a protrusion is encountered, laser deformation can be observed from a specific angle using two-dimensional image data returned by a high-speed camera. Image preprocessing and foreign object feature extraction analysis are then performed based on the two-dimensional data returned by sensors, and a threshold value for the foreign object features is used to determine whether there is foreign object interference on the runway. However, these technologies suffer from low efficiency and accuracy in runway foreign object detection. Summary of the Invention

[0003] The purpose of this invention is to provide an airport foreign object detection method, device, and electronic equipment to improve the efficiency and accuracy of foreign object detection on airport runways.

[0004] This invention provides a method for detecting foreign objects on an airport runway. Multiple lidar sensors are installed at preset intervals on both sides of the airport runway. The method includes: acquiring first lidar point cloud data of the airport runway using each lidar; dividing the first lidar point cloud data into ground point cloud data and non-ground point cloud data; if at least one potential foreign object is detected based on the non-ground point cloud data, filtering each potential foreign object; if a target foreign object is detected from the at least one potential foreign object, locating the target foreign object to determine its position information.

[0005] Furthermore, the steps of dividing the first laser point cloud data into ground point cloud data and non-ground point cloud data include: obtaining a pre-constructed prior distribution model of airport runway ground point cloud; evaluating the simulated point cloud data using the prior distribution model of airport runway ground point cloud to obtain evaluation results; determining the target fitting ground benchmark based on the preset uniform interior point distribution model and the evaluation results; and dividing the first laser point cloud data into ground point cloud data and non-ground point cloud data according to the target fitting ground benchmark.

[0006] Furthermore, if at least one potential foreign object is detected based on non-ground point cloud data, the step of screening each potential foreign object includes: if there is a first point cloud data in the non-ground point cloud data with a point cloud height higher than a first threshold, the object corresponding to the first point cloud data is identified as at least one potential foreign object extracted from the non-ground point cloud data; from the potential foreign objects, objects corresponding to the second point cloud data with a point cloud height higher than a second threshold are excluded, so as to screen each potential foreign object; wherein, the second threshold is greater than the first threshold.

[0007] Furthermore, the method also includes: if a first foreign object is selected from at least one potential foreign object, and the height of the first foreign object is less than a third threshold, the target part of the first foreign object is defined according to the target fitted ground reference; wherein, the third threshold is greater than the first threshold and less than the second threshold; the target part includes: the raised part and / or the collapsed part of the first foreign object; the target part is processed to obtain the local normal corresponding to the target part; the angle between the local normal and the normal of the target fitted ground reference is calculated to obtain the normal angle; if the normal angle is greater than a preset angle, and / or, the height of the first foreign object is higher than a fourth threshold, the first foreign object is confirmed as the target foreign object; wherein, the fourth threshold is greater than the first threshold and less than the third threshold.

[0008] Furthermore, the maximum measurement range of each lidar is greater than half the width of the airport.

[0009] Furthermore, the step of acquiring the first laser point cloud data of the airport runway through each lidar includes: acquiring the original laser point cloud data of the airport runway through each lidar; preprocessing the original laser point cloud data according to a preset preprocessing method to obtain the first laser point cloud data; wherein, the preprocessing method includes at least one of the following: discrete point removal processing, filtering processing, and coordinate transformation processing.

[0010] Furthermore, each lidar is equipped with a corresponding image acquisition device; the image acquisition device is used to identify the type of foreign object; the method also includes: if a target foreign object is selected from at least one potential foreign object, an alarm signal is generated; the size information and latitude and longitude information of the target foreign object are acquired; the first foreign object information of the target foreign object is sent to the terminal device; wherein, the first foreign object information includes at least one of the following: the location information, size information and latitude and longitude information of the target foreign object; and the second foreign object information of the target foreign object is saved; wherein, the second foreign object information includes at least one of the following: the event type and time of the selection of the target foreign object, and the image and / or video of the target foreign object acquired by the image acquisition device.

[0011] This invention provides an airport runway foreign object detection device, in which multiple lidars are installed at preset intervals on both sides of the airport runway. The device includes: a data acquisition module for acquiring first lidar point cloud data of the airport runway through each lidar; a division module for dividing the first lidar point cloud data into ground point cloud data and non-ground point cloud data; a filtering module for filtering each potential foreign object if at least one potential foreign object is detected based on the non-ground point cloud data; and a positioning module for locating the target foreign object if it is detected from the at least one potential foreign object, thereby determining the location information of the target foreign object.

[0012] The present invention provides an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the airport runway foreign object detection method described above.

[0013] The present invention provides a machine-readable storage medium storing machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement any of the above-mentioned airport runway foreign object detection methods.

[0014] The present invention provides a method, apparatus, and electronic equipment for detecting foreign objects on airport runways. This method involves acquiring first laser point cloud data of the airport runway using each lidar; dividing the first laser point cloud data into ground point cloud data and non-ground point cloud data; if at least one potential foreign object is detected based on the non-ground point cloud data, each potential foreign object is screened; if a target foreign object is detected from the at least one potential foreign object, the target foreign object is located to determine its position information. This method divides the first laser point cloud data acquired by multiple lidars into non-ground point cloud data, detects potential foreign objects based on the non-ground point cloud data, and then further screens and detects the target foreign object, thereby improving the efficiency and accuracy of foreign object detection on airport runways. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 A flowchart of an airport foreign object detection method provided in an embodiment of the present invention;

[0017] Figure 2 A schematic diagram of the deployment of a front-end device provided in an embodiment of the present invention;

[0018] Figure 3 This is a schematic diagram illustrating the relative magnitude of a threshold, provided as an embodiment of the present invention.

[0019] Figure 4 This is a schematic diagram of the architecture of an airport runway foreign object detection system provided in an embodiment of the present invention;

[0020] Figure 5 This is a schematic diagram of the structure of an airport runway foreign object detection device provided in an embodiment of the present invention;

[0021] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Currently, airports worldwide are increasingly demanding higher standards for runway foreign object detection, especially with the development of drones and electric aircraft, which will exacerbate the impact of foreign objects on aircraft. Due to its high precision and efficiency, lidar has become one of the primary technologies for airport runway foreign object detection, mainly applied to runways, taxiways, and aprons. Airport runways frequently contain various sizes and shapes of foreign objects; if not detected and removed promptly, they can pose a danger to aircraft takeoff and landing, and in severe cases, even lead to accidents. Therefore, lidar will see wider application and development in airport runway foreign object detection.

[0024] One implementation of this technology involves emitting a single-line laser beam towards the area to be detected. If a protrusion is encountered, the laser deformation can be observed from a specific angle based on two-dimensional image data returned by a high-speed camera. Image preprocessing and foreign object feature extraction analysis are then performed based on the two-dimensional data returned by the sensor. A threshold value for the foreign object features is used to determine whether there is foreign object interference on the airport runway. However, this method uses a single-line laser beam for scanning, resulting in significant errors. Furthermore, the precision of the light beam and the relatively uneven surface of the roadbed can easily lead to false alarms.

[0025] Another implementation of this technology involves fixing a radar module to one side of the airport runway using a tower to achieve automatic detection of foreign objects on the runway. This method uses lidar alone for detection, but due to the discontinuous nature of lidar information acquisition, it is prone to missing smaller foreign objects.

[0026] In another implementation of related technologies, the foreign object detection device includes a 3D LiDAR scanning head, a 3D LiDAR drive controller, and a 3D information processor. Multiple units of this device are distributed at different locations on the runway to cover the entire area. The system can detect foreign objects on the runway by comparing real-time altitude data with baseline data and trigger an alarm when any discrepancies are found. In this approach, a single sensor may struggle to fully reflect the objective environment during data processing, potentially leading to detection failures on the algorithm side. Therefore, this invention provides an airport foreign object detection method, device, and electronic device, which can be applied to applications requiring foreign object detection at airports.

[0027] To facilitate understanding of this embodiment, a foreign object detection method for airports disclosed in this invention will first be introduced. In this method, multiple lidar units are installed at preset intervals on both sides of the airport runway. These preset intervals can be set according to actual needs. For example, since most lidar units have a ranging accuracy of 200m@10%, if installed on a side light frame, they can be installed directly according to the spacing of the side lights. Alternatively, they can be freely deployed according to actual needs. For example, a frame with a flexible structure for the front-end equipment can be used, and different distances can be deployed depending on the type of lidar and the airport runway. For instance, the interval can be set according to 1 / 2 to 1 / 3 of the lidar's measurement range. Generally, it is sufficient to ensure that the measurement range of multiple lidar units can cover the entire airport runway. Figure 1 As shown, the method includes the following steps:

[0028] Step S102: Collect the first laser point cloud data of the airport runway using each lidar.

[0029] Laser point cloud data can be understood as a dataset of spatial points obtained by LiDAR scanning. Each point contains three-dimensional coordinate information, namely the three elements X, Y, and Z. Some also contain color information, reflection intensity information, echo count information, etc. In actual implementation, each LiDAR can scan within its corresponding measurement range to obtain its own corresponding laser point cloud data. By combining the laser point cloud data of each LiDAR, such as through combination or stitching, the first laser point cloud data corresponding to the airport runway can be obtained.

[0030] Step S104: Divide the first laser point cloud data into ground point cloud data and non-ground point cloud data.

[0031] In practical implementation, ground segmentation algorithms can be used to divide the first laser point cloud data into ground point cloud data and non-ground point cloud data. For example, the division can be based on the height information of the point cloud. Ground point cloud data usually has the following characteristics: the height of the ground points is relatively low compared to the height of the sensor; the height of the points around the ground points changes little, showing a flat feature; the distribution of ground points is relatively dense and the distribution is relatively regular.

[0032] Step S106: If at least one potential foreign object is detected based on non-ground point cloud data, each potential foreign object is screened.

[0033] The aforementioned potential foreign objects may be actual foreign objects that need to be detected and removed, or they may be airport facilities or personnel on the airport runway. In practice, the presence of potential foreign objects can be detected based on the height information of non-ground point cloud data. If they exist, each potential foreign object needs to be screened, for example, based on information such as the height of the potential foreign object, in order to further determine whether the potential foreign object is one that actually needs to be removed.

[0034] Step S108: If a target foreign object is detected from at least one potential foreign object, the target foreign object is located to determine its location information.

[0035] The aforementioned target foreign object can be understood as the foreign object that actually needs to be removed. In actual implementation, if the target foreign object is detected from at least one potential foreign object, the target foreign object can be located so that corresponding measures can be taken in a timely manner, such as making it easier for subsequent staff to remove the target foreign object based on the located location information.

[0036] The aforementioned airport runway foreign object detection method involves acquiring first laser point cloud data of the airport runway using each lidar sensor; dividing this first laser point cloud data into ground point cloud data and non-ground point cloud data; if at least one potential foreign object is detected based on the non-ground point cloud data, each potential foreign object is screened; if a target foreign object is detected from at least one potential foreign object, the target foreign object is located to determine its position information. This method, by dividing the first laser point cloud data acquired by multiple lidar sensors into non-ground point cloud data, first detecting potential foreign objects based on the non-ground point cloud data, and then further screening to detect the target foreign object, can improve the efficiency and accuracy of airport runway foreign object detection.

[0037] This invention also provides another method for detecting foreign objects on airport runways, which is implemented based on the method described in the above embodiments. In this method, the maximum measurement range of each lidar is greater than half the width of the airport. See also Figure 2The diagram illustrates the deployment of a front-end device. Taking a 50m wide airport runway as an example, lidar is installed on both sides, and the scanning area covers the entire runway. The runway typically has a raised center and lower sides, with an angle not exceeding 2° vertically. The main front-end data acquisition device, a lidar with a horizontal field of view of 120° and a calibrated measurement range of 200m, is used as an example. Figure 2 For deployment, the lidar measurement range can be selected from 8m to 33m+, with the maximum value being 33m, which is greater than half the width of the airport. The following explains the different airport levels:

[0038] 4F-class airports are the highest level of airport classification. A 4F-class airport can accommodate various large aircraft. A 4E-class airport refers to an airport with a runway length of ≥1800 meters and a wingspan of ≥52 meters but <65 meters, capable of accommodating four-engine long-range wide-body passenger aircraft such as the Boeing 747 and Airbus A340. A 4D-class airport, under standard conditions, has a usable runway length ≥1800 meters, a wingspan of 36-<52 meters for the largest usable aircraft, and a main landing gear outer wheel spacing of 9-<14 meters. A 4C-class airport indicates the civil aviation airport's flight area classification; 4C-class airports are generally used as regional airports.

[0039] Each lidar is equipped with a corresponding image acquisition device; the image acquisition device is used to identify the type of foreign object, such as the appearance and color of the foreign object; the image acquisition device can be a camera, and one camera can be configured for each lidar. The installation position of the camera can be set as needed, for example, it can usually be set in a position close to the corresponding lidar.

[0040] The method includes the following steps:

[0041] Step 1: Collect raw laser point cloud data of the airport runway using each lidar.

[0042] Step 2: Preprocess the original laser point cloud data according to the preset preprocessing method to obtain the first laser point cloud data; wherein, the preprocessing method includes at least one of the following: discrete point removal processing, filtering processing, coordinate transformation processing.

[0043] The aforementioned raw laser point cloud data can be understood as the actual laser point cloud data of an airport runway collected by each lidar. Discrete point removal processing can remove points outside a preset range from the raw laser point cloud data. Filtering can remove noise and redundant points from the raw laser point cloud data while retaining meaningful points. Coordinate transformation processing can transform the raw laser point cloud data from one coordinate system to any associated coordinate system, facilitating subsequent analysis and processing. In practice, during the acquisition of raw laser point cloud data, due to sensor accuracy, environmental interference, and other factors, a significant amount of noise and outliers are often generated. These factors greatly affect the effect and accuracy of subsequent point cloud processing. Preprocessing the raw laser point cloud data can effectively remove noise and outliers, obtaining the first laser point cloud data, thus improving its quality and usability. Furthermore, the preprocessed first laser point cloud data can better support point cloud analysis, recognition, 3D reconstruction, and other applications.

[0044] Step 3: Obtain the pre-constructed prior distribution model of the airport runway ground point cloud.

[0045] Step four: Evaluate the simulated point cloud data using the prior distribution model of the airport runway ground point cloud to obtain the evaluation results.

[0046] The aforementioned prior distribution model of airport runway ground point cloud typically includes a model of the airport runway and the corresponding first laser point cloud data. The 3D point cloud of the lower half of the field of view can be locally rasterized at multiple resolutions, and model parameters can be calculated by randomly sampling from the set of centroids of each raster to avoid the initial frame sampling points being concentrated in a small area. Then, the distribution information of the mean ground point set and obstacle point set of the previous three frames is used to guide the sampling of the current frame to avoid blind sampling. The specific approach is as follows:

[0047] 1) Record the ratio of each rasterized ground point after meaning in the first three frames, and map it to (0, 1];

[0048] 2) Define the sampling probability distribution as a uniform distribution, and calculate the sampling of the current frame according to this distribution;

[0049] 3) Extract the solution parameters from the sample and fit a plane. This fitted plane corresponds to the above evaluation results.

[0050] Step 5: Based on the preset uniform distribution model of interior points and the evaluation results, determine the target fitting ground benchmark.

[0051] Traditional RANSAC (Random Sample Consensus) only evaluates the optimal ground model by the number of inliers. In this scheme, the inliers of the target fitted ground model projected onto the horizontal plane from the prior distribution model of the airport runway ground point cloud possess both the characteristics of having the largest number and exhibiting a uniform distribution. The specific implementation is as follows:

[0052] 1) The number of inliers is represented by the weighting evaluation factor s1 (this number can be determined by referring to the traditional RANSAC method), and the distribution of inliers in the horizontal direction is represented by the covariance matrix. The proportion of the total area of ​​the point cloud in the horizontal direction is taken as the distribution evaluation factor s2.

[0053] 2) Utilize Find s, where a larger s indicates a better fitted ground model; where m is the proportion weight and n is the distribution weight.

[0054] The optimal ratio of inliers in the current frame raster is updated using the first laser point cloud data sampled in the iterative sampling; the probability of obtaining a ground point by sampling is calculated; when the inliers are optimal, three points are arbitrarily selected to form a plane to obtain the target fitted ground reference, and the iteration is terminated at this time.

[0055] Step 6: Based on the ground reference fitted to the target, the first laser point cloud data is divided into ground point cloud data and non-ground point cloud data.

[0056] Once the target-fitted ground reference is determined, the laser point cloud data located within the target-fitted ground reference can be identified as ground point cloud data, and the laser point cloud data located outside the target-fitted ground reference can be identified as non-ground point cloud data.

[0057] Step 7: If there is a first point cloud data in the non-ground point cloud data with a point cloud height higher than the first threshold, the object corresponding to the first point cloud data is identified as at least one potential foreign object extracted from the non-ground point cloud data.

[0058] The first threshold can be set according to actual needs. In actual implementation, after separating ground point cloud data and non-ground point cloud data, objects with height characteristics usually have obvious height information. The first point cloud data with a height higher than the first threshold can be extracted from the non-ground point cloud data, thereby extracting at least one potential foreign object.

[0059] Step 8: From the potential foreign objects, exclude objects corresponding to the second point cloud data whose point cloud height is higher than the second threshold, in order to filter each potential foreign object; wherein, the second threshold is greater than the first threshold.

[0060] The second threshold can be set according to actual needs. In actual implementation, after at least one potential foreign object is extracted, each potential foreign object can be screened and classified to exclude objects corresponding to the second point cloud data with a point cloud height higher than the second threshold. These objects are usually airport facilities or staff, so they can usually be directly excluded.

[0061] Step nine: If a first foreign object is selected from at least one potential foreign object, and the height of the first foreign object is less than the third threshold, the target location of the first foreign object is defined according to the target fitted ground benchmark; wherein, the third threshold is greater than the first threshold and less than the second threshold; the target location includes: the raised part and / or the collapsed part of the first foreign object.

[0062] The aforementioned third threshold is usually a value close to the first threshold, and can be set according to actual needs. The aforementioned first foreign object can be understood as a small obstacle that is not sensitive to height information. Simply using height information to distinguish these objects has a high probability of misjudgment and omission, and a comprehensive judgment is required. Specifically, the aforementioned target can be fitted with a ground benchmark as a baseline to delineate the raised and / or collapsed parts of the first foreign object.

[0063] Step 10: Process the target area to obtain the local normal corresponding to the target area.

[0064] In practical implementation, regression analysis can be performed on the data corresponding to the target area to obtain the local normal line corresponding to that target area. Regression analysis refers to using statistical principles to mathematically process a large amount of statistical data, determine the correlation between the dependent variable and certain independent variables, and establish a regression equation with good correlation.

[0065] Step 11: Calculate the angle between the local normal and the normal of the target fitted to the ground reference, and obtain the normal angle.

[0066] Step 12: If the angle between the normals is greater than a preset angle, and / or the height of the first foreign object is higher than the fourth threshold, the first foreign object is confirmed as the target foreign object; wherein the fourth threshold is greater than the first threshold and less than the third threshold.

[0067] The aforementioned preset angle and fourth threshold can be set according to actual needs. By comparing the local normal with the normal of the target fitted ground reference, the angle between the two normals can be obtained. Combining the angle between the normals and the height of the first foreign object, it can be further confirmed whether the first foreign object is the target foreign object. Specifically, if the angle between the normals is greater than the preset angle, or if the height of the first foreign object is higher than the fourth threshold, or if the angle between the normals is greater than the preset angle and the height of the first foreign object is higher than the fourth threshold, the first foreign object can be considered as the target foreign object.

[0068] For ease of understanding, see [link to relevant documentation]. Figure 3 The diagram shows a representation of the relative magnitudes of thresholds. It can be seen that the value 1 corresponding to the first threshold is the smallest, the value 2 corresponding to the second threshold is the largest, the value 3 corresponding to the third threshold is between the value 1 corresponding to the first threshold and the value 2 corresponding to the second threshold, and the value 4 corresponding to the fourth threshold is between the value 1 corresponding to the first threshold and the value 3 corresponding to the third threshold.

[0069] Step 13: If a target foreign object is detected from at least one potential foreign object, locate the target foreign object to determine its location information.

[0070] Step fourteen: If the target foreign object is selected from at least one potential foreign object, an alarm signal is generated.

[0071] When a foreign object is detected, an alarm signal can be issued to promptly alert relevant personnel. For example, an alarm can be issued using sound or light signals.

[0072] Step 15: Obtain the size and latitude / longitude information of the target foreign object.

[0073] In practice, once the target foreign object is identified, its size and latitude / longitude information can be determined based on the corresponding laser point cloud data.

[0074] Step sixteen: Send the first foreign object information of the target foreign object to the terminal device; wherein the first foreign object information includes at least one of the following: the location information, size information and latitude and longitude information of the target foreign object.

[0075] The aforementioned terminal equipment can be a computer terminal or a handheld terminal, or other devices that are easy for staff to view. In practice, the location, size, latitude and longitude information of the target foreign object can be sent to the designated terminal via network notification, email, or other means, so that the staff corresponding to the designated terminal can receive the notification information in a timely manner and take relevant measures promptly.

[0076] Step 17: Save the second foreign object information of the target foreign object; wherein the second foreign object information includes at least one of the following: the event type and time of the target foreign object, and the image and / or video of the target foreign object acquired by the image acquisition device.

[0077] The above event types can be determined based on the target foreign object. For example, if the target foreign object is a metal device, the event type can be a metal foreign object event; if the target foreign object is concrete or asphalt fragments, such as stones, sand, or ice slag, the event type can be a concrete or asphalt fragment foreign object event, etc. In actual implementation, the events that have been filtered out for the target foreign object can be recorded and archived. For example, the event type and time when the event occurs can be recorded. Alternatively, high-definition photos or videos of the target foreign object can be taken using an image acquisition device, such as a camera, and archived. Users can adjust different archiving modes according to the actual scenario requirements, which are not limited here.

[0078] For ease of understanding, see [link to relevant documentation]. Figure 4 The diagram shows the architecture of an airport runway foreign object detection system. The system consists of a lidar and camera group, a FOD (Foreign Object Debris) detection data processing center, a display, an alarm module, and a control and linkage system. The lidar and camera group collects real-time monitoring data covering the entire airport runway area. The data is processed by algorithms and used for decision-making and early warning in the FOD detection data processing center. Finally, the monitoring results are output to the display in real time, and alarms are broadcast and linkage control is implemented.

[0079] Specifically, data collected by lidar and cameras can be sent to the FOD detection data processing center. The sensor data preprocessing module preprocesses the raw lidar point cloud data to obtain the first lidar point cloud data. The detection data statistical analysis module then divides this first lidar point cloud data into ground point cloud data and non-ground point cloud data. If at least one potential foreign object is detected based on the non-ground point cloud data, each potential foreign object is screened. If a target foreign object is detected from the at least one potential foreign object, it is located to determine its position information. The display screen corresponding to the control center's visualization module shows relevant information about the target foreign object, such as its location, size, latitude and longitude, and images and / or videos. The FOD information management module can save relevant foreign object information, such as the event type and time of the target foreign object detection, and images and / or videos of the target foreign object acquired by the image acquisition device.

[0080] Through external interfaces, relevant statistical information about the target foreign object can be sent to the FOD removal system and the FOD system control center. The wireless positioning device in the FOD removal system can locate the target foreign object, and the foreign object removal device can remove the target foreign object based on the location. The foreign object removal feedback module can provide feedback on the removal results to the user.

[0081] The various modules in the FOD system control center typically operate in parallel, meaning they can each receive relevant statistical information about the target foreign object. The FOD business process module sends this statistical information to relevant business units, nodes, and equipment. The FOD event scheduling and processing module feeds back information about the presence of the target foreign object to the airport and air traffic control information linkage module, which then issues dispatch instructions to the appropriate personnel to handle the object. The FOD event identification module, in conjunction with the airport and air traffic control information linkage module, can determine the urgency of the foreign object event, such as the need for immediate handling. The FOD comprehensive information display module displays relevant statistical information about the target foreign object, such as location, size, latitude and longitude, and images and / or videos of the object.

[0082] The aforementioned foreign object detection method for airport runways relies on data from underlying sensors to perform edge computing first, and then completes the overall system functionality through central computing. Once an obstacle is detected, obstacle information can be output immediately, and multiple obstacle information entries can be output simultaneously. The output includes location and size information, and latitude and longitude information can be output if needed.

[0083] This method enables multi-obstacle detection and tracking. Laser sensors and cameras process the area under their coverage using clustering and segmentation algorithms to track and identify targets, simultaneously detecting multiple objects and outputting alarm signals. It also provides tracking and monitoring capabilities for suddenly added debris and moving targets on the track. This method allows for the fusion of multiple sensor types. After identifying an obstacle, an alarm is triggered. The system administrator can then automatically focus the camera on the alarm location to magnify and identify the foreign object, further confirming its shape and locating it before proceeding with its removal.

[0084] This method enables zoned detection, allowing users to divide the area into multiple zones based on different detection needs. Within each zone, different settings can be used to prioritize detection in key areas. It supports event archiving, high-definition photography, and video archiving, with users able to adjust different archiving modes according to specific scenario requirements. Archiving records the type and time of the event when it occurred. This method also supports FOD (Fault-Oriented Object) tracking and tracing, enabling historical tracking of obstacles. Upon detecting a FOD, relevant information is recorded for analysis and processing, and the source of the FOD can be traced if necessary. Furthermore, it supports obstacle classification, allowing the system to support manual identification and classification of FOD objects, and the submission of processing suggestions to relevant systems based on object characteristics.

[0085] The aforementioned foreign object detection method for airport runways improves ground detection capabilities, avoiding false detections caused by ground slope or construction requirements; enhances the detection capability and accuracy of foreign objects on airport runways; integrates image acquisition devices and lidar, enabling high-precision detection of the surrounding environment and quickly identifying the location, size, and shape of obstacles; and improves environmental adaptability, adapting to different climatic environments, such as rain, snow, and nighttime, maintaining stable operation in various complex environments.

[0086] This invention provides a foreign object detection device for airport runways, wherein multiple lidar sensors are installed at preset intervals on both sides of the airport runway, such as... Figure 5 As shown, the device includes: a data acquisition module 40 for acquiring first laser point cloud data of the airport runway through each lidar; a division module 41 for dividing the first laser point cloud data into ground point cloud data and non-ground point cloud data; a filtering module 42 for filtering each potential foreign object if at least one potential foreign object is detected based on the non-ground point cloud data; and a positioning module 43 for locating the target foreign object if it is detected from the at least one potential foreign object, so as to determine the location information of the target foreign object.

[0087] The aforementioned airport runway foreign object detection device collects first laser point cloud data of the airport runway using each lidar. This first laser point cloud data is divided into ground point cloud data and non-ground point cloud data. If at least one potential foreign object is detected based on the non-ground point cloud data, each potential foreign object is screened. If a target foreign object is detected from the at least one potential foreign object, the target foreign object is located to determine its position information. This device divides the first laser point cloud data collected by multiple lidars into non-ground point cloud data, detects potential foreign objects based on this non-ground point cloud data, and then further screens and detects target foreign objects, which can improve the efficiency and accuracy of airport runway foreign object detection.

[0088] Furthermore, the partitioning module 41 is also used to: obtain a pre-constructed prior distribution model of the airport runway ground point cloud; evaluate the simulated point cloud data through the prior distribution model of the airport runway ground point cloud to obtain the evaluation result; determine the target fitting ground benchmark based on the preset uniform internal point distribution model and the evaluation result; and partition the first laser point cloud data into ground point cloud data and non-ground point cloud data according to the target fitting ground benchmark.

[0089] Furthermore, the filtering module 42 is also used to: if there is a first point cloud data in the non-ground point cloud data with a point cloud height higher than the first threshold, identify the object corresponding to the first point cloud data as at least one potential foreign object extracted from the non-ground point cloud data; exclude the object corresponding to the second point cloud data with a point cloud height higher than the second threshold from the potential foreign objects, so as to filter each potential foreign object; wherein, the second threshold is greater than the first threshold.

[0090] Furthermore, the screening module 42 is also used to: if a first foreign object is selected from at least one potential foreign object, and the height of the first foreign object is less than a third threshold, to divide the target part of the first foreign object according to the target fitted ground reference; wherein, the third threshold is greater than the first threshold and less than the second threshold; the target part includes: the raised part and / or the collapsed part of the first foreign object; to process the target part to obtain the local normal corresponding to the target part; to calculate the angle between the local normal and the normal of the target fitted ground reference to obtain the normal angle; if the normal angle is greater than a preset angle, and / or, the height of the first foreign object is higher than a fourth threshold, to confirm the first foreign object as a target foreign object; wherein, the fourth threshold is greater than the first threshold and less than the third threshold.

[0091] Furthermore, the maximum measurement range of each lidar is greater than half the width of the airport.

[0092] Furthermore, the acquisition module 40 is also used to: acquire raw laser point cloud data of the airport runway through each lidar; preprocess the raw laser point cloud data according to a preset preprocessing method to obtain the first laser point cloud data; wherein the preprocessing method includes at least one of the following: discrete point removal processing, filtering processing, coordinate transformation processing.

[0093] Furthermore, each lidar is equipped with a corresponding image acquisition device; the image acquisition device is used to identify the type of foreign object; the device is also used to: generate an alarm signal if a target foreign object is selected from at least one potential foreign object; acquire the size information and latitude and longitude information of the target foreign object; send the first foreign object information of the target foreign object to the terminal device; wherein the first foreign object information includes at least one of the following: the location information, size information and latitude and longitude information of the target foreign object; save the second foreign object information of the target foreign object; wherein the second foreign object information includes at least one of the following: the event type and time of the selection of the target foreign object, and the image and / or video of the target foreign object acquired by the image acquisition device.

[0094] The airport runway foreign object detection device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned airport runway foreign object detection method embodiment. For the sake of brevity, any parts not mentioned in the airport runway foreign object detection device embodiment can be referred to the corresponding content in the aforementioned airport runway foreign object detection method embodiment.

[0095] This invention also provides an electronic device, see [link to relevant documentation]. Figure 6 As shown, the electronic device includes a processor 130 and a memory 131. The memory 131 stores machine-executable instructions that can be executed by the processor 130. The processor 130 executes the machine-executable instructions to implement the above-described airport runway foreign object detection method.

[0096] Furthermore, Figure 6 The electronic device shown also includes a bus 132 and a communication interface 133, with the processor 130, the communication interface 133 and the memory 131 connected via the bus 132.

[0097] The memory 131 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 133 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network. The bus 132 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0098] Processor 130 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 130 or by instructions in software form. Processor 130 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 131, and processor 130 reads the information in memory 131 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0099] This invention also provides a machine-readable storage medium storing machine-executable instructions. When these machine-executable instructions are called and executed by a processor, they cause the processor to implement the aforementioned airport runway foreign object detection method. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0100] The computer program product of the airport runway foreign object detection method, device and electronic device provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0101] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting foreign objects on an airport runway, characterized in that, Multiple lidar sensors are installed at preset intervals on both sides of the airport runway. The method includes: The first laser point cloud data of the airport runway is acquired by each of the aforementioned lidars; The first laser point cloud data is divided into ground point cloud data and non-ground point cloud data; If at least one potential foreign object is detected based on the non-ground point cloud data, each potential foreign object is screened. If a target foreign object is detected from at least one potential foreign object, the target foreign object is located to determine its location information; If at least one potential foreign object is detected based on the non-ground point cloud data, the steps for screening each potential foreign object include: If there is a first point cloud data in the non-ground point cloud data whose point cloud height is higher than the first threshold, the object corresponding to the first point cloud data is identified as at least one potential foreign object extracted from the non-ground point cloud data. From the potential foreign objects, objects corresponding to the second point cloud data whose point cloud height is higher than the second threshold are excluded, so as to filter each potential foreign object; wherein, the second threshold is greater than the first threshold; The method further includes: If a first foreign object is selected from at least one potential foreign object, and the height of the first foreign object is less than a third threshold, the target location of the first foreign object is defined according to the target fitted ground reference; wherein, the third threshold is greater than the first threshold and less than the second threshold; the target location includes: the raised part and / or the collapsed part of the first foreign object; The target area is processed to obtain the local normal corresponding to the target area; Calculate the angle between the local normal and the normal of the target-fitted ground reference to obtain the normal angle; If the included angle of the normal is greater than a preset angle, and / or the height of the first foreign object is higher than a fourth threshold, the first foreign object is confirmed as a target foreign object; wherein the fourth threshold is greater than the first threshold and less than the third threshold.

2. The method according to claim 1, characterized in that, The steps of dividing the first laser point cloud data into ground point cloud data and non-ground point cloud data include: Obtain a pre-constructed prior distribution model of the airport runway ground point cloud; The simulated point cloud data was evaluated using the prior distribution model of the airport runway ground point cloud, and the evaluation results were obtained. Based on the preset uniform distribution model of interior points and the evaluation results, the target fitting ground benchmark is determined; Based on the target-fitted ground reference, the first laser point cloud data is divided into ground point cloud data and non-ground point cloud data.

3. The method according to claim 1 or 2, characterized in that, The maximum measurement range of each of the lidar sensors is greater than half the width of the airport.

4. The method according to claim 1 or 2, characterized in that, The steps of acquiring first laser point cloud data of the airport runway using each of the aforementioned lidars include: The raw laser point cloud data of the airport runway is collected by each of the aforementioned lidars; The original laser point cloud data is preprocessed according to a preset preprocessing method to obtain the first laser point cloud data; wherein the preprocessing method includes at least one of the following: discrete point removal processing, filtering processing, and coordinate transformation processing.

5. The method according to claim 1 or 2, characterized in that, Each of the lidar sensors is equipped with a corresponding image acquisition device; the image acquisition device is used to identify the type of foreign object; the method further includes: If the target foreign object is selected from at least one potential foreign object, an alarm signal is generated. Obtain the size and latitude / longitude information of the target foreign object; The first foreign object information of the target foreign object is sent to the terminal device; wherein the first foreign object information includes at least one of the following: the location information, the size information, and the latitude and longitude information of the target foreign object; The second foreign object information of the target foreign object is stored; wherein the second foreign object information includes at least one of the following: the event type and time of the target foreign object, and the image and / or video of the target foreign object acquired by the image acquisition device.

6. A foreign object detection device for airport runways, characterized in that, Multiple lidar sensors are installed at predetermined intervals on both sides of the airport runway. The device includes: The acquisition module is used to acquire first laser point cloud data of the airport runway through each of the lidars; A segmentation module is used to divide the first laser point cloud data into ground point cloud data and non-ground point cloud data; The filtering module is used to filter each potential foreign object if at least one potential foreign object is detected based on the non-ground point cloud data. The positioning module is used to locate the target foreign object if a target foreign object is detected from at least one potential foreign object, so as to determine the location information of the target foreign object; The filtering module is also used for: If there is a first point cloud data in the non-ground point cloud data whose point cloud height is higher than the first threshold, the object corresponding to the first point cloud data is identified as at least one potential foreign object extracted from the non-ground point cloud data. From the potential foreign objects, objects corresponding to the second point cloud data whose point cloud height is higher than the second threshold are excluded, so as to filter each potential foreign object; wherein, the second threshold is greater than the first threshold; If a first foreign object is selected from at least one potential foreign object, and the height of the first foreign object is less than a third threshold, the target location of the first foreign object is defined according to the target fitted ground reference; wherein, the third threshold is greater than the first threshold and less than the second threshold; the target location includes: the raised part and / or the collapsed part of the first foreign object; The target area is processed to obtain the local normal corresponding to the target area; Calculate the angle between the local normal and the normal of the target-fitted ground reference to obtain the normal angle; If the included angle of the normal is greater than a preset angle, and / or the height of the first foreign object is higher than a fourth threshold, the first foreign object is confirmed as a target foreign object; wherein the fourth threshold is greater than the first threshold and less than the third threshold.

7. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the airport runway foreign object detection method according to any one of claims 1-5.

8. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the airport runway foreign object detection method according to any one of claims 1-5.

Images