Machine learning-based FOD detection algorithm

Abstract

The invention relates to the field of airport road foreign matter detection, and provides a machine learning-based FOD detection algorithm, which comprises an input module used for receiving basic image data of an airport runway; the processing module is used for carrying out preprocessing and feature extraction on the input data; the storage module is used for storing the original image data and the detection result; the acquisition module is used for acquiring image and sound data of an airport runway in real time; and the recognition module is used for recognizing moving objects and static objects in the collected data based on a machine learning model. And high-precision real-time detection is realized through cooperation of multiple modules. The acquisition module adopts a high-definition camera to ensure that image data is clear; the processing module utilizes a convolutional neural network (CNN) to extract features, and can accurately recognize moving and static objects in combination with a YOLOv5 algorithm of the recognition module. The comparison module compares the real-time image with the original image, abnormity is judged if the real-time image is lower than the threshold value, the runway foreign matter can be found in time, and the system can rapidly position the FOD in the complex environment.

Description

Technical Field

[0001] This invention relates to the field of foreign object detection on airport roads, specifically a machine learning-based FOD detection algorithm. Background Technology

[0002] In the field of air transport, the safe operation of airport runways is a crucial prerequisite for ensuring normal flight takeoffs and landings and overall flight safety. With the rapid development of the global aviation industry, foreign object debris (FOD) on airport runways poses a serious threat to flight safety. FOD may include debris from runway surface damage, parts fallen from vehicles, birds, etc. If these foreign objects are not detected and removed in a timely manner, they may be sucked into aircraft engines or collide with aircraft tires, leading to serious accidents such as engine failure and tire blowouts, causing enormous economic losses and safety hazards.

[0003] According to the announcement number CN116934711A, this invention belongs to the field of FOD detection technology, specifically a machine learning-based FOD detection algorithm. Addressing the problem of existing algorithms' inaccurate detection of foreign objects, the following solution is proposed: It includes an input module, a processing module, a storage module, a data acquisition module, an identification module, a display module, a comparison module, a judgment module, and an alarm module. The input module and processing module are electrically connected; the processing module and storage module are bidirectionally connected; the data acquisition module and identification module are electrically connected; the alarm module and processing module are both electrically connected to the identification module; the display module and processing module are electrically connected; the judgment module and processing module are both electrically connected to the comparison module; and the comparison module, storage module, and alarm module are all electrically connected to the judgment module. This invention can quickly detect debris in airports, enabling comprehensive debris removal and improving airport security. It is simple to use and easy to operate.

[0004] Currently, airport runway FOD (Focus on Debris) detection mainly relies on manual inspections and traditional sensor detection methods. Manual inspections are limited by the visual fatigue of inspectors, the inspection range, and the frequency, making real-time, comprehensive runway monitoring difficult, and suffer from low efficiency and high false negative rates. Traditional sensor detection methods, such as millimeter-wave radar and infrared sensors, while providing some assistance, experience a significant drop in accuracy under complex weather conditions (such as heavy rain and fog), and have limited ability to identify small foreign objects or stationary objects. Furthermore, most existing detection systems lack intelligent analysis and self-learning capabilities, failing to optimize detection models based on historical data and adapt to environmental changes in different airport runways. This results in insufficient accuracy and timeliness of FOD detection, failing to meet the high runway safety requirements of modern airports. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a machine learning-based FOD detection algorithm that solves the problems of low detection efficiency, high false negative rate, and limited ability to identify small foreign objects or stationary objects.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a machine learning-based FOD detection algorithm, comprising: Input module: Used to receive basic image data of the airport runway; Processing module: Used to preprocess and extract features from the input data; Storage module: Used to store raw image data and detection results; Acquisition module: Used for real-time acquisition of image and audio data of the airport runway; Recognition module: Based on machine learning models, it identifies moving and stationary objects in the collected data; Alarm module: Triggers an alarm when a potential FOD is detected; Display module: used to visualize detection results and alarm information; Comparison module: Compares the real-time collected data with the stored raw data; Judgment module: Determines whether an object is a FOD based on the comparison results; Driving module: Used to automatically drive away moving objects such as birds.

[0007] Preferably, the processing module uses a convolutional neural network (CNN) for feature extraction, and its formula is as follows:

[0008] in, For input features, As weight, For bias, This is the activation function.

[0009] Preferably, the storage module includes: Data input unit: Receives data output from the processing module; Temporary storage unit: caches data collected in real time; Permanent storage unit: stores historical data; Data output unit: Provides data to the comparison module.

[0010] Preferably, the comparison module uses a similarity algorithm to calculate the difference between the real-time image and the original image, and the similarity threshold is set to 90%. If the similarity is lower than this threshold, it is judged as abnormal.

[0011] Preferably, the determination module includes: Data receiving unit: Receives the output of the comparison module; Data classification unit: classifies objects into moving objects or stationary objects; Data transmission unit: Triggers alarm module or drive-away module based on classification results.

[0012] Preferably, the alarm module supports the following alarm methods: Voice prompt: Play alarm information through the speaker; Text notification: Display alarm information on the display module; Network notification: Sends alarm information to terminal devices via wireless network.

[0013] Preferably, the bird-repelling module scares away birds by emitting sound waves of a specific frequency, with the sound wave frequency range being 10kHz-20kHz.

[0014] Preferably, it also includes a learning module for optimizing the machine learning model based on each detection result, the optimization formula being:

[0015] in, For model parameters, For learning rate, This is the gradient of the loss function.

[0016] Preferably, the image acquisition unit of the acquisition module uses a high-definition camera with a resolution of not less than 1920×1080 and a frame rate of 30fps.

[0017] Preferably, the recognition module uses the YOLOv5 algorithm for object detection with a detection accuracy of ≥95%.

[0018] This invention provides a FOD detection algorithm based on machine learning. It has the following beneficial effects: 1. This invention achieves high-precision real-time detection through multi-module collaboration. The acquisition module uses a high-definition camera to ensure clear image data; the processing module uses a convolutional neural network (CNN) to extract features, combined with the YOLOv5 algorithm of the recognition module, to accurately identify moving and stationary objects. The comparison module compares real-time and raw images, and determines anomalies if the image is below a threshold, enabling timely detection of foreign objects on the runway. This allows the system to quickly locate FODs in complex environments, providing real-time protection for airport runway safety and preventing flight accidents caused by foreign objects.

[0019] 2. The alarm module of this invention supports three levels of alarm methods: voice, text, and network notification, and adopts a graded alarm mechanism: primary warning with audible and visual alerts, intermediate warning with voice broadcast, and advanced warning linked to the ATC system and triggering runway closure, ensuring that FOD of different hazard levels can be responded to in a timely manner. The judgment module triggers corresponding mechanisms based on object classification. For moving objects such as birds, the deterrence module emits specific frequency sound waves to automatically deter them, reducing the threat of birds to flight. This achieves an automated process from detection to processing, significantly shortening emergency response time, reducing manual intervention costs, and improving airport safety management efficiency.

[0020] 3. The storage module of this invention includes temporary and permanent storage, combined with the Zstandard compression algorithm, to achieve efficient data management and long-term retention, facilitating historical data retrieval and analysis. The learning module optimizes model parameters based on detection results, continuously improving detection accuracy. Furthermore, the system supports multimodal data fusion and dynamic threshold adjustment, maintaining stable detection performance even under complex weather conditions, enhancing the system's adaptability and robustness to different environments. Attached Figure Description

[0021] Figure 1 This is a system diagram of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see the appendix Figure 1 This invention provides a machine learning-based FOD detection algorithm, comprising: Input Module: Used to receive basic image data of the airport runway. The input module adopts multi-source data fusion technology and includes: Visual Input Interface: Supports RTSP / ONVIF protocol and can be connected to cameras; Data Verification Unit: Built-in CRC32 verification algorithm to ensure data transmission integrity; Format Converter: Automatically converts H.265 / H.264 video streams to OpenCV standard Mat format; Bandwidth Adaptive Mechanism: Dynamically adjusts the bitrate according to network conditions, with an adjustment range of 1-50Mbps. Processing module: This module preprocesses the input data and extracts features. It uses a convolutional neural network (CNN) for feature extraction, with the following formula:

[0024] in, For input features, As weight, For bias, For activation functions; Storage Module: Used to store raw image data and detection results. The storage module includes: Data Input Unit: Receives data output from the processing module, supports NVMe SSD high-speed caching, uses Zstandard data compression algorithm, and has a compression ratio of 3:1; Temporary Storage Unit: Caches real-time acquired data, uses Redis in-memory database, uses LRU automatic eviction policy, and has a maximum cache duration of 72 hours; Permanent Storage Unit: Stores historical data, using a distributed Ceph storage cluster. Data output unit: Provides data to the comparison module; Acquisition module: Used to acquire real-time image and sound data of the airport runway. The image acquisition unit of the acquisition module uses a high-definition camera with a resolution of no less than 1920×1080 and a frame rate of 30fps. Recognition Module: Based on a machine learning model, this module identifies moving and stationary objects in the collected data. It uses the YOLOv5 algorithm for object detection with an accuracy of ≥95%. Alarm Module: Triggers an alarm when a potential FOD is detected. The alarm module supports the following alarm methods: Voice alert: Plays alarm information through a speaker; Text alert: Displays alarm information on the display module; Network notification: Sends alarm information to the terminal device via a wireless network, employing a tiered alarm mechanism. Primary warning (potential FOD): Audible and visual alarm: 85dB buzzer + LED flashing; push level is P3 (non-emergency); Intermediate alert (confirmed FOD): Voice broadcast; push notification level is P2 (important); Advanced warning (hazardous FOD): Links with the ATC system, pushes the alert level to P1 (emergency), and automatically triggers the runway closure protocol; Display module: used to visualize detection results and alarm information; The comparison module compares the real-time acquired data with the stored raw data. The comparison module uses a similarity algorithm to calculate the difference between the real-time image and the raw image. The similarity threshold is set at 90%. If the similarity is lower than this threshold, it is judged as abnormal. Judgment Module: Determines whether an object is a FOD based on the comparison results. The judgment module includes: a data receiving unit: receiving the output of the comparison module; a data classification unit: classifying the object as a moving object or a stationary object; and a data transmission unit: triggering the alarm module or the repelling module based on the classification results. Repelling module: Used to automatically scare away moving objects such as birds. The repelling module drives away birds by emitting sound waves of a specific frequency, with the sound wave frequency range being 10kHz-20kHz.

[0025] The learning module is used to optimize the machine learning model based on each detection result. Its optimization formula is as follows:

[0026] in, For model parameters, For learning rate, This is the gradient of the loss function.

[0027] Example 1: FOD Detection System Based on CNN and YOLOv5 System Configuration Hardware: High-definition camera with a resolution of 1920×1080 and a frame rate of 30fps; sound wave transmitter with a frequency range of 10kHz-20kHz; server for model training and inference.

[0028] Software: Convolutional Neural Network (CNN) for feature extraction; YOLOv5 model for object detection; similarity algorithm with a threshold of 90%.

[0029] Workflow: 1. Data Collection a. Cameras capture real-time images of the airport runway; b. Sound sensors detect ambient sounds.

[0030] 2. Data Processing a. Image data is used to extract features via CNN, using the following formula: ,in, b) As a bias, used to adjust the model sensitivity; YOLOv5 detects moving and stationary objects with a detection accuracy of ≥95%.

[0031] 3. Comparison and Judgment a. The comparison module calculates the similarity between the real-time image and the original image. If the similarity is less than 90%, it is judged as abnormal. b. The judgment module classifies objects. Moving objects trigger the driving module, and stationary objects trigger the alarm module.

[0032] 4. Calling the police and driving away a. Alarm methods: voice, text, and network notification; b. The bird deterrence module emits 15kHz sound waves to scare away birds.

[0033] 5. Learning Optimization The learning module updates model parameters and optimizes formulas:

[0034] Example 2: Multi-module collaborative FOD detection process 1. Input module: Receives basic runway images.

[0035] 2. Storage module: Original images are stored in permanent storage units; real-time data is cached in temporary storage units.

[0036] 3. Identification and Comparison: The identification module marks potential FODs; after the comparison module finds differences, it triggers the judgment module.

[0037] 4. Categorized processing: For stationary FODs, such as metal parts, the alarm module notifies the staff; for moving FODs, such as birds, the deterrent module is activated and an alarm is triggered.

[0038] 5. Display and Query: The display module shows the runway status in real time; staff can retrieve historical data through the query module.

[0039] Example 3: Optimization Scheme for FOD Detection under Complex Meteorological Conditions 1. Multimodal data fusion detection a. Visual inspection: An improved YOLOv5s model with anti-interference features is used, and a rainwater noise filtering layer is added; b. Radar-assisted: Deploy millimeter-wave radar (77GHz) to detect underwater metallic foreign objects; c. Infrared compensation: The thermal imaging camera is activated during heavy rain at night, with a resolution of 640×512.

[0040] 2. Dynamic threshold adjustment mechanism a. The weather sensing module acquires meteorological data in real time, such as rainfall and wind speed; b. When rainfall is greater than 50 ml / h, the similarity threshold is adjusted to 85%; 3. Special handling procedures A dual verification mechanism is used for waterlogged areas: a) visual detection of suspected objects; b) radar scanning to confirm metal signals; c) an alarm is triggered only when both verifications are successful.

[0041] Example 4: FOD Detection Solution for Airports in Nighttime and Low-Light Environments 1. Hardware Configuration a. Infrared thermal imaging camera: 640×512 resolution, supports 0.01℃ temperature difference detection, suitable for nighttime scenarios with no light source; b. Illumination system: 850nm infrared LED array, covering a runway width of 60 meters, avoiding visible light interference to pilots; c. Low-light visible light camera: starlight-level sensor, minimum illumination of 0.001 lux, combined with wide dynamic range (WDR) technology.

[0042] 2. Technical Process a. Multi-source data fusion: Thermal imaging data is used to extract temperature anomaly areas through CNN; visible light images are used to identify contour features through YOLOv5, and the detection accuracy remains ≥92% even at night.

[0043] b. Dynamic threshold adjustment: When the ambient light is ≤10 lux, the comparison module will lower the similarity threshold from 90% to 88% to avoid misjudgment caused by shadows; temperature gradient analysis is introduced: if the temperature difference between a certain area and the surrounding temperature is >5℃, a primary warning will be triggered directly.

[0044] c. Alarm and handling mechanism: When stationary FOD (such as tire debris) is detected at night, the alarm module provides visual warning through infrared flashing lights (wavelength 940nm) to avoid strong light interference; for wild animals active at night (such as rabbits), the deterrent module emits 18kHz pulse sound waves with a coverage range of 200 meters, and at the same time activates the electric deterrent device of the runway boundary fence.

[0045] Example 5: Deployment Scheme of Airport FOD Detection Network with Multi-Runway Collaboration 1. System Architecture a. Edge-Cloud Collaborative Architecture: Edge Nodes: Each runway deploys an independent detection unit (including acquisition module, processing module, and recognition module) to process local data in real time and reduce cloud bandwidth pressure; Cloud Center: Aggregates the detection results of each runway and optimizes the global model through the learning module.

[0046] b. Cross-runway data association: Establish a foreign object type database: record the historical FOD types of each runway, such as construction debris commonly found on runway T1 and bird activity frequently on runway T2; when an anomaly is detected on a runway, the system automatically queries historical data from other runways to predict the migration path of foreign objects, such as light FOD drift caused by wind direction.

[0047] 2. Emergency Response Mechanism: If a high-risk FOD is detected on the main runway, the system will automatically set the secondary runway as a backup take-off and landing channel and notify the ATC system via the network to push the level P1. The bird control module works in coordination among multiple runways: If a flock of birds is found on runway T3, the adjacent runways T2 / T4 will simultaneously emit 10-20kHz sound waves to form a "sound barrier" to prevent birds from migrating.

[0048] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A machine learning-based FOD detection algorithm, characterized in that, include: Input module: Used to receive basic image data of the airport runway; Processing module: This module preprocesses and extracts features from the input data. It uses a convolutional neural network (CNN) for feature extraction, and the formula is as follows: in, As input features, As weight, For bias, For activation functions; Storage module: Used to store raw image data and detection results; Acquisition module: Used for real-time acquisition of image and audio data of the airport runway; Recognition module: Based on machine learning models, it identifies moving and stationary objects in the collected data; Alarm module: Triggers an alarm when a potential FOD is detected; Display module: used to visualize detection results and alarm information; The comparison module compares the real-time acquired data with the stored original data. The comparison module uses a similarity algorithm to calculate the difference between the real-time image and the original image. The similarity threshold is set to 90%. If the similarity is lower than this threshold, it is judged as abnormal. Judgment module: Determines whether an object is a FOD based on the comparison results; Drive-away module: Used to automatically drive away moving objects such as birds; It also includes a learning module, which optimizes the machine learning model based on each detection result. The optimization formula is as follows: in, For model parameters, For learning rate, This is the gradient of the loss function.

2. The FOD detection algorithm based on machine learning according to claim 1, characterized in that, The storage module includes: Data input unit: Receives data output from the processing module; Temporary storage unit: caches data collected in real time; Permanent storage unit: stores historical data; Data output unit: Provides data to the comparison module.

3. The FOD detection algorithm based on machine learning according to claim 1, characterized in that, The judgment module includes: Data receiving unit: Receives the output of the comparison module; Data classification unit: classifies objects into moving objects or stationary objects; Data transmission unit: Triggers alarm module or drive-away module based on classification results.

4. The FOD detection algorithm based on machine learning according to claim 1, characterized in that, The alarm module supports the following alarm methods: Voice prompt: Play alarm information through the speaker; Text notification: Display alarm information on the display module; Network notification: Sends alarm information to terminal devices via wireless network.

5. The FOD detection algorithm based on machine learning according to claim 1, characterized in that, The bird-repelling module drives away birds by emitting sound waves of a specific frequency, ranging from 10kHz to 20kHz.

6. The FOD detection algorithm based on machine learning according to claim 1, characterized in that, 。 7. The FOD detection algorithm based on machine learning according to claim 1, characterized in that, The image acquisition unit of the acquisition module uses a high-definition camera with a resolution of no less than 1920×1080 and a frame rate of 30fps.

8. The FOD detection algorithm based on machine learning according to claim 1, characterized in that, The recognition module uses the YOLOv5 algorithm for object detection, with a detection accuracy of ≥95%.

Images