Method for detecting foreign objects on an airport runway based on biomimetic binocular vision

Abstract

This invention relates to a method for detecting foreign objects (FOOs) on airport runways based on biomimetic binocular vision. The method includes: simultaneously receiving images at least two scales from cameras with different focal lengths; performing cooperative target detection based on spatial correspondence; mapping and fusing the detection results to output supervised detection results; employing an unsupervised learning algorithm for anomaly detection and outputting unsupervised detection results; cross-validating and fusing the supervised and unsupervised detection results to generate a comprehensive FEO detection result; receiving left and right view images from a biomimetic binocular camera; performing three-dimensional spatial localization of FEOs in the comprehensive FEO detection result based on a binocular stereo vision algorithm; and fusing the comprehensive FEO detection result with the FEO location information to generate a final detection report. This method integrates multiple technological advantages, forming a complete, high-precision, and robust all-weather FEO detection solution for airport runways.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for detecting foreign objects on airport runways based on bionic binocular vision. Background Technology

[0002] Foreign object debris (FOD) on the runway refers to objects on the runway surface that may pose a threat to the safety of aircraft takeoff and landing, such as metal parts, gravel, plastic products, birds, and wild animals. Timely detection and removal of FOD is a critical step in ensuring flight safety.

[0003] Currently, mainstream methods for detecting foreign objects on runways mainly rely on traditional manual inspections and automated detection based on single technologies. Manual inspections depend on the visual observation of inspectors, resulting in low efficiency, susceptibility to subjective factors and adverse weather conditions, and the inability to achieve 24 / 7 real-time monitoring. Automated detection technologies, such as solutions based on fixed camera video analysis, while enabling uninterrupted monitoring, have significant shortcomings in detection accuracy, environmental adaptability, and the ability to identify small-sized foreign objects. While lidar solutions offer high positioning accuracy, their high equipment cost makes large-scale deployment difficult. Furthermore, existing technologies generally struggle to effectively identify unknown types of foreign objects not present in the training data. Summary of the Invention

[0004] The purpose of this invention is to provide a foreign object detection method for airport runways based on biomimetic binocular vision, so as to solve the problems of insufficient detection accuracy, weak ability to identify unknown foreign objects, and low positioning accuracy of existing airport runway foreign object detection methods.

[0005] To achieve the above objectives, this invention provides a method for detecting foreign objects on airport runways based on biomimetic binocular vision, comprising the following steps:

[0006] S1. Simultaneously receive at least two scale images from cameras with different focal lengths, perform cooperative target detection based on the spatial correspondence between the at least two scale images, and map and fuse the detection results of the at least two scale images to output supervised detection results;

[0007] S2. An unsupervised learning algorithm is used to perform anomaly detection on the images at least two scales, and the unsupervised detection results are output.

[0008] S3. Cross-validate and fuse the supervised detection results with the unsupervised detection results to generate a comprehensive foreign object detection result;

[0009] S4. Receive left and right view images from the bionic binocular camera, and perform three-dimensional spatial positioning of the foreign object in the comprehensive foreign object detection result based on the binocular stereo vision algorithm, and output the position information of the foreign object.

[0010] S5. The comprehensive foreign object detection results are fused with the location information of the foreign object to generate a final detection report.

[0011] Optionally, the different focal length cameras include at least wide-angle cameras and telephoto cameras.

[0012] Optionally, S1 specifically includes:

[0013] Establish geometric mapping relationships between images of different scales;

[0014] Perform rapid target detection on the image from the wide-angle camera to filter candidate regions;

[0015] The candidate regions are mapped onto the image from the telephoto camera for refined detection;

[0016] A feature pyramid network is used to fuse feature information at different scales to output supervised detection results.

[0017] Optionally, S2 specifically includes:

[0018] Construct a convolutional autoencoder network to learn feature representations of a normal runway scene, and identify abnormal regions by reconstructing errors; or...

[0019] Generative adversarial networks are constructed to learn the normal runway scene distribution. An anomaler identifies abnormal samples that deviate from the normal distribution, and an anomaly score calculation mechanism is used to quantify the degree of anomalousness in each region.

[0020] Optionally, S4 specifically includes:

[0021] The disparity maps of the left and right view images are calculated using a semi-global matching or deep learning stereo matching algorithm;

[0022] Based on the camera's intrinsic and extrinsic parameters, the disparity value corresponding to the region where the foreign object is located in the disparity map is converted into the three-dimensional spatial coordinates of the foreign object;

[0023] Establish a runway plane coordinate system and map the three-dimensional spatial coordinates of the foreign object to the runway plane coordinate system to obtain the distance and azimuth of the foreign object relative to the detection device.

[0024] Optionally, S5 specifically includes:

[0025] The comprehensive foreign object detection results are weighted and fused with the location information of the foreign object;

[0026] Decision fusion is performed using Bayesian inference or DS evidence theory;

[0027] Classification and hazard assessment based on foreign object characteristics.

[0028] Optionally, S5 further includes:

[0029] The foreign object is tracked in time using Kalman filtering or multi-target tracking algorithms to predict its trajectory, and the hazard level assessment is dynamically updated based on the trajectory.

[0030] Based on the same inventive concept, this invention also provides an airport runway foreign object detection system based on biomimetic binocular vision, comprising:

[0031] An image receiving module is used to simultaneously receive images at at least two scales from cameras with different focal lengths;

[0032] A multi-scale detection module is used to perform cooperative target detection based on the spatial correspondence between the at least two scale images, and to map and fuse the detection results of the at least two scale images to output supervised detection results;

[0033] The unsupervised detection module is used to perform anomaly detection on the at least two scale images using an unsupervised learning algorithm and output the unsupervised detection results.

[0034] The first fusion module is used to cross-validate and fuse the supervised detection results with the unsupervised detection results to generate a comprehensive foreign object detection result.

[0035] The stereo positioning module is used to perform three-dimensional spatial positioning of foreign objects in the comprehensive foreign object detection results based on a binocular stereo vision algorithm, and output the position information of the foreign object.

[0036] The second fusion module is used to fuse the comprehensive foreign object detection results with the location information of the foreign object to generate a final detection report.

[0037] Optionally, the unsupervised detection module uses an autoencoder, variational autoencoder, or generative adversarial network for anomaly detection.

[0038] Based on the same inventive concept, the present invention also provides a readable storage medium having a computer program stored thereon, which, when executed, can realize the airport runway foreign object detection method based on bionic binocular vision as described above.

[0039] The airport runway foreign object detection method based on bionic binocular vision provided by this invention has at least one of the following beneficial effects:

[0040] (1) Through multi-scale collaborative detection, the advantages of cameras with different focal lengths are complemented, which improves the detection speed and the recognition accuracy of small foreign objects;

[0041] (2) By introducing unsupervised learning algorithms, it is possible to effectively identify unknown types of foreign objects that have not appeared in the training set, thereby enhancing the generalization ability of the system;

[0042] (3) Cross-validation and fusion of supervised and unsupervised detection results effectively integrates the high-precision recognition advantage of supervised learning with the open set detection capability of unsupervised learning, solves the problems of poor environmental adaptability and high false negative rate for unknown types of foreign objects by single detection technology, and significantly improves the overall detection rate and robustness of the system.

[0043] (4) The three-dimensional spatial positioning of foreign objects is achieved by using bionic binocular vision, realizing non-contact, centimeter-level precision three-dimensional positioning of foreign objects. The output of the distance and azimuth angle of the foreign object relative to the detection equipment is intuitive and clear, which can be directly used to guide ground staff to handle the situation quickly and accurately, solving the pain point of traditional detection methods that can only alarm but cannot accurately locate the object;

[0044] (5) The detection accuracy of this invention reaches over 95%; the recognition rate of small foreign objects (2-3cm) is significantly improved. The unsupervised learning algorithm can identify unknown types of foreign objects that have not appeared in the training data, and the binocular stereo vision algorithm achieves centimeter-level positioning accuracy, meeting the airport's 24 / 7 uninterrupted monitoring requirements. Attached Figure Description

[0045] Those skilled in the art will understand that the accompanying drawings are provided to better understand the invention and do not constitute any limitation on the scope of the invention. Wherein:

[0046] Figure 1 This is a flowchart of a foreign object detection method for airport runways based on biomimetic binocular vision, provided as an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0049] In the description of this invention, it should be understood that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0050] Furthermore, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes said element. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0051] Please refer to Figure 1 This invention provides a method for detecting foreign objects on airport runways based on biomimetic binocular vision, comprising the following steps:

[0052] S1. Simultaneously receive at least two scale images from cameras with different focal lengths, perform cooperative target detection based on the spatial correspondence between the at least two scale images, and map and fuse the detection results of the at least two scale images to output supervised detection results;

[0053] S2. An unsupervised learning algorithm is used to perform anomaly detection on the images at least two scales, and the unsupervised detection results are output.

[0054] S3. Cross-validate and fuse the supervised detection results with the unsupervised detection results to generate a comprehensive foreign object detection result;

[0055] S4. Receive left and right view images from the bionic binocular camera, and perform three-dimensional spatial positioning of the foreign object in the comprehensive foreign object detection result based on the binocular stereo vision algorithm, and output the position information of the foreign object.

[0056] S5. The comprehensive foreign object detection results are fused with the location information of the foreign object to generate a final detection report.

[0057] This method utilizes multi-scale collaborative detection to leverage the complementary advantages of cameras with different focal lengths, improving detection speed and accuracy in identifying small foreign objects. By introducing an unsupervised learning algorithm, it effectively identifies unknown types of foreign objects not present in the training set, enhancing the system's generalization ability. Then, supervised and unsupervised detection results are cross-validated and fused, significantly reducing the false alarm and false negative rates of single detection methods and improving detection reliability. Finally, biomimetic binocular vision is used to achieve three-dimensional spatial localization of foreign objects, providing precise location guidance for foreign object removal. This method integrates the advantages of multiple technologies, forming a complete, high-precision, and robust all-weather airport runway foreign object detection solution, fundamentally solving the technical problems mentioned in the background technology, such as low efficiency, poor environmental adaptability, difficulty in detecting small targets, insufficient identification of unknown foreign objects, and lack of precise localization.

[0058] First, execute S1 to simultaneously receive at least two scale images from cameras with different focal lengths. Based on the spatial correspondence between the at least two scale images, perform cooperative target detection, and map and fuse the detection results of the at least two scale images to output supervised detection results.

[0059] In a preferred embodiment, the different focal length cameras include at least a wide-angle camera and a telephoto camera.

[0060] The wide-angle camera has a large horizontal field of view, used to cover a large area of ​​the runway for monitoring. The telephoto camera has a narrower field of view but higher resolution, and can be deployed alongside or adjacent to the wide-angle camera on the same main unit. Its optical axis is calibrated to partially overlap with the field of view of the wide-angle camera. With this configuration, the wide-angle camera achieves a wide field of view for rapid anomaly detection, while the telephoto camera achieves a clearer field of view for detailed analysis of suspicious areas.

[0061] In this embodiment, S1 specifically includes:

[0062] Establish geometric mapping relationships between images of different scales;

[0063] Perform rapid target detection on the image from the wide-angle camera to filter candidate regions;

[0064] The candidate regions are mapped onto the image from the telephoto camera for refined detection;

[0065] A feature pyramid network is used to fuse feature information at different scales to output supervised detection results.

[0066] Specifically, after receiving synchronized image data from the wide-angle camera and the telephoto camera, the two-scale images are spatiotemporally aligned to ensure the correspondence of the same target in images of different scales. Then, a scale mapping relationship is established, mapping the target region in the wide-angle camera image to the corresponding region in the telephoto camera image. Next, a lightweight object detection network (such as a lightweight version of YOLO or SSD) is run on the wide-angle camera image. This network, trained on a large amount of labeled runway foreign object data, can quickly infer multiple candidate region bounding boxes that may contain foreign objects across the entire image, rapidly filtering potential foreign object regions. Due to the relatively low resolution of the wide-angle image and the small size of the target, the goal at this stage is high recall, allowing for a certain number of false positives. Then, each candidate region is projected onto the high-resolution telephoto image according to the aforementioned geometric mapping relationship for refined feature extraction. Finally, a Feature Pyramid Network (FPN) or similar structure is used to fuse feature information at different scales. Confidence-weighted fusion can be used, and non-maximum suppression (NMS) algorithm is used to eliminate duplicate detections. Finally, refined detection results (such as category, confidence, and precise bounding box) are output as supervised detection results, thereby establishing a cross-scale detection consistency verification mechanism and improving detection reliability.

[0067] Next, step S2 is executed, employing an unsupervised learning algorithm to perform anomaly detection on the at least two scale images, outputting the unsupervised detection results. Unsupervised anomaly detection aims to discover pixel regions deviating from the "normal" runway scene, without requiring foreign object annotation data. This embodiment provides two optional implementation methods, which can be used individually or in combination:

[0068] 1) Anomaly Detection Based on Convolutional Autoencoders. First, a convolutional autoencoder is trained using a large number of normal runway images (containing images from different time periods and under different weather conditions) that do not contain any foreign objects. This convolutional autoencoder learns to compress the normal images into a low-dimensional feature space and then reconstruct them. The training objective is to make the reconstructed images as similar to the input images as possible. By calculating the reconstruction error (e.g., mean squared error) between the original image and the reconstructed image at each pixel location, and setting a threshold, pixel regions with reconstruction errors exceeding this threshold are marked as anomaly regions, and the output is the unsupervised detection result. Preferably, a variational autoencoder (VAE) can be used to enhance the modeling capability for complex scenes.

[0069] 2) Anomaly Detection Based on Generative Adversarial Networks (GANs). A GAN is constructed where the generator learns the distribution of a normal runway scene and attempts to generate realistic images; the discriminator learns to distinguish between real normal images and fake images generated by the generator. The GAN is trained on a large number of normal images until it reaches Nash equilibrium. During inference, the generator is fixed, and the image to be tested is input. For a normal region, the generator can generate a similar image well, and the discriminator will give a high "realism" score (i.e., a low anomaly score). For a foreign object region, since it does not belong to the distribution learned by the generator, the generator has difficulty generating a matching image, resulting in the discriminator giving a low "realism" score (i.e., a high anomaly score). Through this mechanism, an anomaly score map can be calculated to quantify the degree of anomaly in each region. By setting a score threshold, unsupervised detection results can be obtained. This method can typically learn more complex normal data manifolds and has better detection capabilities for more subtle anomalies.

[0070] Unsupervised learning algorithms can discover unknown types of foreign objects (such as new types of plastic products or specific animals) that are not present in the training set, which complements supervised detection and greatly enhances the system's generalization and practicality.

[0071] Then, step S3 is executed to cross-validate and fuse the supervised detection results with the unsupervised detection results, generating a comprehensive foreign object detection result. By cross-validating the unsupervised detection results with the supervised detection results and establishing a confidence fusion strategy, the advantages of supervised and unsupervised detection can be combined, allowing for focused annotation and analysis of areas detected by unsupervised detection but not by supervised detection.

[0072] Next, S4 is executed to receive left and right view images from the bionic binocular camera, and to perform three-dimensional spatial positioning of the foreign object in the comprehensive foreign object detection result based on the binocular stereo vision algorithm, and output the position information of the foreign object.

[0073] It should be noted that the "bionic binocular camera" here refers to a stereoscopic vision device designed to mimic the characteristics of the binocular visual system of organisms (such as birds of prey and primates). In a specific embodiment, the bionic binocular camera consists of two high-performance CMOS cameras mounted on a servo gimbal with two degrees of freedom (pitch and yaw). These two cameras can function as a single unit for synchronous active scanning, simulating the "gazing" mechanism of biological eyes to quickly center the region of interest in the field of view; or they can be controlled independently to simulate the switching between "monocular" observation and "binocular" ranging in some organisms. Its "bionic" characteristics are reflected in its active perception and dynamic adjustment capabilities, rather than just a static binocular baseline. In step S4, when it is necessary to acquire stereoscopic image pairs of foreign objects, the system can control the bionic binocular camera to converge its optical axis to the approximate area of ​​the initially detected foreign object, thereby acquiring high-quality left and right view images suitable for stereo matching. This active adjustment mechanism, compared to a fixed binocular system, can achieve better stereoscopic imaging effects over a wider range, thereby improving subsequent positioning accuracy.

[0074] Specifically, after determining the presence of a foreign object based on S3, the system controls the bionic binocular camera to focus and capture left and right view images of the area where the foreign object is located. The core of stereo matching is calculating disparity. Traditional algorithms such as semi-global matching (SGM) can be used. This algorithm aggregates two-dimensional information through multi-path one-dimensional constraints, achieving a good balance between accuracy and efficiency, and performs well in areas with rich texture. Alternatively, deep learning-based stereo matching algorithms (such as GC-Net and PSMNet) can be used. These algorithms directly learn the mapping from image pairs to disparity maps through convolutional networks, exhibiting stronger matching capabilities for areas with weak or repetitive textures, but with relatively high computational cost. After obtaining the disparity map through matching, further post-processing can be performed to eliminate noise and incorrect matching. Then, based on the intrinsic parameters (focal length, principal point, distortion coefficients) and extrinsic parameters (rotation matrix and translation vector describing the relative position and orientation of the left and right cameras) of the bionic binocular camera, the disparity value corresponding to the area where the foreign object is located in the disparity map is converted into the three-dimensional spatial coordinates of the foreign object using the principle of triangulation.

[0075] Finally, a runway plane coordinate system is established, and the three-dimensional spatial coordinates of the foreign object are mapped to this coordinate system to obtain the distance and azimuth of the foreign object relative to the detection device. For example, the center point of one end of the runway entrance is taken as the origin O, the direction of the runway centerline is the positive Y-axis, the horizontal direction to the right perpendicular to the centerline is the positive X-axis, and the vertical upward direction is the Z-axis. Through a known coordinate transformation (rotation and translation), the three-dimensional coordinates of the foreign object in the camera coordinate system are transformed to this runway plane coordinate system. Ultimately, the output position information includes the two-dimensional coordinates (X, Y) of the foreign object on the runway plane, which allows for easy calculation of the lateral distance of the foreign object from the detection device and its azimuth and distance relative to the origin.

[0076] Preferably, multi-frame fusion technology can be used to improve positioning stability, establish a motion compensation mechanism to eliminate the impact of camera shake on positioning, and perform smoothing filtering on positioning results to reduce instantaneous errors.

[0077] Preferably, after the S4 outputs the three-dimensional spatial coordinates of the foreign object (especially its precise location in the runway plane coordinate system), to further eliminate false alarms, verify positioning accuracy, and potentially obtain richer target information, an active verification step can be performed: that is, based on the three-dimensional spatial coordinates, adjacent devices are invoked to rotate to the corresponding angle for verification. This verification generates a "verification confidence score" and potentially updated target attribute information. This result will serve as a key input to subsequent decision fusion steps. For example, in weighted fusion or DS evidence theory decision-making, the final confidence of a target that passes the verification will be significantly improved; conversely, a "target" that fails the verification will be marked as a "suspected false alarm," and its alarm level will be downgraded or suppressed. In a broader system deployment, "adjacent devices" can be two physically adjacent, independent sets of the same devices. The system uses a scheduling algorithm to select the nearest device with the best current perspective and available space to perform the verification task. This demonstrates system-level collaborative operation capabilities, achieving optimized resource utilization and redundant verification.

[0078] Finally, step S5 is executed to fuse the comprehensive foreign object detection results with the location information of the foreign object, generating a final detection report.

[0079] Preferably, S5 specifically includes:

[0080] The comprehensive foreign object detection results are weighted and fused with the location information of the foreign object;

[0081] Decision fusion is performed using Bayesian inference or DS evidence theory;

[0082] Classification and hazard assessment based on foreign object characteristics.

[0083] The comprehensive foreign object detection results from S3 (including foreign object category, detection confidence, target bounding box, etc.) and the foreign object location information (3D coordinates, planar coordinates, etc.) from S4 are weighted and fused. Then, Bayesian inference or DS evidence theory is used for decision fusion. Finally, classification and hazard level assessment are performed based on the fused information. Classification relies not only on visual appearance but also on size (calculated from 3D coordinates) and rough material inference (such as the reflective properties of metal under specific imaging conditions). The hazard level assessment model evaluates the threat level of foreign objects to flight safety based on characteristics such as size, location (runway center landing area vs. runway shoulder), and whether the material is metallic, classifying them into high, medium, and low risk levels and triggering alarms at different response levels.

[0084] Preferably, S5 further includes:

[0085] The foreign object is tracked in time using Kalman filtering or multi-target tracking algorithms to predict its trajectory, and the hazard level assessment is dynamically updated based on the trajectory.

[0086] For example, a tracker is initialized for a foreign object detected in the current frame. In subsequent frames, a Kalman filter algorithm is used to predict the object's position in the next frame based on its motion model in the runway plane coordinate system, and data association is performed near the predicted position to achieve multi-target tracking. For successfully matched trajectories, the state of the Kalman filter is updated with the current observations. By analyzing the trajectory, the movement trajectory of the foreign object can be predicted (e.g., whether it rolls with the wind or is carried away by birds). More importantly, its hazard level can be dynamically updated based on the object's motion status. For example, a small stone stationary on the shoulder may have a low hazard level, but the same stone rolling towards the center of the runway should have its hazard level increased to medium or high in real time. This function enables dynamic and intelligent assessment of runway safety status.

[0087] In this embodiment, the system processing and detection accuracy of the airport runway foreign object detection method based on bionic binocular vision reaches over 95%; the recognition rate of small-sized foreign objects (2-3cm) is significantly improved; the unsupervised learning algorithm can identify unknown types of foreign objects that did not appear in the training data, and the binocular stereo vision algorithm achieves centimeter-level positioning accuracy; meeting the airport's 24 / 7 uninterrupted monitoring requirements.

[0088] Based on the same inventive concept, this invention also proposes an airport runway foreign object detection system based on biomimetic binocular vision, comprising:

[0089] An image receiving module is used to simultaneously receive images at at least two scales from cameras with different focal lengths;

[0090] A multi-scale detection module is used to perform cooperative target detection based on the spatial correspondence between the at least two scale images, and to map and fuse the detection results of the at least two scale images to output supervised detection results;

[0091] The unsupervised detection module is used to perform anomaly detection on the at least two scale images using an unsupervised learning algorithm and output the unsupervised detection results.

[0092] The first fusion module is used to cross-validate and fuse the supervised detection results with the unsupervised detection results to generate a comprehensive foreign object detection result.

[0093] The stereo positioning module is used to perform three-dimensional spatial positioning of foreign objects in the comprehensive foreign object detection results based on a binocular stereo vision algorithm, and output the position information of the foreign object.

[0094] The second fusion module is used to fuse the comprehensive foreign object detection results with the location information of the foreign object to generate a final detection report.

[0095] Preferably, the multi-scale detection module employs a deep learning object detection network, including but not limited to the YOLO series, Faster R-CNN, and SSD algorithm.

[0096] Preferably, the unsupervised detection module uses an autoencoder, variational autoencoder, or generative adversarial network for anomaly detection.

[0097] Since the airport runway foreign object detection system based on bionic binocular vision provided by this invention belongs to the same inventive concept as the airport runway foreign object detection method based on bionic binocular vision described above, the airport runway foreign object detection system based on bionic binocular vision provided by this invention has all the advantages of the airport runway foreign object detection method based on bionic binocular vision described above. Therefore, the beneficial effects of the airport runway foreign object detection system based on bionic binocular vision provided by this invention will not be described in detail here.

[0098] Based on the same inventive concept, embodiments of the present invention also propose a readable storage medium storing a computer program thereon, which, when executed, can realize the airport runway foreign object detection method based on bionic binocular vision as described above.

[0099] A readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device, such as, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer programs described herein can be downloaded from the readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. Networks can include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. Each computing / processing device's network adapter card or network interface receives and forwards a computer program from the network for storage on a readable storage medium within the respective computing / processing device. The computer program used to perform the operations of this invention can be execution instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer program can execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from a computer program. These electronic circuits can execute computer-readable program instructions, thereby realizing various aspects of the present invention.

[0100] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by a computer program. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. These computer programs can also be stored in a readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the readable storage medium storing the computer program comprises an article of manufacture including instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams.

[0101] A computer program may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the computer program executing on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0102] Since the readable storage medium provided by this invention belongs to the same inventive concept as the airport runway foreign object detection method based on bionic binocular vision described above, the readable storage medium provided by this invention has all the advantages of the airport runway foreign object detection method based on bionic binocular vision described above. Therefore, the beneficial effects of the readable storage medium provided by this invention will not be described in detail here.

[0103] In summary, this invention provides a method for detecting foreign objects (FOOs) on airport runways based on biomimetic binocular vision. Through multi-scale collaborative detection, it achieves complementary advantages of different focal length cameras, improving detection speed and accuracy in identifying small FEOs. By introducing an unsupervised learning algorithm, it can effectively identify unknown types of FEOs not found in the training set, enhancing the system's generalization ability. Cross-validation and fusion of supervised and unsupervised detection results significantly reduces the false alarm and false negative rates of single detection methods, improving detection reliability. Finally, biomimetic binocular vision is used to achieve three-dimensional spatial localization of FEOs, providing precise location guidance for FEO removal. This method integrates the advantages of multiple technologies, forming a complete, high-precision, and robust all-weather airport runway FEO detection solution, fundamentally solving the technical problems mentioned in the background technology, such as low efficiency, poor environmental adaptability, difficulty in detecting small targets, insufficient identification of unknown FEOs, and lack of precise localization.

[0104] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for detecting foreign objects on an airport runway based on biomimetic binocular vision, characterized in that, Includes the following steps: S1. Simultaneously receive at least two scale images from cameras with different focal lengths, perform cooperative target detection based on the spatial correspondence between the at least two scale images, and map and fuse the detection results of the at least two scale images to output supervised detection results; S2. An unsupervised learning algorithm is used to perform anomaly detection on the images at least two scales, and the unsupervised detection results are output. S3. Cross-validate and fuse the supervised detection results with the unsupervised detection results to generate a comprehensive foreign object detection result; S4. Receive left and right view images from the bionic binocular camera, and perform three-dimensional spatial positioning of the foreign object in the comprehensive foreign object detection result based on the binocular stereo vision algorithm, and output the position information of the foreign object. S5. The comprehensive foreign object detection results are fused with the location information of the foreign object to generate a final detection report.

2. The airport runway foreign object detection method based on bionic binocular vision according to claim 1, characterized in that, The different focal length cameras include at least wide-angle cameras and telephoto cameras.

3. The method for detecting foreign objects on the airport runway based on the bionic binocular vision according to claim 2, characterized in that, S1 specifically includes: Establish geometric mapping relationships between images of different scales; Perform rapid target detection on the image from the wide-angle camera to filter candidate regions; The candidate regions are mapped onto the image from the telephoto camera for refined detection; A feature pyramid network is used to fuse feature information at different scales to output supervised detection results.

4. The method for detecting foreign objects on an airport runway based on biomimetic binocular vision according to claim 1, characterized in that, S2 specifically includes: Construct a convolutional autoencoder network to learn feature representations of a normal runway scene, and identify abnormal regions by reconstructing errors; or... Generative adversarial networks are constructed to learn the normal runway scene distribution. An anomaler identifies abnormal samples that deviate from the normal distribution, and an anomaly score calculation mechanism is used to quantify the degree of anomalousness in each region.

5. The method for detecting foreign objects on airport runways based on bionic binocular vision according to claim 1, characterized in that, S4 specifically includes: The disparity maps of the left and right view images are calculated using a semi-global matching or deep learning stereo matching algorithm; Based on the camera's intrinsic and extrinsic parameters, the disparity value corresponding to the region where the foreign object is located in the disparity map is converted into the three-dimensional spatial coordinates of the foreign object; Establish a runway plane coordinate system and map the three-dimensional spatial coordinates of the foreign object to the runway plane coordinate system to obtain the distance and azimuth of the foreign object relative to the detection device.

6. The method for detecting foreign objects on airport runways based on bionic binocular vision according to claim 1, characterized in that, S5 specifically includes: The comprehensive foreign object detection results are weighted and fused with the location information of the foreign object; Decision fusion is performed using Bayesian inference or DS evidence theory; Classification and hazard assessment based on foreign object characteristics.

7. The method for detecting foreign objects on airport runways based on bionic binocular vision according to claim 6, characterized in that, The S5 also includes: The foreign object is tracked in time using Kalman filtering or multi-target tracking algorithms to predict its trajectory, and the hazard level assessment is dynamically updated based on the trajectory.

8. A foreign object detection system for airport runways based on biomimetic binocular vision, characterized in that, include: An image receiving module is used to simultaneously receive images at at least two scales from cameras with different focal lengths; A multi-scale detection module is used to perform cooperative target detection based on the spatial correspondence between the at least two scale images, and to map and fuse the detection results of the at least two scale images to output supervised detection results; The unsupervised detection module is used to perform anomaly detection on the at least two scale images using an unsupervised learning algorithm and output the unsupervised detection results. The first fusion module is used to cross-validate and fuse the supervised detection results with the unsupervised detection results to generate a comprehensive foreign object detection result. The stereo positioning module is used to perform three-dimensional spatial positioning of foreign objects in the comprehensive foreign object detection results based on a binocular stereo vision algorithm, and output the position information of the foreign object. The second fusion module is used to fuse the comprehensive foreign object detection results with the location information of the foreign object to generate a final detection report.

9. The airport runway foreign object detection system based on bionic binocular vision according to claim 8, characterized in that, The unsupervised detection module uses autoencoders, variational autoencoders, or generative adversarial networks for anomaly detection.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it can realize the airport runway foreign object detection method based on bionic binocular vision according to any one of claims 1-7.

Images