An airport runway FOD identification method, system and product based on runway bidirectional feature fusion

Abstract

The application discloses an airport runway FOD identification method, system and product based on runway bidirectional feature fusion in the field of airport runway FOD identification, and the method comprises the following steps: S100, a plurality of groups of FOD monitoring camera groups composed of two FOD monitoring cameras are arranged at equal intervals along the length direction of the airport runway; S200, the field of view area of the FOD monitoring camera is calibrated; S300, the weight parameter of the overlapping field of view area is configured; S400, the shooting image of the FOD monitoring camera is acquired in real time, the near-field field of view area is directly input into the FOD identification model, the feature map is output for FOD identification, the area belonging to the far-field field of view area in the overlapping field of view area is input into the FOD identification model based on the weight parameter, and the weighted feature map is output for FOD identification after feature fusion; and S500, the FOD alarm signal is output based on the FOD identification result. The application can significantly improve the accuracy and reliability of the airport runway FOD identification.

Description

Technical Field

[0001] This application relates to an airport runway FOD identification method, system, and product based on runway bidirectional feature fusion for the field of airport runway FOD identification. Background Technology

[0002] Foreign Object Debris (FOD) on airport runways refers to non-specific foreign substances, debris, or objects that may damage aircraft, such as metal parts, machine tools, gravel, and plastic products. FOD poses a serious threat to aviation safety; if ingested into an engine or damaging aircraft components, it could lead to catastrophic accidents, causing enormous economic losses and casualties. Therefore, achieving efficient and accurate runway FOD identification is a core element in ensuring safe airport operations.

[0003] Currently, airport runway FOD (Fear of Destruction) identification primarily employs technologies including manual inspection, radar detection, and machine vision-based automatic identification methods. Among these, machine vision-based identification technology has become a focus of research and application due to its ability to provide rich target detail information, relatively low cost, and ease of deployment. This type of technology typically relies on fixed optical cameras deployed along the runway side to acquire runway images and utilize target detection algorithms for FOD identification, such as deep learning models like YOLOv and Faster R-CNN. However, limited by the actual application scenarios of airport runway FOD, existing machine vision-based FOD identification technologies still face some challenges.

[0004] When dealing with wide runways used by large passenger aircraft, where the runway width can reach 50-60 meters, cameras struggle to clearly capture small objects (FODs) on the opposite side of the runway, such as small rubber fragments or metal parts measuring 2-5 cm in length. Although using high-resolution sensors and telephoto lenses can alleviate this problem to some extent, due to the limitations of physical imaging principles, small distant targets occupy very few pixels in the image, resulting in a significant loss of detail. This makes it difficult for image feature-based recognition algorithms to effectively extract the discriminative features of the targets, leading to a higher false negative rate.

[0005] Furthermore, in the engineering deployment of airport runway FOD monitoring systems, in order to ensure that the cameras themselves do not obstruct aircraft take-off and landing, there are clear restrictions on the installation height of the cameras. This directly restricts the coverage capability of the camera's field of view in the depth direction, thereby affecting its imaging clarity and recognizability of small targets at a distance.

[0006] Some technical solutions attempt to improve recognition rates by using dynamic zoom to magnify distant areas. However, zoom cameras place extremely high demands on control strategies and recognition algorithms. Not only is it necessary to plan the camera's scanning time and scanning space range according to different focal lengths, but it is also necessary to solve the problems of field of view changes, depth of field adjustment, and image stability introduced by zoom, which increases the complexity and maintenance cost of the system.

[0007] In addition, environmental interference factors, such as adverse effects caused by lighting conditions such as backlighting, reflection, and low illumination, or reduced image clarity caused by weather conditions such as rain, fog, and light haze, will further increase the difficulty of identifying small FODs at long distances. Summary of the Invention

[0008] The purpose of this application is to overcome the shortcomings of the prior art and provide an airport runway FOD identification method, system and product based on runway bidirectional feature fusion, which can significantly improve the accuracy and reliability of airport runway FOD identification.

[0009] Firstly, this application provides an airport runway FOD identification method based on runway bidirectional feature fusion, the technical solution of which includes the following steps: S100, several groups of FOD monitoring cameras are deployed at equal intervals along the length of the airport runway. Each group of FOD monitoring cameras consists of two FOD monitoring cameras that are one-to-one and set opposite each other on both sides of the airport runway. S200 calibrates the field of view of FOD surveillance cameras, including calibrating the near-field field of view, far-field field of view, and overlapping field of view of two FOD surveillance cameras in the same group. S300 configures weight parameters for overlapping view areas; The S400 acquires images captured by FOD surveillance cameras in real time, directly inputs the near-field field of view in the captured images into the FOD recognition model, outputs a feature map for FOD recognition, and inputs the regions belonging to the far-field field of view in the overlapping field of view in the captured images into the FOD recognition model based on weight parameters, performs feature fusion, and outputs a weighted feature map for FOD recognition. The S500 outputs a FOD alarm signal based on the FOD identification result.

[0010] By adopting the above technical solution, FOD monitoring cameras are deployed in a relatively fixed and one-to-one manner on both sides of the airport runway. For clear images in the near field of view, the images are directly input into the FOD recognition model for FOD recognition. For images in the far field of view that are relatively blurry, the overlapping fields of view are subjected to weighted feature fusion based on weight parameters for FOD recognition. This can effectively improve the accuracy of FOD recognition, especially the detection rate and accuracy of small FODs in the far field of view.

[0011] As a preferred embodiment, the calibration of the field of view area of ​​the FOD surveillance camera in S200 specifically includes the following steps: S201, perform horizontal calibration on two FOD surveillance cameras in the same group so that their optical axes are perpendicular to the centerline of the airport runway and their horizontal projections are on the same straight line. S202, calibrate the near field of view depth d1 and far field of view depth d2 of the FOD monitoring camera. The area of ​​the field of view depth of the FOD monitoring camera that is less than the near field of view depth d1 is the near field of view area, and the area of ​​the calibrated field of view depth that is greater than the near field of view depth d1 and less than the far field of view depth d2 is the far field of view area. S203 calibrates the overlapping field of view area of ​​two FOD surveillance cameras in the same group based on the field of view depth.

[0012] By adopting the above technical solution, the two FOD monitoring cameras in the same group are horizontally calibrated to ensure that their shooting directions are strictly aligned and perpendicular to the airport runway direction. Then, based on the depth of field, the near-field field of view, far-field field of view, and overlapping field of view are calibrated, realizing the regional division of the images captured by the FOD monitoring cameras, and providing a basis for subsequent FOD identification by distinguishing the field of view regions.

[0013] Preferably, in S202, the calibration method for the near-field field depth d1 and the far-field field depth d2 is as follows: The FOD monitoring camera is used to photograph a calibration reference object of a certain size. The depth distance between the calibration reference object and the FOD monitoring camera is adjusted. When the pixel area of ​​the calibration reference object in the field of view reaches the near field depth threshold, the depth distance is calibrated as the near field depth d1. When the pixel area of ​​the calibration reference object in the field of view reaches the far field depth threshold, the depth distance is calibrated as the far field depth d2.

[0014] By adopting the above technical solution, the near-field depth of view and the far-field depth of view are calculated by calibrating the pixel area of ​​the reference object in the field of view. This ensures that the recognition capability of the minimum FOD of the calibrated near-field field of view and far-field field of view meets the design requirements.

[0015] As a preferred embodiment, S202 also includes verifying the constraints on the near-field and far-field depth of view of the FOD surveillance camera, specifically: ; Where h is the width of the airport runway, and d1' is the near-field field of view depth of the FOD surveillance camera on the opposite side of the same group to be verified.

[0016] Using the above technical solution, based on the constraints of near-field field of view depth d1 and far-field field of view depth d2, the pitch angle of the FOD monitoring camera is adjusted to ensure that the field of view of the FOD monitoring camera can cover the entire range in the width direction of the airport runway. Preferably, in S300, the weight parameter configuration for the overlapping visual region specifically includes algorithms for configuring static weight parameters and configuring dynamic weight parameters; The static weight parameters are configured based on the field of view depth of the FOD surveillance camera. Field of view areas with the same field of view depth have the same static weight parameters, and the static weight parameters decrease as the field of view depth increases. The dynamic weight parameters are configured based on the global quality of the images captured by the FOD surveillance cameras. A dynamic confidence score algorithm is constructed based on the quality index of the captured images. The dynamic confidence score of the two FOD surveillance cameras in the same group at the same time is calculated through the captured images. Based on the dynamic confidence score, the dynamic weight parameters of the two FOD surveillance cameras at the current time are configured. The weight parameter is the product of the static weight parameter and the dynamic weight parameter.

[0017] Preferably, in the algorithm configuration of dynamic weight parameters, the quality indicators of the captured image include brightness, sharpness, and noise level.

[0018] By employing the above technical solution, static and dynamic weight parameters are configured for the overlapping field-of-view regions. The static weight parameters prioritize image regions that are closer to the FOD monitoring camera, have higher resolution, and less distortion, providing more reliable feature recognition for small FODs. The dynamic weight parameters prioritize image regions with higher quality, thus suppressing the influence of environmental interference factors. The synergy of these two parameters improves the accuracy and reliability of FOD recognition in overlapping regions.

[0019] As a preferred option, the FOD identification model in S400 employs a Feature Map Pyramid Network (FPN). Specific methods for FOD identification in S400 include: The system acquires images captured by FOD surveillance cameras in real time. For the near-field field of view, the images are input into the backbone network of the Feature Map Pyramid Network (FPN) for feature extraction to obtain feature maps. The feature maps are then input into the detection head to output the target recognition results of FOD. For regions within the far-field region of the overlapping field of view, the images captured by the two FOD surveillance cameras are multiplied by weight parameters to obtain a weighted image. These images are then input into the backbone of the Feature Map Pyramid Network (FPN) for feature extraction to obtain two feature maps. The two feature maps are then fused to obtain a weighted feature map. Finally, the weighted feature map is input into the detection head to output the target recognition result of FOD.

[0020] By adopting the above technical solution, a standard FPN network is used for the near-field field of view obtained from a single FOD monitoring camera. Each FOD monitoring camera independently processes FOD identification, resulting in high computational efficiency, saving computing resources, and avoiding unnecessary noise introduced by complex fusion. For the far-field overlapping field of view obtained from two FOD monitoring cameras, an enhanced FPN network based on weighted fusion branches is used. This allows for complementary fusion of information from the two FOD monitoring cameras, enabling the superposition and enhancement of weak features in the far-field field of view while suppressing noise and uncertainty. Differentiated FPN network architectures are used to distinguish different field of view regions, balancing detection accuracy and computational efficiency, and employing the most suitable feature recognition processing method for different regions.

[0021] Secondly, this application provides an airport runway FOD identification system based on runway bidirectional feature fusion to implement the above-mentioned identification method, and adopts the following technical solution: Includes a camera module, a FOD recognition module, and a FOD warning output module; The camera module includes a FOD surveillance camera and a camera control module; The FOD surveillance cameras are arranged in pairs, corresponding one to one and opposite each other on both sides of the airport runway, and several groups of FOD surveillance cameras are deployed at equal intervals along the length of the airport runway. The camera control module calibrates the near-field field of view, far-field field of view, and overlapping field of view of the FOD monitoring camera. The FOD recognition module acquires images captured by the FOD monitoring camera in real time, and performs FOD recognition on the near-field field of view area in the captured image and the far-field field of view area in the overlapping field of view area in the captured image. The FOD warning output module outputs a FOD warning signal based on the FOD identification result.

[0022] Preferably, each of the FOD surveillance cameras consists of a visible light camera and an infrared camera, with the visible light camera providing visible light images and the infrared camera providing infrared images, respectively.

[0023] Thirdly, a computer program product of this application adopts a technical solution including a computer program or instructions, which enables the computer program or instructions to perform the steps in the above-mentioned airport runway FOD identification method based on runway bidirectional feature fusion.

[0024] The airport runway FOD identification method, system, and product based on runway bidirectional feature fusion provided by this invention solves the problems of low detection accuracy, poor environmental adaptability, and reliance on complex mechanical adjustments in current technologies for small-sized FOD, especially far-field micro-targets, in wide runway environments by innovatively adopting a dual-camera collaborative detection architecture and an adaptive feature fusion mechanism. The technical effects and advantages achieved by this invention are specifically reflected in the following three aspects: 1. This application effectively distributes the detection task across the runway width by deploying two opposing FOD monitoring cameras to jointly cover the runway area, thus reducing the field of view required by a single camera. Simultaneously, this application divides the detection area into a near-field non-overlapping zone and a far-field overlapping zone. In the near-field zone, the image resolution of a single FOD monitoring camera is sufficiently high; therefore, an independent feature extraction and recognition strategy is employed to achieve rapid and accurate identification of near-field FOD. In the far-field overlapping zone, a bidirectional feature fusion mechanism is activated, weighted and fused to extract feature maps from the differentiated images captured by the two FOD monitoring cameras. This achieves information complementarity, enabling high-precision FOD identification across the entire runway width, significantly enhancing reliability.

[0025] 2. This application can achieve full coverage of the monitoring range in the width direction of the runway without relying on camera head turning for scanning or dynamic zooming. It eliminates the need for a sophisticated zoom control system and dynamic image algorithm scheme. Furthermore, it does not require waiting for mechanical parts to respond and position, which improves the response speed of monitoring and recognition, avoids the inherent failure risks such as wear and jamming of mechanical moving parts, and simplifies the calibration and maintenance process of the system.

[0026] 3. This application utilizes the different perspective information provided by dual cameras to construct an adaptive compensation mechanism. For example, when the image quality (such as brightness and sharpness) of one camera deteriorates due to strong light, momentary occlusion, or lens damage, the system automatically reduces the contribution of that camera's features in the fusion process through dynamic weight calculation, while simultaneously increasing the weight of features from the other camera with better image quality. This dynamic weight allocation strategy enables the system to effectively suppress interference caused by fluctuations in the image quality of a single camera, thereby maintaining stable and reliable detection performance under various complex and changing lighting and weather conditions. Compared to systems relying on a single sensor or simple fusion strategies, this invention significantly enhances robustness to environmental interference. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating an airport runway FOD identification method based on runway bidirectional feature fusion according to an embodiment of this application. Figure 2 This is a flowchart of step S200 in an airport runway FOD identification method based on runway bidirectional feature fusion according to an embodiment of this application. Figure 3 This is a schematic diagram illustrating the calibrated field of view area of ​​the FOD surveillance camera according to an embodiment of this application; Figure 4 This is a schematic diagram of the architecture of an airport runway FOD identification system based on runway bidirectional feature fusion according to an embodiment of this application; Figure 5 This is a schematic diagram of the architecture of an exemplary computer device according to an embodiment of this application. Detailed Implementation

[0028] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of this application.

[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that in the optional embodiments of this application, the object information and other related data involved require the permission or consent of the object when the embodiments of this application are applied to specific products or technologies, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. That is to say, if the embodiments of this application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations, and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required, and the embodiments also need to be implemented with the authorization and consent of the object.

[0030] For safety reasons, the installation height of FOD (Fear of Displacement) surveillance cameras installed along airport runways is usually strictly limited. For example, according to civil aviation technical requirements, the total height of side-light cameras should not exceed 430 millimeters. For large airport runways that may be more than 50 meters wide, FOD surveillance cameras deployed on one side of the runway at this height often struggle to clearly capture small FODs on the opposite side. Small FODs occupy a very small pixel area in the captured image, resulting in significant loss of detail and making them difficult for algorithms to effectively identify. Furthermore, the low installation angle means the camera's field of view is close to the runway surface. The runway's raised structure, the aircraft's projection on the runway, and even the shadows cast by the FODs themselves can all obscure small FODs behind them, creating blind spots in detection.

[0031] To achieve continuous, stable, and comprehensive FOD (Fear of Disasters) identification and monitoring of airport runways with a certain width, embodiments of this application provide an airport runway FOD identification method based on runway bidirectional feature fusion. Please refer to [link to relevant documentation]. Figure 1 Specifically, it includes the following steps.

[0032] S100 deploys several groups of FOD surveillance cameras at equal intervals along the length of the airport runway.

[0033] A FOD (Focus on Discharge) surveillance camera group consists of two FOD cameras positioned opposite each other on either side of the airport runway. The monitoring area of ​​one FOD camera group covers the entire width of the airport runway. The monitoring areas of adjacent FOD camera groups may overlap to a certain extent along the length of the airport runway to ensure that the monitoring areas of all FOD camera groups cover the entire length of the airport runway. This overlap does not affect image acquisition and FOD identification monitoring based on the monitoring areas of each individual FOD camera group.

[0034] S200 calibrates the field of view (FAV) of the FOD surveillance cameras, including the near-field, far-field, and overlapping FAV regions of two FOD surveillance cameras in the same group. The near-field FAV region refers to the furthest distance the FOD surveillance camera can capture while meeting near-field recognition requirements; the far-field FAV region refers to the furthest distance the FOD surveillance camera can capture while meeting far-field recognition requirements; and the overlapping FAV region refers to the area captured by two FOD surveillance cameras in the same group that overlaps with each other.

[0035] For details, please refer to Figure 2 The specific steps for calibrating the field of view area of ​​a FOD surveillance camera include the following.

[0036] S201, perform horizontal calibration on two FOD surveillance cameras in the same group, ensuring that their optical axes are perpendicular to the centerline of the airport runway, and that their horizontal projections lie on the same straight line. This horizontal calibration ensures that the shooting directions of the two FOD surveillance cameras on either side of the runway are strictly aligned and perpendicular to the runway direction, allowing for matching and correspondence of the images captured by the two FOD surveillance cameras in both the width and depth directions during subsequent calibration of the overlapping field of view.

[0037] S202, calibrate the near field of view depth d1 and far field of view depth d2 of the FOD monitoring camera. The area of ​​the field of view depth of the FOD monitoring camera that is less than the near field of view depth d1 is the near field of view area, and the area of ​​the calibrated field of view depth that is greater than the near field of view depth d1 and less than the far field of view depth d2 is the far field of view area.

[0038] The calibration methods for near-field depth of view d1 and far-field depth of view d2 are as follows.

[0039] The FOD monitoring camera is used to photograph a calibration reference object of a certain size. The depth distance between the calibration reference object and the FOD monitoring camera is adjusted. When the pixel area of ​​the calibration reference object in the field of view reaches the near field depth threshold, the depth distance is calibrated as the near field depth d1. When the pixel area of ​​the calibration reference object in the field of view reaches the far field depth threshold, the depth distance is calibrated as the far field depth d2.

[0040] It's important to note that the pixel area of ​​a target object in a captured image is a core quantitative metric for evaluating FOD (Foreign Object Defect) recognition capabilities. Deep learning models (such as YOLO and Faster R-CNN) perform target recognition by analyzing features like edges, textures, and colors. A larger pixel area means more pixels to depict these details, allowing the model to learn more unique and stable feature patterns. Conversely, a target composed of only a few pixels contains very limited feature information and is almost indistinguishable from image noise.

[0041] For the same target object, the pixel area in the captured image gradually decreases as the distance from the camera lens increases. In this application, the pixel area of ​​a known-sized reference object at different depths of view in the captured image is used as the basis for determining the near-field and far-field fields of view. The actual shooting effect is related to the camera's resolution and focal length. In the embodiments of this application, taking a FOD surveillance camera with a resolution of 1080P and a focal length of 18mm as an example, when the near-field recognition requirement is that a 2cm*2cm square object has a pixel area of ​​at least 10*10 pixels in the captured image, the measured near-field depth of view d1 is approximately 15m. When the far-field recognition requirement is that a 2cm*2cm square object has a pixel area of ​​at least 5*5 pixels in the captured image, the measured far-field depth of view is approximately 30m.

[0042] Furthermore, it is necessary to verify the constraints on the near-field and far-field depth of view of the FOD surveillance camera, specifically: ; Where h is the width of the airport runway, and d1' is the near-field field of view depth of the FOD surveillance camera on the opposite side of the same group to be verified.

[0043] The constraints ensure that the far-field field of view of the FOD monitoring cameras on both sides can cover the width of the entire airport runway, avoiding blind spots or the inability of FOD recognition accuracy in the middle of the airport runway to meet the far-field recognition requirements. The constraints ensure that the far-field field of view of any FOD monitoring camera on one side can reach the near-field field of view of the FOD monitoring camera on the other side, ensuring that the overlapping field of view can cover the relatively blurry far-field field of view. The far-field field of view is provided by the images captured by the FOD monitoring cameras on both sides for feature fusion for FOD identification and monitoring. Typically, the FOD monitoring cameras on both sides are symmetrically deployed, then d1'=d1.

[0044] Following the above embodiments, taking an airport runway width of h=40m as an example for verification, it can be seen that the calculated results of d1 and d2 satisfy the constraints. However, if the airport runway width is wider, for example, h reaches 50m, the constraints are not satisfied. It is necessary to adjust the focal length and resolution performance of the FOD monitoring camera, or optimize the FOD recognition algorithm to relax the conditions for the near-field depth threshold and far-field depth threshold of the pixel area.

[0045] S203 calibrates the overlapping field of view area of ​​two FOD surveillance cameras in the same group based on the field of view depth.

[0046] Following the above embodiments, please refer to Figure 3 For a 40m wide airport runway, the near-field field of view (d1) of one FOD (Field of View) camera is 15m, and the far-field field of view (d2) is 30m; the near-field field of view (d1') of the other FOD camera is 15m, and the far-field field of view (d2') is 30m. The overlapping area is a 20m wide region extending from the center of the airport runway to both sides. Since some of the overlapping area enters the near-field shooting area of ​​the FOD camera, it needs to be excluded. Therefore, the depth d of the far-field region within the overlapping field of view in the captured image is a 10m wide region extending from the center of the airport runway to both sides.

[0047] It should be noted that during the factory initialization of the FOD surveillance camera, the relationship between the vertical axis coordinates of the ground-captured image and the field of view depth can be calibrated using methods such as a grid marking board. After calibrating the near-field field of view depth d1 and far-field field of view depth d2 of the FOD surveillance camera, the captured image of the FOD surveillance camera can be divided into near-field field of view area, far-field field of view area, and overlapping field of view area in the vertical axis direction.

[0048] It should also be noted that, for ease of explanation, the above embodiments assume that the FOD monitoring camera is installed at the edge of the airport runway, and the depth of field is calculated from the edge of the airport runway. In actual implementations, there is a reserved distance between the FOD monitoring camera and the edge of the airport runway, so the calibration of the near-field field of view, far-field field of view, and overlapping field of view needs to be adjusted accordingly.

[0049] S300 configures weight parameters for overlapping view regions.

[0050] Specifically, configuring weight parameters for overlapping visual regions includes algorithms for configuring static weight parameters and dynamic weight parameters.

[0051] The static weight parameters are configured based on the field of view depth of the FOD surveillance camera. Field of view areas with the same field of view depth have the same static weight parameters, and the static weight parameters decrease as the field of view depth increases.

[0052] The closer an object is to the camera, the larger the image appears on the image sensor, the more pixels it occupies, and therefore the richer the details and the higher the signal-to-noise ratio. Conversely, the same object appears smaller in a distant area, details are easily lost, and it is more susceptible to environmental disturbances, lens distortion, and other factors.

[0053] In the embodiments of this application, the static weight parameters based on field of view depth actually adopt a weight allocation strategy based on prior knowledge of physical space. This results in higher weights for closer and clearer images from one FOD surveillance camera and lower weights for farther and blurrier images from the other FOD surveillance camera. By assigning higher weights to nearby areas, the fusion algorithm is guided to make fuller use of higher quality and more reliable features, thereby improving the feature quality of the feature fusion process from the source. In the embodiments of this application, in the vertical axis direction of the captured image, a static weight parameter λ1 is configured for each row of pixels according to the field of view depth. The static weight parameter for the pixels in the same row of another FOD surveillance camera at the same location on the runway is λ1', where λ1 + λ1' = 1.

[0054] Dynamic weighting parameters are configured based on the overall image quality captured by the FOD surveillance cameras. Due to the complex and variable environment of airport runways, the lighting and environmental conditions faced by the cameras on both sides are not the same. For example, one camera may be backlit while the other side has good lighting; or one lens may be temporarily contaminated by water droplets while the other remains clean. The dynamic weighting parameters aim to respond in real-time to changes in the image quality captured by the FOD surveillance cameras, automatically reducing the weight of the side with poorer image quality and increasing the contribution of the side with higher quality, thereby ensuring the overall reliability of the fusion result and effectively suppressing interference caused by fluctuations in the image quality of a single camera.

[0055] In the embodiments of this application, a dynamic confidence score algorithm is constructed based on the quality index of the captured images. The dynamic confidence score of two FOD monitoring cameras in the same group at the same time is calculated through the captured images. Based on the dynamic confidence score, dynamic weight parameters for the two FOD monitoring cameras at the current time are configured.

[0056] Specifically, the quality indicators of the images captured in this application include brightness, sharpness, and noise level.

[0057] The brightness index is used to evaluate the overall exposure level of an image and measure the information loss caused by overexposure or underexposure. It is obtained by converting the captured image to grayscale, calculating the average brightness of the grayscale image, and then normalizing the average brightness based on its proximity to a preset ideal brightness to determine the brightness index S1.

[0058] The sharpness index is used to evaluate the sharpness of image edges and textures, measuring the richness of detail in object edges and textures. The sharpness index S2 is obtained by calculating the variance of the Laplace operator of the grayscale image and normalizing it.

[0059] The noise level metric is used to evaluate random noise in an image and measure the purity of the signal. It involves selecting a flat, textureless region of interest (ROI) in the image (typically a smooth runway surface to ensure no field of view, FOD-free), applying a slight Gaussian blur (e.g., 3x3) to eliminate detail interference, and then calculating and normalizing the standard deviation of the grayscale values ​​in that region to obtain the noise level (Noise). A maximum acceptable noise level (Tmax) is then set. Noise level index .

[0060] By weighted summing the three normalized basic quality indicators mentioned above, we obtain the dynamic confidence score K: K = w1*S1 + w2*S2 + w3*S3 Among them, w1, w2, and w3 are the weight parameters of each basic quality indicator, and w1+w2+w3=1. The weight allocation can be adjusted according to the actual scenario.

[0061] For a single FOD surveillance camera and another FOD surveillance camera in the same group, dynamic confidence scores K and K' are calculated for two images captured by them at the same time, respectively. The dynamic weight parameters of the FOD surveillance cameras are also included. .

[0062] The weight parameter λ = λ1 * λ2.

[0063] The S400 acquires images captured by FOD surveillance cameras in real time and inputs these images into the FOD recognition model.

[0064] Specifically, in this embodiment, the FOD recognition model employs a Feature Map Pyramid Network (FPN). Through its bottom-up, top-down, and laterally connected structure, the FPN naturally constructs a multi-scale feature pyramid. Shallow feature maps have high resolution and are responsible for detecting small targets; deep feature maps are rich in semantic information and are responsible for detecting large targets. This means that a single model can efficiently handle foreign objects of different sizes across the entire runway. The feature extraction and recognition process of the FPN is existing technology and will not be elaborated upon here.

[0065] More specifically, specific methods for FOD identification using Feature Map Pyramid Network (FPN) include: The system acquires images from FOD surveillance cameras in real time and extracts the near-field and far-field regions within the overlapping fields of view. To ensure that the images from the two FOD surveillance cameras correspond to each other in the overlapping region, one side of the image needs to be flipped and mirrored.

[0066] For the near-field region, the feature map is extracted by inputting it into the backbone network of the Feature Map Pyramid Network (FPN). The feature map is then input into the detection head to output the target recognition result of FOD.

[0067] For regions within the far-field area of ​​the overlapping field of view, the images captured by the two FOD surveillance cameras are multiplied by their respective weight parameters λ to obtain weighted images. These weighted images are then input into the backbone of the Feature Map Pyramid Network (FPN) for feature extraction, resulting in two feature maps. The two feature maps are then fused to obtain a weighted feature map, which is then input into the detection head to output the FOD target recognition result. Specific methods for feature fusion include element-wise feature addition or feature fusion guided by channel attention mechanisms.

[0068] The S500 outputs a FOD alarm signal based on the FOD identification result.

[0069] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0070] Please see Figure 4 An airport runway FOD identification system based on runway bidirectional feature fusion according to an embodiment of this application is used to implement the above-mentioned identification method, including a camera module 1, a FOD identification module 2 and a FOD early warning output module 3.

[0071] The camera module 1 includes a FOD monitoring camera 11 and a camera control module 12.

[0072] The FOD monitoring cameras are arranged in pairs, one-to-one and opposite each other on both sides of the airport runway. Several groups of FOD monitoring cameras are deployed at equal intervals along the length of the airport runway. More specifically, each FOD monitoring camera consists of a visible light camera and an infrared camera. The visible light camera and the infrared camera provide visible light images and infrared light images respectively, enabling the system to cope with various complex lighting and weather conditions, and achieve all-weather, highly reliable FOD monitoring.

[0073] The camera control module 12 calibrates the near-field field of view, far-field field of view, and overlapping field of view of the FOD monitoring camera 11.

[0074] The FOD recognition module 2 acquires images captured by the FOD monitoring camera 11 in real time, and performs FOD recognition on the near-field field of view area in the captured image and the far-field field of view area in the overlapping field of view area in the captured image.

[0075] The FOD warning output module 3 outputs a FOD warning signal based on the FOD identification results.

[0076] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the airport runway FOD identification system based on runway bidirectional feature fusion described above can be referred to the corresponding process in the aforementioned method embodiments, and will not be repeated here.

[0077] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores weight parameter configuration data for the FOD monitoring camera group and stores images captured by the FOD monitoring cameras and FOD identification signals. The network interface is used to connect to external FOD monitoring cameras. When the computer program is executed by the processor, it implements an airport runway FOD identification method based on runway bidirectional feature fusion.

[0078] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0079] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An airport runway FOD identification method based on runway bidirectional feature fusion, characterized in that, Includes the following steps: S100, several groups of FOD monitoring cameras are deployed at equal intervals along the length of the airport runway. Each group of FOD monitoring cameras consists of two FOD monitoring cameras that are one-to-one and set opposite each other on both sides of the airport runway. S200 calibrates the field of view of FOD surveillance cameras, including calibrating the near-field field of view, far-field field of view, and overlapping field of view of two FOD surveillance cameras in the same group. S300 configures weight parameters for overlapping view areas; The S400 acquires images captured by FOD surveillance cameras in real time, directly inputs the near-field field of view in the captured images into the FOD recognition model, outputs a feature map for FOD recognition, and inputs the regions belonging to the far-field field of view in the overlapping field of view in the captured images into the FOD recognition model based on weight parameters, performs feature fusion, and outputs a weighted feature map for FOD recognition. The S500 outputs a FOD alarm signal based on the FOD identification result.

2. The airport runway FOD identification method based on runway bidirectional feature fusion according to claim 1, characterized in that, In S200, the specific steps for calibrating the field of view area of ​​the FOD surveillance camera include the following: S201, perform horizontal calibration on two FOD surveillance cameras in the same group so that their optical axes are perpendicular to the centerline of the airport runway and their horizontal projections are on the same straight line. S202, calibrate the near field of view depth d1 and far field of view depth d2 of the FOD monitoring camera. The area of ​​the field of view depth of the FOD monitoring camera that is less than the near field of view depth d1 is the near field of view area, and the area of ​​the calibrated field of view depth that is greater than the near field of view depth d1 and less than the far field of view depth d2 is the far field of view area. S203 calibrates the overlapping field of view area of ​​two FOD surveillance cameras in the same group based on the field of view depth.

3. The airport runway FOD identification method based on runway bidirectional feature fusion according to claim 2, characterized in that, In S202, the calibration method for the near-field field depth d1 and the far-field field depth d2 is as follows: The FOD monitoring camera is used to photograph a calibration reference object of a certain size. The depth distance between the calibration reference object and the FOD monitoring camera is adjusted. When the pixel area of ​​the calibration reference object in the field of view reaches the near field depth threshold, the depth distance is calibrated as the near field depth d1. When the pixel area of ​​the calibration reference object in the field of view reaches the far field depth threshold, the depth distance is calibrated as the far field depth d2.

4. The airport runway FOD identification method based on runway bidirectional feature fusion according to claim 3, characterized in that, S202 also includes constraint verification for the near-field and far-field depth of view of the FOD surveillance camera, specifically: ； Where h is the width of the airport runway, and d1' is the near-field field of view depth of the FOD surveillance camera on the opposite side of the same group to be verified.

5. The airport runway FOD identification method based on runway bidirectional feature fusion according to claim 1, characterized in that, In S300, the weight parameter configuration for overlapping visual regions specifically includes algorithms for configuring static weight parameters and configuring dynamic weight parameters; The static weight parameters are configured based on the field of view depth of the FOD surveillance camera. Field of view areas with the same field of view depth have the same static weight parameters, and the static weight parameters decrease as the field of view depth increases. The dynamic weight parameters are configured based on the global quality of the images captured by the FOD surveillance cameras. A dynamic confidence score algorithm is constructed based on the quality index of the captured images. The dynamic confidence score of the two FOD surveillance cameras in the same group at the same time is calculated through the captured images. Based on the dynamic confidence score, the dynamic weight parameters of the two FOD surveillance cameras at the current time are configured. The weight parameter is the product of the static weight parameter and the dynamic weight parameter.

6. The airport runway FOD identification method based on runway bidirectional feature fusion according to claim 5, characterized in that, In the algorithm configuration of dynamic weight parameters, the quality indicators of the captured image include brightness, sharpness, and noise level.

7. The airport runway FOD identification method based on runway bidirectional feature fusion according to claim 1, characterized in that, In S400, the FOD identification model uses a Feature Map Pyramid Network (FPN). Specific methods for FOD identification in S400 include: The system acquires images captured by FOD surveillance cameras in real time. For the near-field field of view, the images are input into the backbone network of the Feature Map Pyramid Network (FPN) for feature extraction to obtain feature maps. The feature maps are then input into the detection head to output the target recognition results of FOD. For regions within the far-field region of the overlapping field of view, the images captured by the two FOD surveillance cameras are multiplied by weight parameters to obtain a weighted image. These images are then input into the backbone of the Feature Map Pyramid Network (FPN) for feature extraction to obtain two feature maps. The two feature maps are then fused to obtain a weighted feature map. Finally, the weighted feature map is input into the detection head to output the target recognition result of FOD.

8. An airport runway FOD identification system based on runway bidirectional feature fusion, used to implement the identification method according to any one of claims 1 to 7, characterized in that, Includes a camera module, a FOD recognition module, and a FOD warning output module; The camera module includes a FOD surveillance camera and a camera control module; The FOD surveillance cameras are arranged in pairs, corresponding one to one and opposite each other on both sides of the airport runway, and several groups of FOD surveillance cameras are deployed at equal intervals along the length of the airport runway. The camera control module calibrates the near-field field of view, far-field field of view, and overlapping field of view of the FOD monitoring camera. The FOD recognition module acquires images captured by the FOD monitoring camera in real time, and performs FOD recognition on the near-field field of view area in the captured image and the far-field field of view area in the overlapping field of view area in the captured image. The FOD warning output module outputs a FOD warning signal based on the FOD identification result.

9. An airport runway FOD identification system based on runway bidirectional feature fusion according to claim 8, characterized in that, Each of the FOD surveillance cameras consists of a visible light camera and an infrared camera, which respectively provide visible light images and infrared images.

10. A computer program product, characterized in that, The computer program product includes a computer program or instructions that enable the computer program or instructions to perform the steps in the airport runway FOD identification method based on runway bidirectional feature fusion as described in any one of claims 1 to 7.

Images