An airport runway foreign object target detection method and device

Abstract

The application discloses an airport runway foreign matter target detection method and device, and the method comprises the following steps: acquiring an airport runway image set; the airport runway image set comprises N airport runway images with different visual angles, and N is a positive integer; fusing the N images in the airport runway image set to obtain a fused image; and performing target detection on the fused image to obtain an airport runway foreign matter target detection result. The application analyzes and fuses the characteristics of images with different visual angles, makes up for the shortcomings of a single image, improves the utilization rate of image information, and obtains more abundant and accurate information; the fused image is processed to obtain an initial candidate box; the initial candidate box is subjected to feature embedding, coding and classification detection to obtain the airport runway foreign matter target detection result.

Description

Technical Field

[0001] This invention relates to the field of image processing and application technology, and in particular to a method and apparatus for detecting foreign objects on airport runways. Background Technology

[0002] Foreign Object Debris (FOD) on airport runways mainly includes metal parts, machine tools, etc. Stone fragments, plastics, and other debris can easily puncture aircraft tires or be forcefully sucked into aircraft engines, causing damage to critical components and resulting in major safety accidents with unimaginable threats to life and property loss.

[0003] Foreign object (FOD) detection on airport runways has always been a critical issue in the aviation industry and a vital part of maintaining safe airport operations. Most airports rely on manual inspections to detect FOD on the runways. However, this method is limited by factors such as staff subjectivity, high time costs, and variable environments, making it impossible to strictly guarantee safe airport operations and leaving significant safety hazards. To ensure safe aircraft takeoffs and landings, traditional manual FOD inspections are no longer sufficient to meet practical needs, necessitating efficient and accurate intelligent detection. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and apparatus for detecting foreign objects on airport runways, which analyzes and fuses the characteristics of images from various perspectives to make up for the deficiencies of a single image, improve the utilization rate of image information, and obtain richer and more accurate information; processes the fused image to obtain initial candidate boxes; and performs feature embedding, encoding, and classification detection on the initial candidate boxes to obtain the detection results of foreign objects on airport runways.

[0005] To address the aforementioned technical problems, a first aspect of the present invention discloses a method for detecting foreign objects on airport runways, the method comprising: S1, Obtain the set of airport runway images; the set of airport runway images includes N airport runway images from different perspectives, where N is a positive integer; S2, fuse N images in the airport runway image set to obtain a fused image; S3, perform target detection on the fused image to obtain the foreign object target detection result for the airport runway.

[0006] As an optional implementation, in the first aspect of the present invention, fusing the N airport runway images from different perspectives to obtain a fused image includes: S21, preprocess the airport runway image set to obtain a preprocessed image set; S22, decompose the preprocessed image set to obtain a first sub-band and a second sub-band; S23, fuse the first sub-band to obtain the first fused image; S24, fuse the second sub-band to obtain the second fused image; S25, transform the first fused image and the second fused image to obtain a fused image.

[0007] As an optional implementation, in the first aspect of the present invention, the preprocessing of the airport runway image set to obtain a preprocessed image set includes: S211, filter any image in the airport runway image set to obtain a denoised image; S212, Amplitude compensation is performed on the denoised image to obtain a compensated image; S213, perform position correction on the compensated image to obtain a preprocessed image set.

[0008] As an optional implementation, in the first aspect of the present invention, the step of decomposing the preprocessed image set to obtain a first sub-band and a second sub-band includes: S221, Preset scale factor M ; S222, order Decompose any image in the preprocessed image set to obtain a high-frequency image. and low-frequency images ; S223, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S224, for the low-frequency image Decomposition is performed to obtain high-frequency images. and low-frequency images ; S225, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S226, Execute S224~S225 until... ; S227, for Information from each direction is integrated to obtain the first information, the low-frequency image. This is the second piece of information; S228, integrate the first information of each image in the preprocessed image set to obtain the first sub-band; S229, integrate the second information of each image in the preprocessed image set to obtain the second sub-band.

[0009] As an optional implementation, in a first aspect of the present invention, fusing the first sub-band to obtain a first fused image includes: The first sub-band is fused using the first sub-band fusion model to obtain the first fused image; The expression for the first sub-band fusion model is: in, The first fused image contains the first sub-band. One element, The first sub-band One element, for exist Pixel value at point for In the neighborhood The average pixel value within the range, and For the preset value, such as =3, =3, For the first The weight of each element.

[0010] As an optional implementation, in a first aspect of the present invention, fusing the second sub-band to obtain a second fused image includes: S241, Process any image in the second sub-band to obtain the observed image; S242, the observed images are fused to obtain a second fused image.

[0011] As an optional implementation, in the first aspect of the present invention, the step of performing target detection on the fused image to obtain the foreign object detection result for the airport runway includes: S31, Process the fused image to obtain initial candidate boxes; S32, Complete the initial candidate boxes to obtain optimized candidate boxes; S33, perform feature embedding on the optimized candidate box to obtain embedded feature information; S34, the embedded feature information is encoded to obtain optimized feature information; S35, classify and detect the optimized feature information to obtain the detection result of foreign object targets on the airport runway.

[0012] A second aspect of this invention discloses a foreign object detection device for airport runways, the device comprising: The data acquisition module is used to acquire a set of airport runway images; the set of airport runway images includes N airport runway images from different perspectives, where N is a positive integer; The image fusion module is used to fuse N images from the airport runway image set to obtain a fused image; The target detection module is used to perform target detection on the fused image to obtain the foreign object target detection result of the airport runway.

[0013] As an optional implementation, in the second aspect of the present invention, the step of fusing the N airport runway images from different perspectives to obtain a fused image includes: S21, preprocess the airport runway image set to obtain a preprocessed image set; S22, decompose the preprocessed image set to obtain a first sub-band and a second sub-band; S23, fuse the first sub-band to obtain the first fused image; S24, fuse the second sub-band to obtain the second fused image; S25, transform the first fused image and the second fused image to obtain a fused image.

[0014] As an optional implementation, in the second aspect of the present invention, the preprocessing of the airport runway image set to obtain a preprocessed image set includes: S211, filter any image in the airport runway image set to obtain a denoised image; S212, Amplitude compensation is performed on the denoised image to obtain a compensated image; S213, perform position correction on the compensated image to obtain a preprocessed image set.

[0015] As an optional implementation, in a second aspect of the present invention, the step of decomposing the preprocessed image set to obtain a first sub-band and a second sub-band includes: S221, Preset scale factor ; S222, order Decompose any image in the preprocessed image set to obtain a high-frequency image. and low-frequency images ; S223, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S224, for the low-frequency image Decomposition is performed to obtain high-frequency images. and low-frequency images ; S225, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S226, Execute S224~S225 until... ; S227, for Information from each direction is integrated to obtain the first information, the low-frequency image. This is the second piece of information; S228, integrate the first information of each image in the preprocessed image set to obtain the first sub-band; S229, integrate the second information of each image in the preprocessed image set to obtain the second sub-band.

[0016] As an optional implementation, in a second aspect of the present invention, fusing the first sub-band to obtain a first fused image includes: The first sub-band is fused using the first sub-band fusion model to obtain the first fused image; The expression for the first sub-band fusion model is: in, The first fused image contains the first sub-band. One element, The first sub-band One element, for exist Pixel value at point for In the neighborhood The average pixel value within the range, and For the preset value, such as =3, =3, For the first The weight of each element.

[0017] As an optional implementation, in a second aspect of the present invention, the fusion of the second sub-band to obtain a second fused image includes: S241, Process any image in the second sub-band to obtain the observed image; S242, the observed images are fused to obtain a second fused image.

[0018] As an optional implementation, in a second aspect of the present invention, the step of performing target detection on the fused image to obtain the foreign object detection result for the airport runway includes: S31, Process the fused image to obtain initial candidate boxes; S32, Complete the initial candidate boxes to obtain optimized candidate boxes; S33, perform feature embedding on the optimized candidate box to obtain embedded feature information; S34, the embedded feature information is encoded to obtain optimized feature information; S35, classify and detect the optimized feature information to obtain the detection result of foreign object targets on the airport runway.

[0019] A third aspect of the present invention discloses another foreign object detection device for airport runways, the device comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute some or all of the steps in the airport runway foreign object detection method disclosed in the first aspect of the present invention.

[0020] The fourth aspect of the present invention discloses a computer-storable medium storing computer instructions, which, when invoked, are used to execute some or all of the steps in the airport runway foreign object detection method disclosed in the first aspect of the present invention.

[0021] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: This invention analyzes and fuses the characteristics of images from various perspectives to compensate for the shortcomings of a single image, improve the utilization rate of image information, and obtain richer and more accurate information; it processes the fused image to obtain initial candidate boxes; it then performs feature embedding, encoding, and classification detection on the initial candidate boxes to obtain the foreign object detection results for airport runways; the method of this invention improves the accuracy of foreign object detection on airport runways. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a method for detecting foreign objects on an airport runway, as disclosed in an embodiment of the present invention. Figure 2 This is a schematic diagram of the detection model structure disclosed in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an airport runway foreign object detection device disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram of another airport runway foreign object detection device disclosed in an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0026] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0027] This invention discloses a method and apparatus for detecting foreign objects on airport runways. The method includes: acquiring a set of airport runway images; the set of airport runway images encompasses N airport runway images from different perspectives, where N is a positive integer; fusing the N images in the set of airport runway images to obtain a fused image; and performing target detection on the fused image to obtain a foreign object detection result for the airport runway. This invention analyzes and fuses the characteristics of images from different perspectives, compensating for the deficiencies of single images, improving the utilization rate of image information, and obtaining richer and more accurate information; processing the fused image to obtain initial candidate boxes; and performing feature embedding, encoding, and classification detection on the initial candidate boxes to obtain the foreign object detection result for the airport runway. These are described in detail below.

[0028] Example 1 Please see Figure 1 , Figure 1 This is a schematic flowchart of a foreign object detection method for airport runways disclosed in an embodiment of the present invention. Figure 1 The described method for detecting foreign objects on airport runways is applied in the field of image processing and application technology, and the embodiments of this invention are not limited thereto. Figure 1 As shown, the foreign object detection method for airport runways may include the following operations: S1, Obtain the set of airport runway images; the set of airport runway images includes N airport runway images from different perspectives, where N is a positive integer; Specifically, it refers to N images for a unified scene; S2, fuse N images in the airport runway image set to obtain a fused image; S3, perform target detection on the fused image to obtain the foreign object target detection result for the airport runway.

[0029] Optionally, fusing the N airport runway images from different perspectives to obtain a fused image includes: S21, preprocess the airport runway image set to obtain a preprocessed image set; S22, decompose the preprocessed image set to obtain a first sub-band and a second sub-band; S23, fuse the first sub-band to obtain the first fused image; S24, fuse the second sub-band to obtain the second fused image; S25, transform the first fused image and the second fused image to obtain a fused image.

[0030] Optionally, the preprocessing of the airport runway image set to obtain a preprocessed image set includes: S211, filter any image in the airport runway image set to obtain a denoised image; Perform noise statistical analysis on the input image to calculate the noise variance (Gaussian noise) and the proportion of noise pixels (salt-and-pepper noise). If the proportion of salt-and-pepper noise is ≤5%, median filtering is preferred; if the variance of Gaussian noise is ≥0.01, Gaussian filtering is superimposed; if both types of noise coexist, a combination of "median filtering + Gaussian filtering" is used.

[0031] S212, Amplitude compensation is performed on the denoised image to obtain a compensated image; 1. Runway region segmentation: Based on the grayscale characteristics of the runway (the grayscale value of the runway body is usually between 120-200, the shoulder is slightly lower, and the background difference is large), threshold segmentation is adopted (the initial threshold is the average grayscale value of the image + 30) to initially segment the suspected runway region and avoid the background region from affecting the calculation of compensation parameters.

[0032] 2. Gray-scale statistics (calculation of compensation benchmark): Calculate the mean gray-scale value within the segmented suspected runway area. and standard deviation Set the target grayscale range as (Ensure uniform grayscale across the runway area after compensation), target mean =160 (Standard grayscale of the runway under normal lighting conditions, which can be adjusted according to actual application).

[0033] 3. Partition adaptive compensation: (1) Runway body area: A combination of grayscale stretching and gamma correction is used. First, the grayscale range is mapped to the target area through linear stretching. Then adjust the gamma value according to the brightness / darkness of the area (when it is brighter). =1.2-1.5, when it's darker =0.6-0.9), correcting nonlinear grayscale deviation.

[0034] (2) Shoulder and edge areas: Mean matching compensation is used. The mean gray value of the shoulder area is calculated and adjusted to the mean value of the runway body target. 0.8 times (maintaining the grayscale difference between the runway and the shoulder to facilitate subsequent edge recognition).

[0035] (3) Background area: Slight compensation, only limiting the gray value to the range of [0,255] to avoid overcompensation and amplification of background noise.

[0036] S213, perform position correction on the compensated image to obtain a preprocessed image set.

[0037] The lens intrinsic parameters (focal length, principal point coordinates) and distortion coefficients are obtained. The image is then distorted by using the inverse process of Zhang Zhengyou's calibration method to eliminate barrel / pincushion distortion.

[0038] Optionally, the step of decomposing the preprocessed image set to obtain a first sub-band and a second sub-band includes: S221, Preset scale factor ; S222, order Decompose any image in the preprocessed image set to obtain a high-frequency image. and low-frequency images ; S223, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S224, for the low-frequency image Decomposition is performed to obtain high-frequency images. and low-frequency images ; S225, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S226, Execute S224~S225 until... ; S227, for Information from each direction is integrated to obtain the first information, the low-frequency image. This is the second piece of information; S228, integrate the first information of each image in the preprocessed image set to obtain the first sub-band; Specifically, the first information of each image is weighted and fused to obtain the first sub-band; S229, integrate the second information of each image in the preprocessed image set to obtain a second sub-band. Specifically, perform weighted fusion of the second information of each image to obtain the second sub-band; Optionally, fusing the first sub-band to obtain the first fused image includes: The first sub-band is fused using the first sub-band fusion model to obtain the first fused image; The expression for the first sub-band fusion model is: in, The first fused image contains the first sub-band. One element, The first sub-band One element, for exist Pixel value at point for In the neighborhood The average pixel value within the range, and For the preset value, such as =3, =3, For the first The weight of each element.

[0039] Optionally, fusing the second sub-band to obtain the second fused image includes: S241, Process any image in the second sub-band to obtain the observed image; Each image in the second sub-band has an observation image; Using the trained Hidden Markov Model, each image in the second sub-band is processed to obtain the observed image of each image; Specifically, the pixel grayscale sequence of the second sub-band image is regarded as the observation sequence input of the HMM. The optimal hidden state (target detail state / background noise state) corresponding to each pixel is determined by probability decoding using the trained HMM. Based on the optimal hidden state, the pixel grayscale value is adaptively adjusted (enhancing target detail pixels and suppressing noise pixels), and finally an observation image that meets the requirements of subsequent processing is generated.

[0040] S242, the observed images are fused to obtain a second fused image.

[0041] The specific fusion method is the canonical correlation fusion method; Optionally, the step of performing target detection on the fused image to obtain the foreign object target detection result for the airport runway includes: S31, Process the fused image to obtain initial candidate boxes; The fused image contains multi-dimensional features (grayscale, edges, details), and needs to be processed in a targeted manner to highlight the target area. Then, an efficient candidate box generation algorithm is used to obtain the initial candidate box to avoid missing small targets (such as tiny cracks) and falsely detecting background areas.

[0042] Image Enhancement: Contrast-limited adaptive histogram equalization is used to enhance the grayscale difference between the target and the background, avoiding local overexposure / underexposure that could cause the target to be obscured. Parameter settings: Block size 8×8, contrast limit threshold 2.0, ensuring that the enhanced target edges are clear and there is no noise amplification.

[0043] Coarse localization of the target region: Suspected target regions are extracted through adaptive threshold segmentation. The threshold is calculated using the triangular thresholding method (adapted to the grayscale distribution characteristics of the runway fusion image) to obtain a binarized image; morphological dilation (3×3 structuring element) is performed on the binarized image to connect the broken target regions, and then erosion (3×3 structuring element) is performed to remove small background noise.

[0044] Initial candidate box generation: A sliding window traversal + region area filtering strategy is adopted. Sliding window settings: Based on the common target sizes of airport runways (such as crack width of 2-5 pixels, marking defects of 10-50 pixels), three sets of windows with different scales are designed: 8×8, 16×16, and 32×32, with a step size of 4 pixels (balancing efficiency and coverage); Window filtering: The proportion of target pixels within each window (the proportion of foreground pixels in the binarized image) is calculated. When the proportion is ≥30%, the window is retained as an initial candidate box; Coordinate recording: The coordinates of the upper left corner (x1, y1) and the lower right corner (x2, y2) of each initial candidate box are recorded to form an initial candidate box set.

[0045] S32, Complete the initial candidate boxes to obtain optimized candidate boxes; The enhanced fused image is subjected to Canny edge detection (low threshold = image grayscale mean × 0.2, high threshold = low threshold × 2.5) to obtain the target edge map. For each initial candidate box, it is determined whether its boundary coincides with the target edge in the edge map. If a boundary of the candidate box does not touch the target edge, the window is expanded in the edge direction (expanded by 2 pixels each time) until the boundary touches the target edge or expands to the image boundary. If the target edge in the candidate box is incomplete (e.g., only contains part of the edge), the candidate box is completed to completely surround the target edge by combining the principle of the minimum bounding rectangle of the target. The weighted NMS algorithm (to increase the probability of retaining small targets) is used to remove redundant edges to obtain optimized candidate boxes.

[0046] S33, perform feature embedding on the optimized candidate box to obtain embedded feature information; All image regions corresponding to the optimized candidate boxes are scaled to a uniform size (e.g., 64×64 pixels). Bilinear interpolation is used to ensure image quality and avoid the impact of size differences on feature extraction. MobileNetV2 is used as the backbone network for feature extraction. The normalized candidate box regions are input, and the output features of the second-to-last layer of the network (128 dimensions) are taken as deep semantic features. HOG features (9 orientations, 8×8 cell size, 16×16 block size, 324-dimensional feature dimension) and LBP features (1 radius, 8 neighborhood points, 256-dimensional feature dimension) are extracted from the normalized regions and fused to obtain a shallow feature vector (580-dimensional feature dimension). The deep semantic features and the shallow feature vector are concatenated to obtain an original feature vector with a dimension of 708. The original feature vector is L2 normalized (making the feature vector magnitude 1) to obtain the final embedded feature information.

[0047] S34, the embedded feature information is encoded to obtain optimized feature information; The embedded feature information is dimensionality reduced to obtain dimensionality-reduced feature information; The reduced-dimensionality feature information is encoded to obtain optimized feature information; The encoding method is as follows: A channel attention mechanism (SENet module) is used to enhance key feature dimensions. Global average pooling is applied to the reduced-dimensional feature vector to obtain 1×1×k global features. Attention weights for each feature dimension are calculated using a two-layer fully connected network (k / 16 neurons in the first layer and k neurons in the second layer) and a sigmoid activation function. Feature weighting involves element-wise multiplication of the reduced-dimensional feature vector with the attention weights to obtain optimized feature information. S35, classify and detect the optimized feature information to obtain the detection result of foreign object targets on the airport runway.

[0048] Specifically, the optimized feature information is classified and detected using a detection model to obtain the detection results of foreign object targets on the airport runway. Figure 2 This is a schematic diagram of the detection model structure disclosed in an embodiment of the present invention. The present invention introduces an SRDNN module to transform the input feature map into a dynamic topological graph with semantic relevance, dynamically optimizing the graph topology and modeling across semantic contexts. Four attention refinement modules are sequentially connected, with the output of the previous level concatenated with the feature map of ResNet-50 as the input to the next level module. Through multi-level refinement of the feature map, high-level and low-level features are fused to refine the details of the network's predicted saliency map. Attention refinement modules ARM1~ARM3 are used to process the feature maps extracted by ResNet50. These attention refinement modules guide the network to selectively process information in each channel, focusing on spatially more meaningful parts. In lower convolutional layers of the network, limited by the receptive field size, contextual information outside the receptive field often cannot be utilized.

[0049] As can be seen, this invention analyzes and fuses the characteristics of images from various perspectives, making up for the deficiencies of a single image, improving the utilization rate of image information, and obtaining richer and more accurate information; the fused image is processed to obtain initial candidate boxes; the initial candidate boxes are then subjected to feature embedding, encoding, and classification detection to obtain the foreign object detection results for airport runways; the method of this invention improves the accuracy of foreign object detection for airport runways.

[0050] Example 2 Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an airport runway foreign object detection device disclosed in an embodiment of the present invention. Figure 3 The described airport runway foreign object detection device is applied in the field of image processing and application technology, and the embodiments of the present invention are not limited thereto. Figure 3 As shown, the airport runway foreign object detection device may include the following operations: S301, Data acquisition module, used to acquire a set of airport runway images; the set of airport runway images includes N airport runway images from different perspectives, where N is a positive integer; S302, Image fusion module, used to fuse N images in the airport runway image set to obtain a fused image; S303, Target Detection Module, is used to perform target detection on the fused image to obtain the foreign object target detection result of the airport runway.

[0051] Example 3 Please see Figure 4 , Figure 4 This is a schematic diagram of another foreign object detection device for airport runways disclosed in an embodiment of the present invention. Figure 4 The described airport runway foreign object detection device is applied in the field of image processing and application technology, and the embodiments of the present invention are not limited thereto. Figure 4 As shown, the airport runway foreign object detection device may include the following operations: Memory 401 storing executable program code; Processor 402 coupled to memory 401; The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the airport runway foreign object detection method described in Embodiment 1.

[0052] Example 5 This invention discloses a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program enables a computer to perform the steps in the airport runway foreign object detection method described in Embodiment 1.

[0053] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0054] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0055] Finally, it should be noted that the airport runway foreign object detection method and apparatus disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting foreign objects on airport runways, characterized in that, The method includes: S1, Obtain the set of airport runway images; the set of airport runway images includes N airport runway images from different perspectives, where N is a positive integer; S2, fuse N images from the airport runway image set to obtain a fused image, including: S21, preprocess the airport runway image set to obtain a preprocessed image set; S22, decompose the preprocessed image set to obtain a first sub-band and a second sub-band, including: S221, Preset scale factor ; S222, order Decompose any image in the preprocessed image set to obtain a high-frequency image. and low-frequency images ; S223, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S224, for the low-frequency image Decomposition is performed to obtain high-frequency images. and low-frequency images ; S225, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S226, Execute S224~S225 until... ; S227, for Information from each direction is integrated to obtain the first information, the low-frequency image. This is the second piece of information; S228, integrate the first information of each image in the preprocessed image set to obtain the first sub-band; S229, integrate the second information of each image in the preprocessed image set to obtain the second sub-band; S23, the first sub-band is fused to obtain a first fused image, including: The first sub-band is fused using the first sub-band fusion model to obtain the first fused image; The expression for the first sub-band fusion model is: in, The first fused image contains the first sub-band. One element, The first sub-band One element, for exist Pixel value at point for In the neighborhood The average pixel value within the range, and For the preset value, such as =3, =3, For the first The weight of each element; S24, the second sub-band is fused to obtain a second fused image, including: S241, Process any image in the second sub-band to obtain the observed image; Each image in the second sub-band has an observation image; Using the trained Hidden Markov Model, each image in the second sub-band is processed to obtain the observed image of each image; Specifically, the pixel grayscale sequence of the second sub-band image is regarded as the observation sequence input of the HMM. The optimal hidden state corresponding to each pixel is determined by probability decoding using the trained HMM. Based on the optimal hidden state, the pixel grayscale value is adaptively adjusted to enhance the target detail pixels and suppress noise pixels, and finally generate an observation image that meets the requirements of subsequent processing. S242, The observed images are fused to obtain a second fused image; S25, transform the first fused image and the second fused image to obtain a fused image; S3, Perform target detection on the fused image to obtain the foreign object detection result for the airport runway, including: S31, Process the fused image to obtain initial candidate boxes; S32, Complete the initial candidate boxes to obtain optimized candidate boxes; S33, perform feature embedding on the optimized candidate box to obtain embedded feature information; All image regions corresponding to the optimized candidate boxes are scaled to a uniform size of 64×64 pixels. Bilinear interpolation is used to ensure image quality and avoid the impact of size differences on feature extraction. MobileNetV2 is used as the backbone network for feature extraction. The normalized candidate box regions are input, and the output features of the second-to-last layer of the network with a dimension of 128 are taken as deep semantic features. HOG features (9 orientations, 8×8 cell size, 16×16 block size, 324-dimensional feature) and LBP features (radius 1, 8 neighborhood points, 256-dimensional feature) are extracted from the normalized regions and fused to obtain a shallow feature vector with a dimension of 580. The deep semantic features and the shallow feature vector are concatenated to obtain an original feature vector with a dimension of 708. The original feature vector is then L2 normalized to make the feature vector magnitude 1, resulting in the final embedded feature information. S34, the embedded feature information is encoded to obtain optimized feature information; The embedded feature information is dimensionality reduced to obtain dimensionality-reduced feature information; The reduced-dimensionality feature information is encoded to obtain optimized feature information; The encoding method is as follows: The SENet module, employing a channel attention mechanism, is used to enhance key feature dimensions. Global average pooling is performed on the reduced-dimensional feature vector to obtain 1×1×k global features. An attention weight for each feature dimension is calculated using a two-layer fully connected network with k / 16 neurons in the first layer and k neurons in the second layer, along with a Sigmoid activation function. Feature weighting involves element-wise multiplication of the reduced-dimensional feature vector with the attention weights to obtain optimized feature information. S35, classify and detect the optimized feature information to obtain the detection result of foreign object targets on the airport runway.

2. The method for detecting foreign objects on airport runways according to claim 1, characterized in that, The preprocessing of the airport runway image set to obtain a preprocessed image set includes: S211, filter any image in the airport runway image set to obtain a denoised image; S212, Amplitude compensation is performed on the denoised image to obtain a compensated image; S213, perform position correction on the compensated image to obtain a preprocessed image set.

3. A foreign object detection device for airport runways, characterized in that, The device includes: The data acquisition module is used to acquire a set of airport runway images; the set of airport runway images includes N airport runway images from different perspectives, where N is a positive integer; The image fusion module is used to fuse N images from the airport runway image set to obtain a fused image, including: S21, preprocess the airport runway image set to obtain a preprocessed image set; S22, decompose the preprocessed image set to obtain a first sub-band and a second sub-band, including: S221, Preset scale factor ; S222, order Decompose any image in the preprocessed image set to obtain a high-frequency image. and low-frequency images ; S223, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S224, for the low-frequency image Decomposition is performed to obtain high-frequency images. and low-frequency images ; S225, using a preset Directional filter for the high-frequency image Processing is performed to obtain Directional information; S226, Execute S224~S225 until... ; S227, for Information from each direction is integrated to obtain the first information, the low-frequency image. This is the second piece of information; S228, integrate the first information of each image in the preprocessed image set to obtain the first sub-band; S229, integrate the second information of each image in the preprocessed image set to obtain the second sub-band; S23, the first sub-band is fused to obtain a first fused image, including: The first sub-band is fused using the first sub-band fusion model to obtain the first fused image; The expression for the first sub-band fusion model is: in, The first fused image contains the first sub-band. One element, The first sub-band One element, for exist Pixel value at point for In the neighborhood The average pixel value within the range, and For the preset value, such as =3, =3, For the first The weight of each element; S24, the second sub-band is fused to obtain a second fused image, including: S241, Process any image in the second sub-band to obtain the observed image; Each image in the second sub-band has an observation image; Using the trained Hidden Markov Model, each image in the second sub-band is processed to obtain the observed image of each image; Specifically, the pixel grayscale sequence of the second sub-band image is regarded as the observation sequence input of the HMM. The optimal hidden state corresponding to each pixel is determined by probability decoding using the trained HMM. Based on the optimal hidden state, the pixel grayscale value is adaptively adjusted to enhance the target detail pixels and suppress noise pixels, and finally generate an observation image that meets the requirements of subsequent processing. S242, The observed images are fused to obtain a second fused image; S25, transform the first fused image and the second fused image to obtain a fused image; The target detection module is used to perform target detection on the fused image to obtain the foreign object target detection result for the airport runway, including: S31, Process the fused image to obtain initial candidate boxes; S32, Complete the initial candidate boxes to obtain optimized candidate boxes; S33, perform feature embedding on the optimized candidate box to obtain embedded feature information; All image regions corresponding to the optimized candidate boxes are scaled to a uniform size of 64×64 pixels. Bilinear interpolation is used to ensure image quality and avoid the impact of size differences on feature extraction. MobileNetV2 is used as the backbone network for feature extraction. The normalized candidate box regions are input, and the output features of the second-to-last layer of the network with a dimension of 128 are taken as deep semantic features. HOG features (9 orientations, 8×8 cell size, 16×16 block size, 324-dimensional feature) and LBP features (radius 1, 8 neighborhood points, 256-dimensional feature) are extracted from the normalized regions and fused to obtain a shallow feature vector with a dimension of 580. The deep semantic features and the shallow feature vector are concatenated to obtain an original feature vector with a dimension of 708. The original feature vector is then L2 normalized to make the feature vector magnitude 1, resulting in the final embedded feature information. S34, the embedded feature information is encoded to obtain optimized feature information; The embedded feature information is dimensionality reduced to obtain dimensionality-reduced feature information; The reduced-dimensionality feature information is encoded to obtain optimized feature information; The encoding method is as follows: The SENet module, employing a channel attention mechanism, is used to enhance key feature dimensions. Global average pooling is performed on the reduced-dimensional feature vector to obtain 1×1×k global features. An attention weight for each feature dimension is calculated using a two-layer fully connected network with k / 16 neurons in the first layer and k neurons in the second layer, along with a Sigmoid activation function. Feature weighting involves element-wise multiplication of the reduced-dimensional feature vector with the attention weights to obtain optimized feature information. S35, classify and detect the optimized feature information to obtain the detection result of foreign object targets on the airport runway.

4. A foreign object detection device for airport runways, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the airport runway foreign object detection method as described in any one of claims 1-2.

5. A computer-storable medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked, are used to execute the airport runway foreign object detection method as described in any one of claims 1-2.

Images