Mobile Airport Pavement Foreign Object Detection Method and System Based on 2D Imaging

Abstract

The present invention provides a mobile airport pavement foreign object detection method and system based on 2D imaging, which relates to the technical field of airport pavement foreign object detection based on 2D imaging. Through the coordination of multi-spectral dynamic threshold segmentation, cross-frame feature fusion, confidence-driven scanning, and hierarchical three-dimensional reconstruction, the present invention significantly improves the accuracy and efficiency of airport pavement foreign object detection. Based on a dynamic threshold strategy that combines global and local segmentation, it effectively suppresses uneven illumination and specular reflection interference, enhancing the ability to distinguish small foreign objects from complex backgrounds; through a cross-frame fusion weight allocation dominated by reflectivity difference values and an independent high-reflectivity region retention mechanism, it avoids the mis-removal of high-value targets during mobile detection, and combines multi-directional texture feature modeling to distinguish metal foreign objects from the inherent structure of the pavement, reducing the false alarm rate; by comprehensively scoring confidence and fusing reflectivity difference values, texture complexity, and spatial consistency features, it realizes the precise screening of foreign objects and efficient airport pavement safety monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of airport pavement foreign object detection based on 2D imaging, and particularly to a mobile airport pavement foreign object detection method and system based on 2D imaging. Background Art

[0002] Real-time and accurate detection of foreign objects on airport pavements, such as metal fragments, rubber chunks, sand and gravel, is a core link in aviation safety operation and maintenance. Traditional methods mainly rely on manual inspections or fixed sensors, which have problems such as low efficiency, poor real-time performance, high false alarm and missed detection rates in complex environments. Detection schemes based on single-band optical imaging are easily interfered by uneven illumination and pavement reflection, resulting in high false alarm rates in non-target areas such as shadows and water stains, and serious missed detections of low-reflectivity foreign objects due to weakened features; traditional threshold segmentation algorithms rely on fixed or global thresholds and cannot dynamically adapt to different material pavements and illumination changes, especially in complex texture areas such as pavement joints and repair marks, where false alarms frequently occur; multi-frame fusion algorithms mostly rely on a single spatial overlap ratio criterion and ignore the temporal stability analysis of reflectivity differences, resulting in high-reflectivity foreign objects being mis-removed due to positioning errors during mobile detection; existing technologies do not sufficiently utilize the multi-modal features of reflectivity and texture in a coordinated manner. The high-reflectivity characteristics and sharp edges of metal foreign objects are not jointly modeled, and airport pavement joints are difficult to distinguish because their reflectivity is close to that of foreign objects but their texture is smooth; the allocation of hardware resources is rigid. Three-dimensional verification uses fixed trajectory scanning and cannot dynamically adjust the path according to the foreign object density. The risk of missed detection in high-density areas is high, and redundant scanning in low-density areas reduces efficiency; three-dimensional reconstruction algorithms have high computational resource requirements and are disconnected from the front-end detection confidence. Low-risk areas occupy computing power, and the reconstruction accuracy of high-risk foreign objects is insufficient, making it difficult to meet the balance requirements of real-time performance and accuracy.

[0003] Therefore, it is necessary to provide a mobile airport pavement foreign object detection method and system based on 2D imaging to solve the above technical problems. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a mobile airport pavement foreign object detection method and system based on 2D imaging, achieving the beneficial effect of detecting foreign objects on airport pavements in real time and accurately.

[0005] The present invention provides a mobile airport pavement foreign object detection method based on 2D imaging, including:

[0006] S1: Continuously perform multi-spectral scanning on the airport pavement through a mobile detection platform to obtain a series of consecutive pavement images, perform dynamic threshold segmentation on each pavement image to obtain an initial candidate region and its reflectivity difference value and texture complexity;

[0007] S2: Based on the pose parameters of the mobile detection platform, the initial candidate regions are de-duplicated and fused through the intersection over union (IoU) matching algorithm to obtain the set of initial candidate regions after de-duplication and fusion, as well as their fused reflectance difference values and fused texture complexities;

[0008] S3: Based on the fused reflectance difference values and fused texture complexities, calculate the comprehensive confidence score of each initial candidate region after de-duplication and fusion, and combine it with a preset confidence score threshold to obtain a set of high-confidence candidate regions;

[0009] S4: Generate spiral scan trajectory parameters based on the spatial distribution density of the high-confidence candidate regions, and control the mobile detection platform to collect multi-view images of the high-confidence candidate regions according to the spiral scan trajectory parameters to obtain multi-view images;

[0010] S5: Perform feature matching and three-dimensional reconstruction on the multi-view images to obtain the foreign object detection results of each high-confidence candidate region.

[0011] Preferably, in step S1, the steps for obtaining the initial candidate regions include:

[0012] Based on a preset global segmentation threshold, perform preliminary binary segmentation on the pavement image frame, and extract all connected components as the rough candidate regions;

[0013] Taking each rough candidate region as the center, expand a rectangular region with a preset size outward as the local background analysis area, and exclude the pixels of the rough candidate region itself;

[0014] Calculate the mean and standard deviation of the reflectance in the near-infrared band within the local background analysis area;

[0015] Generate a local segmentation threshold based on the mean and standard deviation of the reflectance to perform secondary segmentation on the pavement image frame, extract the connected components after secondary segmentation, and merge the connected components that spatially overlap with the rough candidate regions to obtain the set of initial candidate regions.

[0016] Preferably, in step S1, the texture complexity is based on the gray-level co-occurrence matrix to extract the contrast, energy, and homogeneity indexes of the candidate regions, and combine multi-directional texture features to generate the texture complexity.

[0017] Preferably, step S1 further includes distinguishing metal foreign objects from pavement joints through texture pattern classification.

[0018] Preferably, in step S2, the method for determining the fusion weight for fusing the initial candidate regions through the intersection over union (IoU) matching algorithm is:

[0019] For the initial candidate regions that reach the preset spatial overlap ratio threshold, the fusion weight is obtained by normalizing the reflectance difference values of the initial candidate regions.

[0020] Preferably, in step S2, for an initial candidate region that does not reach the preset spatial overlap ratio threshold, if its reflectivity difference value exceeds the preset reflectivity difference threshold, then mark this initial candidate region as an independent initial candidate region.

[0021] Preferably, in step S3, the calculation of the comprehensive confidence score further includes:

[0022] Adding a scoring gain term based on the intersection over union to the calculation of the comprehensive confidence score for the initial candidate regions marked as independent.

[0023] Preferably, in step S4, the steps for generating the spiral scan trajectory parameters include:

[0024] Taking the geographical coordinates of the high-confidence candidate regions as the center, counting the number of high-confidence candidate regions within a preset radius range, and calculating the regional density value per unit area;

[0025] Based on the regional density value, inversely scaling the preset reference detour radius to obtain the actual detour radius and proportionally expanding the preset reference number of scan circles based on the regional density value to obtain the actual number of scan circles;

[0026] Combining the actual detour radius and the actual number of scan circles to generate a spiral scan trajectory coordinate sequence, and obtaining the spiral scan trajectory parameters.

[0027] Preferably, in step S5, the 3D reconstruction includes:

[0028] Dividing the accuracy level based on the comprehensive confidence score, dynamically controlling the density of 3D point cloud generation according to the accuracy level, and selecting different reconstruction algorithms for high-confidence candidate regions of different accuracy levels.

[0029] The present invention also provides a mobile airport pavement foreign object detection system based on 2D imaging, which is applied to a method for mobile airport pavement foreign object detection based on 2D imaging, including:

[0030] A data acquisition and processing module, which is used to perform multi-spectral continuous scanning on the airport pavement through a mobile detection platform, obtain continuous multi-frame pavement images, perform dynamic threshold segmentation on each frame of pavement image, and obtain initial candidate regions, their reflectivity difference values, and texture complexities;

[0031] A cross-frame fusion module, which is used to perform duplicate removal and fusion on the initial candidate regions through an intersection over union matching algorithm based on the pose parameters of the mobile detection platform, and obtain a set of initial candidate regions after duplicate removal and fusion, their fused reflectivity difference values, and fused texture complexities;

[0032] A confidence score screening module, which is used to calculate the comprehensive confidence scores of each initial candidate region after deduplication and fusion based on the fused reflectance difference value and the fused texture complexity, and obtain a set of high-confidence candidate regions in combination with a preset confidence score threshold;

[0033] A multi-view image acquisition module, which is used to generate spiral scan trajectory parameters based on the spatial distribution density of high-confidence candidate regions, and control a mobile detection platform to perform multi-view image acquisition on the high-confidence candidate regions according to the spiral scan trajectory parameters to obtain multi-view images;

[0034] A foreign object verification module, which is used to perform feature matching and three-dimensional reconstruction on multi-view images to obtain the foreign object detection results of each high-confidence candidate region.

[0035] Compared with related technologies, a mobile airport pavement foreign object detection method and system based on 2D imaging provided by the present invention has the following beneficial effects:

[0036] Through the technical collaboration of multi-spectral dynamic threshold segmentation, cross-frame feature fusion, confidence-driven scanning, and hierarchical three-dimensional reconstruction, the present invention significantly improves the accuracy and efficiency of airport pavement foreign object detection. First, based on a dynamic threshold strategy combining global and local segmentation, it effectively suppresses uneven illumination and specular reflection interference, and enhances the ability to distinguish small foreign objects from complex backgrounds; through a cross-frame fusion weight allocation dominated by the reflectance difference value and an independent high-reflectance region retention mechanism, it avoids the mis-removal of high-value targets during mobile detection. At the same time, it uses multi-directional texture feature modeling to distinguish metal foreign objects from the inherent structure of the pavement, greatly reducing the false alarm rate; the confidence score model fuses reflectance, texture, and spatial consistency features to achieve the precise screening of high-probability foreign objects; the spiral scan trajectory is dynamically optimized based on the regional density value to adaptively scan the path, taking into account the refined coverage of high-density regions and the rapid screening of low-density regions, improving the utilization rate of hardware resources; the hierarchical three-dimensional reconstruction mechanism dynamically adapts the point cloud density and algorithm complexity according to the confidence level to ensure the millimeter-level precision reconstruction of high-confidence targets, while simplifying the computational load of low-risk regions. Each module forms a closed-loop system through multi-modal data collaboration of reflectance and texture, confidence feedback optimization, and dynamic resource scheduling, and realizes robust and efficient airport pavement safety monitoring under scenarios of complex illumination, rain and fog interference, and foreign object material diversity. Description of the Drawings

[0037] Figure 1 It is a flowchart of a mobile airport pavement foreign object detection method based on 2D imaging of the present invention;

[0038] Figure 2 It is a module structure diagram of a mobile airport pavement foreign object detection system based on 2D imaging of the present invention. Detailed Embodiments

[0039] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only for explaining the present invention, rather than limiting the present invention. In addition, it should be noted that for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all the structures. In addition, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0040] In addition, it should be noted that for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all the content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as being processed sequentially, many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. When the operations are completed, the process can be terminated, but there can also be additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, and so on.

[0041] Embodiment 1

[0042] A mobile airport pavement foreign object detection method based on 2D imaging. In the specific implementation process, as Figure 1 shown, it shows a flowchart of a mobile airport pavement foreign object detection method based on 2D imaging, including:

[0043] Step S1: Continuously scan the airport pavement in multiple spectra through a mobile detection platform to obtain multiple consecutive frames of pavement images. Perform dynamic threshold segmentation on each frame of pavement image to obtain the initial candidate regions and their reflectivity difference values and texture complexities.

[0044] Specifically, in step S1, the steps for obtaining the initial candidate regions include:

[0045] Perform preliminary binary segmentation on the pavement image frame based on a preset global segmentation threshold, and extract all connected components as the rough candidate regions;

[0046] Take each rough candidate region as the center, and expand a rectangular region with a preset size outward as the local background analysis area, and exclude the pixels of the rough candidate region itself;

[0047] Calculate the mean and standard deviation of the reflectivity in the near-infrared band in the local background analysis area;

[0048] Generate a local segmentation threshold based on the mean and standard deviation of the reflectivity to perform secondary segmentation on the pavement image frame, extract the connected components after the secondary segmentation, and merge the connected components that spatially overlap with the rough candidate regions to obtain the initial candidate region set.

[0049] Specifically, in step S1, based on the gray-level co-occurrence matrix, the texture complexity extracts the contrast, energy, and homogeneity indexes of the candidate regions, and combines multi-directional texture features to generate the texture complexity.

[0050] Specifically, step S1 further includes distinguishing metal foreign objects from pavement joints through texture pattern classification.

[0051] In the specific implementation process, first, the mobile detection platform moves uniformly along the pavement through the multi-spectral imaging device, synchronously collects visible light and near-infrared band images at a preset frame rate. For each frame of image, first, preliminary binary segmentation is performed using a preset global segmentation threshold, and all connected components are screened out as the rough candidate regions; subsequently, to eliminate mis-segmentation caused by pavement reflection or shadow, a local background analysis area is generated by expanding outward with each rough candidate region as the center, and the pixels of the candidate region itself are excluded. The mean and standard deviation of the reflectance in the near-infrared band in this area are calculated, and an adaptive secondary segmentation threshold (exemplarily, the mean plus a multiple of the standard deviation) is dynamically generated based on the local background statistical characteristics. The original image is subjected to secondary segmentation and the connected components are extracted; through spatial overlap analysis, the secondary segmentation connected components are merged with the rough candidate regions, and isolated noise points caused by uneven illumination are removed to form an accurate initial candidate region set. On this basis, for each candidate region, the contrast, energy, and homogeneity indexes in four directions of 0°, 45°, 90°, and 135° are calculated respectively through the gray-level co-occurrence matrix. Among them, the contrast characterizes the texture clarity, the energy characterizes the texture uniformity, and the homogeneity describes the local similarity. Then, a weighted fusion strategy is adopted to generate a multi-directional comprehensive texture complexity value; at the same time, based on the difference in the mean near-infrared reflectance between the candidate region and the local background, combined with the background standard deviation for normalization processing, the reflectance difference value is quantified to characterize the spectral separability between foreign objects and the pavement. Further, through a pre-trained texture pattern classification model, the texture feature vectors of the candidate regions are subjected to pattern discrimination to distinguish highly reflective metal foreign objects from pavement joints. Among them, highly reflective metal foreign objects present discrete textures with high contrast and low homogeneity, and pavement joints have continuous textures with low contrast and high homogeneity, thereby suppressing the misdetection of the inherent structure of the pavement and providing a highly reliable candidate region set and feature data for the subsequent steps.

[0052] Step S2: Based on the pose parameters of the mobile detection platform, the initial candidate regions are de-duplicated and fused through the intersection over union (IoU) matching algorithm to obtain the de-duplicated and fused initial candidate region set, its fused reflectance difference value, and fused texture complexity.

[0053] Specifically, in step S2, the method for determining the fusion weight for fusing the initial candidate regions through the IoU matching algorithm is as follows:

[0054] For the initial candidate regions that reach the preset spatial overlap ratio threshold, the fusion weight is obtained by normalizing the reflectivity difference value of the initial candidate regions.

[0055] Specifically, in step S2, for the initial candidate regions that do not reach the preset spatial overlap ratio threshold, if their reflectivity difference value exceeds the preset reflectivity difference threshold, then mark the initial candidate region as an independent initial candidate region.

[0056] In the specific implementation process, first, based on the pose parameters of the mobile detection platform, such as GPS positioning data and pan-tilt attitude angles by way of example, map the coordinates of the initial candidate regions in consecutive multi-frame pavement images to a unified geographic coordinate system, calculate the spatial overlap ratio of the initial candidate regions in different frames through the intersection over union (IoU) algorithm, determine that the initial candidate regions that reach the preset spatial overlap ratio threshold are repeated observations of the same foreign object in different frames, and then perform feature deduplication and fusion. Taking the reflectivity difference value as the core, perform normalized weight assignment for the successfully matched candidate regions, and perform weighted fusion according to the fusion weight of the normalized weight assignment to obtain the set of initial candidate regions after deduplication and fusion. At the same time, perform weighted averaging on the reflectivity difference value and texture complexity of each pavement image according to the fusion weight of the normalized weight assignment to generate the fused reflectivity difference value and the fused texture complexity; for the initial candidate regions that do not reach the spatial overlap ratio threshold but whose reflectivity difference value is higher than the background statistical value, such as metal fragments that appear independently in a single frame due to high reflectivity by way of example, mark them as independent initial candidate regions and retain the original feature values to avoid losing high-value targets due to cross-frame matching failures; finally, merge the fused initial candidate regions with the independent candidate regions to form the set of initial candidate regions after deduplication.

[0057] Step S3: Based on the fused reflectivity difference value and the fused texture complexity, calculate the comprehensive confidence score of each initial candidate region after deduplication and fusion, and combine it with the preset confidence score threshold to obtain the set of high-confidence candidate regions.

[0058] Specifically, in step S3, the calculation of the comprehensive confidence score further includes:

[0059] Add a scoring gain term based on the intersection over union (IoU) to the calculation of the comprehensive confidence score of the initial candidate regions marked as independent.

[0060] During the specific implementation process, first, the initial candidate area after deduplication fusion is weightedly summed based on its fused reflectivity difference value and fused texture complexity, where the reflectivity difference value is given a dominant weight due to its higher sensitivity to foreign object detection, and texture complexity is calculated as an auxiliary feature; for the candidate area that successfully matches across frames, its fused reflectivity difference value is obtained by multi-frame normalized weighted average, while the independent candidate area, that is, the target that does not reach the spatial overlap ratio threshold but the reflectivity significantly exceeds the limit, retains the original reflectivity difference value of the single frame. On this basis, the intersection-over-union (IoU) score gain term is introduced for the independent candidate area. The design of the IoU score gain term aims to solve the balance between the risk of missed detection and false detection suppression of the initial candidate area marked as independent. Its core logic is: for the independent initial candidate area, count its number of appearances in historical continuous frames and the average IoU with other initial candidate areas. If the initial candidate area appears frequently and independently in multiple frames and the average IoU is continuously lower than the preset threshold, it indicates that it is not covered by other frames and may be a high-speed moving foreign object or instantaneous interference. Then, according to the number of independent appearances, the IoU score gain term is used to calculate the number of independent candidate areas. The product of the gain factor and the mean IoU value generates a dynamic gain factor, which is added to the basic confidence score. For example, if a metal fragment is captured in only a single frame due to the rapid movement of the mobile detection platform, but its reflectivity difference value is extremely high and the number of independent occurrences exceeds the threshold, the system will assign it a higher gain factor, causing the comprehensive confidence score to exceed the scan verification threshold and trigger subsequent multi-view acquisition. Conversely, for isolated noise with low reflectivity difference values and large fluctuations in the mean IoU value, even if the number of independent occurrences is large, its comprehensive score will still be below the filtering threshold because the gain factor is suppressed by the mean IoU value. The weight distribution of the IoU score gain term follows the principle of "high reflectivity dominates, low IoU ratio compensates". In the confidence score formula, the reflectivity difference value occupies the linear dominant weight, texture complexity participates in the calculation as an auxiliary feature, and the IoU score gain term is superimposed with a nonlinear proportional coefficient. This ensures that highly reflective foreign objects can be captured even if cross-frame matching fails, while also preventing low-reflectivity noise from accidentally appearing multiple times and falsely increasing the confidence score. The confidence score threshold is also dynamically adjusted based on ambient lighting conditions. For example, in rainy and foggy weather, where texture feature reliability decreases, the weight of texture complexity is reduced and the weight of reflectance difference is increased to ensure robustness in complex scenes. Ultimately, through the coordinated calculation of reflectance and texture features and independent target gain compensation, a set of high-confidence candidate regions with a comprehensive score exceeding the dynamic threshold is selected.

[0061] Step S4: generating spiral scanning trajectory parameters based on the spatial distribution density of the high-confidence candidate area, and controlling the mobile detection platform to perform multi-view image acquisition on the high-confidence candidate area according to the spiral scanning trajectory parameters to obtain multi-view images.

[0062] Specifically, in step S4, the step of generating the spiral scanning trajectory parameters includes:

[0063] Centering on the geographical coordinates of the high-confidence candidate regions, count the number of high-confidence candidate regions within a preset radius, and calculate the regional density value per unit area;

[0064] Based on the regional density value, perform inverse proportional scaling on the preset reference bypass radius to obtain the actual bypass radius, and perform direct proportional expansion on the preset reference number of scanning circles to obtain the actual number of scanning circles;

[0065] Combine the actual bypass radius and the actual number of scanning circles to generate a spiral scanning trajectory coordinate sequence, and obtain the spiral scanning trajectory parameters.

[0066] In the specific implementation process, first, centering on the geographical coordinates of each high-confidence candidate region, count the number of high-confidence candidate regions within a preset radius and calculate the density value per unit area. The higher the density value, the denser the foreign object distribution is; perform inverse proportional scaling on the preset reference bypass radius based on this density value. Exemplarily, if the density value doubles, the bypass radius is halved. At the same time, perform direct proportional expansion on the reference number of scanning circles. Exemplarily, if the density value doubles, the number of scanning circles doubles, ensuring fine scanning of high-density regions and rapid coverage of low-density regions; subsequently, according to the adjusted actual bypass radius and the number of scanning circles, generate a spiral trajectory coordinate sequence according to the Archimedes spiral mathematical model, control the mobile detection platform to move along the trajectory and keep the pan-tilt unit aligned with the center of the candidate region, and synchronously adjust the platform movement speed dynamically according to the density value. Exemplarily, decelerate when the density value is high to improve the imaging resolution, and accelerate when the density value is low to shorten the scanning time; in addition, optimize the trajectory according to the spatial distribution characteristics of adjacent high-confidence candidate regions. Exemplarily, if the distance between multiple high-confidence candidate regions is less than the bypass radius, merge them to generate a shared spiral trajectory to avoid repeated scanning; at the same time, introduce a priority sorting mechanism, use the product of the density value and the confidence score as the basis for the scanning order, and give priority to scanning high-density and high-confidence regions to ensure the rapid verification of key targets and achieve the optimal allocation of scanning resources, taking into account both detection efficiency and accuracy.

[0067] Step S5: Perform feature matching and 3D reconstruction on the multi-view images to obtain the foreign object detection results of each high-confidence candidate region.

[0068] Specifically, in step S5, the 3D reconstruction includes:

[0069] Divide the accuracy level based on the comprehensive confidence score, and dynamically control the density of 3D point cloud generation according to the accuracy level and select different reconstruction algorithms for high-confidence candidate regions of different accuracy levels.

[0070] In the specific implementation process, first, feature matching is performed on the collected multi-view images, and the accuracy level is divided based on the comprehensive confidence score. For the high-confidence region, dense point cloud reconstruction is adopted. Exemplarily, a three-dimensional model with millimeter-level accuracy is generated by combining structured light coding and stereo vision triangulation algorithms. For the medium and low-confidence regions, sparse point clouds or fast reconstruction algorithms based on structure from motion are used to balance the computational efficiency and accuracy requirements. For high-reflectivity targets such as metal foreign objects, laser-assisted registration technology is adopted to enhance the robustness of feature point matching. The mismatched points caused by reflection are removed through the RANSAC algorithm, and the three-dimensional coordinates and dimensions of the foreign objects are calculated based on multi-view geometric constraints. At the same time, multi-spectral imaging data and three-dimensional point cloud features are used for joint classification to distinguish foreign object types such as metal and rubber, and real-time alarms are triggered for foreign objects whose height exceeds the safety threshold. Finally, a structured foreign object detection result is output. In addition, the three-dimensional reconstruction result is fed back to step S1. Exemplarily, if the three-dimensional reconstruction of a certain high-confidence candidate region fails multiple times but the reflectivity difference value continuously exceeds the preset threshold, the local segmentation threshold sensitivity of this region in step S1 is automatically increased to enhance the detection ability of secondary segmentation. It is fed back to step S3. Exemplarily, for the verified foreign objects, their reflectivity difference values, texture features, and three-dimensional height data are extracted, and the weights in the formula for calculating the comprehensive confidence score in step S3 are dynamically updated. It is fed back to step S4. Exemplarily, for the high-frequency false detection regions, their geographical locations are recorded and the spiral scan density threshold in step S4 is permanently increased, and the subsequent scan priority is reduced. Through feedback adjustment, the long-term stability and detection accuracy of the system in a complex environment are ensured.

[0071] The working principle of a mobile airport pavement foreign object detection method based on 2D imaging provided by the present invention is as follows:

[0072] Based on the collaboration of multispectral imaging and dynamic adaptive algorithms, an efficient and accurate identification of foreign objects on airport pavements is achieved through a multi-stage progressive detection and verification mechanism. First, multispectral scanning is used to obtain pavement images. Combining global and local dynamic threshold segmentation strategies, initial candidate regions are extracted while suppressing light interference, and a multi-modal feature model of foreign objects is constructed through reflectivity difference values and multi-directional texture complexity. Secondly, the initial candidate regions detected repeatedly are fused by cross-frame intersection over union (IoU) matching. With the reflectivity difference value dominating the weight assignment, high-reflectivity independent targets are retained to form a set of initial candidate regions after deduplication and optimization. Subsequently, a comprehensive confidence score is calculated based on the fused reflectivity difference value and fused texture complexity, and high-confidence regions are screened by combining the IoU gain compensation mechanism for independent targets. A density-adaptive spiral scanning trajectory is generated based on the spatial distribution density, dynamically adjusting the detour radius and the number of scanning circles to achieve fine coverage of high-density regions and rapid screening of low-density regions. Finally, the existence of foreign objects is verified through multi-view feature matching and hierarchical three-dimensional reconstruction, and the types of foreign objects are classified by combining multispectral reflectivity and three-dimensional geometric features, forming a closed-loop feedback mechanism to optimize the front-end segmentation and confidence model. The entire invention solution realizes the balance of high robustness and efficiency of foreign object detection in complex environments through the closed-loop iteration of collaborative analysis of reflectivity and texture features, dynamic resource allocation, and multi-view verification.

[0073] Embodiment 2

[0074] A mobile airport pavement foreign object detection system based on 2D imaging. In the specific implementation process, as Figure 2 shown, it shows a module structure diagram of a mobile airport pavement foreign object detection system based on 2D imaging, including:

[0075] A data acquisition and processing module 100, which is used to continuously perform multispectral scanning on the airport pavement through a mobile detection platform, obtain continuous multi-frame pavement images, perform dynamic threshold segmentation on each frame of pavement image, and obtain initial candidate regions and their reflectivity difference values and texture complexity;

[0076] A cross-frame fusion module 200, which is used to perform deduplication and fusion on the initial candidate regions through the IoU matching algorithm based on the pose parameters of the mobile detection platform, and obtain a set of initial candidate regions after deduplication and fusion and their fused reflectivity difference values and fused texture complexity;

[0077] A confidence score screening module 300, which is used to calculate the comprehensive confidence score of each initial candidate region after deduplication and fusion based on the fused reflectivity difference value and fused texture complexity, and obtain a set of high-confidence candidate regions by combining a preset confidence score threshold;

[0078] The multi-view image acquisition module 400 is used to generate spiral scan trajectory parameters based on the spatial distribution density of high-confidence candidate regions, and control the mobile detection platform to perform multi-view image acquisition on the high-confidence candidate regions according to the spiral scan trajectory parameters to obtain multi-view images;

[0079] The foreign object verification module 500 is used to perform feature matching and three-dimensional reconstruction on the multi-view images to obtain the foreign object detection results of each high-confidence candidate region.

[0080] The working principle of a mobile airport pavement foreign object detection system based on 2D imaging provided by the present invention is as follows:

[0081] First of all, the data acquisition and processing module 100 continuously scans the airport pavement through a multi-spectral imaging device that combines visible light and near-infrared bands, extracts the initial candidate regions by using a threshold segmentation method that combines global rough segmentation and local background statistics secondary segmentation, and synchronously calculates the reflectivity difference value and texture complexity; Subsequently, the cross-frame fusion module 200 maps multiple frames of initial candidate regions to a unified geographic coordinate system through the pose parameters of the mobile detection platform, performs cross-frame fusion dominated by the reflectivity difference value on the overlapping initial candidate regions by using the intersection over union matching algorithm, and retains the high-reflectivity independent targets at the same time to form a set of initial candidate regions after deduplication; The confidence score screening module 300 screens high-confidence candidate regions based on a confidence score model that fuses reflectivity and texture features. The multi-view image acquisition module 400 dynamically adjusts the bypass radius and the number of scan circles in combination with a density-adaptive spiral scan trajectory, and controls the detection platform to perform refined multi-view acquisition on high-density regions and quickly cover low-density regions; Finally, the foreign object verification module 500 verifies the existence and classification of foreign objects through feature matching and hierarchical three-dimensional reconstruction of multi-view images, combines multi-spectral reflectivity and three-dimensional geometric features, and at the same time feeds back the verification results to the front-end module to optimize the segmentation threshold and comprehensive confidence score, forming a full-link closed-loop adaptive detection system, realizing high-precision, high-efficiency and strong robustness of foreign object detection in complex environments.

[0082] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to the embodiments of this application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1 a device for the functions specified in one block or multiple blocks.

[0083] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and this program can be stored in a computer-readable storage medium. The storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0084] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of another identical element in the process, method, commodity or device comprising the element.

Claims

1. A mobile airport pavement foreign object detection method based on 2D imaging, characterized in that, The airport pavement foreign object detection method includes the following steps: S1: Continuously scan the airport pavement in multiple spectra through a mobile detection platform to obtain multiple consecutive frames of pavement images. Perform dynamic threshold segmentation on each frame of pavement image to obtain initial candidate regions and their reflectivity difference values and texture complexities; S2: Based on the pose parameters of the mobile detection platform, perform duplicate removal and fusion on the initial candidate regions through the intersection over union (IoU) matching algorithm to obtain a set of initial candidate regions after duplicate removal and fusion and their fused reflectivity difference values and fused texture complexities; S3: Based on the fused reflectivity difference value and the fused texture complexity, calculate the comprehensive confidence score of each initial candidate region after duplicate removal and fusion, and combine it with a preset confidence score threshold to obtain a set of high-confidence candidate regions; S4: Generate spiral scan trajectory parameters based on the spatial distribution density of the high-confidence candidate regions, and control the mobile detection platform to perform multi-view image acquisition on the high-confidence candidate regions according to the spiral scan trajectory parameters to obtain multi-view images; S5: Perform feature matching and three-dimensional reconstruction on the multi-view images to obtain the foreign object detection results of each high-confidence candidate region; In step S1, the steps for obtaining the initial candidate regions include: Perform preliminary binary segmentation on the pavement image frame based on a preset global segmentation threshold, and extract all connected components as the rough candidate regions; Taking each rough candidate region as the center, expand a rectangular region with a preset size outward as the local background analysis area, and exclude the pixels of the rough candidate region itself; Calculate the mean and standard deviation of the reflectivity in the near-infrared band within the local background analysis area; Generate a local segmentation threshold based on the mean and standard deviation of the reflectivity to perform secondary segmentation on the pavement image frame, extract the connected components after the secondary segmentation, and merge the connected components that spatially overlap with the rough candidate regions to obtain a set of initial candidate regions; In step S1, the texture complexity is based on the gray-level co-occurrence matrix to extract the contrast, energy, and homogeneity indexes of the candidate regions, and combines multi-directional texture features to generate the texture complexity; In step S2, perform cross-frame intersection over union (IoU) calculation on the initial candidate regions of multiple consecutive frames of pavement images, and determine the initial candidate regions with an intersection over union reaching the preset spatial overlap ratio threshold as the same foreign object; Based on the reflectivity difference value, perform normalized weight assignment to all the initial candidate regions determined to be the same foreign object; Perform weighted fusion on the reflectivity difference values and texture complexities of all the initial candidate regions determined to be the same foreign object according to the fusion weights assigned by the normalized weight assignment to obtain the fused reflectivity difference value and the fused texture complexity.

2. The mobile airport pavement foreign object detection method based on 2D imaging according to claim 1, characterized in that Step S1 also includes distinguishing metal foreign objects from pavement joints through texture pattern classification.

3. The method for detecting foreign objects on an airport pavement based on 2D imaging according to claim 2, wherein In step S2, the method for determining the fusion weight for fusing the initial candidate regions through the intersection over union (IoU) matching algorithm is: For the initial candidate regions that reach the preset spatial overlap ratio threshold, obtain the fusion weight by normalizing the reflectivity difference values of the initial candidate regions.

4. A mobile airport pavement foreign object detection method based on 2D imaging according to claim 3, characterized in that, In step S2, for the initial candidate regions that do not reach the preset spatial overlap ratio threshold, if their reflectivity difference values exceed the preset reflectivity difference threshold, mark the initial candidate regions as independent initial candidate regions.

5. A mobile airport pavement foreign object detection method based on 2D imaging according to claim 4, characterized in that, In step S3, the calculation of the comprehensive confidence score further includes: For the calculation of the comprehensive confidence score of the initially candidate regions marked as independent, a scoring gain term based on the intersection over union is added.

6. The mobile airport pavement foreign object detection method based on 2D imaging according to claim 5, characterized in that, In step S4, the steps for generating the spiral scan trajectory parameters include: Taking the geographical coordinates of the high-confidence candidate regions as the center, counting the number of high-confidence candidate regions within a preset radius range, and calculating the regional density value per unit area; Inversely scaling the preset reference circumferential radius based on the regional density value to obtain the actual circumferential radius and proportionally expanding the preset reference number of scan circles based on the regional density value to obtain the actual number of scan circles; Combining the actual circumferential radius and the actual number of scan circles to generate a spiral scan trajectory coordinate sequence and obtaining the spiral scan trajectory parameters.

7. A mobile airport pavement foreign object detection method based on 2D imaging according to claim 6, characterized in that In step S5, the 3D reconstruction includes: Dividing the accuracy levels based on the comprehensive confidence score, dynamically controlling the density of the 3D point cloud generation according to the accuracy levels, and selecting different reconstruction algorithms for the high-confidence candidate regions of different accuracy levels.

8. A mobile airport pavement foreign object detection system based on 2D imaging, characterized in that, Applied to a mobile airport pavement foreign object detection method according to any one of claims 1-7, the airport pavement foreign object detection system includes: A data acquisition and processing module, configured to perform multi-spectral continuous scanning on the airport pavement through a mobile detection platform, obtain a continuous multi-frame pavement image, perform dynamic threshold segmentation on each frame of the pavement image, and obtain the initial candidate regions and their reflectivity difference values and texture complexities; A cross-frame fusion module, configured to perform duplicate removal and fusion on the initial candidate regions through an intersection over union matching algorithm based on the pose parameters of the mobile detection platform, and obtain the set of initial candidate regions after duplicate removal and fusion and their fused reflectivity difference values and fused texture complexities; A confidence score screening module, configured to calculate the comprehensive confidence scores of the initial candidate regions after duplicate removal and fusion based on the fused reflectivity difference values and fused texture complexities, and combine a preset confidence score threshold to obtain a set of high-confidence candidate regions; A multi-view image acquisition module, configured to generate spiral scan trajectory parameters based on the spatial distribution density of the high-confidence candidate regions, and control the mobile detection platform to perform multi-view image acquisition on the high-confidence candidate regions according to the spiral scan trajectory parameters to obtain multi-view images; A foreign object verification module, configured to perform feature matching and 3D reconstruction on the multi-view images to obtain the foreign object detection results of the high-confidence candidate regions.

Images