An airport pavement multi-class disease collaborative identification method

Abstract

The application discloses an airport pavement multi-class disease collaborative identification method, S1: collecting original data according to a three-dimensional laser 3D vision acquisition module and a two-dimensional image acquisition module; S2: synchronously calibrating, extracting features and fusing processing three-dimensional laser point cloud data and two-dimensional high-definition image data; S3: carrying out noise reduction and sharpening processing on the cracks in the two-dimensional image, and strengthening crack features; S4: through elevation difference analysis of the three-dimensional laser point cloud data, the height difference and the morphological features of the wrong platform and the settlement are identified. Through two-dimensional and three-dimensional vision dual-mode fusion, the synchronous and collaborative detection of three-dimensional diseases and plane diseases is realized, the dimensional defects of single visual identification are made up, and all kinds of pavement diseases are ensured to be identified without dead angle and full coverage; and the whole identification process is automatically completed, manual on-site research and judgment are not needed, synchronous operation can be carried out with a vehicle-mounted high-speed acquisition device, the disease identification efficiency is greatly improved, the on-site operation time is reduced, and the normal operation of the airport is avoided from being disturbed.

Description

Technical Field

[0001] This invention relates to the field of airport pavement defect detection and identification technology, and in particular to a method for collaborative identification of multiple types of airport pavement defects. Background Technology

[0002] Currently, airport pavement defect identification mainly relies on two methods: manual identification and single-vision identification. Manual identification depends on on-site visual observation by inspectors, combined with simple tools to measure defect dimensions, to determine the type and severity of defects. However, this method is inefficient, easily affected by the experience and conscientiousness of inspectors, leading to missed detections and misjudgments. Furthermore, defect size measurement and severity assessment are highly subjective, resulting in poor data consistency. Additionally, manual on-site work interferes with normal airport operations, failing to meet the airport's high-safety, low-interference operational requirements. Single-vision identification methods are divided into two-dimensional image recognition and three-dimensional laser recognition. Two-dimensional image recognition is mainly used to capture planar defects, while three-dimensional laser recognition is mainly used to detect three-dimensional defects. However, two-dimensional image recognition can only capture planar defects such as cracks and spalling, and cannot identify elevation differences in three-dimensional defects such as misalignment and settlement, resulting in a limited identification dimension. While three-dimensional laser recognition can detect three-dimensional defects, it struggles to capture subtle details of planar defects. Moreover, both identification methods are often simply superimposed, lacking specific dual-modal collaborative identification logic, making it impossible to achieve synchronous coupled detection of planar and three-dimensional defects, and thus difficult to form a complete defect identification system. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for collaborative identification of multiple types of defects on airport pavement.

[0004] The objective of this invention is achieved through the following technical solution: a method for collaborative identification of multiple types of defects on airport pavement, comprising the following steps:

[0005] S1: Acquire raw data based on the 3D laser 3D vision acquisition module and the 2D image acquisition module;

[0006] S2: Simultaneously calibrate, extract features, and fuse 3D laser point cloud data with 2D high-definition image data;

[0007] S3: Denoise and sharpen cracks in 2D images, and enhance crack features;

[0008] S4: Identify the height differences and morphological characteristics of misalignment and settlement by analyzing the elevation differences of 3D laser point cloud data.

[0009] Preferably, in step S1, the three-dimensional laser 3D vision acquisition module is used to acquire the three-dimensional contour, elevation data and morphological features of the pavement; the two-dimensional image acquisition module is used to acquire high-definition images of the pavement surface.

[0010] Preferably, in step S2, the data synchronization calibration specifically involves calibrating the 3D laser point cloud data. With two-dimensional high-definition image data Spatial coordinate and time axis synchronization calibration is performed. Spatial coordinate calibration uses coordinate transformation formulas to convert the world coordinates of the 3D point cloud into the pixel coordinates of the 2D image.

[0011] ;

[0012] ;

[0013] in,( , ) represents the pixel coordinates of a 3D point cloud in a 2D image. , , () represents the world coordinates of a point in a 3D laser point cloud. The horizontal focal length of the 2D image acquisition camera. The vertical focal length of the 2D image acquisition camera. and The coordinates of the principal point in the two-dimensional image;

[0014] Time axis synchronization calibration uses the 3D laser point cloud acquisition time as a benchmark, and aligns the 2D image acquisition time with the point cloud acquisition time using a timestamp matching formula.

[0015] ;

[0016] in, The acquisition time after calibration of the 2D image. The acquisition time for the 3D point cloud is [time]. This is the time deviation compensation value.

[0017] Preferably, in step S2, feature extraction includes extracting elevation features of the 3D point cloud and texture features of the 2D image. Specifically, extracting the elevation features of the 3D point cloud involves calculating the elevation differences between adjacent point clouds and identifying the characteristics of 3D defects.

[0018] ;

[0019] in, For the third point cloud The point and the first The elevation difference between two adjacent points For the first Elevation coordinates of a 3D point cloud. For the first Elevation coordinates of a 3D point cloud;

[0020] when ≥ When a suspected three-dimensional defect is identified in the area, the point cloud cluster of that area is extracted as the feature set of the three-dimensional defect. This is the elevation difference threshold;

[0021] Texture feature extraction from two-dimensional images uses the SIFT algorithm to extract texture features from the image.

[0022] ;

[0023] in, This is a 128-dimensional feature vector of key points in a two-dimensional image, used to describe the texture details of planar defects. ~ These are the components of the feature vector in each dimension, with values ​​ranging from 0 to 255.

[0024] Preferably, in step S2, the specific steps of the fusion process are as follows:

[0025] S21: The extracted 3D elevation features and 2D texture features are fused to generate a fused feature set.

[0026] ;

[0027] in, This is a dual-modal fusion feature vector. This is the weighting coefficient, with a value ranging from 0.4 to 0.6. This is a 3D point cloud elevation feature vector. It is a two-dimensional image texture feature vector;

[0028] S22: The fused feature vector is then denoised and normalized. The normalization formula is as follows:

[0029] ;

[0030] in, This is the normalized fused feature vector, with each component taking values ​​ranging from 0 to 1. To find the minimum value in the fused feature vector, This represents the maximum value among the fused feature vectors.

[0031] Preferably, step S3 further includes the following step:

[0032] S31: Denoising and sharpening are performed on the acquired 2D high-definition images of the pavement to enhance the texture features of fine cracks ≥2mm. The noise reduction formula is as follows:

[0033] ;

[0034] in, For adaptive Gaussian filter template in image coordinates The filtered value at that point, These are the pixel coordinates of a two-dimensional image. The standard deviation is adaptive, with a range of 0.5 to 2.0. Pi;

[0035] The sharpening formula is:

[0036] ;

[0037] in, For the sharpened image in The pixel grayscale value at that location, For the noise-reduced image in The original pixel grayscale value at that location, , , and They are respectively The grayscale values ​​of the four adjacent pixels (top, bottom, left, and right);

[0038] S32: An improved Canny edge detection algorithm is used to extract the edge features of subtle cracks in the preprocessed image. The threshold calculation formula is as follows:

[0039] ;

[0040] ;

[0041] in, The high threshold value ranges from 80 to 120. The threshold value is 30-50. The average gray level of the image. The standard deviation of image grayscale. This is the high threshold coefficient, with a value ranging from 1.2 to 1.5. This is a low threshold coefficient, with a value ranging from 0.4 to 0.5;

[0042] The formula for connecting crack edges is:

[0043] ;

[0044] S33: Retrieve the synchronized calibrated 3D laser point cloud data, and calculate the point cloud elevation difference of the suspected fine crack area extracted in step S32 to help determine whether it is a real crack.

[0045] ;

[0046] in, The point cloud elevation difference is the value of the suspected crack area. This represents the highest elevation value of the 3D point cloud within the suspected crack area. The lowest elevation value of the 3D point cloud within the suspected crack area.

[0047] Let the threshold for crack elevation difference be... ,when ≥ If the crack is genuine and minor, it is considered a real, minor crack; otherwise, it is considered noise or stain and is discarded.

[0048] Preferably, step S4 further includes the following step:

[0049] S41: Divide the synchronized calibrated 3D laser point cloud data into sections according to pavement area, and perform noise reduction and downsampling processing on the point cloud data of each section.

[0050] ;

[0051] in, For the first The average neighborhood distance of a point cloud. The number of neighboring points. For the first The three-dimensional coordinates of a point cloud ( , , ), For the first The first point cloud The three-dimensional coordinates of the neighboring points ( , , ), For the first Point cloud and the first Euclidean distance between neighboring points;

[0052] Set the average distance threshold of the neighborhood ,when > If the point is deemed an outlier, it is removed; otherwise, it is retained as a valid point cloud.

[0053] S42: For the preprocessed effective point cloud, the elevation difference features of the misalignment and settlement are extracted using the regional elevation fitting formula and the elevation difference calculation formula. The regional elevation fitting formula is as follows:

[0054] ;

[0055] in, This represents the fitted theoretical elevation value of the pavement. and Let be the horizontal coordinates of the point cloud. and These are the fitting coefficients. For the fitting constant term;

[0056] The formula for calculating the elevation difference is:

[0057] ;

[0058] in, This represents the difference between the actual elevation of the point cloud and the fitted theoretical elevation. This represents the actual elevation value of the point cloud. The theoretical elevation value is obtained by the plane fitting formula;

[0059] S43: Set the threshold for judging three-dimensional defects, when When the thickness is ≤-2mm, it is considered a settlement defect. | represents the settlement depth; when When the thickness is ≥2mm, it is judged as misalignment defect. The height of the platform is the offset height.

[0060] The present invention has the following advantages: The present invention achieves synchronous and collaborative detection of three-dimensional and two-dimensional pavement defects by fusing two-dimensional and three-dimensional vision, making up for the dimensional defects of single vision recognition, and ensuring that all kinds of pavement defects are identified without blind spots and with full coverage; and the entire identification process is completed automatically, without the need for manual on-site judgment, and can be used in conjunction with vehicle-mounted high-speed acquisition equipment to work synchronously, which greatly improves the efficiency of defect identification, reduces on-site operation time, and avoids interference with the normal operation of the airport. Attached Figure Description

[0061] Figure 1 A schematic diagram of a collaborative identification method for multiple types of defects on airport pavement;

[0062] Figure 2 This is a schematic diagram of a two-dimensional grayscale image;

[0063] Figure 3 This is a schematic diagram of a 3D depth map. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0065] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0066] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined with each other.

[0067] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0068] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0069] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0070] In this embodiment, as Figure 1 As shown, a method for collaborative identification of multiple types of defects on airport pavement includes the following steps:

[0071] S1: Acquire raw data using the 3D laser 3D vision acquisition module and the 2D image acquisition module; specifically, such as... Figure 3 As shown, the 3D laser 3D vision acquisition module is used to acquire the 3D contour, elevation data, and morphological features of the pavement, focusing on capturing the elevation differences of three-dimensional defects such as misalignment, settlement, and potholes; for example... Figure 2As shown, the two-dimensional image acquisition module is used to capture high-definition images of the pavement surface, focusing on capturing detailed features of planar defects such as cracks, peeling, and repair marks.

[0072] S2: Synchronously calibrate, extract features, and fuse 3D laser point cloud data with 2D high-definition image data; specifically, the airport pavement dual-modal data fusion algorithm performs synchronous calibration, feature extraction, and fusion processing on 3D laser point cloud data and 2D high-definition image data. It is not a simple superposition, but rather achieves complementarity between planar disease texture features and three-dimensional disease elevation features through data fusion, ensuring full coverage identification of multiple types of diseases and avoiding the limitations of single visual recognition.

[0073] S3: Denoise and sharpen cracks in 2D images, and enhance crack features;

[0074] S4: By analyzing the elevation differences in 3D laser point cloud data, the height differences and morphological characteristics of misalignments and settlements are identified. Through the fusion of 2D and 3D visual modalities, simultaneous and collaborative detection of three-dimensional and planar defects is achieved, compensating for the dimensional limitations of single-vision recognition and ensuring comprehensive and blind-spot-free identification of all types of pavement defects. Furthermore, the entire identification process is automated, eliminating the need for manual on-site assessment. It can be used in conjunction with vehicle-mounted high-speed data acquisition equipment, significantly improving defect identification efficiency, reducing on-site operation time, and avoiding disruption to normal airport operations.

[0075] Furthermore, in step S2, the data synchronization calibration specifically involves the calibration of the three-dimensional laser point cloud data. With two-dimensional high-definition image data Spatial coordinate and time axis synchronization calibration is performed to eliminate data misalignment caused by equipment installation deviations and timing differences during acquisition. Specifically, spatial coordinate calibration uses coordinate transformation formulas to convert the world coordinates of the 3D point cloud into the pixel coordinates of the 2D image.

[0076] ;

[0077] ;

[0078] in,( , ) represents the pixel coordinates of a 3D point cloud in a 2D image. , , () represents the world coordinates of a point in a 3D laser point cloud. and The coordinates are in the horizontal direction. These are elevation coordinates (relative height of the pavement). The horizontal focal length of the 2D image acquisition camera. The vertical focal length of the 2D image acquisition camera is preset according to camera parameters, with a value ranging from 3500 to 4500 pixels, suitable for wide-area acquisition requirements of airport pavement. and These are the coordinates of the principal point of the 2D image, i.e., the coordinates of the camera's imaging center. The value is taken as half the image resolution, such as for a 1920×1080 resolution image. =960, =540, thus ensuring the accuracy of coordinate transformation.

[0079] Time axis synchronization calibration uses the 3D laser point cloud acquisition time as a benchmark, and aligns the 2D image acquisition time with the point cloud acquisition time using a timestamp matching formula.

[0080] ;

[0081] in, The acquisition time after calibration of the 2D image. The acquisition time for the 3D point cloud is [time]. This is the time deviation compensation value, ranging from -5 to 5ms. It is pre-calibrated based on the timing error of the acquisition equipment to ensure time synchronization of dual-modal data.

[0082] Furthermore, in step S2, feature extraction includes extracting the elevation features of the 3D point cloud and the texture features of the 2D image. Specifically, extracting the elevation features of the 3D point cloud involves calculating the elevation differences between adjacent point clouds and identifying the characteristics of three-dimensional defects.

[0083] ;

[0084] in, For the third point cloud The point and the first The elevation difference between two adjacent points For the first Elevation coordinates of a 3D point cloud. For the first Elevation coordinates of a 3D point cloud;

[0085] when ≥ When a suspected three-dimensional defect is identified in the area, the point cloud cluster of that area is extracted as the feature set of the three-dimensional defect. This is the elevation difference threshold;

[0086] Texture feature extraction of two-dimensional images uses the SIFT algorithm to extract texture features (edges and textures of planar defects such as cracks and peeling) from the image.

[0087] ;

[0088] in, It is a 128-dimensional feature vector of key points in a two-dimensional image, used to describe the texture details of planar defects (such as crack width and spalling range). ~ These are the components of the feature vector, with values ​​ranging from 0 to 255. They correspond to the grayscale, gradient, direction, and other texture information of the key points, and are used to distinguish between planar defects and normal pavement areas.

[0089] In this embodiment, the specific steps of the fusion process in step S2 are as follows:

[0090] S21: The extracted three-dimensional elevation features are fused with two-dimensional texture features to generate a fused feature set, enabling collaborative identification of planar and three-dimensional defects.

[0091] ;

[0092] in, This is a dual-modal fusion feature vector. This is a weighting coefficient, ranging from 0.4 to 0.6, dynamically adjusted according to the type of airport pavement defects. When identifying three-dimensional defects (misalignment, settlement), =0.6, increasing the weight of elevation features; when identifying planar defects (cracks, spalling), =0.4, increasing the weight of texture features. It is a 3D point cloud elevation feature vector, obtained by normalizing the elevation difference feature set, and its dimension is consistent with the 2D texture feature vector (128 dimensions). This is a 2D image texture feature vector, and a 128-dimensional feature vector extracted by the SIFT algorithm.

[0093] S22: The fused feature vector is denoised and normalized to remove redundant features and outliers, improving the accuracy of the fused features and providing high-quality feature support for subsequent disease identification. The normalization formula is as follows:

[0094] ;

[0095] in, This is the normalized fused feature vector, with each component taking values ​​ranging from 0 to 1. To find the minimum value in the fused feature vector, The maximum value in the fused feature vector is used. Specifically, the above steps achieve deep fusion of dual-modal data, ensuring the complementary linkage between planar disease texture features and three-dimensional disease elevation features, completely solving the problem of insufficient single visual recognition dimension, and laying the foundation for full coverage and high-precision identification of multiple types of airport pavement diseases.

[0096] In this embodiment, to address the requirement of identifying minute hidden cracks ≥2mm, the image enhancement and feature extraction algorithms are optimized. Noise reduction and sharpening processes are applied to the minute cracks in the two-dimensional image to enhance crack features. Simultaneously, three-dimensional point cloud data is used to assist in the judgment, avoiding missed detection of minute cracks. Step S3 also includes the following steps:

[0097] S31: Denoising and sharpening are performed on the acquired 2D high-definition images of the pavement to enhance the texture features of fine cracks ≥2mm. The noise reduction formula is as follows:

[0098] ;

[0099] in, For adaptive Gaussian filter template in image coordinates The filtered value at that point is used to achieve image noise reduction while preserving details of subtle cracks; These are the pixel coordinates of a two-dimensional image. To adapt the standard deviation, the value ranges from 0.5 to 2.0, based on the image. The grayscale variance is dynamically adjusted at a certain point. When the grayscale variance is large (crack edge area), Use a value of 0.5 to 1.0 to preserve edge details; when the grayscale variance is small (in areas with smooth pavement), Set the value to 1.0~2.0 to enhance noise reduction. Pi, with a value of approximately 3.1416, is used for calculating the Gaussian filter template.

[0100] The sharpening formula is:

[0101] ;

[0102] in, For the sharpened image in The pixel grayscale value at that location, For the noise-reduced image in The original pixel grayscale value at that location, , , and They are respectively The gray values ​​of the four adjacent pixels (top, bottom, left, and right) are used to enhance the crack edges by the gray value difference between adjacent pixels, making fine cracks ≥2mm more clearly visible.

[0103] S32: An improved Canny edge detection algorithm is used to extract the edge features of subtle cracks in the preprocessed image. The threshold calculation formula is as follows:

[0104] ;

[0105] ;

[0106] in, A high threshold, ranging from 80 to 120, is used to filter pixels with clearly defined crack edges. A low threshold, ranging from 30 to 50, is used to filter pixels suspected of being at the edge of a crack. The average grayscale value reflects the overall brightness of the image and is calculated by averaging the grayscale values ​​of all pixels in the image. The standard deviation of image gray levels reflects the dispersion of the image's gray level distribution and is used to dynamically adjust the threshold to adapt to images under different lighting conditions. This is the high threshold coefficient, with a value ranging from 1.2 to 1.5. It is used to control the difference between the high threshold and the grayscale mean, adapting to the grayscale characteristics of fine crack edges. The low threshold coefficient is set to 0.4~0.5 to ensure a reasonable ratio between the low and high thresholds and avoid missing minute cracks.

[0107] The formula for connecting crack edges is:

[0108] ;

[0109] The edge identifier of a pixel is 1, which indicates a crack edge pixel and 0 indicates a non-edge pixel.

[0110] S33: Retrieve the synchronized calibrated 3D laser point cloud data, and calculate the point cloud elevation difference of the suspected fine crack area extracted in step S32 to help determine whether it is a real crack.

[0111] ;

[0112] in, The point cloud elevation difference is the value of the suspected crack area. This represents the highest elevation value of the 3D point cloud within the suspected crack area. The lowest elevation value of the 3D point cloud within the suspected crack area.

[0113] Let the threshold for crack elevation difference be... ,when ≥ If the crack is real and minute, it is considered a genuine minor crack; otherwise, it is considered noise or stain and discarded to avoid missed detections and misjudgments.

[0114] Furthermore, step S4 also includes the following steps:

[0115] S41: The synchronized calibrated 3D laser point cloud data is divided into sections according to the pavement area. The point cloud data of each section is denoised and downsampled to remove aerial noise and abnormal points caused by equipment vibration, thus retaining the effective point cloud of the pavement.

[0116] ;

[0117] in, For the first The average neighborhood distance of a point cloud. The number of neighboring points. For the first The three-dimensional coordinates of a point cloud ( , , ), For the first The first point cloud The three-dimensional coordinates of the neighboring points ( , , ), For the first Point cloud and the first Euclidean distance between neighboring points;

[0118] Set the average distance threshold of the neighborhood The value is 5-10mm, adapted to the point cloud density of airport pavement. > If the point is deemed an outlier, it is removed; otherwise, it is retained as a valid point cloud.

[0119] S42: For the preprocessed effective point cloud, the elevation difference features of the misalignment and settlement are extracted using the regional elevation fitting formula and the elevation difference calculation formula. The regional elevation fitting formula is as follows:

[0120] ;

[0121] in, This represents the fitted theoretical elevation value of the pavement. and Let be the horizontal coordinates of the point cloud. and The fitting coefficients reflect the slope trend of the pavement and are obtained through least squares fitting. This is the fitting constant term, reflecting the offset of the pavement reference elevation;

[0122] The formula for calculating the elevation difference is:

[0123] ;

[0124] in, This represents the difference between the actual elevation of the point cloud and the fitted theoretical elevation. This represents the actual elevation value of the point cloud. The theoretical elevation value is obtained by the plane fitting formula;

[0125] S43: Set the threshold for judging three-dimensional defects, when When the thickness is ≤-2mm, it is considered a settlement defect. | represents the settlement depth; when When the thickness is ≥2mm, it is judged as misalignment defect. The height of the misalignment is determined; simultaneously, point cloud clusters of the affected area are extracted, and the length and width of the clusters are calculated (using horizontal coordinates). , The invention employs extreme value difference calculation to accurately quantify three-dimensional defects and improve recognition accuracy. Specifically, it constructs a two-dimensional collaborative recognition logic for planar and three-dimensional defects. First, planar defects are identified through two-dimensional images, while three-dimensional defects are identified simultaneously through three-dimensional laser point clouds. Then, a fusion algorithm is used to match the two types of defect data, clarifying the distribution relationship between planar and three-dimensional defects in the same area, forming a complete defect recognition map, ensuring no blind spots in detection. Based on the fused dual-modal data, the invention automatically extracts defect dimensions (length, width, depth, height difference) and morphological features. Combined with airport pavement defect level assessment standards, the invention automatically completes the defect severity classification and outputs a standardized quantitative assessment report without manual intervention.

[0126] To address the different pavement characteristics of airport runways, taxiways, and aprons, we optimized the recognition algorithm parameters. For example, to meet the high flatness requirements of runways, we improved the recognition accuracy of misalignment and settlement; to address the characteristic of aprons with many repair marks, we optimized the feature differentiation algorithm to avoid confusing repair marks with defects; at the same time, we adapted to the complex outdoor lighting and severe weather conditions of airports, and optimized the image acquisition and recognition algorithms to ensure recognition stability.

[0127] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for collaborative identification of multiple types of defects on airport pavement, characterized in that: Includes the following steps: S1: Acquire raw data based on the 3D laser 3D vision acquisition module and the 2D image acquisition module; S2: Simultaneously calibrate, extract features, and fuse 3D laser point cloud data with 2D high-definition image data; S3: Denoise and sharpen cracks in 2D images, and enhance crack features; S4: Identify the height differences and morphological characteristics of misalignment and settlement by analyzing the elevation differences of 3D laser point cloud data.

2. The method for collaborative identification of multiple types of airport pavement defects according to claim 1, characterized in that: In step S1, the three-dimensional laser 3D vision acquisition module is used to acquire the three-dimensional contour, elevation data and morphological features of the pavement; the two-dimensional image acquisition module is used to acquire high-definition images of the pavement surface.

3. The method for collaborative identification of multiple types of airport pavement defects according to claim 2, characterized in that: In step S2, data synchronization calibration specifically involves calibrating the three-dimensional laser point cloud data. With two-dimensional high-definition image data Spatial coordinate and time axis synchronization calibration is performed. Spatial coordinate calibration uses coordinate transformation formulas to convert the world coordinates of the 3D point cloud into the pixel coordinates of the 2D image. ； ； in,( , ) represents the pixel coordinates of a 3D point cloud in a 2D image. , , () represents the world coordinates of a point in a 3D laser point cloud. The horizontal focal length of the 2D image acquisition camera. The vertical focal length of the 2D image acquisition camera. and The coordinates of the principal point in the two-dimensional image; Time axis synchronization calibration uses the 3D laser point cloud acquisition time as a benchmark, and aligns the 2D image acquisition time with the point cloud acquisition time using a timestamp matching formula. ； in, The acquisition time after calibration of the 2D image. The acquisition time for the 3D point cloud is [time]. This is the time deviation compensation value.

4. The method for collaborative identification of multiple types of airport pavement defects according to claim 3, characterized in that: In step S2, feature extraction includes extracting elevation features of the 3D point cloud and texture features of the 2D image. Specifically, extracting the elevation features of the 3D point cloud involves calculating the elevation differences between adjacent point clouds and identifying the characteristics of three-dimensional defects. ； in, For the third point cloud The point and the first The elevation difference between two adjacent points For the first Elevation coordinates of a 3D point cloud. For the first Elevation coordinates of a 3D point cloud; when ≥ When a suspected three-dimensional defect is identified in the area, the point cloud cluster of that area is extracted as the feature set of the three-dimensional defect. This is the elevation difference threshold; Texture feature extraction from two-dimensional images uses the SIFT algorithm to extract texture features from the image. ； in, This is a 128-dimensional feature vector of key points in a two-dimensional image, used to describe the texture details of planar defects. ~ These are the components of the feature vector in each dimension, with values ​​ranging from 0 to 255.

5. The method for collaborative identification of multiple types of airport pavement defects according to claim 4, characterized in that: In step S2, the specific steps of the fusion process are as follows: S21: The extracted 3D elevation features and 2D texture features are fused to generate a fused feature set. ； in, This is a dual-modal fusion feature vector. This is the weighting coefficient, with a value ranging from 0.4 to 0.

6. This is a 3D point cloud elevation feature vector. It is a two-dimensional image texture feature vector; S22: The fused feature vector is then denoised and normalized. The normalization formula is as follows: ； in, This is the normalized fused feature vector, with each component taking values ​​ranging from 0 to 1. To find the minimum value in the fused feature vector, This represents the maximum value among the fused feature vectors.

6. The method for collaborative identification of multiple types of airport pavement defects according to claim 5, characterized in that: Step S3 further includes the following steps: S31: Denoising and sharpening are performed on the acquired 2D high-definition images of the pavement to enhance the texture features of fine cracks ≥2mm. The noise reduction formula is as follows: ； in, For adaptive Gaussian filter template in image coordinates The filtered value at that point, These are the pixel coordinates of a two-dimensional image. The standard deviation is adaptive, with a range of 0.5 to 2.

0. Pi; The sharpening formula is: ； in, For the sharpened image in The pixel grayscale value at that location, For the noise-reduced image in The original pixel grayscale value at that location, , , and They are respectively The grayscale values ​​of the four adjacent pixels (top, bottom, left, and right); S32: An improved Canny edge detection algorithm is used to extract the edge features of subtle cracks in the preprocessed image. The threshold calculation formula is as follows: ； ； in, The high threshold value ranges from 80 to 120. The threshold value is 30-50. The average gray level of the image. The standard deviation of image grayscale. This is the high threshold coefficient, with a value ranging from 1.2 to 1.

5. This is a low threshold coefficient, with a value ranging from 0.4 to 0.5; The formula for connecting crack edges is: ； S33: Retrieve the synchronized calibrated 3D laser point cloud data, and calculate the point cloud elevation difference of the suspected fine crack area extracted in step S32 to help determine whether it is a real crack. ； in, The point cloud elevation difference is the value of the suspected crack area. This represents the highest elevation value of the 3D point cloud within the suspected crack area. The lowest elevation value of the 3D point cloud within the suspected crack area. Let the threshold for crack elevation difference be... ,when ≥ If the crack is genuine and minor, it is considered a real, minor crack; otherwise, it is considered noise or stain and is discarded.

7. The method for collaborative identification of multiple types of airport pavement defects according to claim 6, characterized in that: Step S4 also includes the following steps: S41: Divide the synchronized calibrated 3D laser point cloud data into sections according to pavement area, and perform noise reduction and downsampling processing on the point cloud data of each section. ； in, For the first The average neighborhood distance of a point cloud. The number of neighboring points. For the first The three-dimensional coordinates of a point cloud ( , , ), For the first The first point cloud The three-dimensional coordinates of the neighboring points ( , , ), For the first Point cloud and the first Euclidean distance between neighboring points; Set the average distance threshold of the neighborhood ,when > If the point is deemed an outlier, it is removed; otherwise, it is retained as a valid point cloud. S42: For the preprocessed effective point cloud, the elevation difference features of the misalignment and settlement are extracted using the regional elevation fitting formula and the elevation difference calculation formula. The regional elevation fitting formula is as follows: ； in, This represents the fitted theoretical elevation value of the pavement. and Let be the horizontal coordinates of the point cloud. and These are the fitting coefficients. For the fitting constant term; The formula for calculating the elevation difference is: ； in, This represents the difference between the actual elevation of the point cloud and the fitted theoretical elevation. This represents the actual elevation value of the point cloud. The theoretical elevation value is obtained by the plane fitting formula; S43: Set the threshold for judging three-dimensional defects, when When the thickness is ≤-2mm, it is considered a settlement defect. | represents the settlement depth; when When the thickness is ≥2mm, it is judged as misalignment defect. The height of the platform is the offset height.

Images