A method for detecting the absence of a joint filler material in an airport cement runway based on a depth camera

Abstract

The application discloses an airport cement pavement joint filling material missing detection method based on a depth camera, S1: a two-dimensional contour area of a plate joint is obtained; S2: morphological dilation processing is performed on the output plate joint contour; S3: a transverse and longitudinal skeleton line is fitted, and contour deviation is corrected; S4: mask processing is performed on a depth map; S5: a transverse depth curve and a longitudinal depth curve are obtained; S6: a random forest classification algorithm is used to identify the depth curve; S7: each curve is classified; S8: for a direction with a missing risk, all missing feature curves in the direction are extracted, and a extreme value method is used to calculate a recess depth; and S9: a visual detection report is generated. Through machine learning algorithm and depth analysis processing on the pavement three-dimensional point cloud data collected by the depth camera, automatic detection, depth calculation and personalized judgment of the joint filling material missing are realized, and parameters such as missing positions and depths are output, so that the detection efficiency is improved, and the overall detection cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of cement pavement technology, and in particular to a method for detecting missing joint filler material in airport cement pavement based on a depth camera. Background Technology

[0002] Missing grout refers to the phenomenon where the grout material, originally intended for sealing, buffering, and preventing debris intrusion, in the gaps between airport concrete pavement slabs, partially or completely detaches or wears out due to long-term exposure to aircraft loads, environmental temperature and humidity changes, aging, and wear. This results in gaps within the pavement surface. If this defect is not detected and repaired in a timely manner, rainwater, silt, and other impurities can easily seep into the base layer at the bottom of the gap, exacerbating uneven settlement of the pavement slabs and even causing more serious pavement damage such as slab misalignment and breakage, directly threatening the safety and comfort of aircraft takeoff and landing.

[0003] In the field of grout missing detection, traditional manual inspection was the commonly used detection method in the early days. Inspectors had to carry simple tools (such as rulers, depth gauges, etc.) to inspect the pavement on foot, observe the grout condition of each joint, and manually record the defects. The drawbacks are: (1) the efficiency of manual inspection is extremely low, which is difficult to meet the airport's needs for rapid inspection and timely maintenance of the pavement; (2) the results of manual inspection are highly dependent on the experience judgment of the inspectors. For the measurement of the fine parts or depth of missing grout, subjective errors and tool precision limitations can easily lead to inaccurate measurement data, which cannot provide accurate defect parameters for subsequent repair. In order to solve the efficiency problem of manual inspection, existing technologies have gradually introduced automated detection methods such as pattern matching and AI recognition. These methods collect pavement images and use preset grout missing image templates (pattern matching) or trained AI models (such as convolutional neural networks) to analyze the images and determine whether there is missing grout in the target area. The shortcomings of this method are: (1) It cannot quantify the depth of the missing material: The missing material is not a defect independent of the board joint, but is attached to the state of the board joint. Its degree of harm depends not only on the missing area, but also directly on the missing depth. However, pattern matching and AI recognition can only judge the "presence / absence" of missing material through image grayscale and texture differences, and cannot measure the depth of missing material; (2) The threshold cannot be adaptively adjusted: Due to differences in construction standards, service life, and aircraft take-off and landing frequency, the criteria for judging missing material (i.e. the threshold of "whether it needs to be repaired") of cement pavement of different airports are significantly different. For example, due to the small aircraft load, a small airport pavement may allow shallow local missing material; while large hub airports have stricter requirements for the integrity of the missing material due to the large tonnage and high frequency of take-off and landing aircraft, and shallow missing material must also be judged as a defect. However, the judgment thresholds of existing pattern matching and AI recognition technologies are mostly fixed, and can only output "yes / no" results based on a unified standard. They cannot customize and adjust the judgment thresholds according to the actual needs of different airport pavements, which limits the applicability of the detection results and fails to meet the personalized detection needs of different airports. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for detecting missing sealant in airport cement pavement based on a depth camera.

[0005] The objective of this invention is achieved through the following technical solution: a method for detecting missing sealant in airport cement pavement based on a depth camera, comprising the following steps:

[0006] S1: Based on the 3D point cloud data of the airport cement pavement collected by the depth camera, the pavement image is segmented at the pixel level using a pre-trained semantic segmentation model to obtain the 2D contour region of the slab seam.

[0007] S2: Perform morphological expansion processing on the output board seam contour;

[0008] S3: Fit the horizontal and vertical skeleton lines to correct contour deviations;

[0009] S4: Apply a mask to the depth map;

[0010] S5: The generated mask is depth sampled at fixed intervals to obtain the horizontal depth curve and the vertical depth curve;

[0011] S6: Use the random forest classification algorithm to identify depth curves;

[0012] S7: Input the horizontal depth curve and the vertical depth curve into the trained random forest classification model, and classify each curve one by one;

[0013] S8: For directions with a risk of missing features, extract all missing feature curves in that direction and use the extreme value method to calculate the depth of the depression;

[0014] S9: Generate a visual inspection report.

[0015] Preferably, in step S2, a 5×5 pixel rectangular structural element is used to extend the outline boundary of the board seam outward, with the extension range controlled within 2-3 mm, to obtain a complete analysis area for the main body and transition area of ​​the board seam.

[0016] Preferably, in step S3, a skeleton extraction algorithm is used to remove redundant edge pixels to obtain the central skeleton of the board seam. Then, the preliminary skeleton is fitted with a straight line using the least squares method to generate horizontal skeleton lines. and longitudinal skeleton line .

[0017] Preferably, in step S4, the fitted transverse skeleton line is used. and longitudinal skeleton line Based on this, the width is set to... Within the analysis area, a fixed-width effective detection zone for the board seam is defined, and the depth map is masked to retain only the depth data of the board seam area, thus obtaining a transverse board seam mask. and longitudinal plate seam mask .

[0018] Preferably, in step S6, the training steps for the random forest model are as follows:

[0019] S61: Collect depth curve samples of caulking material from different airports and with different degrees of missing material, as well as depth curve samples of normal caulking areas, and label them as missing and normal, dividing them into training and test sets in a 7:3 ratio.

[0020] S62: Preprocess the depth curve;

[0021] S63: Set the number of decision trees to 100, the maximum tree depth to 10, and the minimum number of sample splits to 5. Use the training set to train the random forest model.

[0022] Preferably, in step S62, the preprocessing includes curve smoothing, normalization, and left / right flipping.

[0023] Preferably, in step S7, if the classification result of the depth curve is missing, then the curve is marked as a missing feature curve; the number of missing feature curves in the horizontal and vertical directions are counted respectively, and if the number of missing feature curves is... ,in If the total number of samples is counted, then it is determined that there is a risk of missing sealant in the joint.

[0024] Preferably, step S8 further includes the following step:

[0025] S81: Determine the baseline depth for a single missing feature curve.

[0026] ;

[0027] in, The left end of the curve Depth data for each point; The right end of the curve Depth data for each point;

[0028] S82: Find the minimum depth value in the curve ;

[0029] S83: Calculate the depth of the depression.

[0030] ;

[0031] like If the condition is met, it is determined that there is a missing sealant in the board seam that needs repair, and the location and depth of the missing sealant are output; if If the seam is deemed unrepairable, only the data is recorded without triggering a repair warning. The threshold for determining missing sealant.

[0032] The present invention has the following advantages: By using machine learning algorithms and depth analysis to process the three-dimensional point cloud data of the pavement collected by the depth camera, the present invention realizes automatic detection, depth calculation and personalized judgment of missing sealant, and outputs parameters such as missing location and depth, thereby improving detection efficiency and reducing overall detection cost. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the process for detecting missing sealant in airport cement pavement based on a depth camera. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0035] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0036] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined with each other.

[0037] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0038] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0039] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0040] In this embodiment, as Figure 1As shown, a method for detecting missing sealant in airport cement pavement based on a depth camera includes the following steps:

[0041] S1: Based on the 3D point cloud data of the airport cement pavement collected by the depth camera, the pavement image is segmented at the pixel level using a pre-trained semantic segmentation model to obtain the 2D contour region of the slab seam; specifically, the semantic segmentation model here is a network such as U-Net or segformer.

[0042] S2: Perform morphological dilation processing on the output board seam contour; specifically, in order to avoid incomplete contours at the board seam edges due to image noise and pavement stains, perform morphological dilation processing on the board seam contour output in step S1. Specifically, use a 5×5 pixel rectangular structural element to expand the board seam contour boundary outward, with the expansion range controlled within 2-3mm, to obtain a complete analysis area for the board seam body and transition area.

[0043] S3: Fitting horizontal and vertical skeleton lines to correct contour deviations; specifically, a skeleton extraction algorithm is used to remove redundant edge pixels to obtain the central skeleton of the board seam, and then the preliminary skeleton is fitted with straight lines using the least squares method to generate horizontal skeleton lines respectively. and longitudinal skeleton line This step can correct local deviations caused by misidentification or contour breakage, ensuring that the subsequent sampling direction is completely matched with the actual direction of the plate seam, and avoiding errors caused by sampling offset.

[0044] S4: Mask the depth map; specifically, use the fitted lateral skeleton lines... and longitudinal skeleton line Based on this, the width is set to... Within the analysis area, a fixed-width effective detection zone for the board seam is defined, and the depth map is masked to retain only the depth data of the board seam area, thus obtaining a transverse board seam mask. and longitudinal plate seam mask The main purpose of this step is to limit the range of subsequent depth sampling and eliminate interference from the non-seam area of ​​the pavement panel.

[0045] S5: The generated mask is depth sampled at fixed intervals to obtain horizontal and vertical depth curves; specifically, the mask generated in step S4... , Depth sampling is performed at fixed intervals, and n depth curves are collected respectively. and Each curve is laid out perpendicular to the direction of the plate seam extension, covering the mask. , Full width, curve and The length 's' corresponds to the mask width, and each curve is a one-dimensional depth vector of length 's'. Cross-sampling in both the horizontal and vertical directions achieves full-coverage depth data acquisition of the board seam area, avoiding missing or undetected data due to sampling in only one direction.

[0046] S6: Use the random forest classification algorithm to identify depth curves; specifically, the training steps for the random forest model are as follows:

[0047] S61: Collect depth curve samples of caulking material from different airports and with different degrees of missing material, as well as depth curve samples of normal caulking areas, and label them as missing and normal, dividing them into training and test sets in a 7:3 ratio.

[0048] S62: Preprocess the depth curve; further, the preprocessing includes curve smoothing, normalization, and left / right flipping.

[0049] S63: Set the number of decision trees to 100, the maximum tree depth to 10, and the minimum number of sample splits to 5. Train the random forest model using the training set and verify the model accuracy using the test set. After ensuring that the classification accuracy is ≥95%, save the model for subsequent missing data identification. Specifically, the random forest model only needs to be trained once. After the model is trained, subsequent preprocessing of the depth curve is all that is required.

[0050] S7: Input the horizontal and vertical depth curves into the trained random forest classification model, and classify each curve one by one; specifically, if the classification result of the depth curve is missing, then mark the curve as a missing feature curve; count the number of missing feature curves in the horizontal and vertical directions respectively, and if the number of missing feature curves is... ,in If the total number of samples is zero, then the joint is considered to have a risk of missing sealant. The threshold value set here is... Based on the results of extensive experimental verification, this threshold can balance the false negative rate and the false positive rate, avoiding misjudgment caused by a small amount of noise curves and preventing risk omissions caused by local missing detections.

[0051] S8: For directions with a risk of missing features, extract all missing feature curves in that direction and calculate the depression depth using the extreme value method; furthermore, step S8 also includes the following steps:

[0052] S81: Determine the baseline depth for a single missing feature curve.

[0053] ;

[0054] in, The left end of the curve Depth data for each point (10% of the depth curve in length); The right end of the curve Depth data for each point (length equal to 10% of the depth curve); both ends are normal pavement or normal joint filling areas, with stable depth;

[0055] S82: Find the minimum depth value in the curve ;

[0056] S83: Calculate the depth of the depression.

[0057] ;

[0058] like If the condition is met, it is determined that there is a missing sealant in the board seam that needs repair, and the location and depth of the missing sealant are output; if If the seam is deemed unrepairable, only the data is recorded without triggering a repair warning. A threshold for determining missing sealant is established. Specifically, by correcting the reference depth, the depth reference deviation caused by the overall slope of the pavement and the camera shooting angle in different slab joint areas is eliminated, ensuring the accuracy of the depression depth calculation results.

[0059] S9: Generates visualized inspection reports, providing airport operations and maintenance personnel with intuitive and accurate repair decision-making support. By processing the 3D point cloud data of the pavement collected by depth cameras using machine learning algorithms and depth analysis, it achieves automatic detection, depth calculation, and personalized determination of missing sealant, outputting parameters such as the location and depth of the missing material, thereby improving inspection efficiency and reducing overall inspection costs.

[0060] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting missing joint filler in airport cement pavement based on a depth camera, characterized in that: Includes the following steps: S1: Based on the 3D point cloud data of the airport cement pavement collected by the depth camera, the pavement image is segmented at the pixel level using a pre-trained semantic segmentation model to obtain the 2D contour region of the slab seam. S2: Perform morphological expansion processing on the output board seam contour; S3: Fit the horizontal and vertical skeleton lines to correct contour deviations; S4: Apply a mask to the depth map; S5: The generated mask is depth sampled at fixed intervals to obtain the horizontal depth curve and the vertical depth curve; S6: Use the random forest classification algorithm to identify depth curves; S7: Input the horizontal depth curve and the vertical depth curve into the trained random forest classification model, and classify each curve one by one; S8: For directions with a risk of missing features, extract all missing feature curves in that direction and use the extreme value method to calculate the depth of the depression; S9: Generate a visual inspection report.

2. The method for detecting missing sealant in airport cement pavement based on a depth camera according to claim 1, characterized in that: In step S2, a 5×5 pixel rectangular structural element is used to extend the outline boundary of the board seam outward, with the extension range controlled within 2-3 mm, to obtain a complete analysis area for the main body and transition area of ​​the board seam.

3. The method for detecting missing sealant in airport cement pavement based on a depth camera according to claim 2, characterized in that: In step S3, a skeleton extraction algorithm is used to remove redundant edge pixels to obtain the central skeleton of the board seam. Then, the preliminary skeleton is fitted with a straight line using the least squares method to generate the horizontal skeleton lines. and longitudinal skeleton line .

4. The method for detecting missing sealant in airport cement pavement based on a depth camera according to claim 3, characterized in that: In step S4, the fitted transverse skeleton line and longitudinal skeleton line Based on this, the width is set to... Within the analysis area, a fixed-width effective detection zone for the board seam is defined, and the depth map is masked to retain only the depth data of the board seam area, thus obtaining a transverse board seam mask. and longitudinal plate seam mask .

5. The method for detecting missing sealant in airport cement pavement based on a depth camera according to claim 4, characterized in that: In step S6, the training steps for the random forest model are as follows: S61: Collect depth curve samples of caulking material from different airports and with different degrees of missing material, as well as depth curve samples of normal caulking areas, and label them as missing and normal, dividing them into training and test sets in a 7:3 ratio. S62: Preprocess the depth curve; S63: Set the number of decision trees to 100, the maximum tree depth to 10, and the minimum number of sample splits to 5. Use the training set to train the random forest model.

6. The method for detecting missing sealant in airport cement pavement based on a depth camera according to claim 5, characterized in that: In step S62, the preprocessing includes curve smoothing, normalization, and left-right flipping.

7. The method for detecting missing sealant in airport cement pavement based on a depth camera according to claim 6, characterized in that: In step S7, if the classification result of the depth curve is missing, then the curve is marked as a missing feature curve; the number of missing feature curves in the horizontal and vertical directions are counted respectively, and if the number of missing feature curves is... ,in If the total number of samples is counted, then it is determined that there is a risk of missing sealant in the joint.

8. The method for detecting missing sealant in airport cement pavement based on a depth camera according to claim 7, characterized in that: Step S8 further includes the following steps: S81: Determine the baseline depth for a single missing feature curve. ； in, The left end of the curve Depth data for each point; The right end of the curve Depth data for each point; S82: Find the minimum depth value in the curve ; S83: Calculate the depth of the depression. ； like If the condition is met, it is determined that there is a missing sealant in the board seam that needs repair, and the location and depth of the missing sealant are output; if If the seam is deemed unrepairable, only the data is recorded without triggering a repair warning. The threshold for determining missing sealant.

Images