Cement pavement slab staggering detection method based on depth camera

Abstract

The invention discloses a cement pavement slab staggering detection method based on a depth camera, and the method comprises the steps: S1, collecting pavement data, and obtaining a slab joint mask; s2, performing expansion processing on the slab seam mask to obtain a slab seam area, and obtaining a depth map of the slab seam area; s3, sampling a plurality of depth curves in the transverse direction and the longitudinal direction of the depth map; s4, preprocessing the depth curve, and training a random forest classification model; s5, performing identification and classification by using a random forest classification model, and judging that slab staggering features exist on the depth curve when the slab staggering category is the slab staggering category; and S6, when the slab staggering is judged to exist, extracting the slab staggering area and depth in the direction in which the slab staggering exists. The pavement data acquired by the depth camera carried by the mobile equipment is analyzed and processed in combination with machine learning and depth analysis, so that the position and depth of the error platform can be quickly and accurately detected, a manual inspection mode is replaced, optical fibers do not need to be buried, the cost is reduced, and large-scale application is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of pavement detection, and in particular to a cement pavement misalignment detection method based on a depth camera. Background Art

[0002] Misalignment is caused by permanent vertical deformation of the foundation, roadbed or base layer, which manifests as relative vertical displacement of the two plates at the joints or cracks of the cement concrete pavement, thus affecting the flatness of the pavement and posing certain safety hazards. At present, manual inspections are often used to detect misalignment, using a ruler or triangle for field measurements. This method is not only inefficient, but also easily affected by the operator's skills and experience, resulting in certain errors in the measurement. In recent years, there have also been monitoring of misalignment based on fiber optic sensing technology, that is, by burying optical fibers in the pavement structure layer to monitor the stress and deformation at the pavement joints, but this method consumes huge resources and is difficult to apply on a large scale. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a cement pavement misalignment detection method based on a depth camera.

[0004] The purpose of the present invention is achieved through the following technical solution: A cement pavement misalignment detection method based on a depth camera, comprising the following steps: S1: The mobile device is equipped with a depth camera to collect road surface data, identify the board seams in the grayscale image, and obtain the board seam mask; S2: dilate the plate seam mask to obtain the plate seam area, then perform mask processing on the depth map according to the plate seam mask, retain only the depth data of the plate seam area, resample the depth map to have the same height and width, and obtain the depth map of the plate seam area; S3: sampling a plurality of depth curves at fixed intervals in the horizontal and vertical directions of the depth map; S4: preprocess the depth curve and train the random forest classification model; S5: Use the random forest classification model to perform identification and classification. When the classification result is the misalignment category, it is determined that there is a misalignment feature on the depth curve; S6: When it is determined that there is a misalignment in the depth in the horizontal or vertical direction, the misalignment area and depth are extracted in the direction where the misalignment exists.

[0005] Preferably, in step S1, the board seams in the grayscale image are identified by a semantic segmentation algorithm.

[0006] Preferably, in step S4, the training steps of the random forest classification model are: S41: collecting a number of depth images with misalignment, marking the depth curves into two categories: misalignment and non-misalignment, to form a training data set; S42: Smoothing and normalizing the depth curve; S43: Perform data augmentation on the preprocessed depth curve and use it as input to train the random forest classification model.

[0007] Preferably, in step S43, data enhancement includes left-right flipping and up-down flipping.

[0008] Preferably, in step S6, the steps of extracting the misalignment area and depth are: S61: skeletonize the plate seam mask and extract the horizontal or vertical skeleton line according to the direction of the misalignment; S62: Use Ransac to perform straight line fitting on the points on the skeleton line in the horizontal and vertical directions respectively, to obtain the separated plate seam straight lines in the horizontal and vertical directions; S63: expanding the plate seam straight line to obtain separated plate seam areas in the transverse direction and the longitudinal direction; S64: Calculate the mean depths on both sides of the center line of the plate seam area where there is misalignment, and take the difference between the mean depths of the areas on both sides as the misalignment depth. If the misalignment depth is greater than the set threshold, it is determined that there is misalignment in the plate seam area in this direction, and then extract the contour of the plate seam area as the misalignment position.

[0009] The present invention has the following advantages: the present invention uses a mobile device equipped with a depth camera to collect pavement data, and then combines machine learning with depth analysis for analysis and processing, so as to quickly and accurately detect the position and depth of the error platform, replacing the manual inspection method. At the same time, there is no need to bury optical fibers, which reduces costs and facilitates large-scale applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a structural schematic diagram of the cement pavement misalignment detection method process. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0012] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0014] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0015] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use, or the positions or positional relationships commonly understood by those skilled in the art, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0016] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0017] In this embodiment, if Figure 1 As shown, a cement pavement misalignment detection method based on a depth camera comprises the following steps: S1: The mobile device is equipped with a depth camera to collect road surface data, identify the board seams in the grayscale image, and obtain the board seam mask; S2: dilate the plate seam mask to obtain the plate seam area, then mask the depth map according to the plate seam mask, retain only the depth data of the plate seam area, resample the depth map to have the same height and width, and obtain the depth map of the plate seam area; preferably, resample the depth map to have the same height and width, and its height and width are .

[0018] S3: sampling a plurality of depth curves at fixed intervals in the horizontal and vertical directions of the depth map; specifically, sampling a plurality of depth curves at fixed intervals in the horizontal and vertical directions of the depth map A depth curve is a length of One-dimensional vector, to obtain the depth curve in the horizontal and vertical directions and , .

[0019] S4: preprocess the depth curve and train the random forest classification model; S5: Use the random forest classification model for identification and classification. When the classification result is misalignment, it is determined that there are misalignment features on the depth curve. Specifically, for all depth curves in the horizontal or vertical direction, if the number of depth curves with misalignment features is greater than *0.4, it is judged that there is a misalignment in this direction.

[0020] S6: When it is determined that there is a misalignment in the horizontal or vertical depth, the misalignment area and depth are extracted in the direction of the misalignment. The pavement data collected by the mobile device equipped with a depth camera is analyzed and processed in combination with machine learning and depth analysis, so that the location and depth of the misalignment can be detected quickly and accurately, replacing the manual inspection method. At the same time, there is no need to bury optical fibers, which reduces costs and facilitates large-scale application.

[0021] Furthermore, in step S1, the board seams in the grayscale image are identified by a semantic segmentation algorithm. Specifically, the semantic segmentation algorithm is selected from Unet or Deeplabv3.

[0022] Furthermore, in step S4, the training steps of the random forest classification model are: S41: collecting a number of depth images with misalignment, marking the depth curves into two categories: misalignment and non-misalignment, to form a training data set; S42: Smoothing and normalizing the depth curve; S43: Perform data enhancement on the preprocessed depth curve and use it as input to train the random forest classification model. Preferably, the data enhancement includes left-right flipping and up-down flipping. Specifically, this step only needs to be performed once, that is, after the random forest classification model is trained, only the depth curve needs to be preprocessed.

[0023] In this embodiment, in step S6, the steps of extracting the misalignment area and depth are: S61: skeletonize the plate seam mask and extract the horizontal or vertical skeleton line according to the direction of the misalignment; S62: Use Ransac to perform straight line fitting on the points on the skeleton line in the horizontal and vertical directions respectively, to obtain the separated plate seam straight lines in the horizontal and vertical directions; S63: expanding the plate seam straight line to obtain separated plate seam areas in the transverse direction and the longitudinal direction; S64: Calculate the mean depths on both sides of the center line of the plate seam area where there is misalignment, and take the difference between the mean depths of the two areas as the misalignment depth. If the misalignment depth is greater than the set threshold, it is determined that there is misalignment in the plate seam area in this direction, and then extract the contour of the plate seam area as the misalignment position. Specifically, the threshold is the industry standard threshold. For details, please refer to the determination of the severity of misalignment in the "Technical Specifications for Civil Airport Pavement Evaluation Management", which will not be repeated here.

[0024] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A cement pavement misalignment detection method based on a depth camera, characterized in that: The following steps are involved: S1: The mobile device is equipped with a depth camera to collect road surface data, identify the board seams in the grayscale image, and obtain the board seam mask; S2: dilate the plate seam mask to obtain the plate seam area, then perform mask processing on the depth map according to the plate seam mask, retain only the depth data of the plate seam area, resample the depth map to have the same height and width, and obtain the depth map of the plate seam area; S3: sampling a plurality of depth curves at fixed intervals in the horizontal and vertical directions of the depth map; S4: preprocess the depth curve and train the random forest classification model; S5: Use the random forest classification model to perform identification and classification. When the classification result is the misalignment category, it is determined that there is a misalignment feature on the depth curve; S6: When it is determined that there is a misalignment in the depth in the horizontal or vertical direction, the misalignment area and depth are extracted in the direction where the misalignment exists.

2. The cement pavement misalignment detection method based on a depth camera according to claim 1, characterized in that: In the step S1, the board seams in the grayscale image are identified by using a semantic segmentation algorithm.

3. The cement pavement misalignment detection method based on a depth camera according to claim 2, characterized in that: In step S4, the training steps of the random forest classification model are: S41: collecting a number of depth images with misalignment, marking the depth curves into two categories: misalignment and non-misalignment, to form a training data set; S42: Smoothing and normalizing the depth curve; S43: Perform data augmentation on the preprocessed depth curve and use it as input to train the random forest classification model.

4. The cement pavement misalignment detection method based on a depth camera according to claim 3, characterized in that: In step S43, data enhancement includes left-right flipping and up-down flipping.

5. The cement pavement misalignment detection method based on a depth camera according to claim 4, characterized in that: In step S6, the steps of extracting the misalignment area and depth are as follows: S61: skeletonize the plate seam mask and extract the horizontal or vertical skeleton line according to the direction of the misalignment; S62: Use Ransac to perform straight line fitting on the points on the skeleton line in the horizontal and vertical directions respectively, to obtain the separated plate seam straight lines in the horizontal and vertical directions; S63: expanding the plate seam straight line to obtain separated plate seam areas in the transverse direction and the longitudinal direction; S64: Calculate the mean depths on both sides of the center line of the plate seam area where there is misalignment, and take the difference between the mean depths of the areas on both sides as the misalignment depth. If the misalignment depth is greater than the set threshold, it is determined that there is misalignment in the plate seam area in this direction, and then extract the contour of the plate seam area as the misalignment position.

Images