Pavement crack detection method based on three-dimensional and time crack model

Abstract

The invention discloses a pavement crack detection method based on a three-dimensional and time crack model, and relates to the field of pavement crack intelligent detection. On the basis of an existing 3D detection structure special for power pavement crack information acquisition, depth information is added on the basis of two-dimensional information, a crack expansion model based on three-dimensional point cloud and time information is established in combination with time sequence data, and a crack expansion model is established according to the depth increase, width change and time change trend of the crack. And predicting the expansion speed and direction of the crack so as to analyze a potential risk area of future expansion of the crack, and dynamically monitoring the crack expansion.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection of pavement cracks, and specifically to a pavement crack detection method based on a three-dimensional + time crack model. Background Art

[0002] Traditional crack detection methods mostly rely on static data collection and lack the ability to track the dynamic evolution process of cracks over time. The expansion and change of cracks have strong time dependence. Especially under the influence of different seasons, climate changes or other environmental factors, the expansion behavior of cracks is more complex, and it is difficult for existing technologies to track the changes of cracks in real time and provide timely warnings. Therefore, how to accurately capture the dynamic evolution of cracks and predict the risk of crack expansion by comparing data at different time points is an urgent problem to be solved by current technologies.

[0003] Secondly, although the registration technology of laser point cloud and two-dimensional image has solved the accuracy problem of crack three-dimensional reconstruction to a certain extent, the registration accuracy of point cloud data and image data is still a difficult point in the data processing process. Especially in the case of complex pavements or complex crack morphologies, it is difficult to achieve completely accurate precise matching of point cloud data and synchronous processing of image data, which will affect the final crack detection effect. In addition, most existing technologies focus on the geometric morphology analysis of cracks, and are relatively weak in capturing the details of crack edges and analyzing dynamic characteristics such as crack propagation speed and direction. Although some deep learning-based technologies attempt to extract crack edge features from texture information, these methods still face great challenges in complex crack environments, especially when the noise interference around the cracks is large, it is difficult to ensure the accuracy of crack detection and analysis. Summary of the Invention

[0004] The present invention proposes a pavement crack detection method based on a three-dimensional + time crack model. For the existing 3D detection structure dedicated to power pavement crack information collection, depth information is added on the basis of two-dimensional information, and the pavement cracks are accurately analyzed through the three-dimensional + time crack model.

[0005] Among them, a pavement crack detection method based on a three-dimensional + time crack model includes the following steps: S1. In the target crack area, data is collected simultaneously by a laser scanner and a camera. The camera collects two-dimensional images of the cracks, and the laser scanner collects three-dimensional laser point cloud data of the cracks by scanning the three-dimensional depth of the cracks; and according to the calibrated camera parameters and laser parameters, the two-dimensional images and the three-dimensional laser point cloud data are registered; S2. Based on the collected laser point cloud data and two-dimensional image data, perform 3D reconstruction of the cracks; and combine the texture and depth information of the crack edges through the depth-texture convolution algorithm; S3. Extract the depth profile of the cracks. By segmenting the crack point cloud, obtain the depth change of the cracks from the starting point to the end; and combine with the time series data to establish a 3D + time crack propagation model for capturing the dynamic evolution of the cracks over time. Based on the depth growth, width change, and time change trend of the cracks, predict the crack propagation risk; S4. Based on the prediction results of the crack propagation model, the system automatically evaluates the risk level of the cracks and generates a real-time warning report.

[0006] Further, in the step S1, according to the calibrated camera parameters and laser parameters, register the two-dimensional image and the three-dimensional laser point cloud data to ensure accurate data registration, which specifically includes the following sub-steps: S101. Transform the laser point cloud from the LiDAR coordinate system to the camera coordinate system through the external parameter matrix; S102. Perform perspective projection using the camera internal parameter matrix; S103. Align the point cloud projection points and the two-dimensional image pixels through nearest neighbor interpolation matching.

[0007] Further, in the step S101, the specific process of coordinate transformation is expressed as: ; wherein, the represents the three-dimensional coordinates of the crack in the camera coordinate system, the represents the three-dimensional coordinates in the LiDAR coordinate system, the represents a 3×3 orthogonal matrix for describing the rotation relationship between the LiDAR coordinate system and the camera coordinate system, and the represents a 3×1 vector, indicating the translational offset of the LiDAR coordinate system relative to the camera coordinate system, used to compensate for the position difference between the LiDAR and the camera in three-dimensional space.

[0008] Further, in the step S102, the specific process of perspective projection is expressed as: ; wherein, the represents the pixel coordinates of the crack on the camera image plane, wherein the corresponds to the pixel value of the horizontal axis of the image, the corresponds to the pixel value of the vertical axis of the image, the represents the internal imaging parameters of the camera, and the represents the normalized plane coordinates.

[0009] Further, in step S2, the three-dimensional reconstruction of the crack based on the collected laser point cloud data and two-dimensional image data specifically includes the following sub-steps: S2011. Through projection transformation, map the two-dimensional image pixel coordinates to the corresponding three-dimensional point cloud coordinates; S2012. Perform three-dimensional meshing on the crack point cloud data through the point cloud triangulation algorithm to reconstruct the three-dimensional surface of the crack.

[0010] Further, in step S2, the combination of the texture and depth information of the crack edge through the depth-texture convolution algorithm specifically includes the following sub-steps: S2021. Process the depth information and texture information respectively through a two-stream convolutional neural network. Among them, extract the texture features of the crack through the texture feature extraction branch of the two-stream convolutional neural network; extract the geometric features of the crack through the depth feature extraction branch of the two-stream convolutional neural network; S2022. Use the weighted fusion method to fuse the depth features and texture features; S2023. Perform upsampling through deconvolution to enhance the crack edge; and use the weighted binary cross-entropy loss as the loss function to train the two-stream convolutional neural network; S2024. Generate the crack edge enhancement result according to the trained two-stream convolutional neural network.

[0011] Further, the loss function is specifically expressed as: ; wherein, the represents the weighted binary cross-entropy loss function, the represents the weighting coefficient at the pixel position (x, y), the represents the abscissa of the pixel position, the represents the ordinate of the pixel position, the represents the true label at the pixel position (x, y), the represents the model prediction result at the pixel position (x, y), and the represents the common logarithmic term in the binary cross-entropy.

[0012] Further, step S3 specifically includes the following sub-steps: S301. In the crack point cloud, sample at a fixed interval from the crack starting point to the end point to obtain the depth profile curve, that is: ; ; ; Among them, the point cloud data set representing cracks, the three coordinate components representing the crack point cloud data are used to represent the spatial position of the crack, the index of the point in the point cloud, the total number of points in the point cloud, the abscissa of the j-th point in the crack point cloud, the starting position of the crack point cloud, that is, the starting abscissa of the crack, the depth profile at position x, the ordinate of the j-th point in the crack point cloud, that is, the depth value, the ordinate of the j-th point in the crack point cloud; the spacing distance between adjacent points in the crack point cloud; S302. Calculate the crack depth change rate and the irregularity index of the crack according to the depth profile curve; S303. Collect the crack depth distributions at different times and establish a crack propagation time series; register the crack point clouds at different time points through the iterative closest point algorithm; S304. Calculate the crack depth propagation rate, the crack width propagation rate, and the crack propagation direction vector respectively; establish an LSTM crack propagation model based on the calculated crack depth propagation rate, crack width propagation rate, and crack propagation direction vector as input data; S305. Predict according to the output data of the LSTM crack propagation model as the crack propagation risk index.

[0013] Furthermore, in the step S304, the specific process of calculating the crack depth propagation rate, the crack width propagation rate, and the crack propagation direction vector is expressed as: ; ; ; Among them, the represents the crack depth propagation rate, the represents the depth value at time, the represents at the depth value at time, the represents the crack width propagation rate, the represents the width value at time, the represents at the width value at time, the represents the crack propagation direction vector at time t, the represents the velocity component in the horizontal axis direction, and the represents the velocity component in the vertical axis direction.

[0014] Furthermore, the specific calculation process of the crack propagation risk index is expressed as: ; wherein, the represents the crack propagation risk index, and the represents the depth value.

[0015] The beneficial effects of the invention are: The present invention combines time series data to establish a crack propagation model based on three-dimensional point cloud and time information. According to the depth growth, width change and time change trend of the crack, the propagation speed and direction of the crack are predicted, so as to analyze the potential risk area of the future crack propagation and dynamically monitor the crack propagation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a method flow chart of a pavement crack detection method based on a three-dimensional + time crack model provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The technical solution of the present invention will be further described in detail below with reference to the drawings, but the protection scope of the present invention is not limited to the following.

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention, that is, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Usually, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0019] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention. It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0020] Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a series of elements includes not only those elements but also other elements not expressly listed, or elements that are inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus that comprises the element.

[0021] The features and performance of the present invention will be further described in detail below in conjunction with embodiments.

[0022] Among them, as Figure 1 , a pavement crack detection method based on a three-dimensional + time crack model includes the following steps: S1. In the target crack area, data is collected simultaneously by a laser scanner and a camera. The camera collects two-dimensional images of the crack, and the laser scanner collects three-dimensional laser point cloud data of the crack by scanning the three-dimensional depth of the crack; and according to the calibrated camera parameters and laser parameters, the two-dimensional image and the three-dimensional laser point cloud data are registered. S2. Based on the collected laser point cloud data and two-dimensional image data, three-dimensional reconstruction of the crack is performed; and through the depth-texture convolution algorithm, the texture and depth information at the crack edge are combined. S3. Extract the depth profile of the crack. By segmenting the crack point cloud, the depth change of the crack from the starting point to the end is obtained; and in combination with time series data, a three-dimensional + time crack propagation model for capturing the dynamic evolution of the crack over time is established. Based on the depth growth, width change and time change trend of the crack, the crack propagation risk is predicted. S4. Based on the prediction result of the crack propagation model, the system automatically evaluates the risk level of the crack and generates a real-time warning report.

[0023] Specifically, in the above embodiment, data is collected simultaneously by a laser scanner and a camera. The laser scanner obtains three-dimensional depth data of the crack area, while the camera captures two-dimensional image data of the crack. The laser scanner generates three-dimensional point cloud data, which includes the geometric shape, depth information, etc. of the crack; the camera provides visual information such as the texture and color of the crack surface. Further, a sawtooth calibration block is used for internal and external calibration of the camera to ensure optical distortion correction of the camera, and the focal length, principal point offset, and distortion parameters are calculated; and a laser calibration reference plate is used to adjust the ranging error compensation of the laser scanner to ensure the accuracy of the laser point cloud data. During the detection process, the calibration accuracy is affected by equipment vibration, road surface slope change, and speed fluctuation. Therefore, an IMU sensor and an incremental encoder are used to monitor the equipment attitude in real time and perform dynamic adjustment of the calibration parameters.

[0024] Further, in the step S1, the registration of the two-dimensional image and the three-dimensional lidar point cloud data according to the calibrated camera parameters and lidar parameters specifically includes the following sub-steps: S101. Convert the lidar point cloud from the lidar coordinate system to the camera coordinate system through the extrinsic matrix; S102. Perform perspective projection using the camera intrinsic matrix; S103. Align the point cloud projection points and the two-dimensional image pixels through nearest neighbor interpolation matching.

[0025] Further, in the step S101, the specific process of the coordinate system conversion is expressed as: ; Among them, the represents the three-dimensional coordinates of the crack in the camera coordinate system, the represents the three-dimensional coordinates in the lidar coordinate system, the represents a 3×3 orthogonal matrix for describing the rotation relationship between the LiDAR coordinate system and the camera coordinate system, and the represents a 3×1 vector, indicating the translational offset of the lidar coordinate system relative to the camera coordinate system, used to compensate for the position difference between the lidar and the camera in three-dimensional space.

[0026] Specifically, in the above embodiment, the two-dimensional image and the three-dimensional lidar point cloud data are aligned through the calibration parameters of the camera and the lidar scanner. And the lidar point cloud is converted from the lidar coordinate system to the camera coordinate system through the extrinsic matrix, and projection is performed using the camera intrinsic matrix to ensure the precise matching of the point cloud projection points and the image pixel points.

[0027] Further, in the step S102, the specific process of the perspective projection is expressed as: ; Among them, the represents the pixel coordinates of the crack on the camera image plane, where the corresponds to the pixel value of the horizontal axis of the image, the corresponds to the pixel value of the vertical axis of the image, the represents the internal imaging parameters of the camera, and the represents the normalized plane coordinates. Specifically, the camera intrinsic matrix is used to perform perspective projection on the point cloud data, and the points in three-dimensional space are projected onto the camera image plane.

[0028] As a preferred implementation manner of the above embodiment, a radial distortion correction model can be used to correct the distortion error caused by the camera perspective projection.

[0029] Further, in step S2, the three-dimensional reconstruction of the crack based on the collected laser point cloud data and two-dimensional image data specifically includes the following sub-steps: S2011. Through projective transformation, map the two-dimensional image pixel coordinates to the corresponding three-dimensional point cloud coordinates; S2012. Perform three-dimensional meshing on the crack point cloud data through the point cloud triangulation algorithm to reconstruct the three-dimensional surface of the crack.

[0030] Specifically, in the above embodiment, the pixel coordinates in the two-dimensional image are mapped into the three-dimensional point cloud data, and the three-dimensional point cloud is converted into a three-dimensional grid model of the crack surface through the point cloud triangulation algorithm. The above process represents the spatial structure of the crack through a digital three-dimensional surface to obtain the three-dimensional geometric information of the crack.

[0031] Further, in step S2, the combination of the texture and depth information of the crack edge through the depth-texture convolution algorithm specifically includes the following sub-steps: S2021. Process the depth information and texture information respectively through a two-stream convolutional neural network. Among them, extract the texture features of the crack through the texture feature extraction branch of the two-stream convolutional neural network; extract the geometric features of the crack through the depth feature extraction branch of the two-stream convolutional neural network; S2022. Use a weighted fusion method to fuse the depth features and texture features; S2023. Perform upsampling through deconvolution to enhance the crack edge; and use the weighted binary cross-entropy loss as the loss function to train the two-stream convolutional neural network; S2024. Generate the crack edge enhancement result according to the trained two-stream convolutional neural network.

[0032] Specifically, in the above embodiment, the depth information and texture information are combined to enhance the crack edge. The depth features and texture features are extracted respectively through a two-stream convolutional neural network, and the two are fused with weights to obtain a more accurate crack edge image. The crack edge is upsampled through the deconvolution process to further enhance the crack features. The training of the network uses the weighted binary cross-entropy loss function to ensure that the model can robustly extract crack features under different image qualities.

[0033] Further, the loss function is specifically expressed as: ; wherein, the represents the weighted binary cross-entropy loss function, the represents the weighting coefficient at the pixel position (x,y), the represents the abscissa of the pixel position, the represents the ordinate of the pixel position, and the represents the true label at the pixel position (x, y), and the represents the model prediction result at the pixel position (x, y), and the represents the logarithmic term commonly used in binary cross-entropy.

[0034] Furthermore, the step S3 specifically includes the following sub-steps: S301. In the crack point cloud, sample at a fixed interval from the crack starting point to the end to obtain a depth profile curve, that is: ; ; ; wherein the represents the point cloud data set of the crack, and the represents the three coordinate components of the crack point cloud, which are used to represent the spatial position of the crack, and the represents the index of the point in the point cloud, and the represents the total number of points in the point cloud, and the represents the abscissa of the j-th point in the crack point cloud, and the represents the starting position of the crack point cloud, that is, the starting abscissa of the crack, and the represents the depth profile at position x, and the represents the ordinate of the j-th point in the crack point cloud, that is, the depth value, and the represents the ordinate of the j-th point in the crack point cloud, and the represents the interval distance between adjacent points in the crack point cloud; specifically, in the point cloud data set, is uniformly distributed along the horizontal direction, that is, the spacing between each adjacent point in the x direction is equal, which is , this interval distance is a fixed value, and this value-taking method is usually used for scanning measurement data, sampling at equal intervals in the horizontal direction, and then measuring the corresponding crack morphology (width, depth) at each . In addition, the is only the number in the point cloud and does not mean that the above points must be linearly distributed. Exemplarily, when there are multiple measurement points at a position (a crack has multiple depth points), then the data can be organized in a list or grouped form.

[0035] S302. Calculate the crack depth change rate and the crack irregularity index according to the depth profile curve; S303. Collect the crack depth distributions at different times to establish a crack propagation time series; register the crack point clouds at different time points through the iterative closest point algorithm; S304. Calculate the crack depth expansion rate, crack width expansion rate, and crack expansion direction vector respectively; establish an LSTM crack expansion model based on the calculated crack depth expansion rate, crack width expansion rate, and crack expansion direction vector as input data; S305. Predict using the output data of the LSTM crack expansion model as the crack expansion risk index.

[0036] Specifically, in the above embodiment, the point cloud is sampled at fixed intervals from the starting point to the ending point of the crack to obtain the depth change curve of the crack at different positions, and the change of the crack depth with position is recorded through the above process. By calculating the depth change rate and width change rate of the crack and combining time series data, a three-dimensional + time expansion model of the crack is established. The time series data is used to capture the dynamic evolution process of the crack over time, including factors such as crack depth growth and width change, and the risk of crack expansion is predicted based on these data. Further, based on the crack expansion model, the system will evaluate the crack expansion risk in real time. By calculating features such as the crack depth expansion rate, width expansion rate, and expansion direction vector, and combining the LSTM (Long Short-Term Memory) model to predict the crack expansion trend; according to the output of the model, a risk index of crack expansion is generated, and a warning report is generated for relevant personnel for necessary prevention and repair work.

[0037] Further, in step S304, the specific process of calculating the crack depth expansion rate, crack width expansion rate, and crack expansion direction vector is expressed as: ; ; ; Among them, the represents the crack depth expansion rate, the represents the depth value at time, the represents the depth value at time, the represents at time, the represents the crack width expansion rate, the represents the width value at time, the represents the width value at time, the represents at time, the represents the crack expansion direction vector at time t, the represents the velocity component in the horizontal axis direction, and the represents the velocity component in the vertical axis direction.

[0038] Further, the specific calculation process of the crack propagation risk index is expressed as: ; wherein, the represents the crack propagation risk index, and the represents the depth value.

[0039] The above embodiments provide a crack detection method based on the combination of dual-camera calibration technology and three-dimensional laser scanning technology, aiming to achieve an automated, intelligent, and high-precision crack detection system through high-precision calibration, data fusion, three-dimensional shape reconstruction, crack propagation analysis and prediction, risk assessment, and visualization.

[0040] Further, as a preferred implementation manner of the above embodiments, a pavement crack detection system based on a three-dimensional + time crack model is proposed. The system is implemented based on any one of the above-mentioned pavement crack detection methods based on a three-dimensional + time crack model, and specifically includes: A data acquisition and registration module, configured to simultaneously acquire data through a laser scanner and a camera in a target crack area. The camera acquires a two-dimensional image of the crack, and the laser scanner acquires three-dimensional laser point cloud data of the crack by scanning the three-dimensional depth of the crack; and register the two-dimensional image and the three-dimensional laser point cloud data according to the calibrated camera parameters and laser parameters. A three-dimensional reconstruction module, configured to perform three-dimensional reconstruction of the crack based on the acquired laser point cloud data and two-dimensional image data; and combine the texture and depth information of the crack edge through a depth-texture convolution algorithm. A model prediction module, configured to extract the depth profile of the crack, obtain the depth change of the crack from the starting point to the end point through segmented processing of the crack point cloud; and establish a three-dimensional + time crack propagation model for capturing the dynamic evolution of the crack over time in combination with time series data, and predict the crack propagation risk based on the depth growth, width change, and time change trend of the crack. A risk warning module, configured to automatically evaluate the risk level of the crack based on the prediction result of the crack propagation model and generate a real-time warning report.

[0041] Further, in the data acquisition and registration module, aligning the two-dimensional image data and the three-dimensional laser point cloud data through the calibrated parameters to ensure accurate data registration specifically includes: An external parameter matrix coordinate conversion unit, configured to convert the laser point cloud from the lidar coordinate system to the camera coordinate system through the external parameter matrix. A perspective projection unit, configured to perform perspective projection using the camera internal parameter matrix. A data alignment unit, configured to align the point cloud projection points and the two-dimensional image pixels through nearest neighbor interpolation matching.

[0042] Furthermore, in the alignment of the parameters, the specific process of coordinate system transformation is expressed as: ; Among them, the represents the three-dimensional coordinates of the crack in the camera coordinate system, and the represents the three-dimensional coordinates in the LiDAR coordinate system. The represents a 3×3 orthogonal matrix, which is used to describe the rotation relationship between the LiDAR coordinate system and the camera coordinate system. The represents a 3×1 vector, which represents the translational offset of the LiDAR coordinate system relative to the camera coordinate system and is used to compensate for the position difference between the LiDAR and the camera in three-dimensional space.

[0043] Furthermore, in the perspective projection unit, the specific process of perspective projection is expressed as: ; Among them, the represents the pixel coordinates of the crack on the camera image plane. Among them, corresponds to the pixel value of the horizontal axis of the image, corresponds to the pixel value of the vertical axis of the image. The represents the internal imaging parameters of the camera, and the represents the normalized plane coordinates.

[0044] Furthermore, the three-dimensional reconstruction module specifically includes: A coordinate mapping unit, which is used to map the two-dimensional image pixel coordinates to the corresponding three-dimensional point cloud coordinates through projective transformation; A three-dimensional reconstruction unit, which is used to perform three-dimensional meshing processing on the crack point cloud data through a point cloud triangulation algorithm to reconstruct the three-dimensional surface of the crack; A feature extraction unit, which is used to process the depth information and texture information respectively through a two-stream convolutional neural network. Among them, the texture feature of the crack is extracted through the texture feature extraction branch of the two-stream convolutional neural network; the geometric feature of the crack is extracted through the depth feature extraction branch of the two-stream convolutional neural network; A feature fusion unit, which is used to fuse the depth feature and the texture feature using a weighted fusion method; A model training unit, which is used to perform upsampling through deconvolution to enhance the crack edge; and use the weighted binary cross-entropy loss as the loss function to train the two-stream convolutional neural network; A crack edge enhancement unit, which is used to generate the crack edge enhancement result according to the trained two-stream convolutional neural network.

[0045] Furthermore, the loss function is specifically expressed as: ; Among them, the represents the weighted binary cross-entropy loss function, and the represents the weighting coefficient at the pixel position (x, y), the represents the abscissa of the pixel position, the represents the ordinate of the pixel position, the represents the ground truth label at the pixel position (x, y), the represents the model prediction result at the pixel position (x, y), and the represents the logarithmic term commonly used in binary cross-entropy.

[0046] Furthermore, the model prediction module specifically includes: A profile curve calculation unit, which samples at a fixed interval from the starting point to the end point of the crack in the crack point cloud to obtain a depth profile curve, that is: ; ; ; Among them, the represents the point cloud data set of the crack, and the represents the three coordinate components of the crack point cloud data, which are used to represent the spatial position of the crack. The represents the index of the point in the point cloud, the represents the total number of points in the point cloud, the represents the abscissa of the j-th point in the crack point cloud, the represents the starting position of the crack point cloud, that is, the starting abscissa of the crack, and the represents that the represents the ordinate of the j-th point in the crack point cloud, that is, the depth value, and the represents the ordinate of the j-th point in the crack point cloud; A crack data calculation unit, which is used to calculate the crack depth change rate and the crack irregularity index according to the depth profile curve; A crack point cloud registration unit, which collects the crack depth distributions at different times and establishes a crack growth time series; and registers the crack point clouds at different time points through the iterative closest point algorithm; An LSTM crack growth model construction unit, which calculates the crack depth growth rate, the crack width growth rate, and the crack growth direction vector respectively; and establishes an LSTM crack growth model based on the calculated crack depth growth rate, crack width growth rate, and crack growth direction vector as input data; An exponential prediction unit is used to make predictions based on the output data of the LSTM crack propagation model as the crack propagation risk index.

[0047] Further, in step S304, the specific processes for calculating the crack depth propagation rate, the crack width propagation rate, and the crack propagation direction vector are expressed as follows: ; ; ; Among them, the represents the crack depth propagation rate, the represents the depth value at time, the represents at the depth value at time, the represents the crack width propagation rate, the represents at the width value at time, the represents at the width value at time, the represents the crack propagation direction vector at time t, the represents the velocity component in the x-axis direction, the represents the velocity component in the y-axis direction.

[0048] Further, the specific calculation process of the crack propagation risk index is expressed as follows: ; Among them, the represents the crack propagation risk index, the represents the depth value.

[0049] The above is only the preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments. Instead, it can be used in various other combinations, modifications, and environments, and can be changed within the scope of the concept described herein through the above teachings or the techniques or knowledge in related fields. Any changes and modifications made by those skilled in the art without departing from the spirit and scope of the present invention shall fall within the protection scope of the appended claims of the present invention.

Claims

1. A pavement crack detection method based on a three-dimensional + time crack model, characterized in that: The following steps are involved: S1. In the target crack area, data is collected simultaneously by a laser scanner and a camera, wherein the camera collects a two-dimensional image of the crack, and the laser scanner collects three-dimensional laser point cloud data of the crack by scanning the three-dimensional depth of the crack; and the two-dimensional image and the three-dimensional laser point cloud data are registered according to the calibrated camera parameters and laser parameters; S2. Reconstruct the crack in 3D based on the collected laser point cloud data and 2D image data; and combine the texture and depth information of the crack edge through the depth-texture convolution algorithm; S3. Extract the depth profile of the crack, obtain the depth change of the crack from the starting point to the end by segmenting the crack point cloud; and combine the time series data to establish a 3D + time crack extension model to capture the dynamic evolution of the crack over time, and predict the crack extension risk based on the depth growth, width change and time change trend of the crack; S4. Based on the prediction results of the crack propagation model, the system automatically evaluates the risk level of the cracks and generates a real-time warning report.

2. A pavement crack detection method based on a three-dimensional + time crack model as claimed in claim 1, characterized in that: In step S1, registering the two-dimensional image and the three-dimensional laser point cloud data according to the calibrated camera parameters and laser parameters specifically includes the following sub-steps: S101. Convert the laser point cloud from the laser radar coordinate system to the camera coordinate system through the external parameter matrix; S102. Perform perspective projection using the camera intrinsic parameter matrix; S103. Align the point cloud projection points and the two-dimensional image pixels through nearest neighbor interpolation matching.

3. A pavement crack detection method based on a three-dimensional + time crack model as claimed in claim 2, characterized in that: In step S101, the specific process of coordinate system conversion is expressed as follows: ； Among them, the represents the three-dimensional coordinates of the crack in the camera coordinate system. represents the three-dimensional coordinates in the laser radar coordinate system, Represents a 3×3 orthogonal matrix, which is used to describe the rotation relationship between the laser radar coordinate system and the camera coordinate system. Represents a 3×1 vector, which represents the translation offset of the lidar coordinate system relative to the camera coordinate system, used to compensate for the position difference between the lidar and the camera in three-dimensional space.

4. A pavement crack detection method based on a three-dimensional + time crack model as claimed in claim 3, characterized in that: In step S102, the specific process of perspective projection is as follows: ； Among them, the represents the pixel coordinates of the crack on the camera image plane, where Corresponding to the horizontal axis pixel value of the image, Corresponding to the vertical axis pixel value of the image, Represents the internal imaging parameters of the camera. Represents normalized plane coordinates.

5. A pavement crack detection method based on a three-dimensional + time crack model as claimed in claim 1, characterized in that: In step S2, based on the collected laser point cloud data and two-dimensional image data, three-dimensional reconstruction of the crack specifically includes the following sub-steps: S2011. Mapping the two-dimensional image pixel coordinates to corresponding three-dimensional point cloud coordinates through projection transformation; S2012. The three-dimensional surface of the crack was reconstructed by processing the crack point cloud data into three-dimensional grids through point cloud triangulation algorithm.

6. A pavement crack detection method based on a three-dimensional + time crack model as claimed in claim 1, characterized in that: In step S2, combining the texture and depth information of the crack edge by using a depth-texture convolution algorithm specifically includes the following sub-steps: S2021. Processing the depth information and the texture information respectively by a two-stream convolutional neural network, wherein the texture feature extraction branch of the two-stream convolutional neural network is used to extract the texture feature of the crack; and the depth feature extraction branch of the two-stream convolutional neural network is used to extract the geometric feature of the crack; S2022. Fusion of depth features and texture features using a weighted fusion method; S2023. Enhance crack edges by upsampling through deconvolution; and use weighted binary cross entropy loss as the loss function to train the two-stream convolutional neural network; S2024. Generate crack edge enhancement results based on the trained two-stream convolutional neural network.

7. A pavement crack detection method based on a three-dimensional + time crack model as claimed in claim 6, characterized in that: The loss function is specifically expressed as: ； Wherein, represents the weighted binary cross entropy loss function, represents the weighting coefficient at the pixel position (x, y), represents the horizontal coordinate of the pixel position, represents the vertical coordinate of the pixel position, represents the true label at the pixel position (x, y), Represents the model prediction result at the pixel position (x, y), Represents the logarithmic term commonly used in binary cross entropy.

8. A pavement crack detection method based on a three-dimensional + time crack model as claimed in claim 1, characterized in that: The step S3 specifically includes the following sub-steps: S301. In the crack point cloud, samples are taken at fixed intervals from the crack start point to the crack end point to obtain a depth profile curve, namely: ； ； ； Among them, the A point cloud dataset representing a crack, The three coordinate components representing the crack point cloud data are used to represent the spatial position of the crack. Represents the index of the point in the point cloud. Represents the total number of points in the point cloud. Indicates the first The horizontal coordinate of the point, Indicates the starting position of the crack point cloud, that is, the starting horizontal coordinate of the crack. represents the depth profile at position x, Indicates the first The vertical coordinate of the point, that is, the depth value, Indicates the first The vertical coordinate of a point; Indicates the interval distance between adjacent points in the crack point cloud; S302. Calculating the crack depth change rate and the crack irregularity index according to the depth profile curve; S303. Collecting the crack depth distribution at different times and establishing a crack extension time series; registering the crack point clouds at different time points by an iterative closest point algorithm; S304. respectively calculating the crack depth extension rate, the crack width extension rate and the crack extension direction vector; and establishing an LSTM crack extension model based on the calculated crack depth extension rate, the crack width extension rate and the crack extension direction vector as input data; S305. Predict the crack extension risk index based on the output data of the LSTM crack extension model.

9. A pavement crack detection method based on a three-dimensional + time crack model as claimed in claim 8, characterized in that: In step S304, the specific process of calculating the crack depth expansion rate, the crack width expansion rate and the crack expansion direction vector is as follows: ； ； ； Among them, the represents the crack depth extension rate, Indicated in The depth value at the moment, Indicated in The depth value at the moment, represents the crack width expansion rate, Indicated in The width value of the moment, Indicated in The width value of the moment, represents the crack extension direction vector at time t, represents the velocity component in the direction of the horizontal axis, Represents the velocity component in the direction of the ordinate axis.

10. A pavement crack detection method based on a three-dimensional + time crack model as claimed in claim 9, characterized in that: The specific calculation process of the crack extension risk index is expressed as follows: ； Among them, the represents the crack extension risk index, Indicates the depth value.

Images