Road crack intelligent detection method

Abstract

The invention provides an intelligent road crack detection method, and belongs to the technical field of electrical digital data processing.The intelligent road crack detection method comprises the steps that firstly, a line structured light scanner is adopted to obtain road surface point cloud data, and attitude compensation processing is conducted in combination with an inertial navigation system; then noise is eliminated through bidirectional moving average filtering, and the depth matrix is converted into an image and segmented; gaussian background modeling is used to screen abnormal points, a connected graph is constructed through a minimum spanning tree algorithm, and pruning is carried out to obtain a crack skeleton; using a feature enhancement function to optimize the crack saliency; and finally, inputting a pre-trained multi-scale pavement damage identification network, and outputting a crack identification result and classification. According to the method, three-dimensional point cloud data, geometric feature analysis and a deep learning technology are fused, the technical problem of poor adaptability of a complex pavement environment is solved, and accurate identification of various crack types is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric digital data processing. Specifically, it relates to an intelligent road crack detection method. Background Art

[0002] Road crack detection is an important part of road maintenance management. Traditional crack detection mainly relies on manual inspections and simple image processing techniques. Currently, widely used detection methods include vision detection based on image processing and three-dimensional detection based on laser scanning. The vision detection technology collects road surface images through high-definition cameras and uses image segmentation and edge detection algorithms to extract crack features; three-dimensional detection uses laser scanning equipment to obtain three-dimensional information of the road surface and identifies road surface anomalies through point cloud data analysis. These technologies have been applied in the maintenance of urban roads and highways.

[0003] However, the existing detection technologies perform poorly in complex road surface environments. The method based on image processing is easily affected by factors such as light changes, shadows, and stains, resulting in a high false detection rate; although the method based on laser scanning can obtain high-precision three-dimensional data, its ability to post-process data is limited, and it is difficult to distinguish road surface textures from real cracks, especially in cases of road surface temperature changes and uneven materials, the detection accuracy drops significantly. At the same time, the existing algorithms have insufficient extraction of crack features and are difficult to effectively identify different types of cracks.

[0004] These defects lead to poor adaptability of the existing technologies in complex road surface environments and cannot meet the requirements of road maintenance automation, high efficiency, and precision. There is an urgent need for an intelligent crack detection method that can adapt to various complex road surface environments. That is to say, there is a technical problem of poor adaptability to complex road surface environments in the detection process of road cracks in the existing technologies. Summary of the Invention

[0005] In view of this, the present invention provides an intelligent road crack detection method, which can solve the technical problem of poor adaptability to complex road surface environments in the detection process of road cracks in the existing technologies.

[0006] The present invention is implemented as follows: The present invention provides an intelligent detection method for road cracks, which includes the following steps: obtaining the original laser point cloud elevation data matrix of the road surface and processing it to obtain the corrected laser point cloud elevation data matrix; performing two-way moving average filtering on the corrected laser point cloud elevation data matrix to eliminate road surface noise and obtain the road surface depth matrix; normalizing the road surface depth matrix into a depth image, and uniformly dividing the depth image into multiple initial detection image blocks according to a preset size; using the Gaussian background modeling method to screen the depth anomaly points in each initial detection image block, extracting the potential crack feature point set and constructing a connected graph structure, calculating the Euclidean distance weights between the potential crack feature points, pruning and removing the abnormal connection edges to obtain the crack skeleton structure; applying the crack feature enhancement function to optimize the crack skeleton structure to generate the optimized crack feature image block; inputting the optimized crack feature image block into the pre-trained multi-scale road surface damage recognition network model to output the crack recognition result and the crack type classification.

[0007] Among them, the two-way moving average filtering refers to performing sliding window average calculation on the corrected laser point cloud elevation data matrix in both the horizontal and vertical directions of the road surface, effectively eliminating the influence of road surface texture and acquisition noise, and highlighting the crack features.

[0008] Among them, the depth image is an image formed by converting each depth value in the road surface depth matrix into a gray value according to a linear mapping method, and the size of the gray value represents the depression degree of the road surface; the initial detection image block is an image sub-region obtained by uniformly dividing the depth image according to a fixed-size grid, which is convenient for extracting crack features in a local area.

[0009] Among them, the Gaussian background modeling assumes that the distribution of road surface depth values conforms to the normal distribution characteristic, estimates the mean and standard deviation of the road surface depth through the expectation maximization algorithm, and sets an adaptive threshold to identify abnormal depth points; the potential crack feature points refer to the set of points whose depth values are abnormal and may form a crack structure identified by the Gaussian background modeling.

[0010] Among them, the crack skeleton structure is the main crack structure obtained by processing through the minimum spanning tree algorithm, which represents the spatial distribution form of the crack.

[0011] Among them, the crack feature enhancement function is used for feature extraction and enhancement processing of the crack region. The inputs include the crack skeleton structure matrix, the crack region depth matrix, the crack region texture gradient matrix, the crack direction vector matrix, and the crack connectivity matrix. The output is the enhanced crack feature image matrix. The crack feature enhancement function realizes the improvement of the contrast between the crack region and the background region through multi-dimensional feature space mapping, and enhances the crack feature recognition effect.

[0012] Among them, the crack area depth matrix is a matrix formed by extracting the depth values corresponding to the crack areas from the road surface depth matrix, which characterizes the crack depth distribution; the crack area texture gradient matrix is a matrix formed by calculating the gray scale gradient values in the crack areas of the depth image, which characterizes the crack edge texture features.

[0013] Among them, the crack direction vector matrix is a matrix formed by calculating the trend information of the local crack areas, which characterizes the crack extension direction features; the crack connectivity matrix is a matrix representing the connectivity relationship between potential crack feature points, which describes the integrity of the crack structure.

[0014] Among them, the enhanced crack feature image matrix is an image data matrix with prominent crack features obtained after being processed by the crack feature enhancement function; the optimized crack feature image block is an image sub-region obtained by re-segmenting the enhanced crack feature image matrix according to the size of the initial detection image block, and is used as the input data for crack recognition.

[0015] Among them, the specific structure of the pre-trained multi-scale road damage recognition network model is a deep neural network composed of a feature extraction layer, a multi-scale feature fusion layer, an attention mechanism layer, and a classification layer; the feature extraction layer uses multiple convolutional layers with different convolutional kernel sizes to extract road surface texture features at different scales, and then the multi-scale feature fusion layer adaptively fuses the feature information at different scales with weights.

[0016] Compared with the prior art, an intelligent road crack detection method provided by the present invention

[0017] The present invention proposes an intelligent road crack detection method that combines line structure light scanning and deep learning. By compensating for the attitude changes during data acquisition through an inertial navigation system and combining bidirectional moving average filtering to eliminate road surface noise, high-precision acquisition of the three-dimensional information of the road surface is achieved. This method uses Gaussian background modeling and the minimum spanning tree algorithm to extract crack features, and optimizes the crack structure representation through a custom crack feature enhancement function.

[0018] Compared with the prior art, the present invention significantly improves the adaptability of crack detection in complex road surface environments. Through the real-time attitude compensation of the inertial navigation system, the influence of vehicle jolts on data acquisition is effectively eliminated; the bidirectional moving average filtering and Gaussian background modeling mechanisms can effectively distinguish road surface textures from real cracks, reducing environmental factor interference; the application of the minimum spanning tree algorithm and the crack feature enhancement function improves the connectivity and saliency of crack features, enabling the system to adapt to different road surface materials and environmental conditions.

[0019] The present invention solves the core problem of poor adaptability to complex road surface environments through multi-dimensional data fusion and multi-scale feature learning, and realizes the accurate recognition of various crack types such as transverse cracks, longitudinal cracks, reticular cracks, and crazing cracks, providing a reliable basis for road maintenance decision-making. Brief Description of the Drawings

[0020] Figure 1 This is a flowchart of the method of the present invention. Detailed Embodiments

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0022] As Figure 1 shown, it is a flowchart of an intelligent road crack detection method provided by the present invention. This method includes the following steps:

[0023] S01. Use a detection vehicle equipped with a line-structured light scanner to continuously scan the road surface to obtain a matrix of original laser point cloud elevation data of the road surface;

[0024] S02. Measure the real-time attitude parameters of the line-structured light scanner through the inertial navigation system equipped on the detection vehicle, and perform attitude compensation processing on the matrix of original laser point cloud elevation data of the road surface to obtain a corrected matrix of laser point cloud elevation data;

[0025] S03. Perform two-way moving average filtering on the corrected matrix of laser point cloud elevation data to eliminate road surface noise and obtain a road surface depth matrix;

[0026] S04. Normalize and convert the road surface depth matrix into a depth image, and evenly divide the depth image into multiple initial detection image blocks according to a preset size;

[0027] S05. Use the Gaussian background modeling method to screen the depth abnormal points in each of the initial detection image blocks, and extract a set of potential crack feature points;

[0028] S06. Use the minimum spanning tree algorithm to construct a connected graph structure for all the potential crack feature points, calculate the Euclidean distance weights between the potential crack feature points, and prune and remove abnormal connection edges to obtain a crack skeleton structure;

[0029] S07. Apply a crack feature enhancement function to optimize the crack skeleton structure, enhance the significance of crack features, and generate an optimized crack feature image block;

[0030] S08. Input the optimized crack feature image block into a pre-trained multi-scale road damage recognition network model, and output a crack recognition result and a crack type classification;

[0031] Among them, the line-structured light scanner is a device for high-precision acquisition of the three-dimensional profile of the road surface based on the principle of laser triangulation. It calculates the road surface elevation information by projecting a linear laser beam and collecting the position change of the reflected light.

[0032] Among them, the inertial navigation system is a device used to measure and detect the motion state and attitude change parameters of a vehicle in real time. It includes sensors such as accelerometers and gyroscopes, and provides data on the pitch angle, roll angle, and heading angle of the vehicle.

[0033] Among them, the bidirectional moving average filtering refers to performing sliding window average calculation on the corrected laser point cloud elevation data matrix in two directions, namely the transverse and longitudinal directions of the road surface, effectively eliminating the influence of road surface texture and acquisition noise, and highlighting the crack characteristics.

[0034] Among them, the depth image is an image formed by converting each depth value in the road surface depth matrix into a gray value according to a linear mapping method, and the size of the gray value represents the degree of depression of the road surface.

[0035] Among them, the initial detection image block is an image sub-region obtained by evenly dividing the depth image according to a fixed-size grid, which is convenient for extracting crack characteristics in a local area.

[0036] Among them, Gaussian background modeling assumes that the distribution of road surface depth values conforms to the characteristics of a normal distribution. The mean and standard deviation of the road surface depth are estimated through the expectation maximization algorithm, and an adaptive threshold is set to identify abnormal depth points.

[0037] Among them, potential crack characteristic points refer to a set of points with abnormal depth values identified through Gaussian background modeling and that may form a crack structure.

[0038] Among them, the minimum spanning tree algorithm is a classic graph theory algorithm. The potential crack characteristic points are used as graph nodes, and the distance between nodes is used as the edge weight. A connected graph that includes all nodes and has the minimum total weight is constructed. By setting a distance threshold to prune non-connected characteristic points, the continuous structural characteristics of cracks are effectively extracted.

[0039] Among them, the crack skeleton structure is the main structure of the crack obtained after processing by the minimum spanning tree algorithm, representing the spatial distribution form of the crack.

[0040] Among them, the crack feature enhancement function is used for feature extraction and enhancement processing of the crack region. The inputs include the crack skeleton structure matrix, the crack region depth matrix, the crack region texture gradient matrix, the crack direction vector matrix, and the crack connectivity matrix. The output is the enhanced crack feature image matrix. The crack feature enhancement function realizes the improvement of the contrast between the crack region and the background region through multi-dimensional feature space mapping, enhancing the crack feature recognition effect.

[0041] Among them, the crack region depth matrix is a matrix formed by extracting the depth values corresponding to the crack region from the road surface depth matrix, representing the crack depth distribution.

[0042] Among them, the crack region texture gradient matrix is a matrix formed by calculating the gray gradient values in the crack region of the depth image, representing the crack edge texture characteristics.

[0043] Among them, the crack direction vector matrix is a matrix formed by calculating the trend information of the local crack area, which characterizes the crack extension direction feature.

[0044] Among them, the crack connectivity matrix is a matrix formed by representing the connectivity relationship between potential crack feature points, which describes the integrity of the crack structure.

[0045] Among them, the enhanced crack feature image matrix is an image data matrix that highlights crack features obtained after being processed by the crack feature enhancement function.

[0046] Among them, the optimized crack feature image block is an image sub-region obtained by re-segmenting the enhanced crack feature image matrix according to the size of the initial detection image block, which is used as the input data for crack recognition.

[0047] Among them, the specific structure of the pre-trained multi-scale road surface damage recognition network model is a deep neural network composed of a feature extraction layer, a multi-scale feature fusion layer, an attention mechanism layer, and a classification layer; the feature extraction layer uses multiple convolutional layers with different convolutional kernel sizes to extract texture features of different scales on the road surface, and then the multi-scale feature fusion layer adaptively fuses the feature information of different scales; the attention mechanism layer introduces a spatial attention module to automatically learn the salient features of the crack area, and the attention weight distribution parameters are determined by the crack depth value and the crack length-width ratio parameters; finally, the classification layer realizes the crack type recognition and localization tasks based on the multi-scale fusion features, and the network structure design includes a skip connection mechanism to effectively solve the problem of scale inconsistency in road surface crack detection.

[0048] Among them, the steps for establishing the training data set in the pre-training process of the pre-trained multi-scale road surface damage recognition network model specifically include first collecting road surface image data of different road types, including various road types such as asphalt roads and concrete roads; then manually annotating the collected data, and the annotation content includes crack position, crack type, and crack severity information; then performing enhancement processing on the annotated data, including rotation transformation, scale transformation, and brightness adjustment methods to generate diverse training samples; then dividing the enhanced training data into a training set, a validation set, and a test set, where the training set is used for training model parameters, the validation set is used for tuning model parameters, and the test set is used for evaluating model performance; finally, establishing a training sample database, including original images, annotation information, and image feature content.

[0049] Among them, the steps of pre-training the multi-scale road surface damage recognition network model specifically include first determining the model optimization objective function, adopting the combined form of cross-entropy loss function and reconstruction loss function; then determining the number of network layers and the hyperparameter settings of each layer, including the convolution kernel size, activation function type, pooling method, and the attention mechanism weight coefficient is determined according to the crack feature significance; then using the mini-batch stochastic gradient descent method for model training, and the learning rate adopts the cosine annealing strategy and is dynamically adjusted according to the performance of the validation set; then introducing an early stopping strategy to avoid model overfitting, and stopping training when the performance of the validation set has not improved for multiple consecutive rounds; finally, saving the model parameter weights after training is completed to obtain the pre-trained multi-scale road surface damage recognition network model.

[0050] Among them, the crack recognition results include crack position information, crack area information, and crack confidence information.

[0051] Among them, the crack type classification includes the classification results of transverse cracks, longitudinal cracks, reticular cracks, and crocodile cracks.

[0052] The following describes the specific implementation manners of the above steps in detail.

[0053] The specific implementation manner of step S01 is to continuously scan the road surface using the line-structured light scanner carried by the detection vehicle. The line-structured light scanner adopts the principle of laser triangulation. By projecting a linear laser beam onto the road surface and using a CCD or CMOS sensor to capture the position change information of the reflected light, the light offset is converted into elevation information to achieve the three-dimensional profile measurement of the road surface. In practical applications, the line-structured light scanner scans the road surface at a frequency of 500Hz to 1000Hz to ensure that road surface elevation data with an accuracy of not less than 1mm can be obtained when the vehicle is traveling at a speed of 60km / h. During the scanning process, a transverse scanning line is recorded every 1cm to 2cm along the vehicle traveling direction, and each transverse scanning line contains 1024 to 2048 elevation measurement points, forming a matrix of the original laser point cloud elevation data of the road surface. The purpose of this step is to obtain high-precision three-dimensional geometric information of the road surface and provide basic data for subsequent crack detection.

[0054] The specific implementation of step S02 is to measure the real-time attitude parameters of the line-structured light scanner by detecting the inertial navigation system carried by the vehicle, and perform attitude compensation processing on the elevation data matrix of the original road surface laser point cloud. The inertial navigation system adopts the device composed of an accelerometer, a gyroscope and a GPS module in the prior art, and the acquisition frequency is not less than 100 Hz, which can provide the vehicle's acceleration and angular velocity information in the three-axis directions. First, perform Kalman filtering on the attitude data collected by the inertial navigation system to eliminate the influence of noise; then calculate the attitude compensation matrix according to the filtered pitch angle, roll angle and heading angle; finally, apply the attitude compensation matrix to the elevation data matrix of the original road surface laser point cloud, and eliminate the influence of vehicle attitude changes on the measurement results through coordinate transformation to obtain the corrected laser point cloud elevation data matrix. The purpose of this step is to eliminate the measurement errors caused by the vehicle's bumps, turns and other movements during driving and improve the accuracy of elevation data.

[0055] The specific implementation of step S03 is to perform two-way moving average filtering on the corrected laser point cloud elevation data matrix. First, apply a one-dimensional moving average filter to each row of elevation data along the transverse direction (X direction) of the road surface, and set the half-width of the filtering window to 5 to 15 data points; then apply the one-dimensional moving average filter again to the transverse filtering result along the longitudinal direction (Y direction) of the road surface, and the half-width of the filtering window is also set to 5 to 15 data points. The two-way moving average filtering smooths the road surface texture fluctuations through local area average operations, retains the large-scale elevation change characteristics, effectively eliminates the influence of measurement noise and road surface texture, and highlights the road surface disease characteristics such as cracks. After the filtering process, a road surface depth matrix is obtained, where each element represents the depth value of the road surface at the corresponding position relative to the ideal plane. The purpose of this step is to extract the abnormal depression characteristics of the road surface and provide basic data for crack identification.

[0056] The specific implementation of step S04 is to normalize the road surface depth matrix into a depth image and perform image segmentation. First, determine the maximum depth value and the minimum depth value in the road surface depth matrix; then linearly map the depth value to the gray scale range of 0 to 255 to form a depth image, where the dark area represents the sunken part of the road surface and the light area represents the flat road surface; finally, evenly divide the depth image into multiple initial detection image blocks according to the preset size, and the size of each initial detection image block is set to 256×256 pixels or 512×512 pixels, and a 25% to 30% overlapping area is set between adjacent image blocks to avoid cracks being cut and unable to be completely identified. The purpose of this step is to convert the continuous road surface depth data into an image format suitable for image processing and deep learning algorithms, and improve the calculation efficiency through block processing.

[0057] The specific implementation of step S05 is to use the Gaussian background modeling method to screen the depth anomaly points in each initial detection image block. First, the pixel gray value statistical histogram of the initial detection image block is calculated; then, it is assumed that the pixel gray value distribution in the normal road surface area conforms to the Gaussian distribution, and the expectation maximization algorithm is used to fit the gray histogram to obtain the gray mean μ and standard deviation σ; then, the absolute difference between the gray value of each pixel point and the mean is calculated; after that, the adaptive threshold D = μ + kσ is set, where k is the adaptive coefficient, and the value range is 2.5 to 3.5; finally, the pixel points with the absolute difference between the gray value and the mean greater than the adaptive threshold are marked as potential crack feature points. The Gaussian background modeling method makes full use of the gray distribution characteristics of the normal road surface and identifies anomaly points through statistical significance testing, with strong anti-interference ability. The purpose of this step is to preliminarily screen out the anomaly depth points that may form cracks and lay a foundation for subsequent connectivity analysis.

[0058] The specific implementation of step S06 is to construct a connected graph structure for all potential crack feature points through the minimum spanning tree algorithm. First, an undirected graph is established with potential crack feature points as graph nodes; then, the Euclidean distance between each pair of nodes is calculated as the edge weight; then, the Kruskal algorithm or Prim algorithm is applied to construct the minimum spanning tree to obtain a connected structure that includes all nodes and has the minimum total weight; after that, a distance threshold (usually 5 to 10 pixels) is set, and the edges with weights greater than the threshold are pruned to decompose the connected graph into multiple subgraphs; finally, the subgraphs with the number of nodes greater than the threshold (usually 20 to 30 nodes) are retained to obtain the crack skeleton structure. The minimum spanning tree algorithm draws on the idea of finding the minimum weight connected subgraph in graph theory, effectively identifies the spatial connected structure in potential crack feature points, excludes isolated anomaly points, and extracts the main skeleton of the crack. The purpose of this step is to use the spatial continuity characteristics of the crack to organize discrete feature points into meaningful structured information.

[0059] The specific implementation of step S07 is to optimize the crack skeleton structure by applying a crack feature enhancement function. The crack feature enhancement function takes the crack skeleton structure matrix, the crack area depth matrix, the crack area texture gradient matrix, the crack direction vector matrix, and the crack connectivity matrix as input parameters and outputs an enhanced crack feature image matrix. First, the main axis of the crack is extracted according to the crack skeleton structure; then, the depth distribution characteristics of the crack area are calculated, including the maximum depth, the average depth, and the depth variance; next, the texture gradient characteristics of the crack edge are analyzed, and the gradient amplitude and direction information are extracted; then, the direction consistency measure is calculated according to the crack orientation in the local area; finally, by integrating the above features, the contrast between the crack area and the background area is enhanced through a non-linear mapping function to generate an optimized crack feature image block. The crack feature enhancement function adopts the principle of multi-feature fusion, makes full use of the features of cracks in terms of depth, texture, direction, and connectivity, etc., and improves the integrity and saliency of the crack structure. The purpose of this step is to strengthen the expression of crack features, suppress background interference, and provide high-quality input for subsequent deep learning network recognition.

[0060] The specific implementation of step S08 is to input the optimized crack feature image block into a pre-trained multi-scale road surface damage recognition network model. This network model adopts a multi-scale convolutional neural network structure, including a feature extraction layer, a multi-scale feature fusion layer, an attention mechanism layer, and a classification layer. The feature extraction layer uses three different sizes of convolutional kernels, 3×3, 5×5, and 7×7, to extract image features in parallel; the multi-scale feature fusion layer integrates features with different receptive fields through cross-scale connection methods; the attention mechanism layer adaptively allocates attention weights according to the crack depth value and aspect ratio; the classification layer consists of fully connected layers and outputs the crack recognition result and the crack type classification. The network training adopts a combination form of cross-entropy loss function and reconstruction loss function, optimizes the network parameters by the mini-batch stochastic gradient descent method, and the learning rate adopts the cosine annealing strategy. The model is pre-trained on a large-scale road surface crack dataset, and the recognition accuracy can reach more than 95%, and it can effectively distinguish different types of road surface cracks such as transverse cracks, longitudinal cracks, mesh cracks, and block cracks. The purpose of this step is to utilize the powerful feature extraction and classification capabilities of the deep learning model to realize the automatic detection and classification of road surface cracks and output the final recognition result.

[0061] Furthermore, the specific implementation of the process for establishing the pre-training dataset of the pre-trained multi-scale road surface damage recognition network model adopts a systematic data collection, annotation, enhancement, and organization process. First, multi-scenario road surface data is collected. A line-structured light scanner is used to conduct full-coverage scans of different types of road surfaces, including asphalt road surfaces, cement concrete road surfaces, composite road surfaces, etc. with different materials. At the same time, it covers different usage environments such as urban roads, highways, and rural roads, as well as road surfaces with different usage years such as newly built road surfaces, in-use road surfaces, and aging road surfaces, ensuring that the collected data has sufficient diversity and representativeness. The total collection mileage is not less than 1000 kilometers, and the resolution of the collected road surface images is not lower than 1mm / pixel.

[0062] Then, data annotation is carried out. An expert team in road engineering is organized to conduct refined annotation on the collected road surface data according to the "Highway Technical Condition Evaluation Standard". The annotation content includes three dimensions: crack location, crack type, and crack severity. The crack location adopts a pixel-level precise boundary annotation method, using polygons or curves to precisely outline the crack boundary; the crack type annotation includes five basic types: transverse cracks, longitudinal cracks, diagonal cracks, reticulated cracks, and crazing; the crack severity is divided into three levels according to the crack width: slight (width < 3mm), medium (3mm ≤ width < 5mm), and severe (width ≥ 5mm). Optionally, a multi-person cross-validation mechanism is implemented in the annotation process. Each sample is independently annotated by more than two experts, and the inconsistent areas are adjudicated by senior experts to ensure that the annotation accuracy is not lower than 98%.

[0063] After that, data enhancement processing is carried out. A variety of data enhancement techniques are applied to the original dataset after annotation to generate an augmented dataset. First, geometric transformation enhancement is applied, including horizontal flipping, vertical flipping, random rotation (within the range of ±15°), random scaling (within the range of 0.8 - 1.2 times), and random cropping (retaining 80% - 100% of the original area); then, optical property enhancement is applied, including brightness adjustment (within the range of ±20%), contrast adjustment (within the range of ±20%), sharpening processing, and Gaussian blur processing (σ = 0.5 - 1.5); afterwards, noise addition enhancement is applied, including Gaussian noise (σ = 5 - 15), salt-and-pepper noise (proportion 0.01 - 0.05), and random occlusion (the proportion of the occluded area is 5% - 10%). Through data enhancement processing, the original dataset is expanded by 10 - 15 times, effectively alleviating the problem of insufficient data volume and at the same time improving the model's adaptability to various changing conditions.

[0064] After that, the dataset is partitioned. The enhanced complete dataset is randomly partitioned into a training set, a validation set, and a test set according to the ratio of 7:2:1, while ensuring that the distribution of crack samples of different types and severities is balanced in each subset. The training set is used for training the model parameters, the validation set is used for parameter tuning and early stopping judgment during the training process, and the test set is used for evaluating the performance of the final model. The stratified sampling strategy is adopted during the partitioning process to ensure that the distribution ratio of various types of cracks in each subset is the same as that of the original dataset, avoiding data bias problems.

[0065] Finally, a training sample database is established. All the processed data is organized and stored in an orderly manner. Each sample contains four parts of information: the original image data, the annotation information data, the image feature data, and the metadata. The original image data stores the preprocessed pavement depth image; the annotation information data stores the crack boundary coordinates, type codes, and severity levels; the image feature data stores the pre-extracted traditional image features such as HOG features and LBP features; the metadata stores the background information such as the sample collection time, location, pavement type, and service life. The database establishes an indexing mechanism to support fast retrieval of samples in multiple dimensions such as crack type, severity, and pavement type. The total capacity of the training sample database contains no less than 1 million valid samples, covering the typical feature manifestations of various types of cracks under different conditions, providing sufficient and diverse learning materials for pre-training the multi-scale pavement damage recognition network model.

[0066] Next, the mathematical models or calculation processes involved in the present invention are described in detail.

[0067] In step S02, the mathematical expression for performing Kalman filtering on the attitude data measured by the inertial navigation system is specifically as follows:

[0068] X k = FX k-1 + w k-1 ;

[0069] Z k = HX k + v k ;

[0070] In the formula, X k is the state vector at the current moment, including the position, velocity, acceleration, and attitude angle information of the line structured light scanner; X k-1 is the state vector at the previous moment; F is the state transition matrix; w k-1 is the process noise, which follows a Gaussian distribution with a mean of zero and a covariance of Q; Z k is the observation vector, including the acceleration, angular velocity, etc. data directly measured by the inertial navigation system; H is the observation matrix; v k is the observation noise, which follows a Gaussian distribution with a mean of zero and a covariance of R.

[0071] The prediction step of the Kalman filter is expressed as:

[0072] X k|k-1 = FX k-1|k-1 ;

[0073] P k|k-1 = FP k-1|k-1 F T + Q;

[0074] In the formula, X k|k-1 is the prior state estimate; P k|k-1 is the prior estimate error covariance matrix; P k-1|k-1 is the posterior estimate error covariance matrix at the previous moment; F T is the transpose of the state transition matrix.

[0075] The update step of the Kalman filter is expressed as:

[0076] K k = P k|k-1 H T (HP k|k-1 H T + R) -1 ;

[0077] X k|k = X k|k-1 + K k (Z k - HX k|k-1 );

[0078] P k|k = (I - K k H)P k|k-1 ;

[0079] In the formula, K k is the Kalman gain matrix; X k|k is the posterior state estimate; P k|k is the posterior estimate error covariance matrix; I is the identity matrix.

[0080] The calculation of the attitude compensation matrix is expressed as:

[0081] R compensation = R z (ψ)R y (θ)R x (φ);

[0082] In the formula, R compensation is the attitude compensation rotation matrix; R x (φ), R y (θ), R z$(ψ)$ are the basic rotation matrices about the X-axis (roll angle φ), Y-axis (pitch angle θ), and Z-axis (heading angle ψ) respectively.

[0083] The calculation expression of the corrected laser point cloud elevation data matrix is:

[0084] P corrected = R compensation P original + T;

[0085] Wherein, P corrected is the corrected point cloud coordinate; P original is the original point cloud coordinate; T is the translation vector, calculated according to the vehicle motion state.

[0086] In step S03, the mathematical expression of the bidirectional moving average filtering process is as follows:

[0087] Lateral filtering calculation formula:

[0088]

[0089] Wherein, Z x (i, j) is the elevation value at the position (i, j) after lateral filtering; Z(k, j) is the original elevation value at the position (k, j) in the corrected laser point cloud elevation data matrix; w x is the half-width of the lateral filtering window, and the value range is 5 to 15.

[0090] Longitudinal filtering calculation formula:

[0091]

[0092] Wherein, Z xy (i, j) is the final elevation value at the position (i, j) after bidirectional filtering, that is, the element in the road surface depth matrix; Z x (i, l) is the elevation value at the position (i, l) after lateral filtering; w y is the half-width of the longitudinal filtering window, and the value range is 5 to 15.

[0093] In step S04, the mathematical expression of the depth normalization conversion is as follows:

[0094]

[0095] Wherein, Z img (i, j) is the gray value at the position (i, j) after normalization; Z xy (i, j) is the depth value at the position (i, j) in the road surface depth matrix; Z max and Z min are the maximum and minimum values in the road surface depth matrix respectively.

[0096] In step S05, the mathematical expression of Gaussian background modeling is as follows:

[0097] The iterative process of the Expectation-Maximization algorithm is expressed as:

[0098] E step (expectation calculation):

[0099]

[0100] where γ ik represents the posterior probability that pixel i belongs to the k-th Gaussian component; π k is the weight of the k-th Gaussian component; represents the probability density of the Gaussian distribution with mean μ k and variance at x i ; K is the number of Gaussian components, and usually K = 1 for road surface background modeling.

[0101] M step (maximization calculation):

[0102]

[0103] where are the updated mean, variance, and weight of the k-th Gaussian component respectively; N is the total number of pixels.

[0104] Adaptive threshold calculation formula:

[0105] D = μ + kσ;

[0106] where D is the adaptive threshold; μ is the mean of the Gaussian distribution, representing the average gray value of the normal road surface; σ is the standard deviation of the Gaussian distribution, indicating the fluctuation range of the gray value of the normal road surface; k is the adaptive coefficient, and its value range is 2.5 - 3.5.

[0107] Determination conditions for potential crack feature points:

[0108] |Z img (i, j) - μ| > D;

[0109] where Z img (i, j) is the gray value of the pixel at position (i, j); μ is the average gray value of the normal road surface; D is the adaptive threshold.

[0110] In step S06, the mathematical expression of the minimum spanning tree algorithm is as follows:

[0111] Euclidean distance calculation formula between nodes:

[0112]

[0113] where d ij is the Euclidean distance between node i and node j; (x i , y i ) and (x j , y j ) are the coordinates of node i and node j, respectively.

[0114] Objective function of Kruskal's algorithm:

[0115] min∑ e∈MST w(e);

[0116] s.t. MST is a connected graph and acyclic;

[0117] where MST represents the minimum spanning tree; e represents an edge; w(e) represents the weight of edge e, which is the corresponding Euclidean distance.

[0118] Judgment condition for pruning operation:

[0119] d ij > d threshold ;

[0120] where d ij is the weight of edge (i, j); d threshold is the distance threshold, and its value range is 5 to 10 pixels.

[0121] Subgraph retention condition:

[0122] |V subgraph | > N threshold ;

[0123] where |V subgraph | represents the number of nodes in the subgraph; N threshold is the node number threshold, and its value range is 20 to 30.

[0124] In step S07, the mathematical expression of the crack feature enhancement function is specifically as follows:

[0125] The main axis of the crack is extracted using the principal component analysis (PCA) method:

[0126]

[0127] where C is the covariance matrix; p i is the coordinate of the i-th crack feature point; is the average coordinate of all crack feature points; n is the total number of crack feature points.

[0128] By solving the characteristic equation:

[0129]

[0130] The eigenvalues λ1≥λ2 and the corresponding eigenvectors are obtained. and where represents the main direction vector of the crack.

[0131] Crack depth feature calculation formula:

[0132] D max = max (i，j)∈Ω Z xy (i, j);

[0133]

[0134] In the formula, D max , D avg , D var are the maximum depth, average depth and depth variance of the crack region Ω, respectively;

[0135] Z xy (i, j) is the pavement depth value at position (i, j); |Ω| is the number of pixels included in the crack region.

[0136] Crack edge gradient feature calculation formula:

[0137] G x (i, j) = Z img (i + 1, j) - Z img (i - 1, j);

[0138] G y (i, j) = Z img (i, j + 1) - Z img (i, j - 1);

[0139]

[0140] In the formula, G x (i, j) and G y (i, j) are the horizontal and vertical gradient components at position (i, j), respectively; G mag (i, j) and G dir (i, j) are the gradient amplitude and direction at position (i, j), respectively.

[0141] Direction consistency measure calculation formula:

[0142]

[0143] In the formula, C dir is the direction consistency measure; θ ij is the gradient direction at position (i, j); θ mainis the angle of the main crack direction; |Ω| is the number of pixels included in the crack area.

[0144] Feature enhancement non - linear mapping function:

[0145] Z enhanced (i, j) = α · Z img (i, j) · (1 + β · G mag (i, j) · C dir ) γ · e -δ·d(i ， j) ;

[0146] In the formula, Z enhanced (i, j) is the feature value at position (i, j) after enhancement; Z img (i, j) is the gray value of the original depth image at position (i, i); G mag (i, j) is the gradient amplitude at position (i, j); C dir is the direction consistency measure; d(i, j) is the distance from position (i, j) to the main crack axis; α, β, γ, δ are control parameters, where α controls the overall enhancement intensity, and its value range is 1.2 - 2.0; β controls the contribution degree of gradient features, and its value range is 0.01 - 0.05; γ controls the non - linear degree, and its value range is 1.5 - 2.5; δ controls the distance attenuation rate, and its value range is 0.05 - 0.15.

[0147] Mathematical expression of the cross - entropy loss function:

[0148]

[0149] In the formula, L CE is the cross - entropy loss; N is the number of samples; C is the number of crack categories; y ic is the true label (0 or 1) of sample i belonging to category c; p ic is the probability that the model predicts sample i belongs to category c.

[0150] Mathematical expression of the reconstruction loss function:

[0151]

[0152] In the formula, L Recon is the reconstruction loss; X i is the input image; is the reconstructed image; ||·||2 represents the Euclidean norm.

[0153] Mathematical expression of the combined loss function:

[0154]

[0155] In the formula, L Total is the total loss; λ is the reconstruction loss weight coefficient, and its value range is 0.1 to 0.5; μ is the regularization coefficient, and its value range is 0.0001 to 0.001; is the L2 regularization term of the model parameters, which is used to prevent overfitting.

[0156] Mathematical expression of the cosine annealing strategy of the learning rate:

[0157]

[0158] In the formula, η t is the learning rate of the t-th iteration; η min and η max are the minimum and maximum learning rates respectively. Usually, η min = 0.00001, η max = 0.001; T is the total number of iterations.

[0159] All these formulas are constructed considering the physical characteristics and mathematical principles of pavement crack detection. Kalman filtering adopts a state space model, effectively fuses multi-source information through two steps of prediction and update, and smooths the influence of noise. Bi-directional moving average filtering utilizes the difference in spatial distribution between cracks and normal pavement, eliminates random noise through local averaging, and retains crack features. Gaussian background modeling is based on the assumption that the gray distribution of normal pavement conforms to the Gaussian distribution, and identifies abnormal points through the principle of statistical significance. The minimum spanning tree algorithm transforms crack detection into a graph theory problem, and effectively organizes feature points by using the spatial continuity characteristics of cracks. The crack feature enhancement function comprehensively considers depth, gradient, direction, and position information, enhances the contrast between cracks and the background through non-linear mapping, where the exponential term and power term introduce non-linear characteristics, which are more in line with the visual perception mechanism, and the distance attenuation term ensures smooth transition and improves the robustness of features.

[0160] Specifically, the principle of the present invention is:

[0161] The core principle of the present invention lies in constructing a complete technical route from data acquisition, feature extraction to crack recognition, and solving the problem of poor adaptability to complex pavement environments. First of all, the line structured light scanner is based on the principle of laser triangulation, and can accurately collect the three-dimensional contour information of the pavement, providing richer depth information compared with traditional two-dimensional image methods; the inertial navigation system measures the vehicle attitude parameters in real time, compensates and processes the collected data, and eliminates the influence of vehicle movement on data quality, which is the basis for improving the system's environmental adaptability.

[0162] The innovation in the data processing stage lies in the organic combination of depth information and geometric features. The bidirectional moving average filter performs sliding window calculations in both the transverse and longitudinal directions of the road surface, which can effectively separate the road surface texture and crack features; the Gaussian background modeling estimates the road surface depth distribution parameters through the expectation maximization algorithm, enabling the system to adaptively determine the anomaly point threshold and overcoming the defect of poor adaptability of the fixed threshold method to complex environments. The minimum spanning tree algorithm constructs a connected graph structure from discrete potential crack feature points and prunes through a distance threshold, retaining the continuity features of the cracks and eliminating the interference of noise points at the same time.

[0163] The design of the crack feature enhancement function is the key innovation of the present invention. It constructs a complete representation of the crack features by fusing multi-dimensional features such as the depth matrix of the crack area, the texture gradient matrix, the direction vector matrix, and the connectivity matrix, enabling the system to distinguish the geometric characteristics of different types of cracks. The pre-trained multi-scale road surface damage recognition network extracts features using multiple scale convolutional kernels, introduces a spatial attention mechanism to automatically learn the saliency of the crack area, and solves the scale inconsistency problem through skip connections. These designs enable the model to have the adaptability to different road surface materials and different crack types.

[0164] In addition, diverse training data sets and data augmentation techniques are adopted during the pre-training process, further improving the generalization ability of the model. The combined optimization objective of cross-entropy loss and reconstruction loss, as well as the cosine annealing learning rate strategy, ensure that the model can effectively capture the essential information of the crack features rather than simply memorizing the surface features of the training samples.

[0165] In summary, the present invention constructs an intelligent crack detection method with high adaptability to complex road surface environments through multi-source data fusion, multi-level feature extraction, and multi-scale deep learning. Its technical route is complete and the principle is clear, which can effectively solve the problem of poor environmental adaptability existing in the prior art.

[0166] A specific embodiment 1 of the present invention is provided below. The specific implementation manners of each step in this embodiment 1 are described in detail as follows.

[0167] The specific implementation manner of step S01 is the same as that described above and will not be elaborated here.

[0168] The specific implementation manner of step S02 is to measure the real-time attitude parameters of the line structure light scanner through the inertial navigation system carried by the detection vehicle and perform attitude compensation processing on the original laser point cloud elevation data matrix of the road surface. The inertial navigation system consists of an accelerometer, a gyroscope, and a GPS module, with a sampling frequency of not less than 100 Hz, providing vehicle three-axis acceleration and angular velocity information. First, perform Kalman filtering on the attitude data collected by the inertial navigation system. The state equation and observation equation of the Kalman filter are respectively:

[0169] X k = FX k-1 + w k-1 ;

[0170] Z k = HX k + v k ;

[0171] In the formula, X k is the state vector at the current moment, including the position, velocity, acceleration, and attitude angle information of the line structured light scanner; X k-1 is the state vector at the previous moment; F is the state transition matrix; w k-1 is the process noise, following a Gaussian distribution with a mean of zero and a covariance of Q; Z k is the observation vector, including data such as the acceleration and angular velocity directly measured by the inertial navigation system; H is the observation matrix; v k is the observation noise, following a Gaussian distribution with a mean of zero and a covariance of R.

[0172] The prediction and update steps of the Kalman filter are as follows:

[0173] Prediction step:

[0174] X k|k-1 = FX k-1|k-1 ;

[0175] P k|k-1 = FP k-1|k-1 F T + Q;

[0176] Update step:

[0177] K k = P k|k-1 H T (HP k|k-1 H T + R) -1 ;

[0178] X k|k = X k|k-1 + K k (Z k - HX k|k-1 );

[0179] P k|k = (I - K k H)P k|k-1 ;

[0180] In the formula, X k|k-1 is the prior state estimate; P k|k-1 is the prior estimate error covariance matrix; P k-1|k-1is the posteriori estimation error covariance matrix at the previous moment; F T is the transpose of the state transition matrix; K k is the Kalman gain matrix; X k|k is the posteriori state estimation; P k|k is the posteriori estimation error covariance matrix; I is the identity matrix.

[0181] Then, calculate the attitude compensation matrix according to the filtered pitch angle, roll angle and heading angle:

[0182] R compensation = R z (ψ)R y (θ)R x ;(φ);

[0183] In the formula, R compensation is the attitude compensation rotation matrix; R x (φ), R y (θ), R z (ψ) are the basic rotation matrices about the X-axis (roll angle φ), Y-axis (pitch angle θ), and Z-axis (heading angle ψ), respectively.

[0184] Finally, apply the attitude compensation matrix to the original road surface laser point cloud elevation data matrix to eliminate the influence of vehicle attitude changes on the measurement results through coordinate transformation:

[0185] P corrected = R compensation P original + T;

[0186] In the formula, P corrected is the corrected point cloud coordinate; P original is the original point cloud coordinate; T is the translation vector, which is calculated according to the vehicle motion state. The purpose of this step is to eliminate the measurement errors caused by vehicle movements such as bumps and steering during driving and improve the accuracy of elevation data.

[0187] The specific implementation of step S03 is to perform two-way moving average filtering on the corrected laser point cloud elevation data matrix. The two-way moving average filtering smooths the road surface texture fluctuations through local area average operations, retains the elevation change characteristics at a larger scale, effectively eliminates the influence of measurement noise and road surface texture, and highlights the road surface disease characteristics such as cracks. First, apply a one-dimensional moving average filter to each row of elevation data along the road surface horizontal direction (X direction):

[0188]

[0189] In the formula, Z x(i, j) is the elevation value at position (i, j) after horizontal filtering; Z(k, j) is the original elevation value at position (k, j) in the corrected laser point cloud elevation data matrix; w x is the half-width of the horizontal filtering window, and its value range is 5 to 15 data points. Then, apply a one-dimensional moving average filter to the horizontal filtering result along the longitudinal direction (Y direction) of the road surface again:

[0190]

[0191] In the formula, Z xy (i, j) is the final elevation value at position (i, j) after bidirectional filtering, that is, the element in the road surface depth matrix; Z x (i, l) is the elevation value at position (i, l) after horizontal filtering; w y is the half-width of the longitudinal filtering window, and its value range is 5 to 15 data points. After the filtering process, a road surface depth matrix is obtained, where each element represents the depth value of the road surface at the corresponding position relative to the ideal plane. The purpose of this step is to extract the abnormal depression features of the road surface and provide basic data for crack identification.

[0192] The specific implementation method of step S04 is to normalize and convert the road surface depth matrix into a depth image and perform image segmentation. First, linearly map the depth values in the road surface depth matrix to the gray scale range of 0 to 255 to form a depth image:

[0193]

[0194] In the formula, Z img (i, j) is the gray scale value at position (i, j) after normalization; Z xy (i, j) is the depth value at position (i, j) in the road surface depth matrix; Z max and Z min are the maximum value and the minimum value in the road surface depth matrix respectively. The dark area represents the sunken part of the road surface, and the light area represents the flat road surface. Then, evenly divide the depth image into multiple initial detection image blocks according to the preset size. The size of each initial detection image block is set to 256×256 pixels or 512×512 pixels, and a 25% - 30% overlapping area is set between adjacent image blocks to avoid cracks being split and unable to be completely recognized. The purpose of this step is to convert the continuous road surface depth data into an image format suitable for image processing and deep learning algorithms and improve the calculation efficiency through block processing.

[0195] The specific implementation of step S05 is to use the Gaussian background modeling method to screen the depth abnormal points in each initial detection image block. First, the pixel gray value statistical histogram of the initial detection image block is obtained; then, it is assumed that the pixel gray value distribution in the normal road surface area conforms to the Gaussian distribution, and the expectation maximization algorithm is used to fit the gray histogram. The iterative process of the expectation maximization algorithm includes the E step and the M step:

[0196] E step (expectation calculation):

[0197]

[0198] M step (maximization calculation):

[0199]

[0200] In the formula, γ ik represents the posterior probability that pixel i belongs to the k-th Gaussian component; π k is the weight of the k-th Gaussian component; represents the probability density of the Gaussian distribution with mean μ k and variance at x i ; K is the number of Gaussian components, and usually K = 1 for road surface background modeling; are the updated mean, variance, and weight of the k-th Gaussian component respectively; N is the total number of pixels.

[0201] After obtaining the gray mean μ and standard deviation σ, calculate the adaptive threshold:

[0202] D = μ + kσ;

[0203] In the formula, D is the adaptive threshold; μ is the mean of the Gaussian distribution, representing the average gray value of the normal road surface; σ is the standard deviation of the Gaussian distribution, indicating the fluctuation range of the normal road surface gray value; k is the adaptive coefficient, and its value range is 2.5 - 3.5. Finally, mark the potential crack feature points according to the judgment condition:

[0204] |Z img (i, j) - μ| > D;

[0205] In the formula, Z img (i, j) is the gray value of the pixel at position (i, j). The Gaussian background modeling method makes full use of the gray distribution characteristics of the normal road surface and identifies abnormal points through statistical significance testing, with strong anti-interference ability. The purpose of this step is to preliminarily screen out the abnormal depth points that may form cracks and lay a foundation for subsequent connectivity analysis.

[0206] The specific implementation of step S06 is to construct a connected graph structure for all potential crack feature points through the minimum spanning tree algorithm. The minimum spanning tree algorithm is a classic graph theory algorithm. Drawing on the idea of finding the minimum-weight connected subgraph in graph theory, it effectively identifies the spatial connection structure among potential crack feature points, excludes isolated outliers, and extracts the main skeleton of the crack. First, an undirected graph is established with potential crack feature points as graph nodes; then, the Euclidean distance between each pair of nodes is calculated as the edge weight:

[0207]

[0208] where d ij is the Euclidean distance between node i and node j; (x i , y i ) and (x j , y j ) are the coordinates of node i and node j respectively. Then, the Kruskal algorithm or Prim algorithm is applied to construct the minimum spanning tree. The objective function of the Kruskal algorithm is:

[0209] min∑ e∈MST w(e);

[0210] s.t. MST is a connected graph and acyclic;

[0211] where MST represents the minimum spanning tree; e represents the edge; w(e) represents the weight of edge e, that is, the corresponding Euclidean distance. After that, a distance threshold is set, and the edges with weights greater than the threshold are pruned to decompose the connected graph into multiple subgraphs:

[0212] d ij >d threshold ;

[0213] where d ij is the weight of edge (i, j); d threshold is the distance threshold, and its value range is 5 - 10 pixels. Finally, the subgraphs with the number of nodes greater than the threshold are retained to obtain the crack skeleton structure:

[0214] |V subgraph |>N threshold ;

[0215] where |V subgraph | represents the number of nodes in the subgraph; N threshold is the node number threshold, and its value range is 20 - 30. The purpose of this step is to organize discrete feature points into meaningful structured information by using the spatial continuity characteristics of the crack.

[0216] The specific implementation of step S07 is to optimize the crack skeleton structure by applying a crack feature enhancement function. The crack feature enhancement function adopts the principle of multi-feature fusion, making full use of the features of cracks in terms of depth, texture, direction, and connectivity to improve the integrity and saliency of the crack structure. First, the main axis of the crack is extracted from the crack skeleton structure using the principal component analysis (PCA) method:

[0217]

[0218] where C is the covariance matrix; p i is the coordinate of the i-th crack feature point; is the average coordinate of all crack feature points; n is the total number of crack feature points. By solving the characteristic equation the eigenvalues λ1≥λ2 and the corresponding eigenvectors and are obtained, where represents the main direction vector of the crack.

[0219] Then, the depth distribution feature of the crack region is calculated:

[0220] D max =max (i，j)∈Ω Z xy (i, j);

[0221]

[0222] where D max , D avg , D var are the maximum depth, average depth, and depth variance of the crack region Ω, respectively; Z xy (i, j) is the pavement depth value at position (i, j); |Ω| is the number of pixels included in the crack region.

[0223] Next, the texture gradient feature of the crack edge is analyzed:

[0224] G x (i, j)=Z img (i + 1, j)-Z img (i - 1, j);

[0225] G y (i, j)=Z img (i, j + 1)-Z img (i, j - 1);

[0226]

[0227] where G x (i, j) and G y(i, j) are the horizontal and vertical gradient components at position (i, j); G mag (i, j) and G dir (i, j) are the gradient magnitude and direction at position (i, j), respectively.

[0228] After that, the direction consistency measure is calculated according to the crack orientation in the local area:

[0229]

[0230] In the formula, C dir is the direction consistency measure; θ ij is the gradient direction at position (i, j); θ main is the crack main direction angle; |Ω| is the number of pixels included in the crack area.

[0231] Finally, by integrating the above features, the contrast between the crack area and the background area is enhanced through a non-linear mapping function:

[0232] Z enhanced (i, j) = α · Z img (i, j) · (1 + β · G mag (i, j) · C dir ) γ · e -δ·d(i ， j) ;

[0233] In the formula, Z enhanced (i, j) is the eigenvalue after enhancement at position (i, j); Z img (i, j) is the gray value of the original depth image at position (i, j); G mag (i, j) is the gradient magnitude at position (i, j); C dir is the direction consistency measure; d(i, j) is the distance from position (i, j) to the crack main axis; α, β, γ, δ are control parameters, where α controls the overall enhancement intensity, and its value range is 1.2 - 2.0; β controls the contribution degree of the gradient feature, and its value range is 0.01 - 0.05; γ controls the non-linear degree, and its value range is 1.5 - 2.5; δ controls the distance attenuation rate, and its value range is 0.05 - 0.15. The purpose of this step is to strengthen the crack feature expression, suppress the background interference, and provide high-quality input for the subsequent deep learning network recognition.

[0234] The specific implementation of step S08 is to input the optimized crack feature image patches into a pre-trained multi-scale road surface damage recognition network model. This network model adopts a multi-scale convolutional neural network structure, including a feature extraction layer, a multi-scale feature fusion layer, an attention mechanism layer, and a classification layer. The feature extraction layer uses three different sizes of convolutional kernels, 3×3, 5×5, and 7×7, to extract image features in parallel; the multi-scale feature fusion layer integrates features with different receptive fields through cross-scale connection methods; the attention mechanism layer adaptively assigns attention weights according to the crack depth value and aspect ratio; the classification layer consists of fully connected layers, and outputs the crack recognition result and the crack type classification. The network training adopts a combined form of cross-entropy loss function and reconstruction loss function:

[0235]

[0236] where L CE is the cross-entropy loss; N is the number of samples; C is the number of crack categories; y ic is the true label (0 or 1) of sample i belonging to category c; p ic is the probability that the model predicts sample i belongs to category c; L Recon is the reconstruction loss; X i is the input image; is the reconstructed image; ||·||2 represents the Euclidean norm; L Total is the total loss; λ is the reconstruction loss weight coefficient, and its value range is 0.1~0.5; μ is the regularization coefficient, and its value range is 0.0001~0.001; is the L2 regularization term of the model parameters, which is used to prevent overfitting.

[0237] The network training optimizes the network parameters through the mini-batch stochastic gradient descent method, and the learning rate adopts the cosine annealing strategy:

[0238]

[0239] where η t is the learning rate of the t-th iteration; η min and η max are the minimum and maximum learning rates respectively, and usually η min =0.00001, η max =0.001; T is the total number of iterations. The model is pre-trained on a large-scale road surface crack dataset, and the recognition accuracy can reach more than 95%, which can effectively distinguish different types of road surface cracks such as transverse cracks, longitudinal cracks, reticular cracks, and crocodile cracks. The purpose of this step is to utilize the powerful feature extraction and classification capabilities of the deep learning model to achieve automatic detection and classification of road surface cracks and output the final recognition result.

[0240] To better understand and implement the present invention, the following provides Example 2 of a specific application scenario of the present invention: Researchers carried out intelligent detection of road cracks on a certain section of highway. This highway has been in use for more than 8 years, and the road surface has varying degrees of aging and damage. The total length of the detection section is 54 kilometers, mainly asphalt concrete pavement. The researchers used a detection vehicle equipped with a line-structured light scanner for road surface detection. This vehicle is equipped with a high-precision line-structured light scanner, an inertial navigation system, and a data processing unit.

[0241] The line-structured light scanner uses the LMS511 laser scanner of SICK company. The scanning frequency is set to 800Hz. Each scanning line contains 1280 measurement points, and the lateral resolution is 1.5mm. The detection vehicle travels at a constant speed of 60km / h and records a lateral scanning line every 1.5cm along the driving direction. The longitudinal resolution is 1.5cm. The inertial navigation system uses the SPAN-CPT integrated navigation system of NovAtel company, which includes a three-axis accelerometer, a three-axis gyroscope, and a GPS module. The acquisition frequency is 100Hz, and the accuracy of the vehicle's three-axis attitude angle data provided is better than 0.05°.

[0242] After the collected original data is processed, a test sample area is selected for algorithm verification. This area is 100 meters long and 3.75 meters wide, containing various typical road surface cracks. The researchers processed and analyzed the detection data of this area and implemented the intelligent road crack detection method described in the present invention. The specific parameter settings are shown in Table 1:

[0243] Table 1 Key parameter settings of the algorithm

[0244]

[0245]

[0246] During the experiment, first, attitude compensation processing is performed on the collected original laser point cloud elevation data matrix. The attitude data measured by the inertial navigation system shows that within the test area, the vehicle's roll angle fluctuates within the range of ±0.8°, and the pitch angle fluctuates within the range of ±1.2°. These attitude changes will cause errors in the original measurement data. The Kalman filter algorithm is applied to smooth the attitude data, filtering out high-frequency noise to obtain more stable attitude data. The original laser point cloud elevation data matrix is corrected using the attitude compensation matrix, effectively eliminating the measurement errors caused by vehicle jolts.

[0247] Next, two-way moving average filtering processing is performed. The half-width of the lateral filtering window is set to 10, and the half-width of the longitudinal filtering window is set to 8. The comparison of road surface depth data before and after filtering is shown in Table 2:

[0248] Table 2 Comparison Table of Pavement Depth Data Before and After Filtering (Unit: mm)

[0249] Data type Maximum depth Minimum depth Average depth Standard deviation Original data before filtering 32.86 -4.53 3.42 5.67 Processed data after filtering 28.42 -1.21 3.38 4.24

[0250] The filtering process effectively reduces the data fluctuation amplitude, eliminates the influence of random noise, and simultaneously retains abnormal structural features such as cracks. The filtered pavement depth matrix is normalized into a grayscale image with a grayscale value range of 0 - 255 to form a depth image. The depth image is evenly sliced into 256×256 pixel sizes, with adjacent image blocks overlapping by 25%, resulting in a total of 42 initial detection image blocks.

[0251] Apply the Gaussian background modeling method to each initial detection image block to extract potential crack feature points. Taking one representative image block as an example, the Gaussian background modeling results are shown in Table 3:

[0252] Table 3 Gaussian Background Modeling Results Table

[0253]

[0254] The Gaussian background modeling method successfully identifies depth anomaly points, which are mainly concentrated in the crack area. Apply the minimum spanning tree algorithm to the extracted potential crack feature points for connectivity analysis. Taking image block #18 as an example, the minimum spanning tree processing results are shown in Table 4:

[0255] Table 4 Minimum Spanning Tree Processing Results Table

[0256] Processing stage Number of nodes Number of edges Number of connected subgraphs Initial graph construction 2136 0 2136 Complete MST construction 2136 2135 1 Distance threshold pruning 2136 1853 37 Node number screening 1936 1728 6

[0257] The minimum spanning tree algorithm successfully organizes the potential crack feature points into a continuous structure, eliminates isolated noise points, and retains the crack structure with obvious connectivity. Apply the crack feature enhancement function to the detected 6 connected subgraphs for optimization processing, and calculate the depth features, gradient features, and direction consistency metrics of each crack structure, as shown in Table 5:

[0258] Table 5 Statistical Table of Crack Region Features

[0259]

[0260]

[0261] Based on the above features, apply the crack feature enhancement function to generate optimized crack feature image blocks. Input the 42 optimized crack feature image blocks into a pre-trained multi-scale pavement damage recognition network model, which is pre-trained on 100,000 pavement crack images and includes various types such as transverse cracks, longitudinal cracks, reticular cracks, and crocodile cracks. The recognition results are shown in Table 6:

[0262] Table 6 Statistical Table of Pavement Crack Recognition Results

[0263]

[0264] To verify the effectiveness of the method of the present invention, the researchers compared the detection results with the manual detection results and also compared the performance with traditional road surface crack detection methods, as shown in Table 7:

[0265] Table 7 Performance comparison table of different detection methods

[0266] Detection method Overall detection rate False detection rate Average processing time (s / km) Measurement accuracy (mm) Manual detection 92.3％ 2.8％ 1800 2.0 Image threshold segmentation method 85.6％ 8.4％ 12 No depth information Traditional machine learning method 88.9％ 6.2％ 24 No depth information Single-scale CNN method 91.7％ 5.3％ 32 No depth information Method of the present invention 94.9％ 3.6％ 28 1.5

[0267] Traditional road surface crack detection mainly uses two-dimensional image processing methods such as image threshold segmentation method, edge detection method, texture analysis method, etc., or methods based on traditional machine learning such as SVM, random forest, etc. These methods mainly rely on the gray-scale features of two-dimensional images, cannot obtain crack depth information, and are easily affected by road surface texture, illumination, shadow, etc., resulting in limited detection accuracy. In recent years, methods based on deep learning such as single-scale CNN have been applied, and the detection accuracy has been improved, but they still rely on two-dimensional image features and cannot provide three-dimensional geometric information of cracks.

[0268] The method of the present invention uses line-structured light three-dimensional measurement technology to obtain high-precision three-dimensional data of the road surface, combines an inertial navigation system to achieve attitude compensation, applies bidirectional moving average filtering to eliminate noise, extracts potential crack feature points through Gaussian background modeling, analyzes the connectivity of feature points using the minimum spanning tree algorithm, optimizes crack features using a crack feature enhancement function with multi-feature fusion, and finally uses a pre-trained multi-scale road surface damage recognition network model to achieve automatic crack recognition and classification. Compared with traditional methods, the present invention has the following advantages: First, based on the three-dimensional measurement principle, it can simultaneously obtain crack position, width and depth information, providing comprehensive data for road surface damage assessment; Second, the attitude compensation technology eliminates the influence of vehicle movement on measurement accuracy and improves data reliability; Third, the multi-scale feature extraction and fusion strategy improves the recognition ability for different types and scales of cracks; Finally, the combination of deep learning and traditional image processing technology gives full play to their respective advantages and realizes high-precision and high-efficiency automatic detection of road surface cracks. Experimental results show that the method of the present invention is superior to traditional methods in terms of detection rate, false detection rate, processing efficiency and measurement accuracy, providing reliable technical support for road maintenance decision-making.

[0269] It should be noted that the detailed explanations of the variables involved in the present invention are shown in Tables 8 and 9 below.

[0270] Table 8 Variable explanation table (the first part)

[0271]

[0272]

[0273] Table 9 Variable Explanation Table (Second Part)

[0274]

[0275]

[0276] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should be covered within the protection scope of the present invention.

Claims

1. A road crack intelligent detection method, characterized in that: The following steps are involved: Obtaining the original laser point cloud elevation data matrix of the road surface and processing it to obtain the corrected laser point cloud elevation data matrix; Perform bidirectional moving average filtering on the corrected laser point cloud elevation data matrix to eliminate road noise and obtain a road depth matrix; normalize the road depth matrix into a depth image, and evenly divide the depth image into multiple initial detection image blocks according to a preset size; The Gaussian background modeling method is used to screen the deep abnormal points in each initial detection image block, extract the set of potential crack feature points and construct a connected graph structure, calculate the Euclidean distance weights between the potential crack feature points, prune and remove the abnormal connecting edges to obtain the crack skeleton structure; the crack feature enhancement function is used to optimize the crack skeleton structure to generate the optimized crack feature image block; the optimized crack feature image block is input into the pre-trained multi-scale pavement damage recognition network model, and the crack recognition results and crack type classification are output.

2. The road crack intelligent detection method according to claim 1, characterized in that: Bidirectional moving average filtering refers to the sliding window average calculation of the corrected laser point cloud elevation data matrix along the horizontal and vertical directions of the road surface, which effectively eliminates the influence of road surface texture and collection noise and highlights the crack characteristics.

3. The road crack intelligent detection method according to claim 2, characterized in that: The depth image is formed by converting each depth value in the road surface depth matrix into a grayscale value according to a linear mapping method. The grayscale value represents the degree of road surface depression. The initial detection image block is an image sub-region obtained by evenly dividing the depth image according to a fixed-size grid, which is convenient for extracting crack features in local areas.

4. The road crack intelligent detection method according to claim 3, characterized in that: Gaussian background modeling assumes that the distribution of road depth values ​​conforms to the normal distribution characteristics, estimates the mean and standard deviation of road depth through the expectation maximization algorithm, and sets an adaptive threshold to identify abnormal depth points.

5. The road crack intelligent detection method according to claim 4, characterized in that: The crack skeleton structure is the main structure of the crack obtained by processing with the minimum spanning tree algorithm, which represents the spatial distribution morphology of the crack.

6. The road crack intelligent detection method according to claim 5, characterized in that: The crack feature enhancement function is used to extract and enhance the features of the crack area. The input includes the crack skeleton structure matrix, the crack area depth matrix, the crack area texture gradient matrix, the crack direction vector matrix and the crack connectivity matrix. The output is the enhanced crack feature image matrix. The crack feature enhancement function realizes the contrast improvement between the crack area and the background area through multi-dimensional feature space mapping, thereby enhancing the crack feature recognition effect.

7. The road crack intelligent detection method according to claim 6, characterized in that: The crack area depth matrix is ​​a matrix composed of the corresponding depth values ​​of the crack area extracted from the pavement depth matrix, which characterizes the crack depth distribution; the crack area texture gradient matrix is ​​a matrix composed of the grayscale gradient values ​​of the crack area in the calculated depth image, which characterizes the texture characteristics of the crack edge.

8. The road crack intelligent detection method according to claim 7, characterized in that: The fracture direction vector matrix is ​​a matrix composed of information about the local fracture direction, which characterizes the fracture extension direction characteristics. The fracture connectivity matrix is ​​a matrix that represents the connectivity relationship between potential fracture feature points and describes the integrity of the fracture structure.

9. The road crack intelligent detection method according to claim 8, characterized in that: The enhanced crack feature image matrix is ​​a prominent crack feature image data matrix obtained after being processed by the crack feature enhancement function; the optimized crack feature image block is an image sub-region obtained by re-dividing the enhanced crack feature image matrix according to the initial detection image block size, which is used as the crack recognition input data.

10. The road crack intelligent detection method according to claim 9, characterized in that: The specific structure of the pre-trained multi-scale road damage recognition network model is a deep neural network composed of a feature extraction layer, a multi-scale feature fusion layer, an attention mechanism layer, and a classification layer; The feature extraction layer uses multiple groups of convolution layers with different convolution kernel sizes to extract texture features of the road surface at different scales, and then the feature information of different scales is adaptively weighted through the multi-scale feature fusion layer.

Images