Road engineering asphalt pavement quality detection method and system based on image recognition

Abstract

The invention discloses a road engineering asphalt pavement quality detection method and system based on image recognition. The method comprises the following steps: registering image data; establishing a multi-task deep learning network model, extracting shared deep features from the registered image data, performing up-sampling and jump connection on the deep features through a decoder of a U-Net structure, outputting pavement defect features, and learning texture depth features of point clouds in the registered image data; fusing the pavement defect features and the texture depth features by using a cross attention mechanism to obtain a fusion quality feature vector; and inputting the quality feature vector into an XGBoost gradient boosting decision tree, outputting a quality detection score, and performing rating according to the quality detection score. The subjective randomness of manual evaluation is avoided, and the detection efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road construction, in particular to a road engineering asphalt pavement quality detection method and system based on image recognition. BACKGROUND

[0002] Asphalt pavement is the most common pavement form in road engineering, and its construction quality is directly related to the service life, driving safety and comfort of the road. Traditional asphalt pavement quality detection mainly relies on manual visual inspection or simple mechanical tool measurement. The manual inspection result is easily affected by factors such as the experience and fatigue degree of the detector, and lacks objective and unified evaluation criteria. Moreover, manual detection is slow, needs to close the traffic, affects the normal operation of the road, and is difficult to achieve large-scale and high-frequency detection. Traditional tools can usually only measure a single index, and cannot comprehensively and comprehensively reflect the relationship between the surface disease and functional performance of the pavement, resulting in low efficiency of asphalt pavement quality detection. SUMMARY

[0003] The purpose of the present application is to solve the above problems, and a road engineering asphalt pavement quality detection method and system based on image recognition are designed.

[0004] To achieve the above purpose, the technical scheme of the present application is that, in the above-mentioned road engineering asphalt pavement quality detection method based on image recognition, the road engineering asphalt pavement quality detection method comprises the following steps: Collect high-resolution visible light images and three-dimensional point cloud data in the road engineering construction process, pre-process the collected data, time-synchronize and space-register the pre-processed data, and obtain registered image data; Establish a multi-task deep learning network model, use ResNet-50 network to extract shared deep features from the registered image data, and use the decoder of the U-Net structure to up-sample and jump connect the deep features, output pavement defect features, and use PointNet++ network to learn the texture depth features of the point cloud in the registered image data; Fuse the pavement defect features and the texture depth features using the cross-attention mechanism to obtain a fused quality feature vector; Input the quality feature vector into the XGBoost gradient boosting decision tree to output a quality detection score, and grade according to the quality detection score.

[0005] Further, in the above-mentioned road engineering asphalt pavement quality detection method based on image recognition, the collection of high-resolution visible light images and three-dimensional point cloud data in the road engineering construction process, the pre-processing of the collected data, the time synchronization and space registration of the pre-processed data, and the obtaining of registered image data comprise: Collect high-resolution visible light images and three-dimensional point cloud data in the process of road engineering construction; Remove image noise of the visible light image by using Gaussian filtering, and correct uneven illumination by using the Retinex algorithm to eliminate the influence of shadows and strong light on image quality. Crop and size-normalize the processed image, enhance the image contrast by using histogram equalization, and highlight pavement texture details to obtain preprocessed image data. Remove discrete noise points in the three-dimensional point cloud data by statistical filtering, remove invalid point clouds below the ground and in the air by straight-through filtering, and perform down-sampling processing on the filtered point cloud data to obtain preprocessed point cloud data.

[0006] Further, in the above-mentioned image recognition-based road engineering asphalt pavement quality detection method, the high-resolution visible light images and three-dimensional point cloud data collected in the process of road engineering construction are preprocessed, and the preprocessed data is time-synchronized and spatially registered to obtain registered image data, including: Extract feature points from the preprocessed image data by using the SIFT algorithm, and extract corresponding three-dimensional feature points from the preprocessed point cloud data. Remove false matching feature point pairs by using the RANSAC algorithm, and solve the optimal transformation matrix containing translation parameters and rotation parameters. Map the three-dimensional point cloud data to the two-dimensional coordinate system of the visible light image by using the transformation matrix to obtain registered image data containing texture information and three-dimensional depth information.

[0007] Further, in the above-mentioned image recognition-based road engineering asphalt pavement quality detection method, the multi-task deep learning network model is established, the ResNet-50 network is used to extract shared deep features from the registered image data, the decoder of the U-Net structure is used to up-sample and jump connect the deep features, the pavement defect features are output, and the PointNet++ network is used to learn the texture depth features of the point cloud in the registered image data, including: Input the visible light image in the registered image data into the pre-trained ResNet-50 network, perform preliminary feature extraction and down-sampling by using a 7x7 convolutional layer, gradually extract deep semantic features by using four residual block groups, and obtain a shared deep feature map. Input the shared deep feature map into the decoder of the U-Net structure, perform up-sampling by using four transposed convolutions, and after each up-sampling, jump connect with the shallow feature map of the corresponding level of the ResNet-50 network to fuse deep semantic features and shallow detail features. Adjust the number of feature map channels to the number of defect categories by using a 1x1 convolutional layer, and output the pavement defect feature map.

[0008] Further, in the above-mentioned image recognition-based road engineering asphalt pavement quality detection method, the multi-task deep learning network model is established, ResNet-50 network is used to extract shared deep features from the registration image data, the decoder of the U-Net structure is used for upsampling and jump connection of the deep features, the pavement defect features are output, the PointNet++ network is used to learn the texture depth features of the point cloud in the registration image data, including: The three-dimensional point cloud data in the registration image data is input into the PointNet++ network for feature extraction, and the FPS farthest point sampling algorithm is used in the sampling layer to sample key point sets from the original point cloud; The local neighborhood is divided by using a spherical query method with each key point as the center through the grouping layer, and the local point cloud subset is obtained, and the MLP multi-layer perceptron operation is performed on each local point cloud subset through the feature extraction layer to extract local features; The local features are subjected to maximum pooling operation to obtain global features, and a feature representation of local and global fusion is obtained, and the features are reduced in dimension through the fully connected layer to output the texture depth features.

[0009] Further, in the above-mentioned image recognition-based road engineering asphalt pavement quality detection method, the cross-attention mechanism is used to fuse the pavement defect features and the texture depth features to obtain a fusion quality feature vector, including: The cross-attention mechanism is used to input the pavement defect features and the texture depth features into two independent linear layers for dimension mapping to obtain a first feature vector; The dot product of the transpose matrix of the first feature vector is calculated to obtain an attention weight matrix, and the weight matrix is normalized by using the Softmax function to obtain a second feature vector; The second feature vector is subjected to dot product operation to generate image feature guided point cloud features, and the dimension of the point cloud features is mapped by using a linear layer to obtain a fusion quality feature vector.

[0010] Further, in the above-mentioned image recognition-based road engineering asphalt pavement quality detection method, the quality feature vector is input into the XGBoost gradient boosting decision tree to output a quality detection score, and the quality detection score is used for rating, including: According to the quality detection score S, a grading standard is formulated, the quality grade is divided into excellent, good, qualified and unqualified four grades according to the road engineering asphalt pavement quality acceptance specification.

[0011] Further, in the image recognition-based road engineering asphalt pavement quality detection system, the road engineering asphalt pavement quality detection system comprises the following modules: The image data acquisition module is configured to acquire high-resolution visible light images and three-dimensional point cloud data in a road engineering construction process, pre-process the acquired data, and perform time synchronization and spatial registration on the pre-processed data to obtain registered image data. The feature vector extraction module is configured to establish a multi-task deep learning network model, extract shared deep features from the registered image data using a ResNet-50 network, perform upsampling and skip connection on the deep features through a decoder of a U-Net structure, output pavement defect features, and learn texture depth features of point clouds in the registered image data using a PointNet++ network. The feature vector fusion module is configured to fuse the pavement defect features and the texture depth features using a cross-attention mechanism to obtain a fused quality feature vector. The pavement quality detection module is configured to input the quality feature vector into an XGBoost gradient boosting decision tree to output a quality detection score and perform grading according to the quality detection score.

[0012] Further, in the road engineering asphalt pavement quality detection system based on image recognition, the feature vector fusion module includes the following sub-modules: The mapping sub-module is configured to input the pavement defect features and the texture depth features into two independent linear layers for dimension mapping using a cross-attention mechanism to obtain a first feature vector. The normalization sub-module is configured to calculate the dot product of the transpose matrix of the first feature vector to obtain an attention weight matrix, and perform normalization processing on the weight matrix through a Softmax function to obtain a second feature vector. The obtaining sub-module is configured to perform dot product operation on the second feature vector to generate image feature guided point cloud features, map the dimensions of the point cloud features using a linear layer, and obtain a fused quality feature vector.

[0013] Further, in the road engineering asphalt pavement quality detection system based on image recognition, the feature vector fusion module includes the following sub-modules: The division sub-module is configured to establish grading standards according to the quality detection score S, combine the road engineering asphalt pavement quality acceptance specification, and divide the quality level into four levels of excellent, good, qualified, and unqualified.

[0014] Its beneficial effect lies in, by combining high-resolution visible light image and three-dimensional point cloud data. The visible light image provides rich texture and color information, which is convenient for identifying cracks, pits and other surface diseases; and the three-dimensional point cloud data provides accurate spatial geometric information, which can directly calculate the texture depth, flatness and other key functional indicators. The problem of single information dimension of traditional method is solved, and the integrated detection of pavement surface and function is realized. Based on the shared backbone feature extraction network of the multi-task deep learning network model, the semantic segmentation and three-dimensional geometric analysis two tasks are executed in parallel. Reducing the repeated consumption of computing resources, improving the processing efficiency, can promote each other and check each other. Through a machine learning-based quality comprehensive evaluation model, the extracted multi-dimensional feature vectors are nonlinearly fused, and finally an easy-to-understand comprehensive quality score and grade are output. The evaluation result is more objective, scientific and reliable, avoiding the subjective randomness of manual evaluation, and improving the detection efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0015] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for purposes of illustration only and are not intended to limit the application thereto.

[0016] Figure 1 The first embodiment schematic diagram of the image recognition-based road engineering asphalt pavement quality detection method in the embodiment of the application; Figure 2 The second embodiment schematic diagram of the image recognition-based road engineering asphalt pavement quality detection method in the embodiment of the application; Figure 3 The first embodiment schematic diagram of the image recognition-based road engineering asphalt pavement quality detection system in the embodiment of the application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical scheme and advantages of the application more clear, the application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0018] Those skilled in the art can understand that, unless specifically stated, the singular form "one", "an" and "the" used herein can also include the plural form. It should be further understood that the wording used in the specification of the application includes the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups.

[0019] The application will be described in detail below in combination with the drawings, Figure 1As shown, the image recognition-based road engineering asphalt pavement quality detection method comprises the following steps: Step 101, collect high-resolution visible light images and three-dimensional point cloud data in the road engineering construction process, pre-process the collected data, time-synchronize and space-register the pre-processed data, and obtain registered image data; Specifically, in the embodiment, high-resolution visible light images and three-dimensional point cloud data in the road engineering construction process are collected; Gaussian filtering is used to remove image noise of the visible light images, Retinex algorithm is used for non-uniform light correction, and the influence of shadows and strong light on image quality is eliminated; the processed images are cropped and size-normalized, histogram equalization is used to enhance the contrast of the images, and pavement texture details are highlighted to obtain pre-processed image data; discrete noise points in the three-dimensional point cloud data are removed through statistical filtering, straight-through filtering is used to remove invalid point clouds below the ground and in the air, the filtered point cloud data is down-sampled, and pre-processed point cloud data is obtained. Feature points are extracted from the pre-processed image data by SIFT algorithm, and corresponding three-dimensional feature points are extracted from the pre-processed point cloud data; RANSAC algorithm is used to remove incorrectly matched feature point pairs, and an optimal transformation matrix containing translation parameters and rotation parameters is solved; the three-dimensional point cloud data is mapped to the two-dimensional coordinate system of the visible light image through the transformation matrix, and registered image data containing texture information and three-dimensional depth information is obtained.

[0020] Multi-source data acquisition and registration processing is the basic link of asphalt pavement quality detection, and its data quality directly determines the subsequent detection accuracy, which needs to be implemented from three aspects of equipment selection, preprocessing optimization and synchronous registration calibration. The equipment selection needs to consider both accuracy and adaptability to construction scene: the mobile acquisition vehicle is equipped with an industrial-grade visible light camera with 50 million pixels or more. Since the minimum width of the fine defects such as pavement cracks and loose can reach 0.1 mm, high pixels can ensure clear imaging of defect details, 24-70 mm adjustable focal length can adapt to different acquisition distances, and F2.8-F11 aperture range can cope with the changes of light in early and late construction. The three-dimensional laser scanner is set to a point cloud density of ≥100 points per square centimeter, which is based on the characteristics of the asphalt pavement texture wavelength of about 0.5 mm, to ensure complete collection of texture details, and the ±2 mm ranging accuracy can meet the requirements of elevation flatness detection. The GPS / IMU combined positioning module adopts a double-antenna design, which can resist electromagnetic interference from metal structures such as tower cranes and steel bars in the construction area, and millisecond-level timestamps provide protection for data synchronization. The acquisition process needs to accurately match the construction process: in the initial pressure stage, the mobile acquisition vehicle speed is reduced to 5 km / h to improve the point cloud density; in the re-pressing stage, the interval between shots is shortened to 0.3 seconds per frame to capture the rolling track; in the final pressing stage, the focus is on surface texture, and the rolling temperature is recorded synchronously, as well as other key parameters such as 110-130℃. The preprocessing stage uses targeted algorithms: the visible light image is first filtered by a 5×5 Gaussian filter with a standard deviation of 1.2 to adapt to the noise characteristics of asphalt texture, and then the Retinex algorithm is used to decompose the light component and the reflection component, effectively eliminating the defect misjudgment caused by construction shadows; the three-dimensional point cloud is filtered by statistics, 20 neighborhood points ensure the accuracy of noise identification, and after removing the flying points, the voxel grid filtering is used, 5 mm voxel balances the precision and efficiency of downsampling, and the extracted normal vector and curvature features can mark the high-elevation mutation area in advance. The synchronous registration adopts a dual strategy of time anchoring and spatial calibration: time synchronization is corrected by GPS timestamp interpolation, which solves the millisecond-level deviation caused by device startup delay; in spatial registration, the SIFT algorithm sets a 6-layer Gaussian pyramid, a 0.03 feature point threshold is used to select stable features, and the RANSAC algorithm is combined with a 2 mm inlier threshold to ensure the accuracy of the transformation matrix, and finally the output registration data has both two-dimensional texture and three-dimensional depth information, laying a foundation for subsequent feature extraction.

[0021] In step 102, a multi-task deep learning network model is established, a shared deep feature is extracted from the registration image data by using a ResNet-50 network, and a decoder of a U-Net structure is used for upsampling and skip connection of the deep feature to output pavement defect features, and a PointNet++ network is used to learn texture depth features of point clouds in the registration image data; Specifically, in the present embodiment, the visible light image in the registered image data is input into the pre-trained ResNet-50 network, preliminary feature extraction and down-sampling are performed through a 7x7 convolutional layer, deep semantic features are gradually extracted through 4 residual block groups, and a shared deep feature map is obtained; the shared deep feature map is input into a decoder of the U-Net structure, up-sampling is performed through 4 times of transposed convolution, after each up-sampling, shallow feature maps of corresponding levels of the ResNet-50 network are connected through a skip connection, and deep semantic features and shallow detail features are fused; the number of channels of the feature map is adjusted to the number of defect categories through a 1x1 convolutional layer, and a pavement defect feature map is output. The three-dimensional point cloud data in the registered image data is input into the PointNet++ network for feature extraction, a key point point set is obtained from the original point cloud through a sampling layer using the FPS farthest point sampling algorithm; a local neighborhood is divided using a ball query method with each key point as the center through a grouping layer, a local point cloud subset is obtained, and a local feature is extracted through a feature extraction layer for each local point cloud subset; global features are obtained through a max-pooling operation on the local features, a feature representation of local and global fusion is obtained, the features are reduced in dimension through a fully connected layer, and texture depth features are output.

[0022] The multi-task deep learning network model construction and feature extraction need to realize double-branch parallel extraction-feature level fusion, and the core is to mine the complementary information of image texture and point cloud depth through network structure optimization. The network architecture design adopts the modularization concept, and the visible light image and point cloud branch share the data input interface. The registration data is split into a two-dimensional image tensor and a three-dimensional point cloud matrix by a data analysis module. The two-dimensional image tensor is 1024×1024×3, and the three-dimensional point cloud matrix is N×3, N being the number of point clouds. The two are input into the corresponding branch. The visible light branch selects the structure combined with ResNet-50 and U-Net decoder. The residual block design of ResNet-50 can solve the problem of gradient disappearance in deep network. The pre-training weight is optimized based on the ImageNet dataset, and the first three residual block groups containing 16 residual blocks are frozen during the transfer learning, and only the fourth group containing three residual blocks and subsequent layers are fine-tuned. The reason is that the basic features such as edges and textures extracted by the first three groups are universal, and the last two groups can adapt to the features of asphalt pavement defects. The specific extraction process is as follows: a 7×7 convolutional layer with a stride of 2 and a padding of 3 is used to downsample the input image to 512×512×64, and batch normalization is used to speed up convergence; the four residual block groups are sequentially passed through the bottleneck structure of 1×1 convolution dimension reduction, 3×3 convolution feature extraction, and 1×1 convolution dimension increase, gradually improving the feature abstraction degree, and the fourth group outputs a deep feature map of 16×16×2048. The U-Net decoder adopts the up-sampling-jump connection architecture, and the feature map size is restored to 1024×1024 through four times of transposed convolution with a 2×2 convolution kernel and a stride of 2. After each up-sampling, the feature map is spliced with the corresponding level feature map of ResNet-50, such as splicing the 16×16×2048 feature map with the 32×32×1024 feature map output by the third group of ResNet-50 after up-sampling to 32×32×1024, to fuse deep semantics and shallow details; finally, a 1×1 convolutional layer is used to adjust the channel number to 6, corresponding to six typical defects such as cracks and potholes, and output the defect probability feature map. The point cloud branch adopts the PointNet++ network, which adapts to the unstructured characteristics of point clouds through hierarchical sampling and grouping mechanisms. After inputting the registered point cloud data, the sampling layer selects the key point set through the FPS algorithm, which selects the farthest point from the selected point each time to ensure that the sampling points cover the entire pavement area, and the sampling number is set to 2048 to balance the calculation amount and representativeness; the grouping layer takes each key point as the center and divides the local neighborhood by a ball query with a radius of 0.05 m, and each neighborhood contains 32 points to form a local point cloud subset. The feature extraction layer consists of three layers of MLP. The first layer has 64 neurons to extract local geometric features, the second layer has 128 neurons to fuse neighborhood correlation information, and the third layer has 256 neurons to generate a local feature vector; the key features of each neighborhood are selected through the max-pooling operation to obtain a global feature matrix of 2048×256.3 full connection layers sequentially reduce the feature dimension to 1024, 512, 1024, add 50% dropout to suppress overfitting, and finally output a 1024-dimensional texture depth feature vector. The vector reflects the flatness through the elevation change gradient and reflects the compaction quality through the texture roughness index, which is complementary to the defect features of the image branch. The training process adopts a joint optimization strategy: the visible light branch uses the cross-entropy loss function to calculate the defect classification loss, and the point cloud branch uses the mean square error loss function to calculate the texture depth fitting loss. The total loss is the weighted sum of the two, and the weight ratio is 1:1.2; the optimizer uses Adam, the initial learning rate is 0.001, and it is attenuated to 1 / 10 of the original every 100 rounds. The batch size is 32, and the model converges after 500 iterations. The defect recognition accuracy of the verification set is more than 92%.

[0023] Step 103, fuse the pavement defect features and the texture depth features by using the cross attention mechanism to obtain a fused quality feature vector; Specifically, in the present embodiment, the cross attention mechanism is used to input the pavement defect features and the texture depth features into two independent linear layers for dimension mapping to obtain a first feature vector. The dot product of the transpose matrix of the first feature vector is calculated to obtain an attention weight matrix. The weight matrix is normalized by using the Softmax function to obtain a second feature vector. The second feature vector is subjected to dot product operation to generate image feature guided point cloud features. The dimension of the point cloud features is mapped by using a linear layer to obtain a fused quality feature vector.

[0024] The core of cross-modal attention mechanism feature fusion is to break through the difference between image and point cloud modalities, realize the precise alignment and efficient fusion of key features, and improve the fusion quality through feature preprocessing standardization and adaptive allocation of attention weight. The core goal of feature preprocessing is to solve the problem of mismatch between the feature dimensions of the two modalities while preserving the key information. The 1024x1024xN defect feature map output by the image branch, where N is the number of defect categories, has a spatial distribution probability for each channel of a defect class. Directly inputting the fusion module will cause computational explosion due to the high dimension, so a global average pooling operation is used, with a pooling kernel size of 1024x1024. The average value of all pixels in each channel is taken to compress the two-dimensional spatial feature into a one-dimensional vector of 1x1xN. This vector retains the overall probability distribution feature of each defect class, avoiding spatial information redundancy. To match the 1024-dimensional feature vector of the point cloud branch, a fully connected layer is designed to realize dimension mapping: the input dimension is N, the output dimension is 1024, the weight is initialized with He normal, and the adaptive function is ReLU. The activation function is LeakyReLU with a negative slope of 0.01 to solve the problem of gradient sparsity. The defect probability feature is converted into a 1024-dimensional F_img vector through this layer. Although the 1024-dimensional F_pc vector output by the point cloud branch is dimensionally matched, it needs to be standardized. The LayerNorm layer is used to normalize the features with a mean of 0 and a variance of 1, eliminating the scale differences of point cloud data in different construction scenarios, such as point cloud inflation error caused by high temperature construction. The fusion calculation uses a cross-attention mechanism, which has the advantage of adaptively learning the correlation weight of the two modalities and highlighting the features that are critical to quality detection. First, the feature dimension is reduced through a linear mapping layer: F_img and F_pc are input into two independent linear layers with an input of 1024 and an output of 512. Reducing the dimension to 512 can reduce the amount of attention calculation while enhancing the non-linear expression of the features. The linear layer weight is initialized with Xavier to ensure the stability of the forward propagation and the backward gradient. The attention weight calculation uses a similarity measurement-normalization process: the dot product of F_img', 512-dimensional, and F_pc', 512-dimensional, is calculated to obtain a 512x512 similarity matrix. The element i,j of the matrix represents the correlation strength of the i-th feature of F_img' and the j-th feature of F_pc'. To avoid weight saturation, a temperature parameter τ=0.1 is introduced to scale the similarity matrix, and then normalized through the Softmax function to obtain the attention weight matrix W. The larger the value of an element in the matrix, the greater the contribution of the corresponding feature to quality detection.The feature-guided fusion stage obtains F_pc_att by dot product of W and F_pc'. This process filters key information in the point cloud depth feature through image defect features. For example, when the image identifies a crack, W will strengthen the weight of the elevation mutation feature at the corresponding position of the point cloud. Then, F_img' and F_pc_att are added element by element to realize complementary fusion of features. The added feature vector is fed back to the input end through the residual connection to enhance gradient propagation. Finally, a linear layer is designed to map the fused features back to 1024 dimensions. This layer uses Dropout with a proportion of 0.3 for regularization. The final F_fuse fusion vector is output. To verify the fusion effect, the feature visualization experiment shows that the single-image modal feature has an accuracy of 85% in identifying surface cracks, but only 62% in identifying deep loose and no surface texture changes. The single-point cloud modal has an accuracy of 88% in identifying deep loose, but only 68% in identifying fine cracks and small elevation changes. The F_fuse vector after fusion has an accuracy of more than 90% in identifying both types of defects, proving that it effectively integrates the advantages of the two modalities. At the same time, the ablation experiment verifies the necessity of the attention mechanism. The simple splicing fusion without the attention mechanism reduces the overall recognition accuracy by 12%, indicating that the adaptive allocation of attention weights is crucial to the fusion quality. This fusion process realizes the precise correlation of defect position-depth features through mathematical modeling, providing high-recognizability feature input for subsequent quality scoring.

[0025] In step 104, the quality feature vector is input into the XGBoost gradient boosting decision tree, and the quality detection score is output. The quality detection score is used for rating.

[0026] Specifically, in this embodiment, the grading standard is formulated according to the quality detection score S, and the quality level is divided into excellent, good, qualified, and unqualified according to the Road Engineering Asphalt Pavement Quality Acceptance Specification.

[0027] An XGBoost gradient boosting decision tree model is constructed, with the fused quality feature vector F_fuse as input, and the asphalt pavement quality score as output label. The score is a standard score formulated by expert scoring combined with actual detection indicators, ranging from 0 to 100. First, the training set and the test set are divided in a ratio of 7:3. The training set is used for model optimization through K-fold cross-validation, K=5. The number of decision trees is set to 100, the maximum tree depth is 8, the learning rate is 0.01, and the minimum sample weight sum is 0.1. After training, the fused quality feature vector to be detected is input into the model, and the corresponding quality detection score S is output by the model. According to the quality detection score S, the grading standards are formulated, combined with the road engineering asphalt pavement quality acceptance specification, the quality grade is divided into excellent, good, qualified and unqualified four grades, the specific standards are as follows: excellent, S is greater than or equal to 90: the pavement has no obvious defects, the texture is uniform, the elevation is flat, and it fully meets the design requirements; Good, 80≤S<90: the pavement has very slight small cracks, the single length is less than 0.5m, and there is no other defect, the overall quality is good; Qualified, 60≤S<80: the pavement has a small amount of cracks, the single length is 0.5-2m, the total length is less than 10m or small pits and grooves, the diameter is less than 0.3m and the depth is less than 0.05m, and after simple repair, it can meet the use requirements; Unqualified, S is less than 60: the pavement has a large number of cracks, the total length is greater than or equal to 10m, large pits and grooves, the diameter is greater than or equal to 0.3m or the depth is greater than or equal to 0.05m or loose, displacement and other serious defects, and it needs to be reconstructed. According to the score S output by the model, the corresponding grade is automatically matched to complete the quality rating.

[0028] The beneficial effects are that the high-resolution visible light image is combined with the three-dimensional point cloud data. The visible light image provides rich texture and color information, which is convenient for identifying surface diseases such as cracks and pits; and the three-dimensional point cloud data provides accurate spatial geometric information, which can directly calculate key functional indicators such as texture depth and flatness. The problem of single information dimension in traditional methods is solved, and integrated detection of pavement appearance and function is realized. Based on the shared backbone feature extraction network of the multi-task deep learning network model, the semantic segmentation and three-dimensional geometric analysis tasks are executed in parallel. The repeated consumption of computing resources is reduced, the processing efficiency is improved, and the tasks can promote and verify each other. Through a machine learning-based quality comprehensive evaluation model, the extracted multi-dimensional feature vectors are nonlinearly fused, and finally an easy-to-understand comprehensive quality score and grade are output. The evaluation result is more objective, scientific and reliable, avoiding the subjective randomness of manual evaluation, and improving the detection efficiency.

[0029] Please refer to Figure 2 In the image recognition-based road engineering asphalt pavement quality detection method, a multi-task deep learning network model is established, a ResNet-50 network is used to extract shared deep features from the registered image data, and a decoder of U-Net structure is used for upsampling and jump connection of the deep features, and the pavement defect features are output, including the following steps: Step 201, inputting the three-dimensional point cloud data in the registered image data into the PointNet++ network for feature extraction, and using the FPS farthest point sampling algorithm in the sampling layer to sample key point sets from the original point cloud; Step 202, using a ball query method to divide the local neighborhood with each key point as the center through the grouping layer, obtaining a local point cloud subset, and performing MLP multi-layer perceptron operation on each local point cloud subset through the feature extraction layer to extract local features; Step 203, maximum pooling operation is performed on the local features to obtain global features, and a local and global fused feature representation is obtained, and the features are reduced in dimension through a full connection layer, and a texture depth feature is output.

[0030] The embodiments of the image recognition-based road engineering asphalt pavement quality detection method are introduced above, please refer to Figure 3 In the image recognition-based road engineering asphalt pavement quality detection system, the road engineering asphalt pavement quality detection system comprises the following modules: An image data acquisition module is configured to acquire high-resolution visible light images and three-dimensional point cloud data in a road engineering construction process, pre-process the acquired data, and perform time synchronization and space registration on the pre-processed data to obtain registered image data. A feature vector extraction module is configured to establish a multi-task deep learning network model, extract shared deep features from the registered image data by using a ResNet-50 network, perform upsampling and skip connection on the deep features by using a decoder of a U-Net structure, output pavement defect features, and learn texture depth features of point clouds in the registered image data by using a PointNet++ network. A feature vector fusion module is configured to fuse the pavement defect features and the texture depth features by using a cross-attention mechanism to obtain a fused quality feature vector. A pavement quality detection module is configured to input the quality feature vector into an XGBoost gradient boosting decision tree to output a quality detection score, and grade according to the quality detection score.

[0031] The present application can be used in a plurality of general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0032] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Various changes and improvements can be made to the present application without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for detecting the quality of road engineering asphalt pavement based on image recognition, characterized in that, The method for testing the quality of asphalt pavement in road engineering includes the following steps: High-resolution visible light images and 3D point cloud data were collected during the road construction process. The collected data were preprocessed, and the preprocessed data was time-synchronized and spatially registered to obtain registered image data. A multi-task deep learning network model is established. The ResNet-50 network is used to extract shared deep features from the registered image data. The deep features are upsampled and skipped through a decoder with a U-Net structure to output pavement defect features. The PointNet++ network is used to learn the texture depth features of the point cloud in the registered image data. A cross-attention mechanism is used to fuse pavement defect features and texture depth features to obtain a fused quality feature vector; The quality feature vector is input into the XGBoost gradient boosting decision tree, and a quality detection score is output. A rating is then given based on the quality detection score.

2. The image recognition-based road engineering asphalt pavement quality detection method according to claim 1, characterized in that, The process involves collecting high-resolution visible light images and 3D point cloud data during road construction. The collected data is preprocessed, and the preprocessed data is then time-synchronized and spatially registered to obtain registered image data, including: Collect high-resolution visible light images and 3D point cloud data during road construction; Gaussian filtering is used to remove image noise from visible light images, and the Retinex algorithm is used to correct uneven illumination and eliminate the effects of shadows and strong light on image quality. The processed image is cropped and normalized, and histogram equalization is used to enhance image contrast and highlight pavement texture details to obtain preprocessed image data. Discrete noise points in the 3D point cloud data are removed by statistical filtering, and invalid point clouds below the ground and in the air are removed by pass-through filtering. The filtered point cloud data is then downsampled to obtain preprocessed point cloud data.

3. The method for inspecting the quality of asphalt pavement in road engineering based on image recognition as described in claim 1, characterized in that, The process involves collecting high-resolution visible light images and 3D point cloud data during road construction. The collected data is preprocessed, and the preprocessed data is then time-synchronized and spatially registered to obtain registered image data, including: Feature points are extracted from preprocessed image data using the SIFT algorithm, and corresponding 3D feature points are extracted from preprocessed point cloud data. The RANSAC algorithm is used to remove erroneously matched feature point pairs, and the optimal transformation matrix, including translation and rotation parameters, is solved. The 3D point cloud data is mapped to the 2D coordinate system of the visible light image by a transformation matrix, resulting in registered image data containing texture information and 3D depth information.

4. The method for inspecting the quality of asphalt pavement in road engineering based on image recognition as described in claim 1, characterized in that, The establishment of a multi-task deep learning network model involves using a ResNet-50 network to extract shared deep features from the registered image data, upsampling and skipping connections on the deep features using a U-Net decoder to output surface defect features, and using a PointNet++ network to learn the texture depth features of the point cloud in the registered image data, including: The visible light image in the registered image data is input into the pre-trained ResNet-50 network. Preliminary feature extraction and downsampling are performed through a 7×7 convolutional layer. Deep semantic features are extracted step by step through 4 residual block groups to obtain a shared deep feature map. The shared deep feature map is input into the decoder of the U-Net structure and upsampled through four transposed convolutions. After each upsampling, it is skipped to the shallow feature map of the corresponding layer of the ResNet-50 network to fuse deep semantic features and shallow detail features. The number of feature map channels is adjusted to the number of defect categories by using a 1×1 convolutional layer, and the pavement defect feature map is output.

5. The method for inspecting the quality of asphalt pavement in road engineering based on image recognition as described in claim 1, characterized in that, The establishment of a multi-task deep learning network model involves using a ResNet-50 network to extract shared deep features from the registered image data, upsampling and skipping connections on the deep features using a U-Net decoder to output surface defect features, and using a PointNet++ network to learn the texture depth features of the point cloud in the registered image data, including: The 3D point cloud data in the registered image data is input into the PointNet++ network for feature extraction. The key point set is obtained by sampling from the original point cloud through the sampling layer using the FPS farthest point sampling algorithm. By using a grouping layer with each key point as the center, a local neighborhood is divided using a ball query method to obtain a subset of local point cloud. Then, a feature extraction layer performs MLP (Multilayer Perceptron) operations on each local point cloud subset to extract local features. The local features are max-pooled to obtain the global features, resulting in a fused feature representation of local and global features. The features are then reduced in dimensionality by a fully connected layer to output the texture depth features.

6. The method for inspecting the quality of asphalt pavement in road engineering based on image recognition as described in claim 1, characterized in that, The method of fusing pavement defect features and texture depth features using a cross-attention mechanism to obtain a fused quality feature vector includes: A cross-attention mechanism is used to input the pavement defect features and texture depth features into two independent linear layers for dimension mapping, thereby obtaining the first feature vector; Calculate the dot product of the transpose of the first eigenvector to obtain the attention weight matrix. Normalize the weight matrix using the Softmax function to obtain the second eigenvector. The second feature vector is subjected to a dot product operation to generate image feature-guided point cloud features. A linear layer is then used to map the dimensions of the point cloud features to obtain a fused quality feature vector.

7. The method for inspecting the quality of asphalt pavement in road engineering based on image recognition as described in claim 1, characterized in that, The process of inputting the quality feature vector into an XGBoost gradient boosting decision tree, outputting a quality detection score, and rating based on the quality detection score includes: Based on the quality inspection score S, a grading standard is established, and combined with the road engineering asphalt pavement quality acceptance specifications, the quality grades are divided into four levels: excellent, good, qualified, and unqualified.

8. A road engineering asphalt pavement quality inspection system based on image recognition, characterized in that, The road engineering asphalt pavement quality inspection system includes the following modules: The image data acquisition module is used to acquire high-resolution visible light images and three-dimensional point cloud data during road construction. The acquired data is preprocessed, and the preprocessed data is time-synchronized and spatially registered to obtain registered image data. The feature vector extraction module is used to build a multi-task deep learning network model. It uses the ResNet-50 network to extract shared deep features from the registered image data, and uses a U-Net structure decoder to upsample and skip connections the deep features to output pavement defect features. It uses the PointNet++ network to learn the texture depth features of the point cloud in the registered image data. The feature vector fusion module is used to fuse pavement defect features and texture depth features using a cross-attention mechanism to obtain a fused quality feature vector; The pavement quality inspection module is used to input the quality feature vector into the XGBoost gradient boosting decision tree, output a quality inspection score, and perform a rating based on the quality inspection score.

9. The image recognition-based asphalt pavement quality inspection system for road engineering asphalt pavement as described in claim 8, characterized in that, The feature vector fusion module includes the following sub-modules: The mapping submodule is used to input the pavement defect features and texture depth features into two independent linear layers for dimension mapping using a cross-attention mechanism to obtain the first feature vector. The normalization submodule is used to calculate the dot product of the transpose matrix of the first feature vector to obtain the attention weight matrix. The weight matrix is ​​then normalized using the Softmax function to obtain the second feature vector. The resulting submodule performs a dot product operation on the second feature vector to generate image feature-guided point cloud features. A linear layer is then used to map the dimensions of the point cloud features to obtain a fused quality feature vector.

10. The image recognition-based asphalt pavement quality inspection system for road engineering as per claim 8, characterized in that, The feature vector fusion module includes the following sub-modules: The sub-module is used to formulate grading standards based on the quality inspection score S, and in conjunction with the road engineering asphalt pavement quality acceptance specifications, the quality grades are divided into four levels: excellent, good, qualified, and unqualified.

Images