Pavement disease size accurate quantification method based on binocular vision

Abstract

The invention discloses a pavement disease size accurate quantification method based on binocular vision, and the method specifically comprises the steps: collecting pavement image data through a vehicle-mounted binocular camera, achieving the real-time detection of a disease target through an improved RT-DETR model, and outputting the disease type and detection frame information; after a pavement disease target is detected, pixel-level segmentation is carried out on a disease area in the detection frame based on a semantic segmentation model, and a disease contour mask is extracted; an improved IGEV-Stereo stereo matching algorithm is used for calculating a disparity map, depth information is output based on camera calibration parameters and a disparity estimation result, and conversion from pixel coordinates to three-dimensional coordinates is achieved; and the binocular depth information and a disease detection segmentation result are combined to realize accurate size quantification of typical road surface diseases with different characters. Through the binocular stereoscopic vision technology, automatic detection and precise quantification of pavement diseases can be realized, the cost can be effectively reduced while the disease quantitative evaluation precision is improved, and decision support is provided for road management and maintenance.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and intelligent transportation infrastructure maintenance, and specifically relates to a method for accurately quantifying the size of pavement diseases based on binocular vision. Background Art

[0002] With the continuous increase of China's highway mileage and the continuous expansion of the road network scale, the management and maintenance problems of the built highways have become increasingly prominent. The timely detection, positioning, and quantitative analysis of pavement diseases are important links in evaluating pavement damage and also important tasks in highway maintenance. At present, certain progress has been made in pavement disease detection technology based on computer vision. For example, the recognition of diseases such as cracks and potholes is achieved through object detection algorithms, or the geometric parameters of diseases are extracted by combining the mapping relationship between the pixel plane and the road plane and image processing technology. However, these vision-based pavement disease detection and quantification technologies still have significant deficiencies: on the one hand, the existing algorithms have poor generalization in complex road environments, the model robustness is insufficient, and problems such as missed detection or false detection are prone to occur; on the other hand, most of the current mainstream algorithms are based on monocular cameras. Due to the lack of the ability to obtain depth information, it is difficult to accurately quantify the size of diseases, and the measurement error is relatively large.

[0003] Therefore, there is an urgent need for a low-cost technical solution that can operate stably in complex road environments and has the ability to accurately quantify multi-dimensional disease parameters to promote the upgrading of highway maintenance towards intelligence and precision. Summary of the Invention

[0004] To solve the technical defects in the existing solutions, the present invention proposes a method for accurately quantifying the size of pavement diseases based on binocular vision.

[0005] The technical solution for achieving the purpose of the present invention is: a method for accurately quantifying the size of pavement diseases based on binocular vision, and the specific steps are as follows:

[0006] Step 1: Collect pavement image data, and use the improved RT-DETR model to achieve real-time detection of disease targets, and obtain pavement disease types and pavement disease target detection box information;

[0007] Step 2: Based on the semantic segmentation model, perform pixel-level segmentation on the disease area within the detection box, and extract the disease contour mask;

[0008] Step 3: Use the improved IGEV-Stereo stereo matching algorithm to calculate the disparity map, and output depth information based on the camera calibration parameters and the disparity estimation result to achieve the conversion from pixel coordinates to three-dimensional coordinates;

[0009] Step 4: Combine the binocular depth information with the disease detection and segmentation results, and use the differential size quantification method to measure different typical pavement diseases with different traits, and obtain the physical size of the diseases.

[0010] Preferably, the types of road surface diseases include transverse cracks, longitudinal cracks, crocodile cracks, and potholes.

[0011] Preferably, the improved RT-DETR model includes a feature extraction network structure, a self-attention single-scale feature interaction module, a cross-scale feature fusion structure, and a decoder module;

[0012] The specific process of using the improved RT-DETR model to achieve real-time detection of disease targets and obtain information on road surface disease types and road surface disease target detection frames is as follows:

[0013] Use the feature extraction network structure to extract multi-scale features of the input image. The highest-level features extracted by the feature extraction network are expanded by a 1×1 convolution and then input into the self-attention single-scale feature interaction module to generate a self-attention feature map; the multi-scale features extracted by the feature extraction network structure and the self-attention feature map are fully fused through the cross-scale feature fusion module and then input into the decoder module to obtain detection targets.

[0014] Preferably, the feature extraction network structure includes a multi-scale edge information guidance module, an edge feature fusion module, and a backbone network module. The specific process of using the feature extraction network structure to extract multi-scale features of the input image is as follows:

[0015] Extract the initial feature P1 through cascaded convolution normalization operations, and after max pooling, input it into the backbone network and the multi-scale edge information guidance module respectively;

[0016] The multi-scale edge information guidance module extracts edge information by combining the input initial feature P1 with the Sobel edge operator, and the edge information is output as multi-scale edge-guided features after three consecutive max pooling processes;

[0017] The backbone network module is composed of ResNet blocks. The input initial feature P1 is processed through four ResNet blocks in sequence, gradually generating three intermediate features P3’, P4’, P5’ from low to high. Starting from the second ResNet block, the output of each block corresponds to generating one intermediate feature, and each intermediate feature is fused with the edge-guided features from the multi-scale edge information guidance module through the edge feature fusion module, finally forming multi-scale features P3, P4, P5 corresponding to three resolution levels from low to high.

[0018] Compared with the prior art, the significant advantages of the present invention are:

[0019] 1. The present invention systematically optimizes the disease target detection model and the binocular stereo matching model for lightweight and precision design. While significantly improving the accuracy of disease detection and disparity estimation, the optimized model is more suitable for the actual deployment requirements of in-vehicle edge devices, with lower computational complexity and storage resource occupancy, thus effectively reducing the system deployment cost.

[0020] 2. The present invention more accurately restores the three-dimensional real information of the disease structure based on binocular stereo vision technology, and proposes a differential size quantization algorithm for diseases with different traits, effectively improving the accuracy and adaptability of disease size quantization and meeting the precise quantization requirements of diverse disease forms. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is the flowchart of the method of the present invention.

[0022] Figure 2 is the network structure diagram of the multi-scale edge information extraction module proposed by the present invention.

[0023] Figure 3 is the pseudo-code flowchart of the bidirectional backtracking algorithm based on node degree attributes proposed by the present invention.

[0024] Figure 4 is the schematic diagram of crack width calculation proposed by the present invention.

[0025] Figure 5 is the flowchart of the calculation of the area of the turtle shell cracks proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] In order to better understand the technical content of the present invention, specific embodiments are given below in conjunction with the accompanying drawings for illustration.

[0027] As Figure 1 shown, a method for accurately quantifying the size of road surface diseases based on binocular vision, the specific steps are as follows:

[0028] Step 1. Collect road surface image data through an in-vehicle binocular camera, and use an improved RT-DETR model to achieve real-time detection of disease targets, and output disease types and detection box information;

[0029] Specifically, the installed baseline length of the in-vehicle binocular camera used is 10 cm, which can effectively obtain depth information within the range of 3 - 20 m; the camera resolution is 1920×1080, which can effectively capture disease images with high definition and rich detail information within the road surface range.

[0030] In a further embodiment, the image data collected by the binocular camera also needs to be preprocessed, such as CLAHE histogram equalization and bilateral filtering denoising, to reduce noise interference in complex road environments; and epipolar rectification is completed based on the Bouguet algorithm to provide available data for subsequent binocular stereo matching.

[0031] In a further embodiment, the improved RT-DETR model proposed by the present invention includes a feature extraction network structure based on global edge information transmission, a self-attention single-scale feature interaction module, a cross-scale feature fusion structure, and a decoder module.

[0032] 1) First, the model extracts multi-scale features of the input image through a feature extraction network structure based on global edge information transmission. This structure includes a multi-scale edge information guidance module, an edge feature fusion module, and a backbone network module. The specific process of this structure is as follows:

[0033] ① Initial features P1 are extracted through cascaded convolutional normalization operations and are respectively input into the backbone network module and the multi-scale edge information guidance module after max pooling.

[0034] ② The multi-scale edge information guidance module extracts edge information by combining the input initial features P1 with the Sobel edge operator. The edge information is output as multi-scale edge-guided features after three consecutive max pooling processes.

[0035] ③ The backbone network module consists of ResNet blocks. The input initial features P1 are sequentially processed through four ResNet blocks, and three layers of intermediate features P3’, P4’, and P5’ are gradually generated from low to high. Starting from the second ResNet block, the output of each block corresponds to the generation of one layer of intermediate features. Each layer of intermediate features is fused with the edge-guided features from the multi-scale edge information guidance module through the edge feature fusion module, and finally, multi-scale features P3, P4, and P5 at the corresponding resolution levels are formed.

[0036] 2) After the highest-level features P5 extracted by the feature extraction network are expanded through a 1×1 convolution, they are input into the self-attention single-scale feature interaction module to generate a self-attention feature map F5. The improved model applies an efficient additive self-attention mechanism to replace the traditional self-attention mechanism in this module to generate the self-attention feature map. The specific process is as follows:

[0037] ① The input matrix x with dimensions n×d is linearly transformed to generate a query Q and a key K:

[0038] Q = xW q , K = xW k

[0039] Where is a linear transformation matrix, x is the two-dimensional matrix after the 1×1 convolution expansion of P5, n is the product of the width and height at the original resolution of P5, and d is the number of channels of P5.

[0040] ② Calculate the attention weight α of each processing unit by querying the matrix Q and a learnable weight vector Calculate the attention weight α of each processing unit:

[0041]

[0042] where α is the weight of each processing unit, and d is the dimension of the query matrix.

[0043] ③ Weight-sum the query matrix according to the attention weight α to generate the global query vector q:

[0044]

[0045] where a i is the weight of the i-th processing unit, Q i is the query matrix of the i-th processing unit, and n is the total number of processing units.

[0046] ④ Multiply the global query vector q element-wise with the key matrix K to generate the global context matrix:

[0047] Global context = K * q

[0048] where * represents element-wise multiplication, and the dimension of the global context matrix is n×d.

[0049] ⑤ Perform a linear transformation on the global context matrix, learn the final context representation, and generate the final output through a residual connection:

[0050]

[0051] where Norm is the normalization operation, is the final output feature, which is the self-attention feature map F5.

[0052] 3) The multi-scale features P3, P4, and F5 generated in the above steps are fully fused through a cross-scale feature fusion module and then input into the decoder module to obtain the detection target. The cross-scale feature fusion module of the improved model is constructed with reference to the Context Guided Reconstruction Feature Pyramid Network (CGRFPN) based on context-guided spatial feature reconstruction, including a pyramid context feature extraction module, a rectangular self-calibration module, and a dynamic interpolation fusion module. The specific feature fusion process is as follows:

[0053] ① Use the P3, P4, and self-attention feature maps obtained by the aforementioned module as the input of the pyramid context feature extraction module. In this module, first uniformly downsample the above features to the lowest resolution through average pooling, stack them to form a pyramid feature map, and then input it into a three-level cascaded rectangular self-calibration module for context semantic enhancement, and output context fusion features at three different resolution levels. At the same time, the P3, P4, and self-attention feature maps are separately input into a rectangular self-calibration module to generate enhanced features at three different resolution levels.

[0054] ② The rectangular self-calibration module performs pooling operations along the horizontal and vertical directions respectively, extracts axial context features, generates two axial vectors, and combines these two vectors through broadcast addition to form a context feature map. Subsequently, two large-kernel strip convolutions are applied to calibrate the feature maps in the horizontal and vertical directions respectively.

[0055] ③ Finally, the above context fusion features and enhanced features are subjected to cross-resolution feature alignment through a dynamic interpolation fusion module to achieve pairwise fusion between feature layers, and finally output a multi-scale fusion feature layer containing global semantics and local details.

[0056] Step 2: After detecting the road surface disease target, perform pixel-level segmentation on the disease area within the detection frame based on the semantic segmentation model, and extract the disease contour mask;

[0057] Specifically, the semantic segmentation model used is preferably the SegFormer model through comparative experimental analysis, with an mIoU of 76.5% and an FPS of up to 24 frames per second, taking into account both accuracy and efficiency.

[0058] In a further embodiment, the specific process of the SegFormer model for disease segmentation is as follows:

[0059] 1) Adopt strategies such as cropping and filling for the disease area within the detection frame obtained in Step 1 to make its dimensions meet the input dimension requirements of the segmentation model, generally 512×512×3. Input the processed image into the model.

[0060] 2) Divide the overall image into several small overlapping blocks, and use an encoder based on MixTransformer (MiT) to perform hierarchical encoding on each small block and then re-integrate it into an overall feature map. After four-stage MiT encoder encoding, output four hierarchical feature maps F1, F2, F3, and F4.

[0061] 3) The four hierarchical feature maps generate a fusion feature projection through operations such as channel unification, upsampling, and splicing fusion, and then use a lightweight multi-layer perceptron (MLP) for processing to output the category prediction of the segmentation.

[0062] 4) The decoder finally outputs a segmentation map with a size of 512×512×2 (2 is the number of categories, i.e., disease or background), and each pixel position corresponds to a category probability distribution. Then, through the argmax operation, the segmentation category at each pixel point is obtained to form a segmentation mask, and the model output mask is restored to the original morphological disease pixel-level segmentation mask through operations such as stitching and de-padding.

[0063] Step 3: Use the improved IGEV-Stereo stereo matching model to calculate the disparity map, and output depth information based on the camera calibration parameters and the disparity estimation results to achieve the conversion from pixel coordinates to three-dimensional coordinates;

[0064] In a further embodiment, the improved IGEV-Stereo stereo matching model includes a reused feature extraction network, a combined geometric encoding volume construction module, and a disparity iterative update module based on ConvGRUs. The specific process of stereo matching based on the input binocular images by this algorithm is as follows:

[0065] The left and right views obtained by the vehicle-mounted binocular camera are respectively input into the reused feature extraction network based on RepViT to extract the feature representations of the left and right views, obtaining two feature maps f r and f l . The two feature maps are calculated in two paths to obtain the all-pair cost volume and the grouped cost volume.

[0066] In the combined geometric encoding volume mechanism module, the grouped cost volume is processed by a lightweight 3D regularization network to obtain the geometric encoding volume. The initial disparity d0 is regressed from the geometric encoding volume through the softargmin operation. At the same time, the geometric encoding volume and the all-pair cost volume are stacked and combined to obtain the combined geometric encoding volume.

[0067] The feature f l extracted from the left view by the reused feature extraction network is reused as the global context feature to initialize the hidden state of each ConvGRU unit in the disparity iterative update module. For each disparity iterative update process, based on the current disparity d k a set of geometric features G f is extracted by linear interpolation in the geometric encoding volume. Subsequently, the hidden state is updated using ConvGRU, and the specific formula is as follows:

[0068] x k = [Encoder g (G f ), Encoder d (d k ), d k

[0069] z k= σ(Conv([h k-1 , x k , W z ) + c z )

[0070] r k = σ(Conv([h k-1 , x k , W r ) + c r )

[0071]

[0072] Where z k is the update gate, r k is the reset gate, is the candidate hidden state, W z , W r , W h are the convolutional weights of the update gate, reset gate, and hidden gate respectively, c z , c r , c h are the context biases extracted from the global context features. According to the updated hidden state h k , a disparity residual can be decoded, and the current disparity can be updated by addition. After multiple iterations, the disparity map is gradually optimized to obtain the final output disparity.

[0073] In view of the difference in the difficulty of disparity prediction for pixels in different regions, the improved model proposes to adaptively weight the loss calculation for each pixel based on the probability distribution of disparity prediction, guiding the model to improve the disparity processing effect on the edge detail regions. The specific formula for the adaptive weight is as follows:

[0074]

[0075] Where p(i, d) is the probability that the disparity value at pixel i is d; D is the preset maximum disparity; H(i) is the uncertainty of disparity prediction at pixel i; logD is the maximum uncertainty measure value under the uniform probability distribution, and dividing by it can first map it to the range of 0 - 1. Dividing the normalized uncertainty value of this pixel by the sum of the uncertainty values of all pixels in the image can ensure that the sum of all pixel weights w(i) is 1.

[0076] In a further embodiment, depth information is output based on the camera calibration parameters and the disparity estimation result. The specific formula is as follows:

[0077]

[0078] Where f is the camera focal length, T is the camera baseline length, d is the disparity, Z Cis the depth distance from the target to the optical center of the camera.

[0079] Step 4: Combine the binocular depth information with the disease detection and segmentation results, and propose a differential size quantization method for typical pavement diseases with different characteristics such as transverse and longitudinal cracks, potholes, and crazing, to obtain accurate physical dimensions such as the length, width, and area of the diseases.

[0080] In a further embodiment, the method for calculating the length of the transverse and longitudinal cracks is specifically as follows:

[0081] 1) Skeletonize the crack segmentation mask obtained in Step 2 based on the Zhang-Suen iterative thinning algorithm. The basic idea of this algorithm is to traverse all the pixel points in the mask area and judge the numerical distribution of the 8 pixel nodes around it. If the surrounding pixel distribution meets the specific conditions described by the following formula, it is regarded as a non-skeleton point and deleted.

[0082]

[0083] In the formula, P1 is any pixel point in the mask area, and the rest of the points are the 8 pixel nodes arranged clockwise around this point; A(·) represents the number of jumps from the mask area to the background area; B(·) represents the number of points belonging to the mask area among the 8 surrounding neighborhood points. Iterate the above steps multiple times until there are no pixel points to be deleted, and finally obtain the initial skeleton S of the crack mask.

[0084] 2) Define the degree attribute of the skeleton node as shown in the following formula.

[0085] Among them

[0086] In the formula, I(·) is a pixel value function, which takes 1 if the pixel point is not 0; p is a point on the skeleton, and i is the pixel number of the 8-neighborhood pixels around this pixel point.

[0087] Based on the node degree attribute, adopt a two-way backtracking strategy to eliminate invalid branches for the initial skeleton, and finally obtain the topologically optimized crack skeleton S'. The specific calculation process is as follows:

[0088] ① According to the above node degree attribute, define and construct a degree mapping matrix D of the same size as the image;

[0089] ② Start backtracking from the end pixel p where D(p)=1, trace the path in reverse, until the nearest branch point q is encountered, and record all the points between p and q;

[0090] ③ Calculate the total path length L according to the spatial coordinate differences between the pixel points on the path b . If the total path length L bIf it is less than the set threshold, it is regarded as an invalid branch, and all points between p and q are deleted, and the degree mapping matrix D is updated at the same time. Otherwise, repeat the above ②.

[0091] ④ Repeat the above ② to ③. When all the short paths that meet the conditions are deleted, the optimized skeleton graph S’ is finally obtained.

[0092] 3) Based on the following coordinate transformation formula, map the set of skeleton pixel points to the three-dimensional space.

[0093]

[0094] where Z C is the depth distance obtained in step 3, K is the camera internal parameter matrix, (x i , y i ) is the pixel coordinate, and (X i , Y i , Z i ) is the three-dimensional coordinate.

[0095] Obtain the transverse and longitudinal crack lengths L 3D .

[0096]

[0097] In a further embodiment, the method for calculating the transverse and longitudinal crack widths is as shown in the appendix Figure 3 and specifically is:

[0098] 1) Further obtain the single-pixel boundary of the crack mask based on the Canny operator, which specifically includes five steps: Gaussian filtering, gradient calculation, non-maximum suppression, double-threshold processing, and edge connection. Its mathematical expression is as shown in the following formula.

[0099]

[0100] where E(x, y) is the final crack boundary; T H and T L are the high and low thresholds for classifying the edge intensity; G(x, y) is the gradient amplitude, which is obtained together with the gradient direction θ(x, y) through gradient calculation.

[0101] 2) Search for boundary point pairs along the normal direction of the skeleton points obtained along the crack length The specific search calculation method is as shown in the following formula.

[0102]

[0103] where is the local tangent direction, and is the set of boundary points in the positive and negative normal directions, and the constraint condition ensures that the angle between the search direction and the normal vector is less than θ th .

[0104] 3) Based on the coordinate transformation formula, map the boundary point pairs to three-dimensional space point pairs

[0105]

[0106] Calculate the average Euclidean distance of all boundary point pairs in three-dimensional space as the average crack width W 3D .

[0107]

[0108] In a further embodiment, the method for calculating the pit area is specifically as follows:

[0109] 1) Calculate the ratio of the pixel area of the pit segmentation mask obtained in step 2 to the pixel area of the region surrounded by the pit detection frame obtained in step 1;

[0110]

[0111] where M ∈ {0, 1} H×W is the binary matrix of the pit disease segmentation mask, (x min , y min ), (x max , y max ) are the coordinates of the upper left vertex and the lower right vertex of the detection frame respectively.

[0112] 2) Based on the coordinate transformation formula, map the vertices of the pit detection frame to the vertices of the pit detection frame in three-dimensional space

[0113]

[0114] Calculate the true projected area of the pit detection frame according to the side length of the pit detection frame in three-dimensional space as follows.

[0115] S box = ||P1 - P2||2 × ||P2 - P3||2

[0116] where P1, P2, and P3 are the three vertices of the pit detection frame mapped to three-dimensional space, and ||·||2 is the L2 norm, whose physical meaning is the geometric straight-line distance between two points in three-dimensional space.

[0117] 3) Based on the product of the two-dimensional pixel area ratio and the true area of the pit detection frame, the true area of the pit can be obtained.

[0118] In a further embodiment, the method for calculating the cracking area is as shown in the appendix Figure 4 and specifically:

[0119] 1) Let the set of cracking disease points Determine the horizontal extreme points p L and p R by scanning. These two points form the initial dividing line L LR and divide the point set into upper and lower half-planes.

[0120] 2) Calculate the Euclidean distances from each cracking disease point in the upper and lower half-plane subsets to the current dividing line L LR . With the point p max with the maximum distance, construct a triangle Δp L p max p R . Continue to divide the points outside the triangle into left and right subsets. Subsequently, repeat the above extreme point positioning and dividing operations for the newly divided subsets, recursively process each subset until there are no separable points in the subset.

[0121] 3) Integrate the convex hull vertices obtained from each layer of recursion through post-order traversal to generate a convex hull vertex sequence and connect them to form the final closed polygon.

[0122] 4) Based on the three-dimensional coordinate transformation formula, map the convex hull vertices to the three-dimensional space.

[0123]

[0124] Finally, calculate the area of the convex polygon by the triangulation method shown in the appendix Figure 5 to approximately obtain the true area of the pothole area.

[0125] Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Those with ordinary knowledge in the technical field to which the present invention pertains can make various modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by what is defined in the claims.

Claims

1. An accurate quantification method for pavement disease size based on binocular vision, characterized in that The specific steps are as follows: Step 1: Collect road surface image data, and use the improved RT-DETR model to achieve real-time detection of disease targets, obtaining road surface disease types and road surface disease target detection box information; Step 2: Based on the semantic segmentation model, perform pixel-level segmentation on the disease area within the detection box to extract the disease contour mask; Step 3: Use the improved IGEV-Stereo stereo matching algorithm to calculate the disparity map, and output depth information based on the camera calibration parameters and the disparity estimation result to achieve the conversion from pixel coordinates to three-dimensional coordinates; Step 4: Combine the binocular depth information and the disease detection and segmentation results, and use the differential size quantization method to measure typical road surface diseases with different characteristics to obtain the physical size of the diseases.

2. The method for accurately quantifying the size of road surface diseases based on binocular vision according to claim 1, characterized in that The road surface disease types include transverse cracks, longitudinal cracks, crocodile cracks, and potholes.

3. The method for accurately quantifying the size of road surface diseases based on binocular vision according to claim 1, characterized in that, The improved RT-DETR model includes a feature extraction network structure, a self-attention single-scale feature interaction module, a cross-scale feature fusion structure, and a decoder module; The specific process of using the improved RT-DETR model to achieve real-time detection of disease targets and obtaining road surface disease types and road surface disease target detection box information is as follows: Use the feature extraction network structure to extract multi-scale features of the input image. The highest-level feature extracted by the feature extraction network is expanded by 1×1 convolution and then input into the self-attention single-scale feature interaction module to generate a self-attention feature map; the multi-scale features extracted by the feature extraction network structure and the self-attention feature map are fully fused through the cross-scale feature fusion module and then input into the decoder module to obtain the detection target.

4. The method for accurately quantifying the size of road surface diseases based on binocular vision according to claim 3, wherein The feature extraction network structure includes a multi-scale edge information guidance module, an edge feature fusion module, and a backbone network module. The specific process of using the feature extraction network structure to extract multi-scale features of the input image is as follows: Extract the initial feature P1 through cascaded convolution normalization operations, and after max pooling, input it into the backbone network and the multi-scale edge information guidance module respectively; The multi-scale edge information guidance module extracts edge information by combining the input initial feature P1 with the Sobel edge operator, and the edge information is output as a multi-scale edge guidance feature after three consecutive max pooling processes; The backbone network module is composed of ResNet blocks. The input initial feature P1 is sequentially processed through four ResNet blocks, and three intermediate features P3’, P4’, and P5’ are gradually generated from low to high. Starting from the second ResNet block, the output of each block corresponds to generating one intermediate feature. Each intermediate feature is fused with the edge guidance feature from the multi-scale edge information guidance module through the edge feature fusion module, and finally forms multi-scale features P3, P4, and P5 corresponding to three resolution levels from low to high.

5. The method for accurately quantifying the size of pavement diseases based on binocular vision according to claim 3, wherein The specific process of expanding the highest-level feature by 1×1 convolution and then inputting it into the self-attention single-scale feature interaction module to generate a self-attention feature map is as follows: ① Generate the query matrix Q and the key K by linearly transforming the input matrix: Q = xW q , K = xW k Among them, is a linear transformation matrix, x is a two-dimensional matrix after the highest-level feature is expanded by 1×1 convolution, n is the product of the width and height at the original resolution of the highest-level feature, and d is the number of channels of the highest-level feature; ② By querying the matrix Q and a learnable weight vector w a , calculate the attention weight α for each processing unit: where α is the weight of each processing unit and d is the dimension of the query matrix; ③ Weight-sum the query matrix according to the attention weight α to generate the global query vector q: where a i is the weight of the i-th processing unit, Q i is the query matrix of the i-th processing unit, and n is the total number of processing units; ④Multiply the global query vector q element-wise with the key matrix K to generate the global context matrix: Global context = K * q where * represents element-wise multiplication; ⑤Perform a linear transformation on the global context matrix, learn the final context representation, and generate the final output through a residual connection: Among them, Norm is the normalization operation, is the final output feature, which is the self-attention feature map.

6. The method for accurately quantifying the size of road surface diseases based on binocular vision according to claim 3, wherein The cross-scale feature fusion module includes a pyramid context feature extraction module, a rectangle self-calibration module, and a dynamic interpolation fusion module. The specific process of cross-scale feature fusion by the cross-scale feature fusion module is as follows: Use the multi-scale features and self-attention feature maps obtained by the multi-scale edge information guidance module as the input of the pyramid context feature extraction module. First, uniformly downsample the above features to the lowest resolution through average pooling, stack them to form a pyramid feature map, and then input it into a three-level cascaded rectangle self-calibration module for context semantic enhancement to output context fusion features at three different resolution levels; At the same time, the multi-scale features and self-attention feature maps obtained by the multi-scale edge information guidance module are separately input into a rectangle self-calibration module to generate enhanced features at three different resolution levels; Finally, the context fusion features and the enhanced features are subjected to cross-resolution feature alignment through the dynamic interpolation fusion module to achieve pairwise fusion between feature layers, and finally output a multi-scale fusion feature layer containing global semantics and local details.

7. The method for accurately quantifying the size of road surface diseases based on binocular vision according to claim 6, characterized in that, The specific process of the rectangle self-calibration module for generating feature maps of fused context information at different scales is as follows: Pool the input features along the horizontal and vertical directions respectively to extract axial context features, generate two axial vectors, and combine these two vectors through broadcast addition to form a context feature map; apply two large-kernel strip convolutions to calibrate the feature maps in the horizontal and vertical directions respectively.

8. The method for accurately quantifying the size of road surface diseases based on binocular vision according to claim 1, wherein The improved IGEV-Stereo stereo matching model includes a reused feature extraction network, a combined geometric coding volume construction module, and a disparity iterative update module based on ConvGRUs. The specific process of calculating the disparity map using the improved IGEV-Stereo stereo matching algorithm is as follows: 1) The left and right views obtained by the vehicle-mounted binocular camera respectively pass through the multiplexed feature extraction network based on RepViT to extract the feature representations of the left and right views, and two feature maps f are obtained. r and f l ; The two feature maps are combined and calculated in two paths to obtain the full pairwise cost volume and the grouped cost volume. 2) In the combined geometric coding volume construction module, the grouped cost volume is processed by a lightweight 3D regularization network to obtain the geometric coding volume; the initial disparity d0 is regressed from the geometric coding volume, and at the same time, the geometric coding volume and the all-pair cost volume are stacked and combined to obtain the combined geometric coding volume; 3) The feature f extracted from the left view through the reuse feature extraction network l is reused as the global context feature to initialize the hidden state of each ConvGRU cell in the disparity iterative update module; for each disparity iterative update process, based on the current disparity d k a set of geometric features G are extracted by linear interpolation in the geometric encoding volume f , and then the hidden state is updated using ConvGRU, and the specific formula is as follows: x k = [Encoder g (G f ), Encoder d (d k ), d k ​ z k = σ(Conv([h k-1 , x k , W z ) + c z ) r k = σ(Conv([h k-1 , x k , W r ) + c r ) where z k is the update gate, r k is the reset gate, is the candidate hidden state, W z , W r , W h are the convolutional weights of the update gate, reset gate, and hidden gate respectively, c z , c r , c h are the context biases extracted from the global context features; according to the updated hidden state h k , a disparity residual is decoded, and the current disparity is updated by addition; After multiple iterations, the disparity map is gradually optimized to obtain the final output disparity; 4) Aiming at the difference in the difficulty of disparity prediction for pixels in different regions, adaptively weight the loss calculation for each pixel based on the probability distribution of disparity prediction. The specific formula for the adaptive weight is as follows: In the formula, p(i, d) is the probability that the disparity value at pixel i is d; D is the preset maximum disparity; H(i) is the uncertainty of disparity prediction at pixel i; logD is the maximum uncertainty measure under the uniform probability distribution. Dividing by it can first map it to the range of 0-1. Dividing the normalized uncertainty value of this pixel point by the sum of the uncertainty values of all pixels in the image can ensure that the sum of all pixel weights w(i) is 1.

9. The method for accurately quantifying the size of road surface diseases based on binocular vision according to claim 8, wherein Output depth information based on camera calibration parameters and parallax estimation results. The specific formula is as follows: where f is the camera focal length, T is the camera baseline length, d is the parallax, and Z C is the depth distance from the target to the camera optical center.

10. The method for accurately quantifying the size of road surface diseases based on binocular vision according to claim 1, wherein, Combine binocular depth information with the disease detection segmentation results, and use the differential size quantization method to measure typical pavement diseases with different characteristics. The specific method for obtaining the physical size of the diseases is as follows: The specific method for calculating the transverse and longitudinal crack lengths is as follows: 1) Perform skeletonization on the crack segmentation mask obtained in step 2 based on the Zhang-Suen iterative thinning algorithm. Specifically, traverse all the pixel points in the mask area, and judge the numerical distribution of the 8 pixel nodes around it. If the surrounding pixel distribution meets the specific conditions described by the following formula, it is regarded as a non-skeleton point and deleted: In the formula, P1 is any pixel point in the mask area, and the rest of the points are the 8 pixel nodes arranged clockwise around this point; A(·) represents the number of jumps from the mask area to the background area; B(·) represents the number of points belonging to the mask area among the 8 neighboring points around; Iterate the above steps multiple times until there are no pixel points to be deleted, and finally obtain the initial skeleton S of the crack mask; 2) Define the degree attribute of the skeleton node as shown in the following formula: wherein In the formula, I(·) is the pixel value function, which takes 1 if the pixel point is not 0; p is a point on the skeleton, and i is the pixel number of the 8-neighborhood pixels around this pixel point; Based on the node degree attribute, adopt a two-way backtracking strategy to eliminate invalid branches for the initial skeleton, and finally obtain the topologically optimized crack skeleton S'. The specific calculation process is as follows: ① According to the above node degree attribute, define and construct a degree mapping matrix D of the same size as the image; ② Start backtracking from the end pixel p where D(p)=1, trace the path backwards until the nearest branch point q is encountered, and record all the points between p and q; ③ Calculate the total path length L based on the spatial coordinate differences between the pixel points on the path b , if the total path length L b is less than the set threshold, it is regarded as an invalid branch, delete all the points between p and q, and update the degree mapping matrix D at the same time, otherwise repeat step ②; ④ Repeat steps ② to ③. When all the short paths that meet the conditions are deleted, finally obtain the optimized skeleton graph S'; 3) Map the set of skeleton pixel points to three-dimensional space: where Z C is the depth distance obtained in step 3, K is the camera intrinsic matrix, (x i , y i ) is the pixel coordinate, (X i , Y i , Z i ) is the three-dimensional coordinate; The sum of integrals along the path is used to obtain the transverse and longitudinal crack lengths L 3D : The method for calculating the transverse and longitudinal crack widths is as follows: 1) Obtain the single-pixel boundary E(x, y) of the crack mask based on the Canny operator; 2) Search for boundary point pairs by calculating the normal direction of the skeleton points obtained along the crack length 3) Map the boundary point pairs to 3D space point pairs based on the coordinate transformation formula Calculate the average Euclidean distance of all boundary point pairs in three-dimensional space as the average crack width W 3D : The method for calculating the pothole area is as follows: 1) Calculate the ratio of the pixel area of the pothole segmentation mask obtained in step 2 to the pixel area of the area surrounded by the pothole detection frame obtained in step 1; where \(M\in\{0,1\}\) H×W is the binary matrix of the pothole disease segmentation mask, \((x min , y min ), (x max , y max ) are the coordinates of the upper left vertex and the lower right vertex of the detection box respectively; 2) Map the vertices of the pothole detection frame to the vertices of the pothole detection frame in three-dimensional space Calculate the true projected area of the pothole detection frame according to the side lengths of the pothole detection frame in three-dimensional space: S box = ||P1 - P2||^2 × ||P2 - P3||^2 In the formula, P1, P2, and P3 are the three vertices of the pothole detection frame mapped to three-dimensional space, and ||·||2 is the L2 norm, whose physical meaning is the geometric straight-line distance between two points in three-dimensional space; 3) Based on the product of the two-dimensional pixel area ratio relationship and the true area of the pothole detection frame, obtain the true area of the pothole; The method for calculating the area of map cracking is as follows: 1) Set the crack disease point set Determine the horizontal extreme point p by scanning L and p R , these two points form the initial segmentation line L LR Divide the point set into upper and lower half-planes; 2) Calculate the Euclidean distances from each crack disease point in the upper and lower half-plane subsets to the current segmentation line L LR to find the point p with the maximum distance max and construct a triangle Δp L p max p R p. Then, divide the points outside the triangle into two subsets, the left and the right. Subsequently, repeat the above extreme point positioning and division operations for the newly divided subsets, recursively process each subset until there are no separable points in the subset; 3) Integrate the convex hull vertices obtained by recursively processing each layer through post-order traversal to generate a convex hull vertex sequence Connect them to form the final closed polygon; 4) Based on the three-dimensional coordinate transformation formula, map the vertices of the convex hull to three-dimensional space; 5) Calculate the area of the convex polygon by the triangulation method to approximately obtain the true area of the pothole area.

Images