Road patrol image disease three-dimensional size and maintenance material consumption estimation method and device

Abstract

The invention discloses a road patrol image disease three-dimensional size and maintenance material estimation method and device, and the method comprises the steps: collecting road patrol image data and laser radar three-dimensional point cloud data, carrying out the visual angle correction, target recognition and feature extraction of the image data, carrying out the screening, cutting, meshing and graying of the three-dimensional point cloud data, and carrying out the detection of the three-dimensional size and maintenance material. Constructing a pavement disease image and a pavement three-dimensional disease data set through feature matching, training an adversarial generative network model, inputting the pavement disease image, generating a corresponding pavement disease image with elevation information, and calculating the three-dimensional size of the pavement disease. And performing three-dimensional reconstruction on the pavement disease image with the elevation information, and estimating the use amount of the asphalt mixture by adopting discrete element simulation. According to the method, the three-dimensional size and the maintenance material of the pavement disease can be estimated by using the efficiently acquired road patrol image data, and support is provided for maintenance strategy formulation and cost estimation.

Description

Technical Field

[0001] The present invention belongs to the technical field of road engineering, and in particular relates to a method and device for estimating the three-dimensional size of defects and maintenance materials in road inspection images. Background Art

[0002] The presence of pavement defects not only reduces road performance but also significantly increases driving safety risks. Therefore, regular inspections of pavement defects are particularly important. Traditional road defect inspections rely mainly on manual labor, which is not only inefficient but also costly. With the development of sensor and automation technologies, automatic recognition technology based on road inspection vehicles has made rapid progress in recent years, and numerous road defect target recognition and semantic segmentation algorithms have been proposed and applied. These technologies can identify the location and type of defects in two-dimensional images, but they have limitations in describing the three-dimensional size information of pavement defects, which is crucial for formulating road maintenance strategies, estimating materials and costs, and so on.

[0003] As an advanced sensor technology, LiDAR (LiDAR) can capture global 3D point cloud data of road infrastructure, including its elevation information. This is crucial for accurately describing the 3D dimensions of road surface defects. However, LiDAR equipment is relatively expensive, and in practice, it's difficult to achieve full road coverage, limiting its widespread application in road defect detection. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for estimating the three-dimensional size of road defects and maintenance materials based on road inspection images.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for estimating the three-dimensional size of road defects and maintenance materials from road inspection images, comprising:

[0007] Step S1, collecting road inspection image data D1 and road surface three-dimensional point cloud data L2;

[0008] Step S2: Perform perspective correction on the road inspection image data D1 to obtain D2, obtain road surface disease image data D3 through road surface disease target recognition and disease data screening, and extract disease image location and boundary information;

[0009] Step S3: Filter one-to-one corresponding pavement defect three-dimensional point cloud data L3 from L2 according to D3 and the defect image location and boundary information;

[0010] Step S4, gridding, grayscale conversion, feature matching, and cropping L3 are performed to obtain a dataset DL having a one-to-one correspondence between the pavement disease image data D4 and the pavement disease three-dimensional point cloud data L5 after feature matching correction;

[0011] Step S5: Build and train a generative adversarial network model, input the perspective-corrected pavement damage test data D*, and generate a corresponding pavement damage image L* with elevation information;

[0012] Step S6: inversely calculating the three-dimensional size of the pavement defect based on the pavement defect image L* with elevation information;

[0013] Step S7: Perform three-dimensional reconstruction on the pavement damage image L* with elevation information, and use discrete element simulation to estimate the amount of asphalt mixture.

[0014] Preferably, in step S2, a pre-trained neural network pavement disease target recognition model is used to identify pavement diseases on the pavement inspection data D2 from a bird's-eye view, and a classification of whether it is a disease is output; where 0 represents no disease and 1 represents disease. For samples with an output result of 1, the number of diseases, the boundary information b = [x1, y1, x2, y2] of each disease, and the location information c collected by the image data are output at the same time; the image data with an output result of 1 is filtered out to form pavement disease image data D3, where the location information corresponding to each disease image is C [c1, c2, ... c n ], the boundary information is B[b1,b2,…b n ].

[0015] As an example, in step S3, the position information corresponding to each disease image is C[c1,c2,…c n ], preliminarily locate the road surface disease area in the road surface three-dimensional point cloud data L2; according to the boundary information B[b1,b2,…b n ], combined with spatial relationships, a larger range of pavement 3D point cloud data is cut to form pavement disease 3D point cloud data L3.

[0016] As a preferred method, in step S4, the pavement disease three-dimensional point cloud data L3 is gridded, and the grid is established with the x and y coordinates of the point cloud as the plane, and the z height is used as the grid value, and interpolation calculation is performed. The z height value is used as the grayscale value, and the pavement disease grayscale image data L4 with elevation information is output, and the data is numbered [l1, l2, ... l n ].

[0017] As a preference, in step S4, the pavement damage image data D3[d1, d2, ...d n ], according to the boundary information B[b1,b2,…b n], spatial relationship, and disease feature matching method, feature matching and correction are performed on L4 and D3 to obtain the pavement disease image data D4 and pavement disease three-dimensional point cloud data L5 after feature matching correction, forming a one-to-one corresponding data set DL.

[0018] Preferably, in step S1, an image sensor installed on a road inspection vehicle is used to collect road inspection image data D1; and on the same road section, a vehicle-mounted laser radar is used to collect the entire road three-dimensional point cloud data L1; the laser radar three-dimensional point cloud data L1 is processed to extract only the road surface, excluding the roadside ancillary infrastructure, to obtain the road surface three-dimensional point cloud data L2.

[0019] Preferably, in step S5, a generative adversarial network model is constructed and trained. The model input is three-channel image data, and the output is a one-channel grayscale image of the same pixel size. Training is performed using dataset DL, with pavement damage image data D4 corrected by feature matching as input and pavement damage data L5 with elevation information as ground truth. This allows the model to generate the corresponding pavement damage image L* with elevation information when fed with perspective-corrected pavement damage test data D*.

[0020] The present invention also provides a device for estimating the three-dimensional size of road defects and maintenance materials from road inspection images, comprising:

[0021] The first processing module is used to collect road inspection image data D1 and road surface three-dimensional point cloud data L2;

[0022] The second processing module is used to perform perspective correction on the road inspection image data D1 to obtain D2, obtain road surface disease image data D3 through road surface disease target recognition and disease data screening, and extract disease image location and boundary information;

[0023] The third processing module is used to select the corresponding pavement disease three-dimensional point cloud data L3 from L2 according to D3 and the disease image location and boundary information;

[0024] The fourth processing module is used to perform gridding, grayscale conversion, feature matching and cropping on L3 to obtain a dataset DL having a one-to-one correspondence between the pavement disease image data D4 and the pavement disease three-dimensional point cloud data L5 after feature matching correction;

[0025] The fifth processing module is used to build and train a generative adversarial network model, input the perspective-corrected pavement damage test data D*, and generate a corresponding pavement damage image L* with elevation information;

[0026] A sixth processing module, configured to inversely calculate the three-dimensional size of the pavement disease based on the pavement disease image L* with elevation information;

[0027] The seventh processing module is used to perform three-dimensional reconstruction of the pavement damage image L* with elevation information, and use discrete element simulation to estimate the amount of asphalt mixture.

[0028] This invention, based on a generative adversarial network, uses a constructed training set and road inspection imagery of pavement damage data as input to generate estimated pavement damage data with elevation information, calculate three-dimensional information about the damage, and estimate maintenance materials. This invention can estimate the three-dimensional size of pavement damage using existing road inspection image data, enabling low-cost and lightweight acquisition of three-dimensional road damage information. It also uses discrete element simulation to estimate asphalt mixture usage, providing important technical support for the formulation of road maintenance strategies and the estimation of maintenance materials and costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0030] Figure 1 This is a flow chart of a method for estimating the three-dimensional size of defects and maintenance materials in road inspection images according to an embodiment of the present invention;

[0031] Figure 2 The process of generating the three-dimensional disease data prediction of the present invention;

[0032] Figure 3 pavement damage image data corrected for perspective;

[0033] Figure 4 The three-dimensional point cloud data L3 of road surface damage;

[0034] Figure 5 The pavement damage grayscale image data L4 with elevation information;

[0035] Figure 6 is the three-dimensional size of the pavement disease;

[0036] Figure 7 Estimating process for curing materials;

[0037] Figure 8 Schematic diagram of aggregate filling based on discrete element simulation. DETAILED DESCRIPTION

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Example 1:

[0041] like Figure 1 As shown, an embodiment of the present invention provides a method for estimating the three-dimensional size of defects and maintenance materials from road inspection images, comprising:

[0042] Step S1, collecting road inspection image data D1 and laser radar three-dimensional point cloud data L1, and performing element extraction on L1 to obtain road surface three-dimensional point cloud data L2;

[0043] Step S2: Perform perspective correction on the road inspection image data D1 to obtain D2, obtain road surface disease image data D3 through road surface disease target recognition and disease data screening, and extract disease image location and boundary information;

[0044] Step S3: Filter one-to-one corresponding pavement defect three-dimensional point cloud data L3 from L2 according to D3 and the defect image location and boundary information;

[0045] Step S4, gridding, grayscale conversion, feature matching, and cropping L3 are performed to obtain a dataset DL having a one-to-one correspondence between the pavement disease image data D4 and the pavement disease three-dimensional point cloud data L5 after feature matching correction;

[0046] Step S5: Build and train a generative adversarial network model. Input the pavement damage test data D* after view correction. The model can generate a corresponding pavement damage image L* with elevation information.

[0047] Step S6: inversely calculating the three-dimensional size of the pavement defect based on the pavement defect image L* with elevation information;

[0048] Step S7: Perform three-dimensional reconstruction on the pavement damage image L* with elevation information, and use discrete element simulation to estimate the amount of asphalt mixture.

[0049] As one implementation of an embodiment of the present invention, in S1, an image sensor mounted on a road inspection vehicle, such as a front-facing camera, a rear-facing camera, or a dashcam, is used to collect road inspection image data D1. On the same road section, a vehicle-mounted laser radar is used to collect full-area 3D point cloud data L1. The laser radar 3D point cloud data L1 is processed using methods such as height thresholding, 3D point cloud clustering, and manual cropping to extract the road surface, excluding roadside infrastructure, to produce 3D road surface point cloud data L2.

[0050] As an implementation method of an embodiment of the present invention, in step S2, the road inspection image data D1 is subjected to perspective correction. The original perspective is the oblique perspective of the vehicle-mounted image sensor. The checkerboard method is used to calibrate the internal parameters of the image sensor, and the pixels of the image data are subjected to coordinate conversion. The cropped and corrected image has formed a unified rectangular image, and the road inspection training data D2 and test data D* of the top-down perspective are obtained. D2 is used to construct a data set, and D* is used for model testing. Further, in step S2, a pre-trained neural network road surface disease target recognition model is used to identify road surface diseases on the road inspection data D2 of the top-down perspective. Output is a classification of whether it is a disease (0 for no disease, 1 for disease). For samples with an output result of 1, the number of diseases, the boundary information b = [x1, y1, x2, y2] of each disease, and the location information c collected by the image data are output at the same time. The image data with an output result of 1 is filtered out to form road surface disease image data D3, where the location information corresponding to each disease image is C[c1, c2, ...c n ], the boundary information is B[b1,b2,…b n ].

[0051] As an implementation method of the embodiment of the present invention, in step S3, the position information corresponding to each disease image is C[c1, c2, ...c n ], preliminarily locate the road surface disease area in the road surface three-dimensional point cloud data L2; according to the boundary information B[b1,b2,…b n ], combined with spatial relationships, a larger range of pavement 3D point cloud data is cut to form pavement disease 3D point cloud data L3.

[0052] As an implementation method of the present invention, in step S4, the pavement disease three-dimensional point cloud data L3 is gridded, and the grid is established with the x and y planes in the point cloud coordinates, and the z height is used as the grid value for interpolation calculation. The z height value is used as the grayscale value, and the pavement disease grayscale image data L4 with elevation information is output, and the data is numbered [l1, l2, ... l n ]. The pavement disease image data D3[d1,d2,…d n ], according to the boundary information B[b1,b2,…bn ], spatial relationship, and disease feature matching method, L4 and D3 are feature matched and corrected to obtain the road surface disease image data D4 and road surface disease 3D point cloud data L5 after feature matching and correction, forming a one-to-one corresponding data set DL.

[0053] As one implementation of an embodiment of the present invention, in step S5, a generative adversarial network model is constructed and trained. The model input is three-channel image data, and the output is a one-channel grayscale image of the same pixel size. Training is performed using dataset DL, with pavement defect image data D4 corrected by feature matching as input and pavement defect data L5 with elevation information as ground truth. This allows the model to generate the corresponding pavement defect image L* with elevation information when fed with perspective-corrected pavement defect test data D*.

[0054] As an implementation method of the present invention, in step S6, the elevation value of the road surface disease is calculated based on the conversion relationship between the road surface elevation z and the image gray value, and the maximum length L of the road surface disease is calculated. max , maximum width W max , maximum depth D max , projection area A, volume V and other three-dimensional size indicators.

[0055] Maximum length L max The calculation formula is as follows:

[0056]

[0057] in, is the set of boundary points of the disease, which is extracted by the convex hull algorithm.

[0058] Maximum width W max The calculation formula is as follows:

[0059]

[0060] Among them, θ is the main direction, Π(ρ,θ ⊥ ) represents the normal force θ ⊥ , the section plane with an offset of ρ.

[0061] Maximum depth D max The calculation formula is as follows:

[0062]

[0063] z ref (x,y)=ax+by+c

[0064] Among them, z ref(x, y) represents the elevation value of the pavement reference plane. The ideal plane model established by fitting the point cloud data of the normal pavement around the defect area represents the theoretical pavement elevation when there is no defect.

[0065] The calculation formula for the projected area A is as follows:

[0066]

[0067] The volume V is calculated as follows:

[0068] V=∫∫ Ω |z(x,y)-z ref (x,y)|dxdy

[0069] As an implementation method of the present invention, in step S7, for cracks, cracks, etc., which do not require a mixture and only use a grouting process, the calculated volume is the asphalt material used for its maintenance. For pothole deformation diseases, the road surface disease image L* with elevation information is 3D reconstructed and converted into a 3D grid data format L s Discrete element simulation is used to estimate the asphalt mixture usage. The model is built using several groups of maximum nominal aggregates with larger particle sizes in the design gradation of the target curing asphalt mixture, and the potholes are filled within the road surface level. The proportion of each material in the asphalt mixture is estimated based on the designed gradation ratio and asphalt-aggregate ratio. The calculation formula is as follows:

[0070] Total mass of mixture M total :

[0071]

[0072] Where Va is the air voids ratio (%), which is the percentage of air volume in the mixture to the total volume; f is the adjustment factor (%), which is set to 1.5; Gmm is the maximum theoretical relative density of asphalt mixture (unitless); ρw is the density of water (usually 1 g / cm 3 or 1000kg / m 3 , for unit conversion).

[0073] Grading is usually expressed as passing percentage, that is, the passing percentage Pi (%) of each sieve size.

[0074] The proportion of each particle size range:

[0075] p i =P i -P i-1

[0076] Pi: the pass rate of the i-th level sieve hole (%), Pi+1: the pass rate of the next level (coarser) sieve hole (%), pi: the proportion (%) of the particle size range (such as 4.75~9.5mm).

[0077] Mass of each particle size range m i formula:

[0078]

[0079] Asphalt quality mb i formula:

[0080]

[0081] For the oil-stone ratio.

[0082] Example 2:

[0083] like Figure 2 As shown, an embodiment of the present invention provides a method for estimating the three-dimensional size and maintenance materials of road inspection image defects. First, a front camera installed on a road inspection vehicle is used to collect road inspection image data D1; on the same road section, a vehicle-mounted laser radar is used to collect the three-dimensional point cloud data L1 of the entire road area; the laser radar three-dimensional point cloud data L1 is processed, and the road surface is extracted by manual cropping with software, excluding roadside ancillary infrastructure, to obtain the road surface three-dimensional point cloud data L2.

[0084] The road inspection image data D1 is corrected for perspective. The internal parameters of the image sensor are calibrated using the checkerboard method. The pixels of the image data are converted into coordinates. The cropped and corrected image forms a unified rectangular image, and the road inspection training data D2 and test data D* from a bird's-eye view are obtained.

[0085] The pre-trained YoloV7 convolutional neural network road surface disease target recognition model is used to identify road surface diseases on the road inspection data D2 from a bird's-eye view. Figure 3 As shown. Output the classification of whether it is a disease (0 is no disease, 1 is disease). For samples with an output result of 1, the number of diseases, the boundary information of each disease b = [x1, y1, x2, y2] and the location information c collected by the image data are output at the same time. The image data with an output result of 1 are filtered out to form the road surface disease image data D3, where the location information corresponding to each disease image is C[c1, c2, ... c n ], the boundary information is B[b1,b2,…b n ].

[0086] According to the position information corresponding to each disease image, C[c1,c2,…c n], preliminarily locate the road surface disease area in the road surface three-dimensional point cloud data L2; according to the boundary information B[b1,b2,…b n ], combined with the spatial relationship, a larger range of pavement 3D point cloud data is cut to form pavement disease 3D point cloud data L3, such as Figure 4 shown.

[0087] Grid the pavement disease 3D point cloud data L3, establish a grid with x and y in the point cloud coordinates as the plane, and use the z height as the grid value for interpolation calculation. Use the z height value as the grayscale value to output the pavement disease grayscale image data L4 with elevation information, as shown in the following example: Figure 5 As shown, the data are numbered [l1,l2,…l n ].

[0088] The pavement disease image data D3[d1,d2,…d n ], according to the boundary information B[b1,b2,…b n ], spatial relationship, and disease feature matching method are used to perform feature matching and correction on L4 and D3 to obtain the pavement disease image data D4 and pavement disease three-dimensional point cloud data L5 after feature matching correction, forming a one-to-one corresponding data set DL, and a total of 2000 training samples are obtained.

[0089] A Pixel2Pixel generative adversarial network model was constructed and trained. The model inputs three-channel image data and outputs a one-channel grayscale image of the same pixel size. Training was performed using the dataset DL, with pavement damage image data D4 corrected by feature matching as input and pavement damage data L5 with elevation information as ground truth. The model, when fed with perspective-corrected pavement damage test data D*, generates the corresponding pavement damage image L* with elevation information.

[0090] by Figure 4 、 Figure 5 Taking the road subsidence as an example, the road subsidence after perspective correction is input, and the model generates the corresponding road disease image with elevation information. According to the conversion relationship between the road elevation z and the image grayscale value, the elevation value of the road disease is inversely calculated, such as Figure 6 As shown, the three-dimensional size indicators of the calculated road subsidence are 1.163 meters in length, 1.021 meters in width, 0.106 meters in maximum depth, 0.887 square meters in projected area, and 0.012 cubic meters in volume.

[0091] according to Figure 7 Estimated curing materials shown.

[0092] The disease is subsidence, so it is necessary to use the discrete element simulation method to estimate the maintenance material. The pavement disease image L* with elevation information is reconstructed in three dimensions and converted into a three-dimensional grid data format L s .

[0093] The model is built using several groups of maximum nominal aggregates with larger particle sizes in the design gradation of the target curing asphalt mixture. The proposed asphalt mixture gradation is shown in Table 1:

[0094] Table 1

[0095] Sieve diameter (mm) percentage 16 100 13.2 100 9.5 100 4.75 90 2.36 55 1.18 35 0.6 20 0.3 12 0.15 7 0.075 5

[0096] The maximum nominal particle sizes of 9.5mm, 4.75mm and 2.36mm were selected for aggregate modeling, and the potholes were filled in without exceeding the road surface level, such as Figure 8 shown.

[0097] The asphalt mixture material is estimated based on the designed gradation ratio and asphalt-stone ratio, as shown in Table 2.

[0098] Table 2

[0099]

[0100] Example 3:

[0101] An embodiment of the present invention further provides a device for estimating the three-dimensional size of defects and maintenance materials from road inspection images, comprising:

[0102] The first processing module is used to collect road inspection image data D1 and road surface three-dimensional point cloud data L2;

[0103] The second processing module is used to perform perspective correction on the road inspection image data D1 to obtain D2, obtain road surface disease image data D3 through road surface disease target recognition and disease data screening, and extract disease image location and boundary information;

[0104] The third processing module is used to select the corresponding pavement disease three-dimensional point cloud data L3 from L2 according to D3 and the disease image location and boundary information;

[0105] The fourth processing module is used to perform gridding, grayscale conversion, feature matching and cropping on L3 to obtain a dataset DL having a one-to-one correspondence between the pavement disease image data D4 and the pavement disease three-dimensional point cloud data L5 after feature matching correction;

[0106] The fifth processing module is used to build and train a generative adversarial network model, input the perspective-corrected pavement damage test data D*, and generate a corresponding pavement damage image L* with elevation information;

[0107] A sixth processing module, configured to inversely calculate the three-dimensional size of the pavement disease based on the pavement disease image L* with elevation information;

[0108] The seventh processing module is used to perform three-dimensional reconstruction of the pavement damage image L* with elevation information, and use discrete element simulation to estimate the amount of asphalt mixture.

[0109] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A method for estimating the three-dimensional size and maintenance material requirements of road inspection images, characterized by: include: Step S1, collecting road inspection image data D1 and road surface three-dimensional point cloud data L2; Step S2: Perform perspective correction on the road inspection image data D1 to obtain D2, obtain road surface disease image data D3 through road surface disease target recognition and disease data screening, and extract disease image location and boundary information; Step S3: Filter one-to-one corresponding pavement defect three-dimensional point cloud data L3 from L2 according to D3 and the defect image location and boundary information; Step S4, gridding, grayscale conversion, feature matching, and cropping L3 are performed to obtain a dataset DL having a one-to-one correspondence between the pavement disease image data D4 and the pavement disease three-dimensional point cloud data L5 after feature matching correction; Step S5: Build and train a generative adversarial network model, input the perspective-corrected pavement damage test data D*, and generate a corresponding pavement damage image L* with elevation information; Step S6: inversely calculating the three-dimensional size of the pavement defect based on the pavement defect image L* with elevation information; Step S7: Perform three-dimensional reconstruction on the pavement damage image L* with elevation information, and use discrete element simulation to estimate the amount of asphalt mixture.

2. The method for estimating the three-dimensional size of road defects and maintenance materials from road inspection images according to claim 1, characterized in that: In step S2, the pre-trained neural network pavement disease target recognition model is used to identify pavement diseases on the top-view pavement inspection data D2, and outputs a classification of whether it is a disease; where 0 represents no disease and 1 represents disease. For samples with an output result of 1, the number of diseases, the boundary information of each disease b = [x1, y1, x2, y2] and the location information c collected by the image data are also output; the image data with an output result of 1 is filtered out to form pavement disease image data D3, where the location information corresponding to each disease image is C [c1, c2, ... c n ], the boundary information is B[b1,b2,…b n ].

3. The method for estimating the three-dimensional size of road defects and maintenance materials from road inspection images according to claim 2, characterized in that: In step S3, the position information corresponding to each disease image is C[c1,c2,…c n ], preliminarily locate the road surface disease area in the road surface three-dimensional point cloud data L2; according to the boundary information B[b1,b2,…b n ], combined with spatial relationships, a larger range of pavement 3D point cloud data is cut to form pavement disease 3D point cloud data L3.

4. The method for estimating the three-dimensional size of road defects and maintenance materials from road inspection images according to claim 3, characterized in that: In step S4, the pavement disease three-dimensional point cloud data L3 is gridded, and the grid is established with the x and y coordinates of the point cloud as the plane, and the z height is used as the grid value for interpolation calculation. The z height value is used as the grayscale value, and the pavement disease grayscale image data L4 with elevation information is output. The data is numbered [l1, l2, ... l n ].

5. The method for estimating the three-dimensional size of road defects and maintenance materials from road inspection images according to claim 4, characterized in that: In step S4, the pavement disease image data D3[d1, d2, ...d n ], according to the boundary information B[b1,b2,…b n ], spatial relationship, and disease feature matching method, feature matching and correction are performed on L4 and D3 to obtain the pavement disease image data D4 and pavement disease three-dimensional point cloud data L5 after feature matching correction, forming a one-to-one corresponding data set DL.

6. The method for estimating the three-dimensional size of road defects and maintenance materials from road inspection images according to claim 5, characterized in that: In step S1, an image sensor installed on a road inspection vehicle is used to collect road inspection image data D1; and on the same road section, a vehicle-mounted laser radar is used to collect full-area three-dimensional point cloud data L1 of the road; the laser radar three-dimensional point cloud data L1 is processed to extract only the road surface, excluding the roadside ancillary infrastructure, to obtain the road surface three-dimensional point cloud data L2.

7. The method for estimating the three-dimensional size of road defects and maintenance materials from road inspection images according to claim 5, characterized in that: In step S5, a generative adversarial network model is constructed and trained. The model input is three-channel image data, and the output is a one-channel grayscale image of the same pixel size. Training is performed using dataset DL, with pavement damage image data D4 corrected by feature matching as input and pavement damage data L5 with elevation information as ground truth. This allows the model to generate the corresponding pavement damage image L* with elevation information when fed with perspective-corrected pavement damage test data D*.

8. A device for estimating the three-dimensional size and maintenance material requirements of road inspection images, characterized in that: include: The first processing module is used to collect road inspection image data D1 and road surface three-dimensional point cloud data L2; The second processing module is used to perform perspective correction on the road inspection image data D1 to obtain D2, obtain road surface disease image data D3 through road surface disease target recognition and disease data screening, and extract disease image location and boundary information; The third processing module is used to select the corresponding pavement disease three-dimensional point cloud data L3 from L2 according to D3 and the disease image location and boundary information; The fourth processing module is used to perform gridding, grayscale conversion, feature matching and cropping on L3 to obtain a dataset DL having a one-to-one correspondence between the pavement disease image data D4 and the pavement disease three-dimensional point cloud data L5 after feature matching correction; The fifth processing module is used to build and train a generative adversarial network model, input the perspective-corrected pavement damage test data D*, and generate a corresponding pavement damage image L* with elevation information; A sixth processing module, configured to inversely calculate the three-dimensional size of the pavement disease based on the pavement disease image L* with elevation information; The seventh processing module is used to perform three-dimensional reconstruction of the pavement damage image L* with elevation information, and use discrete element simulation to estimate the amount of asphalt mixture.

Images