Road surface pothole repair material engineering quantity calculation method based on digital twinborn model

Abstract

The invention discloses a pavement pit slot repair material engineering quantity calculation method based on a digital twinborn model, and belongs to the technical field of pavement repair engineering. In order to accurately calculate the engineering quantity of the pavement pothole repair material, the method comprises the following steps: acquiring a pavement image by adopting a vehicle-mounted three-dimensional structured light camera, and then carrying out image pixel updating processing on the acquired pavement image to obtain a processed pavement image; extracting road surface images which all contain the road surface potholes and the disease types most similar to the image features of the road surface potholes; constructing a data set used for pavement pit slot identification; a lightweight pavement pit slot recognition method is constructed, and a deep learning model for pavement pit slot recognition is obtained; constructing a digital twinborn model of the deep learning model based on pavement pit slot recognition; and S5, based on the digital twinborn model obtained in the step S5, establishing a pavement pit slot repair material engineering quantity calculation method based on the digital twinborn model. According to the method, sufficient and accurate materials can be used for repairing work, and the repairing cost can be controlled.

Description

Technical Field

[0001] The present invention belongs to the technical field of road surface repair engineering, and specifically relates to a method for calculating the engineering quantity of road pothole repair materials based on a digital twin model. Background Art

[0002] Potholes are depressions on the road surface, usually caused by factors such as traffic load, water intrusion, and material aging. Potholes can cause driving discomfort and affect the vehicle's driving stability and safety. Frequent driving through potholes may damage the vehicle's suspension system, increasing vehicle maintenance costs. In addition, potholes may accumulate water, increasing the risk of sliding when driving in rainy days. Long-standing potholes that are not repaired will gradually expand, affecting a larger area of ​​the road structure, increasing the difficulty and cost of repair.

[0003] The method of calculating the quantity of materials for pothole repair can achieve accurate quantity calculation, ensure that sufficient materials are used during the repair process, and avoid repair quality problems caused by insufficient materials. In addition, accurate calculation helps to estimate the repair cost, making budget allocation more scientific and reasonable. In the long run, this will help improve the quality and efficiency of road maintenance and reduce the additional costs caused by repeated repairs. Reasonable quantity calculation can also optimize the procurement and deployment of materials in the repair plan, reduce waste, and improve resource utilization efficiency.

[0004] The invention patent with application number 202010872629.0 and invention name “A method, device and equipment for identifying potholes on a road surface based on a three-dimensional light and shadow model” introduces a method, device and equipment for identifying potholes on a road surface using a three-dimensional light and shadow model. The method obtains three-dimensional image data of the road surface and sets up multiple bidirectional projections at different angles to generate road surface shadow areas. By analyzing these shadow areas, the common road surface shadow areas under all projections are identified, and then the final road surface projection area is determined. Through image connectivity domain analysis, independent and connected shadow areas can be identified to determine possible road surface potholes. Further, by measuring the physical area and average depth of these potholes, the potholes that constitute road surface diseases can be determined.

[0005] However, none of the above inventions consider the method for calculating the amount of materials required to repair potholes in the road. If the material calculation is inaccurate, it may lead to insufficient materials during the repair process, thereby affecting the quality of the repair, making the road surface still prone to damage after repair, and reducing the service life of the road surface. Excessive use of materials may lead to waste of resources and increase maintenance costs. The inability to accurately predict material requirements will make project budgets difficult to control, increase management difficulties and economic burdens. These problems will affect the overall safety and functionality of the road, increase public safety risks and maintenance costs. Summary of the invention

[0006] The problem to be solved by the present invention is to accurately calculate the engineering quantity of road pothole repair materials, and proposes a method for calculating the engineering quantity of road pothole repair materials based on a digital twin model.

[0007] To achieve the above object, the present invention is implemented through the following technical solutions: A method for calculating the engineering quantity of road pothole repair materials based on a digital twin model comprises the following steps: S1. Using a vehicle-mounted three-dimensional structured light camera to collect a road image, and then performing image pixel update processing on the collected road image to obtain a processed road image; S2. performing comparative processing on the processed road surface image obtained in step S1, and extracting all road surface images containing road potholes and road surface images of the disease type most similar to the image features of road potholes; S3. Based on all the road images obtained in step S2, a data set for road pothole identification is constructed; S4. Construct a lightweight road pothole recognition method and obtain a deep learning model for road pothole recognition; S5. Build a digital twin model based on the deep learning model for pothole recognition; S6. Based on the digital twin model obtained in step S5, a method for calculating the quantity of road pothole repair materials based on the digital twin model is established;

[0008] Furthermore, the specific implementation method of step S1 includes the following steps: S1.1. Use a vehicle-mounted 3D structured light camera to collect road images, and control the vehicle speed within 70 km / h during the collection process; S1.2. compressing the collected road surface image using Huffman coding to obtain a compressed road surface image, and then performing plane projection to obtain a projected road surface image P; S1.3. Divide the pixel grayscale value 0~255 into 16 grayscale intervals, denoted as , nu is the gray level number, nu =1,2,3,…,16; S1.4. Divide the projected road surface image P obtained in step S1.2 into N equal parts, and record the image matrix after equal division as Pnum, where num is the number of the image matrix after equal division, num=1,2,…,N; S1.5. Count each grayscale interval in the equally divided image The frequency of the corresponding pixel ; S1.6. Calculate each gray level in Pnum The corresponding grayscale frequency cumulative value , the calculation formula is: ; in, no is less than or equal to nu The gray level number, no =0,1,2,…, nu ; S1.7. Construct a pixel value update method, the calculation formula is: ; in, is the minimum non-zero value among all frequency accumulation values, The image after equal division P The total number of pixels corresponding to num, is the maximum value of the gray level, Grayscale The corresponding pixel update value; S1.8 Perform pixel value update processing on all equally divided images to obtain processed road surface images.

[0009] Furthermore, the specific implementation method of step S2 includes the following steps: S2.1. Similar defects of road potholes include longitudinal cracks, transverse cracks, cracking, rutting, bulging, spalling, oiling, subsidence, and undercutting; S2.2. Manually select the processed road surface images obtained in step S1, select all road surface images including potholes and similar diseases, and select the corresponding disease areas, and mark the road surface potholes, longitudinal cracks, transverse cracks, cracks, rutting, bumps, peeling, oil spills, subsidence, and edge gnawing areas in the road surface images in turn. PHO 1. PHO 2. PHO 3. PHO 4. PHO 5. PHO 6. PHO 7. PHO 8. PHO 9. PHO 10; S2.3. Divide the pixel grayscale value 0~255 into 16 grayscale intervals, denoted as , nu =1,2,3,…,16; S2.4. Count the number of pixels corresponding to each gray level in the potholes Then, the number of pixels corresponding to each gray level of similar diseases of potholes such as longitudinal cracks, transverse cracks, cracks, rutting, bulging, peeling, oiling, subsidence, and edge gnawing is counted. , , , , , , , , ; S2.5. Standardize the number of pixels corresponding to each gray level to obtain the standard number of pixels, where the standard number of pixels corresponding to the potholes is , the calculation formula is: ; The standard number of pixels corresponding to longitudinal cracks, transverse cracks, cracks, rutting, bulging, peeling, oil spilling, subsidence, and edge biting is , , , , , , , , ; S2.6. As a benchmark, the relationship function between road potholes and other diseases is established in turn, and the calculation formula is: ; Among them, ii represents the serial number of other similar diseases, ii=2,3,4,5…10, and we get , , , , , , , , Respectively represent the relationship functions between road potholes and longitudinal cracks, transverse cracks, cracks, rutting, bumps, spalling, oiling, subsidence, and undercutting; S2.7. Comparison , , , , , , , , , and sort them in order from large to small, select the disease types corresponding to the top 3 relationship functions as the disease types most similar to the image features of road potholes, and then select all road surface images containing road potholes and the disease types most similar to the image features of road potholes for use.

[0010] Furthermore, the specific implementation method of step S3 includes the following steps: S3.1. By manually labeling, the pothole area and the other three diseases are selected from all the road images containing potholes and the disease types with the most similar image features to the potholes obtained in step S2, and the potholes are labeled as Pit, and the other three diseases are labeled as other1, other2, and other3 respectively, to obtain a data set for pothole recognition; S3.2. The dataset for pothole identification is divided into a training set and a test set in a ratio of 8:2.

[0011] Furthermore, the specific implementation method of step S4 for constructing a lightweight road pothole recognition method includes the following steps: S4.1. Constructing a basic model for pothole identification: S4.1.1. The basic model A consists of 1 input layer, 14 convolutional layers, 14 activation layers, 5 pooling layers, 4 concatenation layers, 4 convolutional upsampling layers, and 1 output layer. S4.1.2. Input the training set obtained in step S3 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: ; in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the ia The true labels of samples, Based on the prediction of model A ia The probability that a sample is class 1; S4.1.3. The node output values ​​of the four output units corresponding to the basic model A are a 1. a 2. a 3. a 4. Perform initial conversion on the node output values ​​of the four output units to obtain the output values The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, Gis the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S4.1.4. Adoption The training set obtained in step S3 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ; S4.2. Constructing an auxiliary model for pothole identification: S4.2.1. Auxiliary model B consists of 1 input layer, 18 convolutional layers, 18 activation layers, 5 pooling layers, 4 concatenation layers, 4 convolutional upsampling layers, and 1 output layer; S4.2.2. Input the training set obtained in step S3 into the auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the auxiliary model B, and obtain the trained auxiliary model B; The calculation formula of the loss function of auxiliary model B is: ; in, is the loss function of auxiliary model B; is the total number of data samples in auxiliary model B; For the ib The true labels of samples, For auxiliary model B, predict ib The probability that a sample is class 1; S4.2.3. The node output values ​​of the four output units corresponding to auxiliary model B are b 1. b 2. b 3. b 4. Perform initial conversion on the node output values ​​of the four output units to obtain the output values b i The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value b i The corresponding enhancement probability value The calculation formula is: ; S4.2.4. Adoption The training set obtained in step S3 is combined with the error chain derivation rule to train the auxiliary model B. After the training is completed, the disease output value corresponding to the auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , i =1,2,3,4; S4.3. Building an enhanced model for pothole identification: S4.3.1. The enhanced model C consists of 1 input layer, 22 convolutional layers, 22 activation layers, 6 pooling layers, 5 concatenation layers, 5 convolutional upsampling layers, and 1 output layer; S4.3.2. Input the training set obtained in step S3 into the enhanced model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the enhanced model C, and obtain the trained enhanced model C; The loss function of the reinforcement model C is calculated as: ; in, To strengthen the loss function of model C; is the total number of data samples in the reinforcement model C; For the ic The true labels of samples, To strengthen the model C to predict the ic The probability that a sample is class 1; S4.3.3. The node output values ​​of the four output units corresponding to the enhanced model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S4.3.4. Adoption The training set obtained in step S3 is combined with the error chain derivation rule to train the enhanced model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the enhanced model C are obtained as follows: ; S4.4. Based on and , establish the loss function between the auxiliary model B and the output layer of the basic model A, and the calculation formula is: ; based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is: ; based on and , establish the relationship loss function between the auxiliary model B and the output layer of the basic model A , the calculation formula is: ; in, for The weight coefficient of for The weight coefficient of S4.5. Establish the relationship loss function between the output layer of the reinforcement model C and the auxiliary model B , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S4.6. Establish the relationship loss function between the output layer of the enhanced model C and the basic model A , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function is Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S4.7. Define the overall loss function corresponding to the basic model A, auxiliary model B, and enhanced model C , the calculation formula is: ; S4.8. Using the data set for pothole recognition obtained in step S3, the basic model, auxiliary model and enhanced model are trained by error back propagation based on the chain rule. When the overall loss function obtained in step S4.7 is When is minimum, the corresponding model parameters are optimal, and the trained basic model A is a deep learning model for road pothole recognition.

[0012] Furthermore, the specific implementation method of step S5 includes the following steps: S5.1. Export the deep learning model for pothole recognition obtained in step S4 as a deployment file for independent services, including one of a pt file, a pth file, a json file, and a txt file; S5.2. Construct a road information model; S5.2.1. Collect information on the location, length, width, and layout of roads; S5.2.2. The data obtained in step S5.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S5.2.3. Based on step S5.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S5.2.4. Export the road information model to a format file supported by the web application, including a json file or a js file; S5.3. In the digital twin platform, import the deployment file of the deep learning model for pothole recognition obtained in step S5.1, and then import the road information model format file obtained in step S5.2; S5.4. Use Python to construct an API interface between image data and the deep learning model for pothole recognition, which is used for intelligent pothole recognition. Use Python to construct an API interface between the deep learning model for pothole recognition and the road information model to complete the construction of the digital twin model.

[0013] Furthermore, the specific implementation method of step S6 includes the following steps:

[0014] S6.1. Use a 3D structured light camera to collect road surface data and upload it to the digital twin model established in step S5. Then use the deep learning model for pothole recognition deployed in the system to intelligently identify pothole areas in the road image, and select all images containing potholes to obtain pothole data. S6.2. Based on the images containing road potholes obtained in step S6.1, identify the number of road potholes in each image and calculate the total number of potholes NM ; S6.3. Construct a method to randomly select the number of pits and obtain the number of randomly selected pits The calculation formula is: ; in, The confidence level is selected z The score is obtained by looking up the z-value table; is an estimate of the proportion of units in a population that have a particular characteristic; is an acceptable error; S6.4. According to the number of randomly selected pits The pits are randomly selected. After the selection is completed, the size information of the pits is measured on site and the volume of the pits is calculated. The specific method is to measure the length of the i-th pit on site. Then divide the pit into 10 equal parts along its length, and measure the width of the section at each equal section. and depth ,in, Refers to i The width of the ijth cross section of a pit; Refers to i The depth of the ijth section of the pit, ij=1,2,3,…,9; Then calculate the i The volume of the pit Vi , the calculation formula is: ; S6.5. Calculate all The pit volume of the pit , , …, , and then calculate the mean pit volume , the calculation formula is: ; The standard deviation of the pit volume is calculated as : ; S6.6. Construct a 95% confidence interval for the range of pit volumes: , is the z score selected according to the confidence level, obtained by looking up the z value table; Then the volume range of the pothole repair material is obtained as follows: .

[0015] Beneficial effects of the present invention: The present invention discloses a method for calculating the engineering quantity of road pothole repair materials based on a digital twin model. Road potholes are usually irregularly shaped damages that have different sizes, depths, and edge features on the road surface. In combination with the characteristics of potholes, the model structure is designed to include an input layer, a convolution layer, an activation layer, a pooling layer, a splicing layer, a convolution upsampling layer, and an output layer. In the model design structure, the convolution layer is used to extract image features at different levels, the activation layer is used to increase nonlinearity to identify complex pothole patterns, the pooling layer is used to reduce feature dimensions and enhance robustness to position changes, the splicing layer and the convolution upsampling layer are used to combine features of different resolutions, maintain edge details, and finally achieve accurate pixel-level pothole recognition through the output layer. This design allows the model to capture the subtle differences in potholes on the road surface while maintaining high computational efficiency and accuracy, and is suitable for processing complex road image data provided by on-board cameras.

[0016] The present invention describes a method for calculating the engineering quantity of road pothole repair materials based on a digital twin model, and proposes a lightweight road pothole identification method, which can compress the model to the required capacity while ensuring the accuracy of road pothole identification, thereby improving the operating speed and reducing the consumption of computing resources, while maintaining the recognition performance of the model.

[0017] The method for calculating the engineering quantity of road pothole repair materials based on a digital twin model described in the present invention can combine a high-precision digital model with real-time data to achieve instant and accurate diagnosis of road conditions. By integrating sensor data, image recognition technology, and machine learning models, it can not only detect and locate road potholes in a timely manner, but also predict future damage development, thereby optimizing maintenance plans and resource allocation. This helps improve road safety, reduce maintenance costs, and extend the service life of roads, which is of great value in improving public safety and economic efficiency.

[0018] The method for calculating the engineering quantity of road pothole repair materials based on a digital twin model described in the present invention can ensure that sufficient and accurate materials are used in the repair work, improve the repair quality and the service life of the road, help control the repair cost, optimize budget allocation, and reduce resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a method for calculating the engineering quantity of road pothole repair materials based on a digital twin model described in the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the specific embodiments described are only part of the embodiments of the present invention, rather than all of the specific embodiments. The components of the specific embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations, and the present invention can also have other embodiments.

[0021] Therefore, the following detailed description of the specific embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected specific embodiments of the present invention. Based on the specific embodiments of the present invention, all other specific embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0022] In order to further understand the content, features and effects of the present invention, the following specific implementation methods are given as examples, and the attached Figure 1 The detailed instructions are as follows:

[0023] Embodiment 1: A method for calculating the engineering quantity of road pothole repair materials based on a digital twin model comprises the following steps: S1. Using a vehicle-mounted three-dimensional structured light camera to collect a road image, and then performing image pixel update processing on the collected road image to obtain a processed road image; Furthermore, the specific implementation method of step S1 includes the following steps: S1.1. Use a vehicle-mounted 3D structured light camera to collect road images, and control the vehicle speed within 70 km / h during the collection process; S1.2. compressing the collected road surface image using Huffman coding to obtain a compressed road surface image, and then performing plane projection to obtain a projected road surface image P; S1.3. Divide the pixel grayscale value 0~255 into 16 grayscale intervals, denoted as , nu is the gray level number, nu =1,2,3,…,16; S1.4. Divide the projected road surface image P obtained in step S1.2 into N equal parts, and record the image matrix after equal division as Pnum, where num is the number of the image matrix after equal division, num=1,2,…,N; S1.5. Count each grayscale interval in the equally divided image The frequency of the corresponding pixel ; S1.6. Calculate each gray level in Pnum The corresponding gray frequency cumulative value , the calculation formula is: ; in, no is less than or equal to nu The gray level number, no =0,1,2,…, nu ; S1.7. Construct a pixel value update method, the calculation formula is: ; in, is the minimum non-zero value among all frequency accumulation values, The image after equal division P The total number of pixels corresponding to num, is the maximum value of the gray level, Grayscale The corresponding pixel update value; S1.8 Perform pixel value update processing on all equally divided images to obtain processed road surface images.

[0024] S2. performing comparative processing on the processed road surface image obtained in step S1, and extracting all road surface images containing road potholes and road surface images of the disease type most similar to the image features of road potholes; Furthermore, the specific implementation method of step S2 includes the following steps: S2.1. Similar defects of road potholes include longitudinal cracks, transverse cracks, cracking, rutting, bulging, spalling, oiling, subsidence, and undercutting; S2.2. Manually select the processed road surface images obtained in step S1, select all road surface images including potholes and similar diseases, and select the corresponding disease areas, and mark the road surface potholes, longitudinal cracks, transverse cracks, cracks, rutting, bumps, peeling, oil spills, subsidence, and edge gnawing areas in the road surface images in turn. PHO 1. PHO 2. PHO 3. PHO 4. PHO 5. PHO 6. PHO 7. PHO 8. PHO 9. PHO 10; S2.3. Divide the pixel grayscale value 0~255 into 16 grayscale intervals, denoted as , nu =1,2,3,…,16; S2.4. Count the number of pixels corresponding to each gray level in the potholes Then, the number of pixels corresponding to each gray level of similar diseases of potholes such as longitudinal cracks, transverse cracks, cracks, rutting, bulging, peeling, oiling, subsidence, and edge gnawing is counted. , , , , , , , , ; S2.5. Standardize the number of pixels corresponding to each gray level to obtain the standard number of pixels, where the standard number of pixels corresponding to the potholes is , the calculation formula is: ; The standard number of pixels corresponding to longitudinal cracks, transverse cracks, cracks, rutting, bulging, peeling, oil spilling, subsidence, and edge biting is , , , , , , , , ; S2.6. As a benchmark, the relationship function between road potholes and other diseases is established in turn, and the calculation formula is: ; Among them, ii represents the serial number of other similar diseases, ii=2,3,4,5…10, and we get , , , , , , , , Respectively represent the relationship functions between road potholes and longitudinal cracks, transverse cracks, cracks, rutting, bumps, spalling, oiling, subsidence, and undercutting; S2.7. Comparison , , , , , , , , , and sort them in order from large to small, select the disease types corresponding to the top 3 relationship functions as the disease types most similar to the image features of road potholes, and then select all road surface images containing road potholes and the disease types most similar to the image features of road potholes for use.

[0025] S3. Based on all the road images obtained in step S2, a data set for road pothole identification is constructed; Furthermore, the specific implementation method of step S3 includes the following steps: S3.1. By manually labeling, the pothole area and the other three diseases are selected from all the road images containing potholes and the disease types with the most similar image features to the potholes obtained in step S2, and the potholes are labeled as Pit, and the other three diseases are labeled as other1, other2, and other3 respectively, to obtain a data set for pothole recognition; S3.2. The dataset for pothole identification is divided into a training set and a test set in a ratio of 8:2.

[0026] S4. Construct a lightweight road pothole recognition method and obtain a deep learning model for road pothole recognition; Furthermore, the specific implementation method of step S4 for constructing a lightweight road pothole recognition method includes the following steps: S4.1. Constructing a basic model for pothole identification: S4.1.1. The basic model A consists of 1 input layer, 14 convolutional layers, 14 activation layers, 5 pooling layers, 4 concatenation layers, 4 convolutional upsampling layers, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolution layer 1: convolution kernel size 3×3; number of convolution kernels 32; stride: 1; Activation layer 1: The activation function is LeakyReLU; Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 32; Stride: 1; Activation layer 2: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 64; Stride: 1; Activation layer 4: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 5: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 5: The activation function is LeakyReLU; Convolution layer 6: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 6: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 7: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; Activation layer 7: The activation function is LeakyReLU; Convolution layer 8: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; Activation layer 8: The activation function is LeakyReLU; Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 9: convolution kernel size 3×3; number of convolution kernels 512; stride: 1; Activation layer 9: The activation function is LeakyReLU; Convolution layer 10: kernel size 3×3; number of kernels 512; stride: 1; Activation layer 10: The activation function is LeakyReLU; Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution upsampling 1: Use 2×2 transposed convolution upsampling, and the output channels are halved to 256; Concatenation layer 1: connects the feature map with the output of activation layer 8, and the number of channels increases to 512; Convolution layer 11: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; Activation layer 11: The activation function is LeakyReLU; Convolution upsampling 2: Use 2×2 transposed convolution upsampling, and the output channels are halved to 128; Concatenation layer 2: connects the feature map with the output of activation layer 6, and the number of channels increases to 256; Convolution layer 12: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 12: The activation function is LeakyReLU; Convolution upsampling 3: Use 2×2 transposed convolution upsampling, and the output channels are halved to 64; Concatenation layer 3: connects the feature map with the output of activation layer 4, and the number of channels increases to 128; Convolution layer 13: kernel size 3×3; number of kernels 64; stride: 1; Activation layer 13: The activation function is LeakyReLU; Convolution upsampling 4: Use 2×2 transposed convolution upsampling, and the output channels are halved to 32; Concatenation layer 4: connects the feature map with the output of activation layer 2, and the number of channels increases to 64; Convolution layer 14: kernel size 3×3; number of kernels 32; stride: 1; Activation layer 14: the activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; number of neurons: 4; S4.1.2. Input the training set obtained in step S3 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: ; in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the ia The true labels of samples, Based on the prediction of model A ia The probability that a sample is class 1; S4.1.3. The node output values ​​of the four output units corresponding to the basic model A are a 1. a 2. a 3. a 4. Perform initial conversion on the node output values ​​of the four output units to obtain the output values The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as , specifically, for potholes in the road 1, for other diseases is 0; S4.1.4. Adoption The training set obtained in step S3 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ; S4.2. Constructing an auxiliary model for pothole identification: S4.2.1. Auxiliary model B consists of 1 input layer, 18 convolutional layers, 18 activation layers, 5 pooling layers, 4 concatenation layers, 4 convolutional upsampling layers, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolution layer 1: convolution kernel size 3×3; number of convolution kernels 64; Stride: 1; Activation layer 1: The activation function is LeakyReLU; Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; Activation layer 2: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 3: The activation function is LeakyReLU; Convolution layer 4: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 4: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 5: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; Activation layer 5: The activation function is LeakyReLU; Convolution layer 6: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; Activation layer 6: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 7: kernel size 3×3; number of kernels 512; stride: 1; Activation layer 7: The activation function is LeakyReLU; Convolution layer 8: convolution kernel size 3×3; number of convolution kernels 512; stride: 1; Activation layer 8: The activation function is LeakyReLU; Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 9: convolution kernel size 3×3; number of convolution kernels 1024; stride: 1; Activation layer 9: The activation function is LeakyReLU; Convolution layer 10: kernel size 3×3; number of kernels 1024; stride: 1; Activation layer 10: The activation function is LeakyReLU; Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution upsampling 1: Use 2×2 transposed convolution upsampling, and the output channels are halved to 512; Concatenation layer 1: connects the feature map with the output of activation layer 8, and the number of channels increases to 1024; Convolution layer 11: convolution kernel size 3×3; number of convolution kernels 512; stride: 1; Activation layer 11: The activation function is LeakyReLU; Convolution layer 12: convolution kernel size 3×3; number of convolution kernels 512; stride: 1; Activation layer 12: The activation function is LeakyReLU; Convolution upsampling 2: Use 2×2 transposed convolution upsampling, and the output channels are halved to 256; Concatenation layer 2: connects the feature map with the output of activation layer 6, and the number of channels increases to 512; Convolution layer 13: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; Activation layer 13: The activation function is LeakyReLU; Convolution layer 14: kernel size 3×3; number of kernels 256; stride: 1; Activation layer 14: the activation function is LeakyReLU; Convolution upsampling 3: Use 2×2 transposed convolution upsampling, and the output channels are halved to 128; Concatenation layer 3: connects the feature map with the output of activation layer 4, and the number of channels increases to 256; Convolution layer 15: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 15: The activation function is LeakyReLU; Convolution layer 16: kernel size 3×3; number of kernels 128; stride: 1; Activation layer 16: The activation function is LeakyReLU; Convolution upsampling 4: Use 2×2 transposed convolution upsampling, and the output channels are halved to 64; Concatenation layer 4: connects the feature map with the output of activation layer 2, and the number of channels increases to 128; Convolution layer 17: kernel size 3×3; number of kernels 64; stride: 1; Activation layer 17: The activation function is LeakyReLU; Convolution layer 18: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; Activation layer 18: The activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; number of neurons: 4; S4.2.2. Input the training set obtained in step S3 into the auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the auxiliary model B, and obtain the trained auxiliary model B; The calculation formula of the loss function of auxiliary model B is: ; in, is the loss function of auxiliary model B; is the total number of data samples in auxiliary model B; For the ib The true labels of samples, For auxiliary model B, predict ib The probability that a sample is class 1; S4.2.3. The node output values ​​of the four output units corresponding to auxiliary model B are b 1. b 2. b 3. b 4. Perform initial conversion on the node output values ​​of the four output units to obtain the output values b i The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value b i The corresponding enhancement probability value The calculation formula is: ; S4.2.4. Adoption The training set obtained in step S3 is combined with the error chain derivation rule to train the auxiliary model B. After the training is completed, the disease output value corresponding to the auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , i =1,2,3,4, specifically, for potholes in the road 1, for other diseases is 0; S4.3. Building an enhanced model for pothole identification: S4.3.1. The enhanced model C consists of 1 input layer, 22 convolutional layers, 22 activation layers, 6 pooling layers, 5 concatenation layers, 5 convolutional upsampling layers, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolution layer 1: convolution kernel size 3×3; number of convolution kernels 32; Stride: 1; Activation layer 1: The activation function is LeakyReLU; Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 32; Stride: 1; Activation layer 2: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; Activation layer 3: The activation function is LeakyReLU; Convolution layer 4: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; Activation layer 4: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 5: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 5: The activation function is LeakyReLU; Convolution layer 6: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 6: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 7: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; Activation layer 7: The activation function is LeakyReLU; Convolution layer 8: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; Activation layer 8: The activation function is LeakyReLU; Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 9: convolution kernel size 3×3; number of convolution kernels 512; stride: 1; Activation layer 9: The activation function is LeakyReLU; Convolution layer 10: kernel size 3×3; number of kernels 512; stride: 1; Activation layer 10: The activation function is LeakyReLU; Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 11: convolution kernel size 3×3; number of convolution kernels 1024; stride: 1; Activation layer 11: The activation function is LeakyReLU; Convolution layer 12: kernel size 3×3; number of kernels 1024; stride: 1; Activation layer 12: The activation function is LeakyReLU; Pooling layer 6: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution upsampling 1: Use 2×2 transposed convolution upsampling, and the output channels are halved to 512; Concatenation layer 1: connects the feature map with the output of activation layer 10, and the number of channels increases to 1024; Convolution layer 13: kernel size 3×3; number of kernels 512; stride: 1; Activation layer 13: The activation function is LeakyReLU; Convolution layer 14: kernel size 3×3; number of kernels 512; stride: 1; Activation layer 14: the activation function is LeakyReLU; Convolution upsampling 2: Use 2×2 transposed convolution upsampling, and the output channels are halved to 256; Concatenation layer 2: connects the feature map with the output of activation layer 8, and the number of channels increases to 512; Convolution layer 15: kernel size 3×3; number of kernels 256; stride: 1; Activation layer 15: The activation function is LeakyReLU; Convolution layer 16: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; Activation layer 16: The activation function is LeakyReLU; Convolution upsampling 3: Use 2×2 transposed convolution upsampling, and the output channels are halved to 128; Concatenation layer 3: connects the feature map with the output of activation layer 6, and the number of channels increases to 256; Convolution layer 17: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 17: The activation function is LeakyReLU; Convolution layer 18: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; Activation layer 18: The activation function is LeakyReLU; Convolution upsampling 4: Use 2×2 transposed convolution upsampling, and the output channels are halved to 64; Concatenation layer 4: connects the feature map with the output of activation layer 4, and the number of channels increases to 128; Convolution layer 19: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; Activation layer 19: The activation function is LeakyReLU; Convolutional layer 20: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; Activation layer 20: The activation function is LeakyReLU; Convolution upsampling 5: Use 2×2 transposed convolution upsampling, and the output channels are halved to 32; Concatenation layer 5: connects the feature map with the output of activation layer 2, and the number of channels increases to 64; Convolution layer 21: convolution kernel size 3×3; number of convolution kernels 32; stride: 1; Activation layer 21: The activation function is LeakyReLU; Convolutional layer 22: convolution kernel size 3×3; number of convolution kernels 32; stride: 1; Activation layer 22: The activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; number of neurons: 4; S4.3.2. Input the training set obtained in step S3 into the enhanced model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the enhanced model C, and obtain the trained enhanced model C; The loss function of the reinforcement model C is calculated as: ; in, To strengthen the loss function of model C; is the total number of data samples in the reinforcement model C; For the ic The true labels of samples, To strengthen the model C to predict the ic The probability that a sample is class 1; S4.3.3. The node output values ​​of the four output units corresponding to the enhanced model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as , specifically, for potholes in the road 1, for other diseases is 0; S4.3.4. Adoption The training set obtained in step S3 is combined with the error chain derivation rule to train the enhanced model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the enhanced model C are obtained as follows: ; S4.4. Based on and , establish the loss function between the auxiliary model B and the output layer of the basic model A, and the calculation formula is: ; based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is: ; based on and , establish the relationship loss function between the auxiliary model B and the output layer of the basic model A , the calculation formula is: ; in, for The weight coefficient of for The weight coefficient of S4.5. Establish the relationship loss function between the output layer of the reinforcement model C and the auxiliary model B , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S4.6. Establish the relationship loss function between the output layer of the enhanced model C and the basic model A , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function is Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S4.7. Define the overall loss function corresponding to the basic model A, auxiliary model B, and enhanced model C , the calculation formula is: ; S4.8. Using the data set for pothole recognition obtained in step S3, the basic model, auxiliary model and enhanced model are trained by error back propagation based on the chain rule. When the overall loss function obtained in step S4.7 is When is minimum, the corresponding model parameters are optimal, and the trained basic model A is a deep learning model for road pothole recognition.

[0027] S5. Build a digital twin model based on the deep learning model for pothole recognition;

[0028] Furthermore, the specific implementation method of step S5 includes the following steps: S5.1. Export the deep learning model for pothole recognition obtained in step S4 as a deployment file for independent services, including one of a pt file, a pth file, a json file, and a txt file; S5.2. Construct a road information model; S5.2.1. Collect information on the location, length, width, and layout of roads; S5.2.2. The data obtained in step S5.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S5.2.3. Based on step S5.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S5.2.4. Export the road information model to a format file supported by the web application, including a json file or a js file; S5.3. In the digital twin platform, import the deployment file of the deep learning model for pothole recognition obtained in step S5.1, and then import the road information model format file obtained in step S5.2; S5.4. Use Python to construct an API interface between image data and the deep learning model for pothole recognition, which is used for intelligent pothole recognition. Use Python to construct an API interface between the deep learning model for pothole recognition and the road information model to complete the construction of the digital twin model.

[0029] S6. Based on the digital twin model obtained in step S5, a method for calculating the quantity of road pothole repair materials based on the digital twin model is established; The specific implementation method of step S6 includes the following steps: S6.1. Use a 3D structured light camera to collect road surface data and upload it to the digital twin model established in step S5. Then use the deep learning model for pothole recognition deployed in the system to intelligently identify pothole areas in the road image, and select all images containing potholes to obtain pothole data. S6.2. Based on the images containing road potholes obtained in step S6.1, identify the number of road potholes in each image and calculate the total number of potholes NM ; S6.3. Construct a method to randomly select the number of pits and obtain the number of randomly selected pits The calculation formula is: ; in, The confidence level is selected z The score is obtained by looking up the z-value table; is an estimate of the proportion of units in a population that have a particular characteristic; is an acceptable error; S6.4. According to the number of randomly selected pits The pits are randomly selected. After the selection is completed, the size information of the pits is measured on site and the volume of the pits is calculated. The specific method is to measure the length of the i-th pit on site. Then divide the pit into 10 equal parts along its length, and measure the width of the section at each equal section. and depth ,in, Refers to i The width of the ijth cross section of a pit; Refers to i The depth of the ijth section of the pit, ij=1,2,3,…,9; Then calculate the i The volume of the pit Vi , the calculation formula is: ; S6.5. Calculate all The pit volume of the pit , , …, , and then calculate the mean pit volume , the calculation formula is: ; The standard deviation of the pit volume is calculated as : ; S6.6. Construct a 95% confidence interval for the range of pit volumes: , is the z score selected according to the confidence level, obtained by looking up the z value table; Then the volume range of the pothole repair material is obtained as follows: .

[0030] The key points and intended protection points of the present invention are: (1) A method for updating image pixels; (2) A lightweight method for identifying potholes in roads.

[0031] (3) A method for calculating the quantity of road pothole repair materials based on digital twins.

[0032] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0033] Although the present application has been described above with reference to specific embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present application. In particular, as long as there is no structural conflict, the various features in the specific embodiments disclosed in the present application may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for calculating the engineering quantity of road pothole repair materials based on a digital twin model, characterized in that: The steps include: S1. Using a vehicle-mounted three-dimensional structured light camera to collect a road image, and then performing image pixel update processing on the collected road image to obtain a processed road image; S2. performing comparative processing on the processed road surface image obtained in step S1, and extracting all road surface images containing road potholes and road surface images of the disease type most similar to the image features of road potholes; S3. Based on all the road images obtained in step S2, a data set for road pothole identification is constructed; S4. Construct a lightweight road pothole recognition method and obtain a deep learning model for road pothole recognition; S5. Build a digital twin model based on the deep learning model for pothole recognition; S6. Based on the digital twin model obtained in step S5, a method for calculating the engineering quantity of road pothole repair materials based on the digital twin model is established.

2. According to claim 1, a method for calculating the engineering quantity of road pothole repair materials based on a digital twin model is characterized in that: The specific implementation method of step S1 includes the following steps: S1.

1. Use a vehicle-mounted 3D structured light camera to collect road images, and control the vehicle speed within 70 km / h during the collection process; S1.

2. compressing the collected road surface image using Huffman coding to obtain a compressed road surface image, and then performing plane projection to obtain a projected road surface image P; S1.

3. Divide the pixel grayscale value 0~255 into 16 grayscale intervals, denoted as , NU is the gray level number, NU =1,2,3,…,16; S1.

4. Divide the projected road surface image P obtained in step S1.2 into N equal parts, and record the image matrix after equal division as Pnum, where num is the number of the image matrix after equal division, num=1,2,…,N; S1.

5. Count each grayscale interval in the equally divided image The frequency of the corresponding pixel ; S1.

6. Calculate each gray level in Pnum The corresponding grayscale frequency cumulative value , the calculation formula is: ； in, no is less than or equal to NU The gray level number, no =0,1,2,…, NU ; S1.

7. Construct a pixel value update method, the calculation formula is: ； in, is the minimum non-zero value among all frequency accumulation values, The image after equal division P The total number of pixels corresponding to num, is the maximum value of the gray level, Grayscale The corresponding pixel update value; S1.8 Perform pixel value update processing on all equally divided images to obtain processed road surface images.

3. A method for calculating the engineering quantity of road pothole repair materials based on a digital twin model according to claim 1 or 2, characterized in that: The specific implementation method of step S2 includes the following steps: S2.

1. Similar defects of road potholes include longitudinal cracks, transverse cracks, cracking, rutting, bulging, spalling, oiling, subsidence, and undercutting; S2.

2. Manually select the processed road surface images obtained in step S1, select all road surface images including potholes and similar diseases, and select the corresponding disease areas, and mark the road surface potholes, longitudinal cracks, transverse cracks, cracks, rutting, bumps, peeling, oil spills, subsidence, and edge gnawing areas in the road surface images in turn. PHO 1. PHO 2. PHO 3. PHO 4. PHO 5. PHO 6. PHO 7. PHO 8. PHO 9. PHO 10; S2.

3. Divide the pixel grayscale value 0~255 into 16 grayscale intervals, denoted as , NU =1,2,3,…,16; S2.

4. Count the number of pixels corresponding to each gray level in the potholes Then, the number of pixels corresponding to each gray level of similar diseases of potholes such as longitudinal cracks, transverse cracks, cracks, rutting, bulging, peeling, oiling, subsidence, and edge gnawing is counted. , , , , , , , , ; S2.

5. Standardize the number of pixels corresponding to each gray level to obtain the standard number of pixels, where the standard number of pixels corresponding to the potholes is , the calculation formula is: ； The standard number of pixels corresponding to longitudinal cracks, transverse cracks, cracks, rutting, bulging, peeling, oil spilling, subsidence, and edge biting is , , , , , , , , ; S2.

6. As a benchmark, the relationship function between road potholes and other diseases is established in turn, and the calculation formula is: ； Among them, ii represents the serial number of other similar diseases, ii=2,3,4,5…10, and we get , , , , , , , , Respectively represent the relationship functions between road potholes and longitudinal cracks, transverse cracks, cracks, rutting, bumps, spalling, oiling, subsidence, and undercutting; S2.

7. Comparison , , , , , , , , , and sort them in order from large to small, select the disease types corresponding to the top 3 relationship functions as the disease types most similar to the image features of road potholes, and then select all road surface images containing road potholes and the disease types most similar to the image features of road potholes for use.

4. The method for calculating the engineering quantity of road pothole repair materials based on a digital twin model according to claim 3 is characterized in that: The specific implementation method of step S3 includes the following steps: S3.

1. By manually labeling, the pothole area and the other three diseases are selected from all the road images containing potholes and the disease types with the most similar image features to the potholes obtained in step S2, and the potholes are labeled as Pit, and the other three diseases are labeled as other1, other2, and other3 respectively, to obtain a data set for pothole recognition; S3.

2. The dataset for pothole identification is divided into a training set and a test set in a ratio of 8:

2.

5. The method for calculating the engineering quantity of road pothole repair materials based on a digital twin model according to claim 4 is characterized in that: Step S4: A specific implementation method of constructing a lightweight road pothole recognition method includes the following steps: S4.

1. Constructing a basic model for pothole identification: S4.1.

1. The basic model A consists of 1 input layer, 14 convolutional layers, 14 activation layers, 5 pooling layers, 4 concatenation layers, 4 convolutional upsampling layers, and 1 output layer. S4.1.

2. Input the training set obtained in step S3 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: ； in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the ia The true labels of samples, Based on the prediction of model A ia The probability that a sample is class 1; S4.1.

3. The node output values ​​of the four output units corresponding to the basic model A are a 1. a 2. a 3. a 4. Perform initial conversion on the node output values ​​of the four output units to obtain the output values The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ； in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S4.1.

4. Adoption The training set obtained in step S3 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ; S4.

2. Constructing an auxiliary model for pothole identification: S4.2.

1. Auxiliary model B consists of 1 input layer, 18 convolutional layers, 18 activation layers, 5 pooling layers, 4 concatenation layers, 4 convolutional upsampling layers, and 1 output layer. The specific network structure is the following structure connected in sequence: S4.2.

2. Input the training set obtained in step S3 into the auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the auxiliary model B, and obtain the trained auxiliary model B; The calculation formula of the loss function of auxiliary model B is: ； in, is the loss function of auxiliary model B; is the total number of data samples in auxiliary model B; For the ib The true labels of samples, For auxiliary model B, predict ib The probability that a sample is class 1; S4.2.

3. The node output values ​​of the four output units corresponding to auxiliary model B are b 1. b 2. b 3. b 4. Perform initial conversion on the node output values ​​of the four output units to obtain the output values b i The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value b i The corresponding enhancement probability value The calculation formula is: ； S4.2.

4. Adoption The training set obtained in step S3 is combined with the error chain derivation rule to train the auxiliary model B. After the training is completed, the disease output value corresponding to the auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , i =1,2,3,4; S4.

3. Building an enhanced model for pothole identification: S4.3.

1. The enhanced model C consists of 1 input layer, 22 convolutional layers, 22 activation layers, 6 pooling layers, 5 concatenation layers, 5 convolutional upsampling layers, and 1 output layer. The specific network structure is the following structure connected in sequence: S4.3.

2. Input the training set obtained in step S3 into the enhanced model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the enhanced model C, and obtain the trained enhanced model C; The loss function of the reinforcement model C is calculated as: ； in, To strengthen the loss function of model C; is the total number of data samples in the reinforcement model C; For the ic The true labels of samples, To strengthen the model C to predict the ic The probability that a sample is class 1; S4.3.

3. The node output values ​​of the four output units corresponding to the enhanced model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ； in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S4.3.

4. Adoption The training set obtained in step S3 is combined with the error chain derivation rule to train the enhanced model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the enhanced model C are obtained as follows: ; S4.

4. Based on and , establish the loss function between the auxiliary model B and the output layer of the basic model A, and the calculation formula is: ； based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is: ； based on and , establish the relationship loss function between the auxiliary model B and the output layer of the basic model A , the calculation formula is: ； in, for The weight coefficient of for The weight coefficient of S4.

5. Establish the relationship loss function between the output layer of the reinforcement model C and the auxiliary model B , the expression is: ； in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of The calculation formula is: ； The calculation formula is: ； S4.

6. Establish the relationship loss function between the output layer of the enhanced model C and the basic model A , the expression is: ； in, for The weight coefficient of for The weight coefficient of Based on and The loss function is Based on and The loss function of The calculation formula is: ； The calculation formula is: ； S4.

7. Define the overall loss function corresponding to the basic model A, auxiliary model B, and enhanced model C , the calculation formula is: ； S4.

8. Using the data set for pothole recognition obtained in step S3, the basic model, auxiliary model and enhanced model are trained by error back propagation based on the chain rule. When the overall loss function obtained in step S4.7 is When is minimum, the corresponding model parameters are optimal, and the trained basic model A is a deep learning model for road pothole recognition.

6. The method for calculating the engineering quantity of road pothole repair materials based on a digital twin model according to claim 5 is characterized in that: The specific implementation method of step S5 includes the following steps: S5.

1. Export the deep learning model for pothole recognition obtained in step S4 as a deployment file for independent services, including one of a pt file, a pth file, a json file, and a txt file; S5.

2. Construct a road information model; S5.2.

1. Collect information on the location, length, width, and layout of roads; S5.2.

2. The data obtained in step S5.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S5.2.

3. Based on step S5.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S5.2.

4. Export the road information model to a format file supported by the web application, including a json file or a js file; S5.

3. In the digital twin platform, import the deployment file of the deep learning model for pothole recognition obtained in step S5.1, and then import the road information model format file obtained in step S5.2; S5.

4. Use Python to build an API interface between image data and the deep learning model for pothole recognition, which is used for intelligent pothole recognition. Use Python to build an API interface between the deep learning model for pothole recognition and the road information model to complete the construction of the digital twin model.

7. A method for calculating the engineering quantity of road pothole repair materials based on a digital twin model according to claim 6, characterized in that: The specific implementation method of step S6 includes the following steps: S6.

1. Use a 3D structured light camera to collect road surface data and upload it to the digital twin model established in step S5. Then use the deep learning model for pothole recognition deployed in the system to intelligently identify pothole areas in the road image, and select all images containing potholes to obtain pothole data. S6.

2. Based on the images containing road potholes obtained in step S6.1, identify the number of road potholes in each image and calculate the total number of potholes NM ; S6.

3. Construct a method to randomly select the number of pits and obtain the number of randomly selected pits The calculation formula is: ； in, The confidence level is selected z The score is obtained by looking up the z-value table; is an estimate of the proportion of units in a population that have a particular characteristic; is an acceptable error; S6.

4. According to the number of randomly selected pits The pits are randomly selected. After the selection is completed, the size information of the pits is measured on site and the volume of the pits is calculated. The specific method is to measure the length of the i-th pit on site. Then divide the pit into 10 equal parts along its length, and measure the width of the section at each equal section. and depth ,in, Refers to i The width of the ijth cross section of a pit; Refers to i The depth of the ijth section of the pit, ij=1,2,3,…,9; Then calculate the i The volume of the pit Vi , the calculation formula is: ； S6.

5. Calculate all The pit volume of the pit , , …, , and then calculate the mean pit volume , the calculation formula is: ； The standard deviation of the pit volume is calculated as : ； S6.

6. Construct a 95% confidence interval for the range of pit volumes: , is the z score selected according to the confidence level, obtained by looking up the z value table; Then the volume range of the pothole repair material is obtained as follows: .

Images