Method for evaluating influence factors of development state of internal void of road based on digital twinning

Abstract

The invention discloses a road internal void development state influence factor evaluation method based on digital twinning, and belongs to the technical field of road evaluation technologies. In order to accurately evaluate the development state of the internal void of the road, the method comprises the steps of collecting an internal image of the road; constructing a road internal void recognition model which comprises a basic model, an auxiliary model, a first reinforcement model and a second reinforcement model; constructing a loss function between the basic model and the auxiliary model of the road internal void identification and the first reinforcement model and the second reinforcement model, then constructing an overall loss function, and training the basic model and the auxiliary model of the road internal void identification and the first reinforcement model and the second reinforcement model by using the data set of the road internal void identification, obtaining a trained road internal void recognition basic model as a road internal void intelligent recognition model; constructing a digital twinborn model based on the road internal void intelligent identification model; and establishing a road internal void development state influence factor evaluation method based on digital twinning.

Description

Technical Field

[0001] The present invention belongs to the technical field of road evaluation, and specifically relates to a method for evaluating factors affecting the internal void development state of a road based on digital twins. Background Art

[0002] Internal voids in roads refer to the voids or separations between road structure layers due to various reasons. This phenomenon usually occurs below the base or surface layer of the road surface and is not easily observed directly. Voids can lead to a decrease in the bearing capacity of the road structure, which in turn causes a series of structural damages such as road subsidence and cracking. Long-term voids can also aggravate the wear and damage of the road surface, shorten the service life of the road, and increase the cost of repair and reconstruction.

[0003] The uneven road surface caused by the gap will increase the bumps during driving, affecting the vehicle's driving stability and riding comfort. The road surface in the gap area is prone to further damage under the repeated load of the vehicle, forming potholes and other dangerous conditions, which may cause traffic accidents in serious cases. In addition, the gap problem inside the road will also lead to a decrease in the drainage performance of the road surface, increase the risk of rainwater accumulation, and further affect traffic safety.

[0004] Research on the evaluation method of factors affecting the development state of road internal voids can more accurately identify and evaluate various risk factors that lead to voids, such as material properties, road design, construction technology, environmental conditions, and traffic loads. This will not only help to take preventive measures during the road design and construction stages to reduce the occurrence of voids, but also timely detect and repair voids through regular inspections and maintenance during the road operation stage, thereby extending the service life of the road and reducing maintenance costs.

[0005] The invention patent with application number 201510037496.4 and invention name "A device and method for fine detection of loose and empty roads in urban subway sections" proposes an efficient and non-destructive device for fine detection of loose and empty roads in urban subway sections, which can achieve detection at a depth of more than 5 meters without damaging the road. This system consists of non-destructive and minimally destructive testing devices. The non-destructive part realizes three-dimensional scanning through orthogonal shock detection and digital panoramic peek imaging, while the minimally destructive part uses small hole digital drilling and digital borehole pressure static penetration for deeper detection. In conjunction with the data processing unit, this device can provide accurate detection results.

[0006] However, the above invention lacks research on the evaluation method of factors affecting the development state of internal voids in roads, which will lead to the void problem being ignored or misassessed, and thus failure to take timely and appropriate maintenance measures. This not only increases the difficulty and cost of road maintenance, but also leads to premature failure of the pavement structure, affecting road safety and transportation efficiency. Summary of the invention

[0007] The problem to be solved by the present invention is to accurately evaluate the development status of internal voids in a road, and propose an evaluation method for factors affecting the development status of internal voids in a road based on digital twins.

[0008] To achieve the above object, the present invention is implemented through the following technical solutions: A method for evaluating factors affecting the internal void development state of a road based on digital twins includes the following steps: S1. Using 3D ground penetrating radar to collect road interior images, and then annotating the collected images to obtain a dataset for identifying road interior voids; S2. Construct a road internal void identification model, including a basic model, an auxiliary model, a first enhanced model, and a second enhanced model; S3. construct a loss function between the basic model, auxiliary model, first reinforcement model, and second reinforcement model of road interior void identification, and then construct an overall loss function, and use the data set of road interior void identification obtained in step S1 to train the basic model, auxiliary model, first reinforcement model, and second reinforcement model of road interior void identification, and obtain the trained basic model of road interior void identification as the road interior void intelligent identification model; S4. Construct a digital twin model based on the intelligent recognition model of road internal voids; S5. Based on the digital twin model obtained in step S4, a method for evaluating factors affecting the internal void development state of the road based on digital twin is established.

[0009] Furthermore, the specific implementation method of step S1 includes the following steps: S1.1. Use three-dimensional ground penetrating radar to collect images of the road interior; S1.2. Manually select all images containing internal voids, cracks, looseness, and water-rich diseases from the collected images; S1.3. By manually labeling, the hollow area and the other three diseases are selected in the image selected in step S1.2, and the hollow area is marked as Hole. The other three diseases are marked as other1, other2, and other3 respectively, to construct a dataset for identifying hollow areas inside roads; S1.4. The data set for road interior void identification obtained in step S1.3 is divided into a training set and a test set in a ratio of 8:2.

[0010] Furthermore, the specific implementation method of step S2 includes the following steps: S2.1. Constructing the basic model A: S2.1.1. The basic model A consists of 1 input layer, 4 convolutional layers, 5 activation layers, 3 pooling layers, 1 flatten layer, 1 fully connected layer, and 1 output layer; S2.1.2. Input the training set obtained in step S1 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: ; in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the The true labels of samples, Based on the prediction of model A The probability that a sample is class 1; S2.1.3. The node output values ​​of the four output units corresponding to the basic model A are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S2.1.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ; S2.2. Construct auxiliary model B: S2.2.1. Auxiliary model B consists of 1 input layer, 8 convolutional layers, 9 activation layers, 5 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer; S2.2.2. Input the training set obtained in step S1 into the auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the auxiliary model B, and obtain the trained auxiliary model B; The calculation formula of the loss function of auxiliary model B is: ; in, is the loss function of auxiliary model B; is the total number of data samples in auxiliary model B; For the The true labels of samples, For auxiliary model B, predict The probability that a sample is class 1; S2.2.3. The node output values ​​of the four output units corresponding to auxiliary model B are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; S2.2.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the auxiliary model B. After the training is completed, the disease output value corresponding to the auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , i =1,2,3,4; S2.3. Construct the first reinforcement model C: S2.3.1. The first enhanced model C consists of 1 input layer, 10 convolutional layers, 11 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer; S2.3.2. Input the training set obtained in step S1 into the first reinforcement model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the first reinforcement model C, and obtain the trained reinforcement model C; The calculation formula of the loss function of the first reinforcement model C is: ; in, is the loss function of the first reinforcement model C; is the total number of data samples in the first reinforcement model C; For the The true labels of samples, The first reinforcement model C predicts The probability that a sample is class 1; S2.3.3. The node output values ​​of the four output units corresponding to the first reinforcement model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S2.3.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the first enhanced model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the first enhanced model C are obtained as follows: ; S2.4. Construct the second reinforcement model D: S2.4.1. The second enhanced model D consists of 1 input layer, 12 convolutional layers, 13 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer; S2.4.2. Input the training set obtained in step S1 into the second reinforcement model D, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the second reinforcement model D, and obtain the trained reinforcement model C; The calculation formula of the loss function of the second reinforcement model D is: ; in, is the loss function of the second reinforcement model D; is the total number of data samples in the second enhanced model D; For the The true labels of samples, The second reinforcement model D predicts The probability that a sample is class 1; S2.4.3. The node output values ​​of the four output units corresponding to the second enhanced model D are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S2.4.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the second enhanced model D. After the training is completed, the output values ​​of the four types of diseases corresponding to the second enhanced model D are obtained as follows: .

[0011] Furthermore, the specific implementation method of step S3 includes the following steps: S3.1. Based on and , establish the loss function between the auxiliary model B and the output layer of the basic model A, and the calculation formula is: ; based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is: ; based on and , establish the relationship loss function between the auxiliary model B and the output layer of the basic model A , the calculation formula is: ; in, for The weight coefficient of for The weight coefficient of S3.2. Establish the relationship loss function between the output layer of the first reinforcement model C and the auxiliary model B , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S3.3. Establish the relationship loss function between the output layer of the first reinforcement model C and the basic model A , the expression is: ; in, for The weight coefficient of Based on and The loss function of The calculation formula is: ; S3.4. Establish the relationship loss function between the output layer of the second reinforcement model D and the basic model A , the expression is: ; in, for The weight coefficient of Based on and The loss function of The calculation formula is: ; S3.5. Establish the relationship loss function between the output layer of the second reinforcement model D and the first reinforcement model C , the expression is: ; in, for The weight coefficient of Based on and The loss function is for The weight coefficient of Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S3.6. Establishing the overall loss function , the calculation formula is: ; S3.7. Using the road internal gap recognition data set obtained in step S1, based on the chain rule and through the error back propagation method, the basic model, auxiliary model and enhanced model for rutting recognition are trained. When the overall loss function is When it is minimum, the corresponding model parameters are optimal, and the trained basic model A is the final road internal void intelligent recognition model.

[0012] Furthermore, the specific implementation method of step S4 includes the following steps: S4.1. Export the road internal void intelligent identification model obtained in step S3 as a deployment file for independent services, including one of a pt file, a pth file, a json file, and a txt file; S4.2. Constructing a road information model; S4.2.1. Collect information on the location, length, width, and layout of roads; S4.2.2. The data obtained in step S4.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S4.2.3. Based on step S4.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S4.2.4. Export the road information model to a format file supported by the web application, including a json file or a js file; S4.3. In the digital twin platform, import the deployment file of the middle road internal void intelligent identification model obtained in step S4.1, and import the road information model format file obtained in step S4.2; S4.4. Use Python to construct an API interface between image data and the intelligent recognition model for road internal voids for intelligent identification of ruts; use Python to construct an API interface between the intelligent recognition model for road internal voids and the road information model to complete the construction of the digital twin model.

[0013] Furthermore, the specific implementation method of step S5 includes the following steps: S5.1. Use a three-dimensional ground penetrating radar to collect road interior image data and upload it to the digital twin platform obtained in step S4. Then use the road interior void intelligent recognition model to intelligently identify the void area in the road surface image, and select all images containing the road interior void; S5.2. For the image of the hollow inside the road identified in step S5.1, drill holes in the hollow area on site and measure the volume of the hollow area by water injection. VV , that is, the volume of the injected water is the volume of the depleted area; S5.3. For the same area, select two different time nodes t 1. t 2. The volumes of the empty areas obtained by the test are VV 1. VV 2. Calculate the growth rate of the empty area : ; By analogy, we can get the development speed of the empty area corresponding to the time nodes t2~t11 ; S5.4. The factors affecting the generation and development of internal voids in roads include traffic flow, ambient temperature, precipitation, elastic modulus of pavement materials, pavement thickness, service life, number of repairs, service time of pipe network, pavement temperature, and number of surrounding engineering activities; Select 10 time nodes t1, t2, t3, ... t10, and test the values ​​of the influencing factors of the generation and development of internal voids in the corresponding 10 time nodes: Traffic flow JL: The traffic flow at 10 time nodes corresponds to JL1, JL2, JL3, JL4, JL5, JL6, JL7, JL8, JL9, and JL10; Ambient temperature HW: The ambient temperatures at 10 time nodes correspond to HW1, HW2, HW3, HW4, HW5, HW6, HW7, HW8, HW9, and HW10; Precipitation JS: The precipitation at 10 time nodes corresponds to JS1, JS2, JS3, JS4, JS5, JS6, JS7, JS8, JS9, and JS10; Pavement material elastic modulus TM: The pavement material elastic modulus at 10 time nodes corresponds to TM1, TM2, TM3, TM4, TM5, TM6, TM7, TM8, TM9, and TM10; Pavement thickness HD: The pavement thickness at 10 time nodes corresponds to HD1, HD2, HD3, HD4, HD5, HD6, HD7, HD8, HD9, and HD10; Service life FY: The service life of the 10 time nodes corresponds to FY1, FY2, FY3, FY4, FY5, FY6, FY7, FY8, FY9, FY10; Maintenance times WX: The maintenance times of the 10 time nodes correspond to WX1, WX2, WX3, WX4, WX5, WX6, WX7, WX8, WX9, and WX10; Pipeline network service time GF: The pipeline network service time of 10 time nodes corresponds to GF1, GF2, GF3, GF4, GF5, GF6, GF7, GF8, GF9, and GF10; Pavement temperature LW: The pavement temperatures at 10 time nodes correspond to LW1, LW2, LW3, LW4, LW5, LW6, LW7, LW8, LW9, and LW10; Number of surrounding engineering activities ZD: The number of surrounding engineering activities at 10 time nodes corresponds to ZD1, ZD2, ZD3, ZD4, ZD5, ZD6, ZD7, ZD8, ZD9, and ZD10; The volume of the empty area: The volume of the empty area at 10 time nodes corresponds to VV 1. VV 2. VV 3. VV 4. VV 5. VV 6. VV 7. VV 8. VV 9. VV 10, the data are obtained by the middle method measured in step S5.2; The development speed of the empty area: The development speed of the empty area at 10 time nodes corresponds to , the data are measured using the method in step S5.3; S5.5. Establish the relationship equation between the volume of the void area and traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and number of surrounding engineering activities at 10 time nodes. The calculation formula is as follows: ; in, m The value range is 1~10. These are the weight coefficients of the impact of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and number of surrounding engineering activities on the volume of the void area; Solving the above equations together, we get The value of , and arranged in order from large to small, the corresponding order of influencing factors is the degree of influence on the volume of the void area; S5.6. Establish the relationship equation between the volume growth rate of the void area and traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and the number of surrounding engineering activities at 10 time nodes. The calculation formula is as follows: ; in, m The value range is 1~10. These are the weight coefficients of the impact of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and number of surrounding engineering activities on the development speed of the void area; Solving the above equations together, we get The value of , and arranged in order from large to small, the corresponding ranking of influencing factors is the degree of influence on the development speed of the vacant area; S5.7. Based on the above steps, analyze the comprehensive impact of all influencing factors on the volume of the void area and the growth rate of the void area; Set the comprehensive impact factor of traffic flow on the volume of the void area and the growth speed of the void area ZY The calculation formula for 1 is: ; in, is the correction factor of the weight coefficient of the impact of traffic flow on the volume of the empty area, bb 1 is the correction factor of the weight coefficient of the impact of traffic flow on the development speed of the empty area; Then the comprehensive influencing factors of ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, number of surrounding engineering activities, volume of depleted area and development speed of depleted area are calculated. ZY 2. ZY 3. ZY 4. ZY 5. ZY 6. ZY 7. ZY 8. ZY 9.ZY 10; Compare ZY 1. ZY 2. ZY 3. ZY 4. ZY 5. ZY 6. ZY 7. ZY 8. ZY 9. ZY 10, and arranged in order from large to small, which corresponds to the degree of comprehensive influence of various factors on the internal voids of the road.

[0014] Beneficial effects of the present invention: The present invention discloses a method for evaluating factors affecting the development state of road internal voids based on digital twins. The construction of a digital twin model plays a key role in evaluating factors affecting the development state of road internal voids. Digital twin technology can monitor road conditions in real time and simulate behavior under various conditions by creating a virtual copy of a road. This model enables road engineers to test and analyze the impact of different design schemes, material selections, and maintenance strategies on road performance in a virtual environment. The integration of digital twin technology and a method for rapid identification of road internal voids can combine high-precision digital models with real-time data to achieve instant and accurate diagnosis of road conditions. By integrating sensor data, image recognition technology, and machine learning models, not only can internal voids of roads be detected and located in a timely manner, but future damage development can also be predicted, thereby optimizing maintenance plans and resource allocation. This helps improve road safety, reduce maintenance costs, and extend the service life of roads, which is of great value in improving public safety and economic efficiency.

[0015] The present invention discloses a method for evaluating factors affecting the development status of road internal voids based on digital twins. In the intelligent recognition model for road internal voids, the convolution layer captures void features and reflection patterns at different depths, the activation layer introduces nonlinearity to distinguish subtle void signals from noise, the pooling layer improves the robustness of feature detection and reduces noise interference, the Flatten layer converts spatial features into one-dimensional vectors, the fully connected layer integrates features to perform classification decisions, and finally the output layer makes a probability prediction of the disease status based on the learned features.

[0016] The method for evaluating factors affecting the development status of internal road voids based on digital twins described in the present invention can help engineers identify and predict void problems in a timely manner, thereby taking preventive measures or timely repairs to ensure the safety of road use and extend its service life. In addition, this method can also optimize the allocation of maintenance and repair budgets and improve the economic efficiency of road maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a method for evaluating factors affecting the internal void development state of a road based on digital twins described in the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the specific embodiments described are only part of the embodiments of the present invention, rather than all of the specific embodiments. The components of the specific embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations, and the present invention can also have other embodiments.

[0019] Therefore, the following detailed description of the specific embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected specific embodiments of the present invention. Based on the specific embodiments of the present invention, all other specific embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0020] In order to further understand the content, features and effects of the present invention, the following specific implementation methods are given as examples, and the attached Figure 1 The detailed instructions are as follows:

[0021] Embodiment 1: A method for evaluating factors affecting the internal void development state of a road based on digital twins includes the following steps: S1. Using 3D ground penetrating radar to collect road interior images, and then annotating the collected images to obtain a dataset for identifying road interior voids; Furthermore, the specific implementation method of step S1 includes the following steps: S1.1. Use three-dimensional ground penetrating radar to collect images of the road interior; S1.2. Manually select all images containing internal voids, cracks, looseness, and water-rich diseases from the collected images; S1.3. By manually labeling, the hollow area and the other three diseases are selected in the image selected in step S1.2, and the hollow area is marked as Hole. The other three diseases are marked as other1, other2, and other3 respectively, to construct a dataset for identifying hollow areas inside roads; S1.4. The data set for road interior void identification obtained in step S1.3 is divided into a training set and a test set in a ratio of 8:2.

[0022] S2. Construct a road internal void identification model, including a basic model, an auxiliary model, a first enhanced model, and a second enhanced model; Furthermore, the specific implementation method of step S2 includes the following steps: S2.1. Constructing the basic model A: S2.1.1. The basic model A consists of 1 input layer, 4 convolutional layers, 5 activation layers, 3 pooling layers, 1 flatten layer, 1 fully connected layer, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 32; stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 2: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Flatten layer: flattens the output of the last convolutional layer; Fully connected layer: 1024 neurons; Activation layer 5: The activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; number of neurons: 4; S2.1.2. Input the training set obtained in step S1 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: ; in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the The true labels of samples, Based on the prediction of model A The probability that a sample is class 1; S2.1.3. The node output values ​​of the four output units corresponding to the basic model A are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as Specifically, for the internal void of the road 1, for other diseases is 0; S2.1.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ; S2.2. Construct auxiliary model B: S2.2.1. Auxiliary model B consists of 1 input layer, 8 convolutional layers, 9 activation layers, 5 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 5: kernel size 3×3; number of kernels 512; stride: 1; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 6: The activation function is LeakyReLU; Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 7: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 8: The activation function is LeakyReLU; Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2; Flatten layer: flattens the output of the last convolutional layer; Fully connected layer: 1024 neurons; Activation layer 9: The activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; number of neurons: 4; S2.2.2. Input the training set obtained in step S1 into the auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the auxiliary model B, and obtain the trained auxiliary model B; The calculation formula of the loss function of auxiliary model B is: ; in, is the loss function of auxiliary model B; is the total number of data samples in auxiliary model B; For the The true labels of samples, For auxiliary model B, predict The probability that a sample is class 1; S2.2.3. The node output values ​​of the four output units corresponding to auxiliary model B are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; S2.2.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the auxiliary model B. After the training is completed, the disease output value corresponding to the auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , i =1,2,3,4, specifically, for the internal void of the road 1, for other diseases is 0; S2.3. Construct the first reinforcement model C: S2.3.1. The first enhanced model C consists of 1 input layer, 10 convolutional layers, 11 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 5: kernel size 3×3; number of kernels 512; stride: 1; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 6: The activation function is LeakyReLU; Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 7: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 8: The activation function is LeakyReLU; Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 9: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization; Activation layer 9: The activation function is LeakyReLU; Convolution layer 10: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization; Activation layer 10: The activation function is LeakyReLU; Pooling layer 6: Type is maximum pooling; pooling size: 2×2; Stride: 2; Flatten layer: flattens the output of the last convolutional layer; Fully connected layer: 1024 neurons; Activation layer 11: The activation function is LeakyReLU; Output layer: type is Softmax; number of neurons: 4; S2.3.2. Input the training set obtained in step S1 into the first reinforcement model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the first reinforcement model C, and obtain the trained reinforcement model C; The calculation formula of the loss function of the first reinforcement model C is: ; in, is the loss function of the first reinforcement model C; is the total number of data samples in the first reinforcement model C; For the The true labels of samples, The first reinforcement model C predicts The probability that a sample is class 1; S2.3.3. The node output values ​​of the four output units corresponding to the first reinforcement model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as Specifically, for the internal void of the road 1, for other diseases is 0; S2.3.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the first enhanced model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the first enhanced model C are obtained as follows: ; S2.4. Construct the second reinforcement model D: S2.4.1. The second enhanced model D consists of 1 input layer, 12 convolutional layers, 13 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Convolutional layer 2: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 5: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 6: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 7: kernel size 3×3; number of kernels 512; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 8: The activation function is LeakyReLU; Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 9: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 9: The activation function is LeakyReLU; Convolution layer 10: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 10: The activation function is LeakyReLU; Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 11: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization; Activation layer 11: The activation function is LeakyReLU; Convolutional layer 12: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization; Activation layer 12: The activation function is LeakyReLU; Pooling layer 6: Type is maximum pooling; pooling size: 2×2; Stride: 2; Flatten layer: flattens the output of the last convolutional layer; Fully connected layer: 1024 neurons; Activation layer 13: The activation function is LeakyReLU; Output layer: type is Softmax; number of neurons: 4; S2.4.2. Input the training set obtained in step S1 into the second reinforcement model D, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the second reinforcement model D, and obtain the trained reinforcement model C; The calculation formula of the loss function of the second reinforcement model D is: ; in, is the loss function of the second reinforcement model D; is the total number of data samples in the second enhanced model D; For the The true labels of samples, The second reinforcement model D predicts The probability that a sample is class 1; S2.4.3. The node output values ​​of the four output units corresponding to the second enhanced model D are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as Specifically, for the internal void of the road 1, for other diseases is 0; S2.4.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the second enhanced model D. After the training is completed, the output values ​​of the four types of diseases corresponding to the second enhanced model D are obtained as follows: .

[0023] S3. construct a loss function between the basic model, auxiliary model, first reinforcement model, and second reinforcement model of road interior void identification, and then construct an overall loss function, and use the data set of road interior void identification obtained in step S1 to train the basic model, auxiliary model, first reinforcement model, and second reinforcement model of road interior void identification, and obtain the trained basic model of road interior void identification as the road interior void intelligent identification model; Furthermore, the specific implementation method of step S3 includes the following steps: S3.1. Based on and , establish the loss function between the auxiliary model B and the output layer of the basic model A, and the calculation formula is: ; based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is: ; based on and , establish the relationship loss function between the auxiliary model B and the output layer of the basic model A , the calculation formula is: ; in, for The weight coefficient of for The weight coefficient of S3.2. Establish the relationship loss function between the output layer of the first reinforcement model C and the auxiliary model B , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S3.3. Establish the relationship loss function between the output layer of the first reinforcement model C and the basic model A , the expression is: ; in, for The weight coefficient of Based on and The loss function of The calculation formula is: ; S3.4. Establish the relationship loss function between the output layer of the second reinforcement model D and the basic model A , the expression is: ; in, for The weight coefficient of Based on and The loss function of The calculation formula is: ; S3.5. Establish the relationship loss function between the output layer of the second reinforcement model D and the first reinforcement model C , the expression is: ; in, for The weight coefficient of Based on and The loss function is for The weight coefficient of Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S3.6. Establishing the overall loss function , the calculation formula is: ; S3.7. Using the road internal gap recognition data set obtained in step S1, based on the chain rule and through the error back propagation method, the basic model, auxiliary model and enhanced model for rutting recognition are trained. When the overall loss function is When it is minimum, the corresponding model parameters are optimal, and the trained basic model A is the final road internal void intelligent recognition model.

[0024] S4. Construct a digital twin model based on the intelligent recognition model of road internal voids; Furthermore, the specific implementation method of step S4 includes the following steps: S4.1. Export the road internal void intelligent identification model obtained in step S3 as a deployment file for independent services, including one of a pt file, a pth file, a json file, and a txt file; S4.2. Constructing a road information model; S4.2.1. Collect information on the location, length, width, and layout of roads; S4.2.2. The data obtained in step S4.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S4.2.3. Based on step S4.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S4.2.4. Export the road information model to a format file supported by the web application, including a json file or a js file; S4.3. In the digital twin platform, import the deployment file of the middle road internal void intelligent identification model obtained in step S4.1, and import the road information model format file obtained in step S4.2; S4.4. Use Python to construct an API interface between image data and the intelligent recognition model for road internal voids for intelligent identification of ruts; use Python to construct an API interface between the intelligent recognition model for road internal voids and the road information model to complete the construction of the digital twin model.

[0025] S5. Based on the digital twin model obtained in step S4, a method for evaluating factors affecting the internal void development state of the road based on digital twin is established.

[0026] Furthermore, the specific implementation method of step S5 includes the following steps: S5.1. Use a three-dimensional ground penetrating radar to collect road interior image data and upload it to the digital twin platform obtained in step S4. Then use the road interior void intelligent recognition model to intelligently identify the void area in the road surface image, and select all images containing the road interior void; S5.2. For the image of the hollow inside the road identified in step S5.1, drill holes in the hollow area on site and measure the volume of the hollow area by water injection. VV , that is, the volume of the injected water is the volume of the depleted area; S5.3. For the same area, select two different time nodes t 1. t 2. The volumes of the empty areas obtained by the test are VV 1. VV 2. Calculate the growth rate of the empty area : ; By analogy, we can get the development speed of the empty area corresponding to the time nodes t2~t11 ; S5.4. The factors affecting the generation and development of internal voids in roads include traffic flow, ambient temperature, precipitation, elastic modulus of pavement materials, pavement thickness, service life, number of repairs, service time of pipe network, pavement temperature, and number of surrounding engineering activities; Select 10 time nodes t1, t2, t3, ... t10, and test the values ​​of the influencing factors of the generation and development of internal voids in the corresponding 10 time nodes. The above data are obtained from public channels; Traffic flow JL: The traffic flow at 10 time nodes corresponds to JL1, JL2, JL3, JL4, JL5, JL6, JL7, JL8, JL9, and JL10; Ambient temperature HW: The ambient temperatures at 10 time nodes correspond to HW1, HW2, HW3, HW4, HW5, HW6, HW7, HW8, HW9, and HW10; Precipitation JS: The precipitation at 10 time nodes corresponds to JS1, JS2, JS3, JS4, JS5, JS6, JS7, JS8, JS9, and JS10; Pavement material elastic modulus TM: The pavement material elastic modulus at 10 time nodes corresponds to TM1, TM2, TM3, TM4, TM5, TM6, TM7, TM8, TM9, and TM10; Pavement thickness HD: The pavement thickness at 10 time nodes corresponds to HD1, HD2, HD3, HD4, HD5, HD6, HD7, HD8, HD9, and HD10; Service life FY: The service life of the 10 time nodes corresponds to FY1, FY2, FY3, FY4, FY5, FY6, FY7, FY8, FY9, FY10; Maintenance times WX: The maintenance times of the 10 time nodes correspond to WX1, WX2, WX3, WX4, WX5, WX6, WX7, WX8, WX9, and WX10; Pipeline network service time GF: The pipeline network service time of 10 time nodes corresponds to GF1, GF2, GF3, GF4, GF5, GF6, GF7, GF8, GF9, and GF10; Pavement temperature LW: The pavement temperatures at 10 time nodes correspond to LW1, LW2, LW3, LW4, LW5, LW6, LW7, LW8, LW9, and LW10; Number of surrounding engineering activities ZD: The number of surrounding engineering activities at 10 time nodes corresponds to ZD1, ZD2, ZD3, ZD4, ZD5, ZD6, ZD7, ZD8, ZD9, and ZD10; The volume of the empty area: The volume of the empty area at 10 time nodes corresponds to VV 1. VV 2. VV 3. VV 4. VV 5. VV 6. VV 7. VV 8. VV 9. VV 10, the data are obtained by the middle method measured in step S5.2; The development speed of the empty area: The development speed of the empty area at 10 time nodes corresponds to , the data are measured using the method in step S5.3; S5.5. Establish the relationship equation between the volume of the void area and traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and number of surrounding engineering activities at 10 time nodes. The calculation formula is as follows: ; in, m The value range is 1~10. These are the weight coefficients of the impact of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and number of surrounding engineering activities on the volume of the void area; Solving the above equations together, we get The value of , and arranged in order from large to small, the corresponding order of influencing factors is the degree of influence on the volume of the void area; S5.6. Establish the relationship equation between the volume growth rate of the void area and traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and the number of surrounding engineering activities at 10 time nodes. The calculation formula is as follows: ; in, m The value range is 1~10. These are the weight coefficients of the impact of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and number of surrounding engineering activities on the development speed of the void area; Solving the above equations together, we get The value of , and arranged in order from large to small, the corresponding ranking of influencing factors is the degree of influence on the development speed of the vacant area; S5.7. Based on the above steps, analyze the comprehensive impact of all influencing factors on the volume of the void area and the growth rate of the void area; Set the comprehensive impact factor of traffic flow on the volume of the void area and the growth speed of the void area ZY The calculation formula for 1 is: ; in, is the correction factor of the weight coefficient of the impact of traffic flow on the volume of the empty area, bb 1 is the correction factor of the weight coefficient of the impact of traffic flow on the development speed of the empty area; Then the comprehensive influencing factors of ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, number of surrounding engineering activities, volume of depleted area and development speed of depleted area are calculated. ZY 2. ZY 3. ZY 4.ZY 5. ZY 6. ZY 7. ZY 8. ZY 9. ZY 10; Compare ZY 1. ZY 2. ZY 3. ZY 4. ZY 5. ZY 6. ZY 7. ZY 8. ZY 9. ZY 10, and arranged in order from large to small, which corresponds to the degree of comprehensive influence of various factors on the internal voids of the road.

[0027] The key points and intended protection points of the present invention are: (1) A multi-model fusion method for intelligent identification of road interior voids.

[0028] (2) A method for evaluating factors affecting the internal void development status of roads based on digital twins.

[0029] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0030] Although the present application has been described above with reference to specific embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present application. In particular, as long as there is no structural conflict, the various features in the specific embodiments disclosed in the present application may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for evaluating factors affecting the internal void development state of a road based on digital twins, characterized in that: The steps include: S1. Using 3D ground penetrating radar to collect road interior images, and then annotating the collected images to obtain a dataset for identifying road interior voids; S2. Construct a road internal void identification model, including a basic model, an auxiliary model, a first enhanced model, and a second enhanced model; S3. construct a loss function between the basic model, auxiliary model, first reinforcement model, and second reinforcement model of road interior void identification, and then construct an overall loss function, and use the data set of road interior void identification obtained in step S1 to train the basic model, auxiliary model, first reinforcement model, and second reinforcement model of road interior void identification, and obtain the trained basic model of road interior void identification as the road interior void intelligent identification model; S4. Construct a digital twin model based on the intelligent recognition model of road internal voids; S5. Based on the digital twin model obtained in step S4, a method for evaluating factors affecting the internal void development state of the road based on digital twin is established.

2. According to the method for evaluating factors affecting the internal void development state of a road based on digital twins in claim 1, it is characterized in that: The specific implementation method of step S1 includes the following steps: S1.

1. Use three-dimensional ground penetrating radar to collect images of the road interior; S1.

2. Manually select all images containing internal voids, cracks, looseness, and water-rich diseases from the collected images; S1.

3. By manually labeling, the hollow area and the other three diseases are selected in the image selected in step S1.2, and the hollow area is marked as Hole. The other three diseases are marked as other1, other2, and other3 respectively, to construct a dataset for identifying hollow areas inside roads; S1.

4. The data set for road interior void identification obtained in step S1.3 is divided into a training set and a test set in a ratio of 8:

2.

3. According to the method for evaluating factors affecting the internal void development state of a road based on digital twins in claim 2, it is characterized in that: The specific implementation method of step S2 includes the following steps: S2.

1. Constructing the basic model A: S2.1.

1. The basic model A consists of 1 input layer, 4 convolutional layers, 5 activation layers, 3 pooling layers, 1 flatten layer, 1 fully connected layer, and 1 output layer; S2.1.

2. Input the training set obtained in step S1 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: ； in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the The true labels of samples, Based on the prediction of model A The probability that a sample is class 1; S2.1.

3. The node output values ​​of the four output units corresponding to the basic model A are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ； in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S2.1.

4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ; S2.

2. Construct auxiliary model B: S2.2.

1. Auxiliary model B consists of 1 input layer, 8 convolutional layers, 9 activation layers, 5 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer; S2.2.

2. Input the training set obtained in step S1 into the auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the auxiliary model B, and obtain the trained auxiliary model B; The calculation formula of the loss function of auxiliary model B is: ； in, is the loss function of auxiliary model B; is the total number of data samples in auxiliary model B; For the The true labels of samples, For auxiliary model B, predict The probability that a sample is class 1; S2.2.

3. The node output values ​​of the four output units corresponding to auxiliary model B are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ； S2.2.

4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the auxiliary model B. After the training is completed, the disease output value corresponding to the auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , i =1,2,3,4; S2.

3. Construct the first reinforcement model C: S2.3.

1. The first enhanced model C consists of 1 input layer, 10 convolutional layers, 11 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer; S2.3.

2. Input the training set obtained in step S1 into the first reinforcement model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the first reinforcement model C, and obtain the trained reinforcement model C; The calculation formula of the loss function of the first reinforcement model C is: ； in, is the loss function of the first reinforcement model C; is the total number of data samples in the first reinforcement model C; For the The true labels of samples, The first reinforcement model C predicts The probability that a sample is class 1; S2.3.

3. The node output values ​​of the four output units corresponding to the first reinforcement model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ； in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S2.3.

4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the first enhanced model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the first enhanced model C are obtained as follows: ; S2.

4. Construct the second reinforcement model D: S2.4.

1. The second enhanced model D consists of 1 input layer, 12 convolutional layers, 13 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer; S2.4.

2. Input the training set obtained in step S1 into the second enhanced model D, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the second enhanced model D, and obtain the trained enhanced model C; The calculation formula of the loss function of the second reinforcement model D is: ； in, is the loss function of the second reinforcement model D; is the total number of data samples in the second enhanced model D; For the The true labels of samples, The second reinforcement model D predicts The probability that a sample is class 1; S2.4.

3. The node output values ​​of the four output units corresponding to the second enhanced model D are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ； in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S2.4.

4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the second enhanced model D. After the training is completed, the output values ​​of the four types of diseases corresponding to the second enhanced model D are obtained as follows: .

4. According to claim 3, a method for evaluating factors affecting the internal void development state of a road based on digital twins is characterized in that: The specific network structure of the basic model A in step S2.1.1 is the following structures connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 32; stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 2: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Flatten layer: flattens the output of the last convolutional layer; Fully connected layer: 1024 neurons; Activation layer 5: The activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; Number of neurons:

4.

5. According to the method for evaluating factors affecting the internal void development state of a road based on digital twins in claim 4, it is characterized in that: The specific network structure of the auxiliary model B in step S2.2.1 is the following structures connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 5: kernel size 3×3; number of kernels 512; stride: 1; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 6: The activation function is LeakyReLU; Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 7: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 8: The activation function is LeakyReLU; Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2; Flatten layer: flattens the output of the last convolutional layer; Fully connected layer: 1024 neurons; Activation layer 9: The activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; Number of neurons:

4.

6. The method for evaluating factors affecting the internal void development state of a road based on digital twin according to claim 5 is characterized in that: The specific network structure of the first enhanced model C in step S2.3.1 is the following structures connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 5: kernel size 3×3; number of kernels 512; stride: 1; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 6: The activation function is LeakyReLU; Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 7: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 8: The activation function is LeakyReLU; Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 9: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization; Activation layer 9: The activation function is LeakyReLU; Convolution layer 10: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization; Activation layer 10: The activation function is LeakyReLU; Pooling layer 6: Type is maximum pooling; pooling size: 2×2; Stride: 2; Flatten layer: flattens the output of the last convolutional layer; Fully connected layer: 1024 neurons; Activation layer 11: The activation function is LeakyReLU; Output layer: type is Softmax; Number of neurons:

4.

7. The method for evaluating factors affecting the internal void development state of a road based on digital twin according to claim 6 is characterized in that: The specific network structure of the second enhanced model D in step S2.4.1 is the following structures connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Convolutional layer 2: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 5: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 6: The activation function is LeakyReLU; Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 7: kernel size 3×3; number of kernels 512; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 8: The activation function is LeakyReLU; Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolutional layer 9: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 9: The activation function is LeakyReLU; Convolution layer 10: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization; Activation layer 10: The activation function is LeakyReLU; Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2; Convolution layer 11: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization; Activation layer 11: The activation function is LeakyReLU; Convolutional layer 12: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization; Activation layer 12: The activation function is LeakyReLU; Pooling layer 6: Type is maximum pooling; pooling size: 2×2; Stride: 2; Flatten layer: flattens the output of the last convolutional layer; Fully connected layer: 1024 neurons; Activation layer 13: The activation function is LeakyReLU; Output layer: type is Softmax; Number of neurons:

4.

8. The method for evaluating factors affecting the internal void development state of a road based on digital twin according to claim 7 is characterized in that: The specific implementation method of step S3 includes the following steps: S3.

1. Based on and , establish the loss function between the auxiliary model B and the output layer of the basic model A, and the calculation formula is: ； based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is: ； based on and , establish the relationship loss function between the auxiliary model B and the output layer of the basic model A , the calculation formula is: ； in, for The weight coefficient of for The weight coefficient of S3.

2. Establish the relationship loss function between the output layer of the first reinforcement model C and the auxiliary model B , the expression is: ； in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of The calculation formula is: ； The calculation formula is: ； S3.

3. Establish the relationship loss function between the output layer of the first reinforcement model C and the basic model A , the expression is: ； in, for The weight coefficient of Based on and The loss function of The calculation formula is: ； S3.

4. Establish the relationship loss function between the output layer of the second reinforcement model D and the basic model A , the expression is: ； in, for The weight coefficient of Based on and The loss function of The calculation formula is: ； S3.

5. Establish the relationship loss function between the output layer of the second reinforcement model D and the first reinforcement model C , the expression is: ； in, for The weight coefficient of Based on and The loss function is for The weight coefficient of Based on and The loss function of The calculation formula is: ； The calculation formula is: ； S3.

6. Establishing the overall loss function , the calculation formula is: ； S3.

7. Using the road internal gap recognition data set obtained in step S1, based on the chain rule and through the error back propagation method, the basic model, auxiliary model and enhanced model for rutting recognition are trained. When the overall loss function is When it is minimum, the corresponding model parameters are optimal, and the trained basic model A is the final road internal void intelligent recognition model.

9. The method for evaluating factors affecting the internal void development state of a road based on digital twin according to claim 8 is characterized in that: The specific implementation method of step S4 includes the following steps: S4.

1. Export the road internal void intelligent identification model obtained in step S3 as a deployment file for independent services, including one of a pt file, a pth file, a json file, and a txt file; S4.

2. Constructing a road information model; S4.2.

1. Collect information on the location, length, width, and layout of roads; S4.2.

2. The data obtained in step S4.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S4.2.

3. Based on step S4.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S4.2.

4. Export the road information model to a format file supported by the web application, including a json file or a js file; S4.

3. In the digital twin platform, import the deployment file of the middle road internal void intelligent identification model obtained in step S4.1, and import the road information model format file obtained in step S4.2; S4.

4. Use Python to construct an API interface between image data and the intelligent recognition model for road internal voids for intelligent identification of ruts; use Python to construct an API interface between the intelligent recognition model for road internal voids and the road information model to complete the construction of the digital twin model.

10. The method for evaluating factors affecting the internal void development state of a road based on digital twin according to claim 9 is characterized in that: The specific implementation method of step S5 includes the following steps: S5.

1. Use a three-dimensional ground penetrating radar to collect road interior image data and upload it to the digital twin platform obtained in step S4. Then use the road interior void intelligent recognition model to intelligently identify the void area in the road surface image, and select all images containing the road interior void; S5.

2. For the image of the hollow inside the road identified in step S5.1, drill holes in the hollow area on site and measure the volume of the hollow area by water injection. VV , that is, the volume of the injected water is the volume of the depleted area; S5.

3. For the same area, select two different time nodes t 1. t 2. The volumes of the empty areas obtained by the test are VV 1. VV 2. Calculate the growth rate of the empty area : ； By analogy, we can get the development speed of the empty area corresponding to the time nodes t2~t11 ; S5.

4. The factors affecting the generation and development of internal voids in roads include traffic flow, ambient temperature, precipitation, elastic modulus of pavement materials, pavement thickness, service life, number of repairs, service time of pipe network, pavement temperature, and number of surrounding engineering activities; Select 10 time nodes t1, t2, t3, ... t10, and test the values ​​of the influencing factors of the generation and development of internal voids in the corresponding 10 time nodes: Traffic flow JL: The traffic flow at 10 time nodes corresponds to JL1, JL2, JL3, JL4, JL5, JL6, JL7, JL8, JL9, and JL10; Ambient temperature HW: The ambient temperatures at 10 time nodes correspond to HW1, HW2, HW3, HW4, HW5, HW6, HW7, HW8, HW9, and HW10; Precipitation JS: The precipitation at 10 time nodes corresponds to JS1, JS2, JS3, JS4, JS5, JS6, JS7, JS8, JS9, and JS10; Pavement material elastic modulus TM: The pavement material elastic modulus at 10 time nodes corresponds to TM1, TM2, TM3, TM4, TM5, TM6, TM7, TM8, TM9, and TM10; Pavement thickness HD: The pavement thickness at 10 time nodes corresponds to HD1, HD2, HD3, HD4, HD5, HD6, HD7, HD8, HD9, and HD10; Service life FY: The service life of the 10 time nodes corresponds to FY1, FY2, FY3, FY4, FY5, FY6, FY7, FY8, FY9, FY10; Maintenance times WX: The maintenance times of the 10 time nodes correspond to WX1, WX2, WX3, WX4, WX5, WX6, WX7, WX8, WX9, and WX10; Pipeline network service time GF: The pipeline network service time of 10 time nodes corresponds to GF1, GF2, GF3, GF4, GF5, GF6, GF7, GF8, GF9, and GF10; Pavement temperature LW: The pavement temperatures at 10 time nodes correspond to LW1, LW2, LW3, LW4, LW5, LW6, LW7, LW8, LW9, and LW10; Number of surrounding engineering activities ZD: The number of surrounding engineering activities at 10 time nodes corresponds to ZD1, ZD2, ZD3, ZD4, ZD5, ZD6, ZD7, ZD8, ZD9, and ZD10; The volume of the empty area: The volume of the empty area at 10 time nodes corresponds to VV 1. VV 2. VV 3. VV 4. VV 5. VV 6. VV 7. VV 8. VV 9. VV 10, the data are obtained by the middle method measured in step S5.2; The development speed of the empty area: The development speed of the empty area at 10 time nodes corresponds to , the data are measured using the method in step S5.3; S5.

5. Establish the relationship equation between the volume of the void area and traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and number of surrounding engineering activities at 10 time nodes. The calculation formula is as follows: ； in, m The value range is 1~10. These are the weight coefficients of the impact of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and number of surrounding engineering activities on the volume of the void area; Solving the above equations together, we get The value of , and arranged in order from large to small, the corresponding order of influencing factors is the degree of influence on the volume of the void area; S5.

6. Establish the relationship equation between the volume growth rate of the void area and traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and the number of surrounding engineering activities at 10 time nodes. The calculation formula is as follows: ； in, m The value range is 1~10. These are the weight coefficients of the impact of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of the pipeline network, pavement temperature, and number of surrounding engineering activities on the development speed of the void area; Solving the above equations together, we get The value of , and arranged in order from large to small, the corresponding ranking of influencing factors is the degree of influence on the development speed of the vacant area; S5.

7. Based on the above steps, analyze the comprehensive impact of all influencing factors on the volume of the void area and the growth rate of the void area; Set the comprehensive impact factor of traffic flow on the volume of the void area and the growth speed of the void area ZY The calculation formula for 1 is: ； in, is the correction factor of the weight coefficient of the impact of traffic flow on the volume of the empty area, bb 1 is the correction factor of the weight coefficient of the impact of traffic flow on the development speed of the empty area; Then the comprehensive influencing factors of ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, number of surrounding engineering activities, volume of depleted area and development speed of depleted area are calculated. ZY 2. ZY 3. ZY 4. ZY 5. ZY 6. ZY 7. ZY 8. ZY 9. ZY 10; Compare ZY 1. ZY 2. ZY 3. ZY 4. ZY 5. ZY 6. ZY 7. ZY 8. ZY 9. ZY 10, and arranged in order from large to small, which corresponds to the degree of comprehensive influence of various factors on the internal voids of the road.

Images