Road internal void development state early warning method, electronic device and storage medium

Abstract

The invention relates to a method for early warning of the development state of road internal voids, an electronic device and a storage medium, and belongs to the technical field of road internal monitoring. In order to solve the problem of timely and accurate early warning of road internal voids, the invention includes using a three-dimensional ground penetrating radar to collect road internal images, annotating the collected images, obtaining a data set for road internal void identification, constructing a road internal void identification model, constructing a loss function between a basic model, an auxiliary model, a first reinforcement model and a second reinforcement model for road internal void identification, and constructing an overall loss function; using the data set for road internal void identification to train the basic model, the auxiliary model, the first reinforcement model and the second reinforcement model for road internal void identification, and obtaining a trained basic model for road internal void identification as a road internal void intelligent identification model; constructing a digital twin model based on the road internal void intelligent identification model; and establishing a method for early warning of the development state of road internal voids.

Description

Technical Field

[0001] The present invention belongs to the technical field of road interior monitoring, and in particular relates to a method for early warning of a road interior hollowing development state, an electronic device and a storage medium. Background Art

[0002] Internal voids in roads refer to voids formed inside the road structure, especially between the pavement and the base or between the base and the subbase. This phenomenon is usually caused by a variety of factors, including improper construction, material degradation, excessive traffic loads, water intrusion, etc. Voids can weaken the overall strength and stability of the pavement structure because the presence of voids weakens the bond between pavement materials, making the pavement more vulnerable to damage from external forces.

[0003] The road surface in the debonding area is prone to settlement, resulting in uneven road surface, which affects the stability and safety of vehicle driving. In addition, debonding may cause serious damage to the road surface, such as cracks and potholes. These damages not only increase the risk of vehicle driving, but may also increase vehicle wear and tear, thereby increasing operating costs. In the rainy season, water discharge problems in the debonding area may also intensify, further deteriorating road conditions.

[0004] Early warning methods for the development of road voids can help road maintenance teams identify and respond to potential void areas before the void problem becomes serious road damage. Through early identification and intervention, maintenance costs can be significantly reduced, large-scale repair work can be avoided, and road driving safety can be ensured. Summary of the invention

[0005] The problem to be solved by the present invention is to give timely and accurate warning of the internal void of a road, and proposes a method for early warning of the development state of the internal void of a road, an electronic device and a storage medium.

[0006] To achieve the above object, the present invention is implemented through the following technical solutions:

[0007] A method for early warning of the internal hollowing development state of a road comprises the following steps:

[0008] S1. Using 3D ground penetrating radar to collect road interior images, and then annotating the collected images to obtain a dataset for identifying road interior voids;

[0009] S2. Construct a road internal void identification model, including a basic model, an auxiliary model, a first enhanced model, and a second enhanced model;

[0010] S3. constructing a loss function among the basic model, the auxiliary model, the first reinforcement model, and the second reinforcement model for identifying internal gaps in the road, and then constructing an overall loss function;

[0011] S4. Using the data set obtained in step S1 for road internal empty recognition, the basic model, auxiliary model, first enhanced model and second enhanced model for road internal empty recognition are trained to obtain the basic model for road internal empty recognition as the road internal empty intelligent recognition model;

[0012] S5. Construct a digital twin model based on the intelligent recognition model of road internal voids;

[0013] S6. Based on the digital twin model obtained in step S5, a method for early warning of the internal void development status of the road is established.

[0014] Furthermore, the specific implementation method of step S1 includes the following steps:

[0015] S1.1. Use three-dimensional ground penetrating radar to collect images of the road interior;

[0016] S1.2. Manually select all images containing internal voids, cracks, looseness, and water-rich diseases from the collected images;

[0017] S1.3. By manually labeling, the hollow area and the other three diseases are selected in the image selected in step S1.2, and the hollow area is marked as Hole. The other three diseases are marked as other1, other2, and other3 respectively, to construct a dataset for identifying hollow areas inside roads;

[0018] S1.4. The data set for road interior void identification obtained in step S1.3 is divided into a training set and a test set in a ratio of 8:2.

[0019] Furthermore, the specific implementation method of step S2 also includes the following steps:

[0020] S2.1. Constructing the basic model A:

[0021] S2.1.1. The basic model A consists of 1 input layer, 4 convolutional layers, 5 activation layers, 3 pooling layers, 1 flatten layer, 1 fully connected layer, and 1 output layer;

[0022] S2.1.2. Input the training set obtained in step S1 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A;

[0023] The calculation formula of the loss function of the basic model A is:

[0024] ;

[0025] in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the The true labels of samples, Based on the prediction of model A The probability that a sample is class 1;

[0026] S2.1.3. The node output values ​​of the four output units corresponding to the basic model A are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is:

[0027] ;

[0028] Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is:

[0029] ;

[0030] in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ;

[0031] S2.1.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ;

[0032] S2.2. Construct auxiliary model B:

[0033] S2.2.1. Auxiliary model B consists of 1 input layer, 8 convolutional layers, 9 activation layers, 5 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer;

[0034] S2.2.2. Input the training set obtained in step S1 into the auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the auxiliary model B, and obtain the trained auxiliary model B;

[0035] The calculation formula of the loss function of auxiliary model B is:

[0036] ;

[0037] in, is the loss function of auxiliary model B; is the total number of data samples in auxiliary model B; For the The true labels of samples, For auxiliary model B, predict The probability that a sample is class 1;

[0038] S2.2.3. The node output values ​​of the four output units corresponding to auxiliary model B are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is:

[0039] ;

[0040] Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is:

[0041] ;

[0042] S2.2.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the auxiliary model B. After the training is completed, the disease output value corresponding to the auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , =1,2,3,4;

[0043] S2.3. Construct the first reinforcement model C:

[0044] S2.3.1. The first enhanced model C consists of 1 input layer, 10 convolutional layers, 11 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer;

[0045] S2.3.2. Input the training set obtained in step S1 into the first reinforcement model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the first reinforcement model C, and obtain the trained reinforcement model C;

[0046] The calculation formula of the loss function of the first reinforcement model C is:

[0047] ;

[0048] in, is the loss function of the first reinforcement model C; is the total number of data samples in the first reinforcement model C; For the The true labels of samples, The first reinforcement model C predicts The probability that a sample is class 1;

[0049] S2.3.3. The node output values ​​of the four output units corresponding to the first reinforcement model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is:

[0050] ;

[0051] Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is:

[0052] ;

[0053] in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ;

[0054] S2.3.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the first enhanced model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the first enhanced model C are obtained as follows: ;

[0055] S2.4. Construct the second reinforcement model D:

[0056] S2.4.1. The second enhanced model D consists of 1 input layer, 12 convolutional layers, 13 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer;

[0057] S2.4.2. Input the training set obtained in step S1 into the second reinforcement model D, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the second reinforcement model D, and obtain the trained reinforcement model C;

[0058] The calculation formula of the loss function of the second reinforcement model D is:

[0059] ;

[0060] in, is the loss function of the second reinforcement model D; is the total number of data samples in the second enhanced model D; For the The true labels of samples, The second reinforcement model D predicts The probability that a sample is class 1;

[0061] S2.4.3. The node output values ​​of the four output units corresponding to the second enhanced model D are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is:

[0062] ;

[0063] Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is:

[0064] ;

[0065] in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ;

[0066] S2.4.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the second enhanced model D. After the training is completed, the output values ​​of the four types of diseases corresponding to the second enhanced model D are obtained as follows: .

[0067] Furthermore, the specific implementation method of step S3 also includes the following steps:

[0068] S3.1. Based on and , establish the loss function between the auxiliary model B and the output layer of the basic model A, and the calculation formula is:

[0069] ;

[0070] based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is:

[0071] ;

[0072] based on and , establish the relationship loss function between the auxiliary model B and the output layer of the basic model A , the calculation formula is:

[0073] ;

[0074] in, for The weight coefficient for The weight coefficient of

[0075] S3.2. Establish the relationship loss function between the output layer of the first reinforcement model C and the auxiliary model B , the expression is:

[0076] ;

[0077] in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of

[0078] The calculation formula is:

[0079] ;

[0080] The calculation formula is:

[0081] ;

[0082] S3.3. Establish the relationship loss function between the output layer of the first reinforcement model C and the basic model A , the expression is:

[0083] in,

[0084] ;

[0085] for The weight coefficient of Based on and The loss function of

[0086] The calculation formula is:

[0087] ;

[0088] S3.4. Establish the relationship loss function between the output layer of the second reinforcement model D and the basic model A , the expression is:

[0089] ;

[0090] in, for The weight coefficient of Based on and The loss function of

[0091] The calculation formula is:

[0092] ;

[0093] S3.5. Establish the relationship loss function between the output layer of the second reinforcement model D and the first reinforcement model C , the expression is:

[0094] ;

[0095] in, for The weight coefficient of Based on and The loss function is for The weight coefficient of Based on and The loss function of

[0096] The calculation formula is:

[0097] ;

[0098] The calculation formula is:

[0099] ;

[0100] S3.6. Establishing the overall loss function , the calculation formula is:

[0101] .

[0102] Furthermore, the specific implementation method of step S4 also includes the following steps: The specific implementation method of step S4 is to use the data set of road internal gap recognition obtained in step S1, based on the chain derivative rule, through the error back propagation method, to train the basic model, auxiliary model and enhanced model for rutting recognition. When the overall loss function is obtained When it is minimum, the corresponding model parameters are optimal, and the trained basic model A is the final road internal void intelligent recognition model.

[0103] Furthermore, the specific implementation method of step S5 also includes the following steps:

[0104] S5.1. Export the road internal void intelligent identification model obtained in step S4 as a deployment file for independent service, including one of a pt file, a pth file, a json file, and a txt file;

[0105] S5.2. Construct a road information model;

[0106] S5.2.1. Collect information on the location, length, width, and layout of roads;

[0107] S5.2.2. The data obtained in step S5.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road;

[0108] S5.2.3. Based on step S5.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model;

[0109] S5.2.4. Export the road information model to a format file supported by the web application, including a json file or a js file;

[0110] S5.3. In the digital twin platform, import the deployment file of the middle road internal void intelligent identification model obtained in step S5.1, and import the road information model format file obtained in step S5.2;

[0111] S5.4. Use Python to construct an API interface between image data and the intelligent recognition model for road internal voids for intelligent identification of ruts; use Python to construct an API interface between the intelligent recognition model for road internal voids and the road information model to complete the construction of the digital twin model.

[0112] Furthermore, the specific implementation method of step S6 also includes the following steps:

[0113] S6.1. Use a three-dimensional ground penetrating radar to collect road interior image data and upload it to the digital twin platform obtained in step S4. Then use the road interior void intelligent recognition model to intelligently identify the void area in the road surface image, and select all images containing the road interior void;

[0114] S6.2. According to the differences in the factors affecting the road service environment, the road internal air gap warning status is divided into four standards;

[0115] S6.2.1. Setting standard 1: When the values ​​of the factors of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, and number of surrounding engineering activities are JLB1, HWB1, JSB1, TMB1, HDB1, FYB1, WXB1, GFB1, LWB1, and ZDB1 respectively, the corresponding volume critical value of the void area is set to VVB1;

[0116] S6.2.2. Setting standard 2: When the values ​​of the factors of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, and number of surrounding engineering activities are JLB2, HWB2, JSB2, TMB2, HDB2, FYB2, WXB2, GFB2, LWB2, and ZDB2 respectively, the corresponding volume critical value of the void area is set to VVB2;

[0117] S6.2.3. Setting standard 3: When the values ​​of traffic flow, ambient temperature, precipitation, pavement material elastic modulus, pavement thickness, service life, number of repairs, pipe network service time, pavement temperature, and number of surrounding engineering activities are JLB3, HWB3, JSB3, TMB3, HDB3, FYB3, WXB3, GFB3, LWB3, and ZDB3 respectively, the corresponding volume critical value of the void area is set to VVB3;

[0118] S6.2.4. Setting standard 4: When the values ​​of the factors of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, and number of surrounding engineering activities are JLB4, HWB4, JSB4, TMB4, HDB4, FYB4, WXB4, GFB4, LWB4, and ZDB4 respectively, the corresponding volume critical value of the void area is set to VVB4;

[0119] S6.3. Select the target road and collect data such as the corresponding traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipe network, pavement temperature, and number of surrounding engineering activities during the operation of the target road, and record them in sequence as: JL, HW, JS, TM, HD, FY, WX, GF, LW, ZD. The volume of the void area measured on-site is recorded as: VV;

[0120] S6.4. Calculate the correlation between the state of the target road selected in step S6.3 and the four criteria set in step S6.2, and determine the critical value corresponding to the warning criterion applicable to the target road;

[0121] Calculate the correlation value GL between the state of the target road and the k-th criterion k , and the calculation formula is

[0122] ;

[0123] Compare the calculated GL1, GL2, GL3, and GL4, and obtain the minimum value among them, denoted as GLmin; then record the critical value of the void area volume corresponding to GLmin as VVBmin;

[0124] S6.5. Optimize the void area volume value to obtain the optimized void area volume value VVn, and the calculation formula is:

[0125] ;

[0126] Among them, is the volume adjustment factor;

[0127] Compare the calculated optimized void area volume value VVn with the critical value VVBmin of the void area volume corresponding to GLmin. When VVn ≥ VVBmin, it indicates that the void area volume has reached or exceeded the allowable range, and an alarm needs to be issued to repair the void area;

[0128] When VVn < VVBmin, it indicates that the void area volume has not reached the critical value, and no alarm is required, but attention needs to be paid during the maintenance process.

[0129] An electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method for warning the development state of voids inside a road are implemented.

[0130] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for warning the development state of voids inside a road is implemented.

[0131] Beneficial effects of the present invention:

[0132] The present invention discloses a method for early warning of the development status of road internal voids. In the road internal void recognition model, the convolution layer captures void features and reflection patterns at different depths, the activation layer introduces nonlinearity to distinguish subtle void signals from noise, the pooling layer improves the robustness of feature detection and reduces noise interference, the Flatten layer converts spatial features into one-dimensional vectors, the fully connected layer integrates features to perform classification decisions, and the final output layer makes a probability prediction of the disease status based on the learned features.

[0133] The present invention discloses a method for early warning of the development status of road voids and a multi-model fusion method for intelligent identification of road voids, which can reduce the complexity of the model and improve the identification speed while ensuring the accuracy of identifying road voids.

[0134] The method for early warning of road internal void development status described in the present invention, the integration of digital twin technology and the method for rapid identification of road internal voids, can utilize high-precision digital models combined with real-time data to achieve instant and accurate diagnosis of road conditions. By integrating sensor data, image recognition technology and machine learning models, it is not only possible to detect and locate road internal voids in a timely manner, but also to predict future damage development, thereby optimizing maintenance plans and resource allocation. This helps to improve road safety, reduce maintenance costs, and extend the service life of roads, which is of great value in improving public safety and economic efficiency.

[0135] The method for early warning of the development state of internal voids in a road disclosed in the present invention can timely identify potential void areas and take maintenance or reinforcement measures in advance, thereby avoiding or reducing road damage. It can also help road maintenance teams optimize maintenance plans and resource allocation, improve road safety and reliability, and reduce the time and cost required for emergency repairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0136] Figure 1 The present invention is a flow chart of a method for early warning of the internal void development state of a road. DETAILED DESCRIPTION

[0137] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the specific embodiments described are only part of the embodiments of the present invention, rather than all of the specific embodiments. The components of the specific embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations, and the present invention can also have other embodiments.

[0138] Therefore, the following detailed description of the specific embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected specific embodiments of the present invention. Based on the specific embodiments of the present invention, all other specific embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0139] In order to further understand the content, features and effects of the present invention, the following specific implementation methods are given as examples, and the attached Figure 1 The detailed instructions are as follows:

[0140] Embodiment 1:

[0141] A method for early warning of the internal hollowing development state of a road comprises the following steps:

[0142] S1. Using 3D ground penetrating radar to collect road interior images, and then annotating the collected images to obtain a dataset for identifying road interior voids;

[0143] Furthermore, the specific implementation method of step S1 includes the following steps:

[0144] S1.1. Use three-dimensional ground penetrating radar to collect images of the road interior;

[0145] S1.2. Manually select all images containing internal voids, cracks, looseness, and water-rich diseases from the collected images;

[0146] S1.3. By manually labeling, the hollow area and the other three diseases are selected in the image selected in step S1.2, and the hollow area is marked as Hole. The other three diseases are marked as other1, other2, and other3 respectively, to construct a dataset for identifying hollow areas inside roads;

[0147] S1.4. The data set for road interior void identification obtained in step S1.3 is divided into a training set and a test set in a ratio of 8:2.

[0148] S2. Construct a road internal void identification model, including a basic model, an auxiliary model, a first enhanced model, and a second enhanced model;

[0149] S2.1. Constructing the basic model A:

[0150] S2.1.1. The basic model A consists of 1 input layer, 4 convolutional layers, 5 activation layers, 3 pooling layers, 1 flatten layer, 1 fully connected layer, and 1 output layer. The specific network structure is the following structure connected in sequence:

[0151] Input layer: image size 227×227×3;

[0152] Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 32; stride: 1; batch normalization;

[0153] Activation layer 1: The activation function is LeakyReLU;

[0154] Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0155] Convolutional layer 2: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization;

[0156] Activation layer 2: The activation function is LeakyReLU;

[0157] Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0158] Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization;

[0159] Activation layer 3: The activation function is LeakyReLU;

[0160] Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization;

[0161] Activation layer 4: The activation function is LeakyReLU;

[0162] Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0163] Flatten layer: flattens the output of the last convolutional layer;

[0164] Fully connected layer: 1024 neurons;

[0165] Activation layer 5: The activation function is LeakyReLU;

[0166] Output layer: type is normalized exponential function Softmax; number of neurons: 4;

[0167] S2.1.2. Input the training set obtained in step S1 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A;

[0168] The calculation formula of the loss function of the basic model A is:

[0169] ;

[0170] in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the The true labels of samples, Based on the prediction of model A The probability that a sample is class 1;

[0171] S2.1.3. The node output values ​​of the four output units corresponding to the basic model A are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is:

[0172] ;

[0173] Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is:

[0174] ;

[0175] in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as Specifically, for the internal void of the road 1, for other diseases is 0;

[0176] S2.1.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ;

[0177] S2.2. Construct auxiliary model B:

[0178] S2.2.1. Auxiliary model B consists of 1 input layer, 8 convolutional layers, 9 activation layers, 5 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer. The specific network structure is the following structure connected in sequence:

[0179] Input layer: image size 227×227×3;

[0180] Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; batch normalization;

[0181] Activation layer 1: The activation function is LeakyReLU;

[0182] Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0183] Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization;

[0184] Activation layer 2: The activation function is LeakyReLU;

[0185] Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0186] Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization;

[0187] Activation layer 3: The activation function is LeakyReLU;

[0188] Convolutional layer 4: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization;

[0189] Activation layer 4: The activation function is LeakyReLU;

[0190] Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0191] Convolution layer 5: kernel size 3×3; number of kernels 512; stride: 1; batch normalization;

[0192] Activation layer 5: The activation function is LeakyReLU;

[0193] Convolutional layer 6: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization;

[0194] Activation layer 6: The activation function is LeakyReLU;

[0195] Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0196] Convolutional layer 7: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization;

[0197] Activation layer 7: The activation function is LeakyReLU;

[0198] Convolutional layer 8: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization;

[0199] Activation layer 8: The activation function is LeakyReLU;

[0200] Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0201] Flatten layer: flattens the output of the last convolutional layer;

[0202] Fully connected layer: 1024 neurons;

[0203] Activation layer 9: The activation function is LeakyReLU;

[0204] Output layer: type is normalized exponential function Softmax; number of neurons: 4;

[0205] S2.2.2. Input the training set obtained in step S1 into the auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the auxiliary model B, and obtain the trained auxiliary model B;

[0206] The calculation formula of the loss function of auxiliary model B is:

[0207] ;

[0208] in, is the loss function of auxiliary model B; is the total number of data samples in auxiliary model B; For the The true labels of samples, For auxiliary model B, predict The probability that a sample is class 1;

[0209] S2.2.3. The node output values ​​of the four output units corresponding to auxiliary model B are , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is:

[0210] ;

[0211] Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is:

[0212] ;

[0213] S2.2.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the auxiliary model B. After the training is completed, the disease output value corresponding to the auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , =1,2,3,4, specifically, for the internal void of the road 1, for other diseases is 0;

[0214] S2.3. Construct the first reinforcement model C:

[0215] S2.3.1. The first enhanced model C consists of 1 input layer, 10 convolutional layers, 11 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer. The specific network structure is the following structure connected in sequence:

[0216] Input layer: image size 227×227×3;

[0217] Convolutional layer 1: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization;

[0218] Activation layer 1: The activation function is LeakyReLU;

[0219] Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0220] Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization;

[0221] Activation layer 2: The activation function is LeakyReLU;

[0222] Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0223] Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization;

[0224] Activation layer 3: The activation function is LeakyReLU;

[0225] Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization;

[0226] Activation layer 4: The activation function is LeakyReLU;

[0227] Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0228] Convolution layer 5: kernel size 3×3; number of kernels 512; stride: 1; batch normalization;

[0229] Activation layer 5: The activation function is LeakyReLU;

[0230] Convolutional layer 6: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization;

[0231] Activation layer 6: The activation function is LeakyReLU;

[0232] Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0233] Convolutional layer 7: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization;

[0234] Activation layer 7: The activation function is LeakyReLU;

[0235] Convolutional layer 8: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization;

[0236] Activation layer 8: The activation function is LeakyReLU;

[0237] Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0238] Convolutional layer 9: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization;

[0239] Activation layer 9: The activation function is LeakyReLU;

[0240] Convolution layer 10: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization;

[0241] Activation layer 10: The activation function is LeakyReLU;

[0242] Pooling layer 6: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0243] Flatten layer: flattens the output of the last convolutional layer;

[0244] Fully connected layer: 1024 neurons;

[0245] Activation layer 11: The activation function is LeakyReLU;

[0246] Output layer: type is Softmax; number of neurons: 4;

[0247] S2.3.2. Input the training set obtained in step S1 into the first reinforcement model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the first reinforcement model C, and obtain the trained reinforcement model C;

[0248] The calculation formula of the loss function of the first reinforcement model C is:

[0249] ;

[0250] in, is the loss function of the first reinforcement model C; is the total number of data samples in the first reinforcement model C; For the The true labels of samples, The first reinforcement model C predicts The probability that a sample is class 1;

[0251] S2.3.3. The node output values ​​of the four output units corresponding to the first reinforcement model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is:

[0252] ;

[0253] Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is:

[0254] ;

[0255] in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as Specifically, for the internal void of the road 1, for other diseases is 0;

[0256] S2.3.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the first enhanced model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the first enhanced model C are obtained as follows: ;

[0257] S2.4. Construct the second reinforcement model D:

[0258] S2.4.1. The second enhanced model D consists of 1 input layer, 12 convolutional layers, 13 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer. The specific network structure is the following structure connected in sequence:

[0259] Input layer: image size 227×227×3;

[0260] Convolutional layer 1: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization;

[0261] Activation layer 1: The activation function is LeakyReLU;

[0262] Convolutional layer 2: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization;

[0263] Activation layer 2: The activation function is LeakyReLU;

[0264] Pooling layer 1: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0265] Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization;

[0266] Activation layer 3: The activation function is LeakyReLU;

[0267] Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization;

[0268] Activation layer 4: The activation function is LeakyReLU;

[0269] Pooling layer 2: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0270] Convolution layer 5: kernel size 3×3; number of kernels 256; stride: 1; batch normalization;

[0271] Activation layer 5: The activation function is LeakyReLU;

[0272] Convolutional layer 6: kernel size 3×3; number of kernels 256; stride: 1; batch normalization;

[0273] Activation layer 6: The activation function is LeakyReLU;

[0274] Pooling layer 3: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0275] Convolutional layer 7: kernel size 3×3; number of kernels 512; stride: 1; batch normalization;

[0276] Activation layer 7: The activation function is LeakyReLU;

[0277] Convolutional layer 8: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization;

[0278] Activation layer 8: The activation function is LeakyReLU;

[0279] Pooling layer 4: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0280] Convolutional layer 9: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization;

[0281] Activation layer 9: The activation function is LeakyReLU;

[0282] Convolution layer 10: kernel size 3×3; number of kernels 1024; stride: 1; batch normalization;

[0283] Activation layer 10: The activation function is LeakyReLU;

[0284] Pooling layer 5: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0285] Convolution layer 11: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization;

[0286] Activation layer 11: The activation function is LeakyReLU;

[0287] Convolutional layer 12: kernel size 3×3; number of kernels 2048; stride: 1; batch normalization;

[0288] Activation layer 12: The activation function is LeakyReLU;

[0289] Pooling layer 6: Type is maximum pooling; pooling size: 2×2; Stride: 2;

[0290] Flatten layer: flattens the output of the last convolutional layer;

[0291] Fully connected layer: 1024 neurons;

[0292] Activation layer 13: The activation function is LeakyReLU;

[0293] Output layer: type is Softmax; number of neurons: 4;

[0294] S2.4.2. Input the training set obtained in step S1 into the second reinforcement model D, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the second reinforcement model D, and obtain the trained reinforcement model C;

[0295] The calculation formula of the loss function of the second reinforcement model D is:

[0296] ;

[0297] in, is the loss function of the second reinforcement model D; is the total number of data samples in the second enhanced model D; For the The true labels of samples, The second reinforcement model D predicts The probability that a sample is class 1;

[0298] S2.4.3. The node output values ​​of the four output units corresponding to the second enhanced model D are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is:

[0299] ;

[0300] Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is:

[0301] ;

[0302] in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as Specifically, for the internal void of the road 1, for other diseases is 0;

[0303] S2.4.4. Adoption The training set obtained in step S1 is combined with the error chain derivation rule to train the second enhanced model D. After the training is completed, the output values ​​of the four types of diseases corresponding to the second enhanced model D are obtained as follows: .

[0304] S3. constructing a loss function among the basic model, the auxiliary model, the first reinforcement model, and the second reinforcement model for identifying internal gaps in the road, and then constructing an overall loss function;

[0305] Furthermore, the specific implementation method of step S3 also includes the following steps:

[0306] S3.1. Based on and , establish the loss function between the auxiliary model B and the output layer of the basic model A, and the calculation formula is:

[0307] ;

[0308] based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is:

[0309] ;

[0310] based on and , establish the relationship loss function between the auxiliary model B and the output layer of the basic model A , the calculation formula is:

[0311] ;

[0312] in, for The weight coefficient of for The weight coefficient of

[0313] S3.2. Establish the relationship loss function between the output layer of the first reinforcement model C and the auxiliary model B , the expression is:

[0314] ;

[0315] in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of

[0316] The calculation formula is:

[0317] ;

[0318] The calculation formula is:

[0319] ;

[0320] S3.3. Establish the relationship loss function between the output layer of the first reinforcement model C and the basic model A , the expression is:

[0321] ;

[0322] in, for The weight coefficient of Based on and The loss function of

[0323] The calculation formula is:

[0324] ;

[0325] S3.4. Establish the relationship loss function between the output layer of the second reinforcement model D and the basic model A , the expression is:

[0326] ;

[0327] in, for The weight coefficient of Based on and The loss function of

[0328] The calculation formula is:

[0329] ;

[0330] S3.5. Establish the relationship loss function between the output layer of the second reinforcement model D and the first reinforcement model C , the expression is:

[0331] ;

[0332] in, for The weight coefficient of Based on and The loss function is for The weight coefficient of

[0333] Based on and The loss function of

[0334] The calculation formula is:

[0335] ;

[0336] The calculation formula is:

[0337] ;

[0338] S3.6. Establishing the overall loss function , the calculation formula is:

[0339] .

[0340] S4. Using the data set obtained in step S1 for road internal empty recognition, the basic model, auxiliary model, first enhanced model and second enhanced model for road internal empty recognition are trained to obtain the basic model for road internal empty recognition as the road internal empty intelligent recognition model;

[0341] Furthermore, the specific implementation method of step S4 also includes the following steps: The specific implementation method of step S4 is to use the data set of road internal gap recognition obtained in step S1, based on the chain derivative rule, through the error back propagation method, to train the basic model, auxiliary model and enhanced model for rutting recognition. When the overall loss function is obtained When it is minimum, the corresponding model parameters are optimal, and the trained basic model A is the final road internal void intelligent recognition model.

[0342] S5. Construct a digital twin model based on the intelligent recognition model of road internal voids;

[0343] Furthermore, the specific implementation method of step S5 also includes the following steps:

[0344] S5.1. Export the road internal void intelligent identification model obtained in step S4 as a deployment file for independent service, including one of a pt file, a pth file, a json file, and a txt file;

[0345] S5.2. Construct a road information model;

[0346] S5.2.1. Collect information on the location, length, width, and layout of roads;

[0347] S5.2.2. The data obtained in step S5.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road;

[0348] S5.2.3. Based on step S5.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model;

[0349] S5.2.4. Export the road information model to a format file supported by the web application, including a json file or a js file;

[0350] S5.3. In the digital twin platform, import the deployment file of the middle road internal void intelligent identification model obtained in step S5.1, and import the road information model format file obtained in step S5.2;

[0351] S5.4. Use Python to construct an API interface between image data and the intelligent recognition model for road internal voids for intelligent identification of ruts; use Python to construct an API interface between the intelligent recognition model for road internal voids and the road information model to complete the construction of the digital twin model.

[0352] S6. Based on the digital twin model obtained in step S5, a method for early warning of the internal void development status of the road is established.

[0353] Under different environments, the corresponding traffic flow, ambient temperature, precipitation, pavement material elastic modulus, pavement thickness, service life, number of repairs, network service time, pavement temperature, and number of surrounding engineering activities during the road service process are different, and these factors will affect the evaluation results of the road status. For example, for the same size of void areas, the impact is relatively small in areas with relatively small traffic flow, while in areas with large traffic flow, the impact is large and may even reach the warning standard. Therefore, this embodiment proposes a method for early warning of the development status of voids inside roads.

[0354] Furthermore, the specific implementation method of step S6 also includes the following steps:

[0355] S6.1. Use a three-dimensional ground penetrating radar to collect road interior image data and upload it to the digital twin platform obtained in step S4. Then use the road interior void intelligent recognition model to intelligently identify the void area in the road surface image, and select all images containing the road interior void;

[0356] S6.2. According to the differences in the factors affecting the road service environment, the road internal air gap warning status is divided into four standards;

[0357] S6.2.1. Setting standard 1: When the values ​​of the factors of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, and number of surrounding engineering activities are JLB1, HWB1, JSB1, TMB1, HDB1, FYB1, WXB1, GFB1, LWB1, and ZDB1 respectively, the corresponding volume critical value of the void area is set to VVB1;

[0358] S6.2.2. Setting standard 2: When the values ​​of the factors of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, and number of surrounding engineering activities are JLB2, HWB2, JSB2, TMB2, HDB2, FYB2, WXB2, GFB2, LWB2, and ZDB2 respectively, the corresponding volume critical value of the void area is set to VVB2;

[0359] S6.2.3. Setting standard 3: When the values ​​of traffic flow, ambient temperature, precipitation, pavement material elastic modulus, pavement thickness, service life, number of repairs, pipe network service time, pavement temperature, and number of surrounding engineering activities are JLB3, HWB3, JSB3, TMB3, HDB3, FYB3, WXB3, GFB3, LWB3, and ZDB3 respectively, the corresponding volume critical value of the void area is set to VVB3;

[0360] S6.2.4. Setting standard 4: When the values ​​of the factors of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, and number of surrounding engineering activities are JLB4, HWB4, JSB4, TMB4, HDB4, FYB4, WXB4, GFB4, LWB4, and ZDB4 respectively, the corresponding volume critical value of the void area is set to VVB4;

[0361] S6.3. Select the target road and collect the corresponding data such as traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, number of surrounding engineering activities during the operation of the target road, and record them as: JL, HW, JS, TM, HD, FY, WX, GF, LW, ZD in sequence. The volume of the void area measured on site is recorded as: VV;

[0362] S6.4. Calculate the correlation between the target road condition selected in step S6.3 and the four criteria set in step S6.2, and determine the critical value corresponding to the warning criterion applicable to the target road;

[0363] Calculate the correlation value GL between the target road condition and the k-th criterion k , and the calculation formula is

[0364] ;

[0365] Compare the calculated GL1, GL2, GL3, and GL4, and obtain the minimum value among them, denoted as GLmin; then denote the critical value of the void area volume corresponding to GLmin as VVBmin;

[0366] S6.5. Optimize the void area volume value to obtain the optimized void area volume value VVn, and the calculation formula is:

[0367] ;

[0368] where is the volume adjustment factor;

[0369] Compare the calculated optimized void area volume value VVn with the critical value VVBmin of the void area volume corresponding to GLmin. When VVn ≥ VVBmin, it indicates that the void area volume has reached or exceeded the allowable range, and an alarm needs to be issued to repair the void area;

[0370] When VVn < VVBmin, it indicates that the void area volume has not reached the critical value, and no alarm is required. Attention needs to be paid during the maintenance process.

[0371] Embodiment 2:

[0372] An electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of a method for warning the development state of internal voids in a road described in Embodiment 1.

[0373] The computer device of the present invention may be a device including a processor and a memory, such as a single-chip microcomputer including a central processing unit. And, when the processor executes the computer program stored in the memory, it implements the steps of the above method for warning the development state of internal voids in a road.

[0374] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0375] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0376] Embodiment 3:

[0377] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a method for early warning of a road internal void development state as described in Example 1.

[0378] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by the processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. The computer-readable storage medium stores a computer program. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned method for warning the internal void development status of a road can be implemented.

[0379] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0380] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0381] Although the present application has been described above with reference to specific embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present application. In particular, as long as there is no structural conflict, the various features in the specific embodiments disclosed in the present application may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for early warning of the internal hollowing development state of a road, characterized in that: The steps include: S1. Using a three-dimensional ground penetrating radar, collect images of the interior of the road, and then annotate the collected images to obtain a data set for identifying gaps inside the road; S2. Construct a road internal void recognition model, including a basic model, an auxiliary model, a first enhanced model, and a second enhanced model; The specific implementation method of step S2 also includes the following steps: S2.

1. Constructing the basic model A: S2.1.

1. The basic model A consists of 1 input layer, 4 convolutional layers, 5 activation layers, 3 pooling layers, 1 flatten layer, 1 fully connected layer, and 1 output layer; S2.1.

2. Input the training set obtained in step S1 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: Among them, La is the loss function of the basic model A; Na is the total number of data samples in the basic model A; y ia is the true label of the iath sample, p ia Predict the probability that the iath sample is class 1 based on the basic model A; S2.1.

3. The node output values ​​of the four output units corresponding to the basic model A are a1, a2, a3, and a4. The node output values ​​of the four output units are initially converted to obtain the output value a i The corresponding probability value f ai The calculation formula is: Then the node output values ​​of the four output units are further optimized and converted to obtain the output value a i The corresponding enhancement probability value g ai The calculation formula is: Among them, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as h ai ; S2.1.

4. Use g ai The training set obtained in step S1 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as j a1 , j a2 , j a3 , j a4 ; S2.

2. Construct auxiliary model B: S2.2.

1. Auxiliary model B consists of 1 input layer, 8 convolutional layers, 9 activation layers, 5 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer; S2.2.

2. Input the training set obtained in step S1 into the auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the auxiliary model B, and obtain the trained auxiliary model B; The calculation formula of the loss function of auxiliary model B is: Where Lb is the loss function of auxiliary model B; Nb is the total number of data samples in auxiliary model B; y ib is the true label of the ib-th sample, p ib Predict the probability that the ibth sample is class 1 for auxiliary model B; S2.2.

3. The node output values ​​of the four output units corresponding to the auxiliary model B are b1, b2, b3, and b4. The node output values ​​of the four output units are initially converted to obtain the output value b i The corresponding probability value f bi The calculation formula is: Then the node output values ​​of the four output units are further optimized and converted to obtain the output value b i The corresponding enhancement probability value g bi The calculation formula is: S2.2.

4. Use g bi The training set obtained in step S1 is combined with the error chain derivation rule to train the auxiliary model B. After the training is completed, the disease output value corresponding to the auxiliary model B is j bi , among the 4 nodes in the output layer, the actual disease corresponding to each node is labeled as h bi , i=1,2,3,4; S2.

3. Construct the first reinforcement model C: S2.3.

1. The first enhanced model C consists of 1 input layer, 10 convolutional layers, 11 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer; S2.3.

2. Input the training set obtained in step S1 into the first reinforcement model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the first reinforcement model C, and obtain the trained reinforcement model C; The calculation formula of the loss function of the first reinforcement model C is: Where Lc is the loss function of the first enhanced model C; Nc is the total number of data samples in the first enhanced model C; y ic is the true label of the icth sample, p ic Predict the probability that the icth sample is class 1 for the first reinforcement model C; S2.3.

3. The node output values ​​of the four output units corresponding to the first reinforcement model C are c i , the node output values ​​of the four output units are initially converted to obtain c i The corresponding probability value f ci The calculation formula is: Then the node output values ​​of the four output units are further optimized and converted to obtain the output value c i The corresponding enhancement probability value g ci The calculation formula is: Among them, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as h ci ; S2.3.

4. Use g ci The training set obtained in step S1 is combined with the error chain derivation rule to train the first enhanced model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the first enhanced model C are j ci ; S2.

4. Construct the second reinforcement model D: S2.4.

1. The second enhanced model D consists of 1 input layer, 12 convolutional layers, 13 activation layers, 6 pooling layers, 1 Flatten layer, 1 fully connected layer, and 1 output layer; S2.4.

2. Input the training set obtained in step S1 into the second enhanced model D, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the second enhanced model D, and obtain the trained enhanced model C; The calculation formula of the loss function of the second reinforcement model D is: Where Ld is the loss function of the second enhanced model D; Nd is the total number of data samples in the second enhanced model D; y id is the true label of the idth sample, p id Predict the probability that the idth sample is class 1 for the second reinforcement model D; S2.4.

3. The node output values ​​of the four output units corresponding to the second reinforcement model D are d i , the node output values ​​of the four output units are initially converted to obtain d i The corresponding probability value f di The calculation formula is: Then the node output values ​​of the four output units are further optimized and converted to obtain the output value d i The corresponding enhanced probability value g di The calculation formula is: Among them, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as h di ; S2.4.

4. Use g di The training set obtained in step S1 is combined with the error chain derivation rule to train the second enhanced model D. After the training is completed, the output values ​​of the four types of diseases corresponding to the second enhanced model D are j di ; S3. construct the loss function between the basic model for road internal void identification, the auxiliary model, the first reinforcement model, and the second reinforcement model, and then construct the overall loss function; S4. Using the data set of road interior void identification obtained in step S1, the basic model, auxiliary model, first reinforcement model and second reinforcement model of road interior void identification are trained to obtain the trained basic model of road interior void identification as the road interior void intelligent identification model; S5. Construct a digital twin model based on the intelligent recognition model of road internal voids; S6. Based on the digital twin model obtained in step S5, a method for early warning of the internal void development status of the road is established.

2. A road internal void development state early warning method according to claim 1, characterized in that: The specific implementation method of step S1 includes the following steps: S1.

1. Use three-dimensional ground penetrating radar to collect images of the interior of the road; S1.

2. Manually select all images containing internal voids, cracks, looseness, and water-rich diseases from the collected images; S1.

3. Through the manual labeling method, the hollow area and the other three diseases are selected in the image selected in step S1.2, and the hollow area is marked as Hole, and the other three diseases are marked as other1, other2, and other3 respectively, to construct a dataset for identifying hollow areas inside roads; S1.

4. Divide the data set for road interior void identification obtained in step S1.3 into a training set and a test set in a ratio of 8:

2.

3. A road internal void development state early warning method according to claim 2, characterized in that: The specific implementation method of step S3 also includes the following steps: S3.

1. Based on j bi and g ai , establish the loss function between the auxiliary model B and the output layer of the basic model A, and the calculation formula is: Based on f ai and h ai , construct the output value a i The corresponding probability value f ai The loss function L of the base model A AA , the calculation formula is: Based on L BA and L AA , establish the relationship loss function L between the auxiliary model B and the output layer of the basic model A BAA , the calculation formula is: L BAA =c BA ·L BA +g AA ·L AA Among them, γ BA For L BA The weight coefficient, γ AA For L AA The weight coefficient of S3.

2. Establish the relationship loss function L between the output layer of the first reinforcement model C and the auxiliary model B CBB , the expression is: L CBB =c CB ·L CB +g BB ·L BB Among them, γ CB For L CB The weight coefficient, γ BB For L BB The weight coefficient, L CB Based on j ci and g bi The loss function of L BB Based on f bi and h bi The loss function of L CB The calculation formula is: L BB The calculation formula is: S3.

3. Establish the relationship loss function L between the first reinforcement model C and the output layer of the basic model A CAA , the expression is: L CAA =c CA ·L CA +g AA ·L AA Among them, γ CA For L CA The weight coefficient, L CA Based on j ci and g ai The loss function of L CA The calculation formula is: S3.

4. Establish the relationship loss function L between the output layer of the second reinforcement model D and the basic model A DAA , the expression is: L DAA =c DA ·L DA +g AA ·L AA Among them, γ DA For L DA The weight coefficient, L DA Based on j ci and g ai The loss function of L DA The calculation formula is: S3.

5. Establish the relationship loss function L between the output layer of the second reinforcement model D and the first reinforcement model C DCC , the expression is: L DCC =c DC ·L DC +g CC ·L CC Among them, γ DC For L DC The weight coefficient, L DC Based on j di and g ci The loss function, γ CC For L CC The weight coefficient, L CC Based on f ci and h ci The loss function of L DC The calculation formula is: L CC The calculation formula is: S3.

6. Establishing the overall loss function L total , the calculation formula is: L total =L DCC +L CBB +L BAA +L DAA +L CAA .

4. A road internal void development state early warning method according to claim 3, characterized in that: The specific implementation method of step S4 also includes the following steps: The specific implementation method of step S4 is to use the data set of road internal gap recognition obtained in step S1, based on the chain derivation rule, and train the basic model, auxiliary model and enhanced model for rutting recognition by error back propagation. When the overall loss function L total When it is minimum, the corresponding model parameters are optimal, and the trained basic model A is the final road internal void intelligent recognition model.

5. A road internal hollow development state early warning method according to claim 4, characterized in that: The specific implementation method of step S5 also includes the following steps: S5.

1. Export the road internal void intelligent identification model obtained in step S4 as a deployment file for independent services, including one of a pt file, a pth file, a json file, and a txt file; S5.

2. Construct a road information model; S5.2.

1. Collect information on the location, length, width, and layout of roads; S5.2.

2. The data obtained in step S5.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S5.2.

3. Based on step S5.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S5.2.

4. Export the road information model to a format file supported by the Web application, including a json file or a js file; S5.

3. In the digital twin platform, import the deployment file of the intelligent recognition model for internal voids in the middle road obtained in step S5.1, and import the road information model format file obtained in step S5.2; S5.

4. Use Python to build an API interface between image data and the intelligent recognition model for road voids, which is used for intelligent identification of ruts. Use Python to build an API interface between the intelligent recognition model for road voids and the road information model to complete the construction of the digital twin model.

6. A method for early warning of road internal void development status according to claim 5, characterized in that: The specific implementation method of step S6 also includes the following steps: S6.

1. Use a three-dimensional ground penetrating radar to collect road interior image data and upload it to the digital twin platform obtained in step S4. Then use the road interior void intelligent recognition model to intelligently identify the void area in the road surface image, and select all images containing the road interior void; S6.

2. According to the differences in the factors affecting the road service environment, the road internal air gap warning status is divided into four standards; S6.2.

1. Setting standard 1: When the values ​​of traffic flow, ambient temperature, precipitation, pavement material elastic modulus, pavement thickness, service life, number of repairs, pipe network service time, pavement temperature, and number of surrounding engineering activities are JLB1, HWB1, JSB1, TMB1, HDB1, FYB1, WXB1, GFB1, LWB1, and ZDB1 respectively, the corresponding volume critical value of the void area is set to VVB1; S6.2.

2. Setting standard 2: When the values ​​of the factors of traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, and number of surrounding engineering activities are JLB2, HWB2, JSB2, TMB2, HDB2, FYB2, WXB2, GFB2, LWB2, and ZDB2 respectively, the corresponding volume critical value of the void area is set to VVB2; S6.2.

3. Setting standard 3: When the values ​​of traffic flow, ambient temperature, precipitation, pavement material elastic modulus, pavement thickness, service life, number of repairs, pipe network service time, pavement temperature, and number of surrounding engineering activities are JLB3, HWB3, JSB3, TMB3, HDB3, FYB3, WXB3, GFB3, LWB3, and ZDB3 respectively, the corresponding volume critical value of the void area is set to VVB3; S6.2.

4. Setting standard 4: When the values ​​of traffic flow, ambient temperature, precipitation, pavement material elastic modulus, pavement thickness, service life, number of repairs, pipe network service time, pavement temperature, and number of surrounding engineering activities are JLB4, HWB4, JSB4, TMB4, HDB4, FYB4, WXB4, GFB4, LWB4, and ZDB4 respectively, the corresponding volume critical value of the void area is set to VVB4; S6.

3. Select the target road and collect the corresponding data on traffic flow, ambient temperature, precipitation, elastic modulus of pavement material, pavement thickness, service life, number of repairs, service time of pipeline network, pavement temperature, and number of surrounding engineering activities during the operation of the target road. The data are recorded as: JL, HW, JS, TM, HD, FY, WX, GF, LW, ZD. The volume of the empty area measured on site is recorded as: VV; S6.

4. Calculate the correlation between the target road state selected in step S6.3 and the four standards set in step S6.2, and determine the critical value corresponding to the warning standard applicable to the target road; Calculate the correlation value GL between the target road state and the kth criterion k , the calculation formula is Compare the calculated GL1, GL2, GL3, and GL4 to obtain the minimum value, which is recorded as GLmin; then record the critical value of the void area volume corresponding to GLmin as VVBmin; S6.

5. Optimize the volume value of the hollow area to obtain the optimized volume value of the hollow area VVn, which is calculated as follows: VVn=χ·GLmin·VV Where, χ is the volume adjustment factor; Compare the optimized value VVn of the void area volume obtained by calculation with the volume critical value VVBmin of the void area corresponding to GLmin. When VVn ≥ VVBmin, it indicates that the void area volume reaches or exceeds the allowable range, and an alarm needs to be issued to repair the void area. When VVn < VVBmin, it indicates that the void area volume has not reached the critical value, and no alarm is required. Attention needs to be paid during the maintenance process.

7. An electronic device, characterized in that: It includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of a method for warning the development state of internal voids in a road according to any one of claims 1-6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, a method for warning the development state of internal voids in a road according to any one of claims 1-6 is implemented.

Images