Driving performance evaluation method under road upheaval action based on digital twinning

Abstract

The invention discloses a driving performance evaluation method under the road upheaval effect based on digital twinning, and belongs to the technical field of road engineering. In order to accurately evaluate the driving performance under the action of the road surface upheaval, the method comprises the following steps: collecting a road surface image, carrying out compression processing and similar disease identification, and constructing a data set for rapid identification of the road surface upheaval; constructing a basic model, a first auxiliary model, a second auxiliary model and a strengthening model for rapid identification of the pavement upheaval; constructing a loss function among output layers of the basic model, the first auxiliary model, the second auxiliary model and the enhanced model based on the rapid identification of the road upheaval; constructing an interlayer loss function and an overall loss function of the reinforcement model and the basic model for rapid identification of the pavement upheaval; performing improvement processing on the pavement upheaval rapid identification model to obtain a lightweight pavement upheaval rapid identification model, and constructing a digital twinborn model; and constructing a driving performance evaluation method under the road upheaval action based on digital twinning. According to the invention, resource distribution is optimized and maintenance cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of road engineering, and in particular relates to a method for evaluating driving performance under road congestion based on digital twins. Background Art

[0002] Pavement bumps are a common road structural problem that manifests as irregular ridges or wavy bumps on the road surface. This phenomenon is usually caused by factors such as uneven settlement of the underlying soil, water penetration, inappropriate roadbed material selection or improper construction methods. Pavement bumps not only lead to unsightly pavement appearance, but more importantly, they can damage the structural integrity of the pavement, shorten the service life of the road, and increase maintenance costs. Bumps can also cause vehicles to accelerate or brake unexpectedly, increasing the risk of traffic accidents, especially at high speeds or on slippery roads.

[0003] The invention patent with application number 202110030593.6 and invention name "Pre-warning method for potholes and congestion of intelligent driving vehicles" describes a road pothole and congestion warning system for intelligent driving vehicles. The system uses an on-board infrared light curtain group to collect road images and obtains driving state parameters through a vehicle state perception module to identify the type, location and size of road potholes and congestion, and determine whether the car will encounter these road abnormalities. Based on this information, the system can start the corresponding warning algorithm to achieve autonomous lane change and active obstacle avoidance, thereby increasing driving safety. However, the above invention does not consider the driving performance evaluation method under the effect of road congestion, which makes it difficult for road managers to accurately identify and evaluate driving discomfort caused by road congestion, which will affect road maintenance decisions and may result in unnecessary maintenance costs or delay necessary repair work. Secondly, when driving on an unevaluated congested road section, the driver may encounter unexpected driving difficulties, increase the risk of traffic accidents, and increase vehicle wear and tear due to the uneven road surface, thereby increasing long-term maintenance costs. In addition, driving discomfort may also affect the overall traffic flow and efficiency of the road, aggravating traffic congestion problems. Summary of the invention

[0004] The problem to be solved by the present invention is to accurately evaluate the driving performance under the influence of road congestion, and propose a driving performance evaluation method under the influence of road congestion based on digital twin.

[0005] To achieve the above object, the present invention is implemented through the following technical solutions: A driving performance evaluation method under road congestion based on digital twins includes the following steps: S1. Using a vehicle-mounted three-dimensional structured light camera to collect road images, and then compressing the collected road images to obtain a compressed road image; S2. Perform similar disease identification on the compressed road surface image obtained in step S1, extract all road surface images containing the bag and the disease type most similar to the image features of the bag, and construct a data set for rapid identification of road bag; S3. Construct the basic model, first auxiliary model, second auxiliary model and enhanced model for rapid identification of road congestion; S4. Construct a loss function among the basic model, the first auxiliary model, the second auxiliary model and the output layer of the enhanced model based on the rapid identification of road congestion; S5. Constructing the inter-layer loss function of the enhanced model and the basic model for rapid identification of road congestion; then combining the loss function between the basic model, the first auxiliary model, the second auxiliary model and the output layer of the enhanced model obtained in step S4 to construct the overall loss function of the road congestion rapid identification model; S6. Using the data set obtained in step S2 for rapid identification of road congestion, the constructed basic model, the first auxiliary model, the second auxiliary model, and the enhanced model are trained to obtain a trained basic model as a road congestion rapid identification model; S7. The road congestion rapid identification model obtained in step S6 is improved to obtain a lightweight road congestion rapid identification model; S8. Constructing a digital twin model based on the lightweight road surface fast identification model obtained in step S7; S9. Based on the digital twin model obtained in step S8, a driving performance evaluation method under road congestion based on digital twin is constructed.

[0006] Furthermore, the specific implementation method of step S1 includes the following steps: S1.1. Use a vehicle-mounted 3D structured light camera to collect road images, and control the vehicle speed within 70 km / h during the collection process; S1.2. compressing the collected road surface image by using Huffman coding to obtain a compressed road surface image; S1.3. Perform a plane projection on the compressed road surface image, and obtain a projected road surface image which is a compressed road surface image.

[0007] Furthermore, the specific implementation method of step S3 includes the following steps: S3.1. Constructing a basic model for rapid identification of road congestion: S3.1.1. The basic model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 path aggregation network layer PANet, 1 pooling splicing layer, 1 spatial pyramid pooling layer SPP, and 1 output layer; S3.1.2. Input the training set obtained in step S2 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: ; in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the ia The true labels of samples, Based on the prediction of model A ia The probability that a sample is class 1; S3.1.3. The node output values ​​of the four output units corresponding to the basic model A are a 1 , a 2 , a 3 , a 4 , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S3.1.4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ; S3.2. Constructing the first auxiliary model for rapid identification of road congestion: S3.2.1. The first auxiliary model B consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer: S3.2.2. Input the training set obtained in step S2 into the first auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the first auxiliary model B, and obtain the trained first auxiliary model B; The calculation formula of the loss function of the first auxiliary model B is: ; in, is the loss function of the first auxiliary model B; is the total number of data samples in the first auxiliary model B; For the ib The true labels of samples, The first auxiliary model B predicts ib The probability that a sample is class 1; S3.2.3. The node output values ​​of the four output units corresponding to the first auxiliary model B are b 1 , b 2 , b 3 , b 4 , the node output values ​​of the four output units are initially converted to obtain the output value b i The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value b i The corresponding enhancement probability value The calculation formula is: ; S3.2.4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the first auxiliary model B. After the training is completed, the disease output value corresponding to the first auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , i =1,2,3,4; S3.3. Constructing the second auxiliary model for rapid identification of road congestion: S3.3.1. The second auxiliary model C consists of 1 input layer, 12 convolutional layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer: S3.3.2. Input the training set obtained in step S2 into the second auxiliary model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the second auxiliary model C, and obtain the trained second auxiliary model C; The calculation formula of the loss function of the second auxiliary model C is: ; in, is the loss function of the second auxiliary model C; is the total number of data samples in the second auxiliary model C; For the ic The true labels of samples, The second auxiliary model C predicts ic The probability that a sample is class 1; S3.3.3. The node output values ​​of the four output units corresponding to the second auxiliary model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S3.3.4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the second auxiliary model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the second auxiliary model C are obtained as follows: ; S3.4. Construct an enhanced model for rapid identification of road congestion: S3.4.1. The enhanced model D consists of 1 input layer, 14 convolutional layers, 14 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer: S3.4.2. Input the training set obtained in step S2 into the enhanced model D, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the enhanced model D, and obtain the trained enhanced model D; The loss function of the enhanced model D is calculated as: ; in, is the loss function of the enhanced model D; is the total number of data samples in the reinforcement model D; For the id The true labels of samples, To strengthen the model D to predict the id The probability that a sample is class 1; S3.4.3. The node output values ​​of the four output units corresponding to the enhanced model D are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S3.4.4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the enhanced model D. After the training is completed, the output values ​​of the four types of diseases corresponding to the enhanced model D are obtained as follows: .

[0008] Furthermore, the specific implementation method of step S4 includes the following steps: S4.1. Based on and , establish the loss function between the first auxiliary model B and the output layer of the basic model A, and the calculation formula is: ; based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is: ; based on and , establish the relationship loss function between the output layer of the first auxiliary model B and the basic model A , the calculation formula is: ; in, for The weight coefficient of for The weight coefficient of S4.2. Establish the relationship loss function between the output layer of the second auxiliary model C and the first auxiliary model B , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S4.3. Establish the relationship loss function between the output layer of the second auxiliary model C and the basic model A , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function is Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S4.4. Establishing the relationship loss function between the output layer of the enhanced model D and the base model A , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function of The calculation formula is: ; S4.5. Establish the relationship loss function between the output layer of the reinforcement model D and the second auxiliary model C , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function is Based on and The loss function of The calculation formula is: ; The calculation formula is: .

[0009] Furthermore, the specific implementation method of step S5 includes the following steps: S5.1. Assuming a small model a and large models d , build a small model a The second, third, fourth and output layers in the large model are d Establish connections between the fourth, sixth, tenth layers and the output layer; S5.2. For the second layer in the small model and the large model d The fourth layer in the small model establishes relationships a The output of the second layer is recorded as , the corresponding size of width × length × number of channels is Large Model d The output of the fourth layer is recorded as , the corresponding size of width × length × number of channels is ; For size comparison, When , a convolution operation is added to the second layer output of the small model, the convolution kernel size is 1×1, and the number of convolution kernels is , the output is ,at this time, The corresponding output values ​​of each channel are , k i is the channel number, The corresponding output values ​​of each channel are ; Build on Small model a The second floor with the large model d The loss function between the fourth layers , the calculation formula is: ; when When in the large model d In the output of the fourth layer, a convolution operation is added, the convolution kernel size is 1×1, and the number of convolution kernels is , the output is ;at this time, The corresponding output values ​​of each channel are: , The corresponding output values ​​of each channel are ; Establish Small model a The second floor with the large model d The loss function between the fourth layers , the calculation formula is: ; S5.3. For the third layer in the small model and the large model b The sixth layer in the relationship is established to obtain a small model a The third floor and the large model b Loss function between the sixth layers ; S5.4. For the fourth layer in the small model and the large model b The tenth layer in the relationship is established to obtain a small model a Fourth floor and large model b Loss function between the tenth layer ; S5.5. When the small model is the base model A, the small model a The second, third, and fourth layers in the graph are the output of pooling layer 1, pooling layer 2, and pooling layer 4 of the basic model A, respectively. When the large model is the enhanced model D, the large model d The fourth, sixth, and tenth layers in are the output of pooling layer 2, the output of pooling layer 3, and the output of pooling layer 5 of the enhanced model D respectively; S5.6. Based on the loss function between the output layer of the enhanced model D and the output layer of the base model A and the inter-layer loss function, define the total loss function between models , the calculation formula is: ; in, for The corresponding weight coefficient is, for The corresponding weight coefficient is, for The corresponding weight coefficient; Furthermore, the specific implementation method of step S6 is to use the data set obtained in step S2 based on the chain derivative rule to train the basic model, the first auxiliary model, the second auxiliary model and the enhanced model through error back propagation. When the overall loss function When is the minimum, the corresponding model parameters are optimal, and the trained basic model is obtained as the road congestion fast recognition model; Furthermore, the specific implementation method of step S7 includes the following steps: S7.1. For the road congestion rapid identification model obtained in step S6, set the enhancement coefficient G The values ​​are: G =1,3,5,7,9,11,15,20,30,40,50; S7.2. Based on different enhancement factors G 11 working conditions are obtained, and 11 models are obtained by training the road congestion rapid recognition model. The accuracy rates of the 11 road congestion rapid recognition models are set in order. , i=1,2,3,…,11, the calculation formula is: ; in, Represents the first i The number of samples that a model predicts as having a bag and actually has a bag when identifying the data in the test set; Represents the first i The number of samples that a model predicts as having a packet but is not actually having a packet when identifying the data in the test set; Comparison The maximum value in ; The corresponding road surface bump quick identification model is the final lightweight road surface bump quick identification model; Furthermore, the specific implementation method of step S8 includes the following steps: S8.1. The lightweight road congestion rapid identification model obtained in step S7 is exported as a deployment file for independent services, including a pt file, a pth file, a json file, or a txt file; S8.2. Construct a road information model; S8.2.1. Collect information on the location, length, width, and layout of roads; S8.2.2. The data obtained in step S8.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S8.2.3. Based on step S8.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S8.2.4. Export the road information model to a format file supported by the web application, including a json file or a js file; S8.3. In the digital twin platform, import the deployment file of the lightweight road congestion rapid identification model obtained in step S7.1, and import the road information model format file obtained in step S7.2; S8.4. Use Python to construct an API interface between image data and a lightweight road bump rapid identification model for intelligent rutting identification. Use Python to construct an API interface between a lightweight road bump rapid identification model and a road information model to complete the construction of a digital twin model.

[0010] Furthermore, the specific implementation method of step S9 includes the following steps: S9.1. Use a 3D structured light camera to collect road surface data and upload it to the digital twin model established in step S7. Then use the lightweight road bump rapid identification model deployed in the system to intelligently identify the road bump area in the road image, select all images containing road bumps, conduct on-site review of the bumps, and measure the area GM and height BD of the bumps. S9.2. Establish driving evaluation indicators, including driving comfort evaluation indicators and driving safety evaluation indicators: Driving comfort evaluation indicators include road surface flatness PZ, driving vibration index CZ, noise level ZY, and in-vehicle vibration amplitude ZD; Driving safety evaluation indicators include accident rate SGL, emergency braking frequency ZP, and vehicle handling stability index CW; S9.3. Select 8 time nodes t1, t2, t3, ... t8, and collect the values ​​of the corresponding driving evaluation indicators at the 8 time nodes: S9.4. Based on the values ​​of each driving evaluation index collected in step S9.3, calculate the average value of each driving evaluation index, including the average value of road surface smoothness , average value of driving vibration index , average noise level , Average value of vibration amplitude inside the vehicle , average accident rate , Average emergency braking frequency , vehicle handling stability index average , Average area of ​​the package , Average value of bag height ; S9.5. Construct the correlation between the area and height of the enclosed area and each driving evaluation index , the calculation formula is as follows; ; in, Any one of the eight time nodes; ZB represents the evaluation index, including road surface smoothness, driving vibration index, noise level, in-vehicle vibration amplitude, accident rate, emergency braking frequency, and vehicle handling stability index. That is, the correlation between the enveloping area and enveloping height and road surface smoothness, driving vibration index, noise level, in-vehicle vibration amplitude, accident rate, emergency braking frequency, and vehicle handling stability index is respectively , , , , , , ; S9.6. Correlation between driving comfort evaluation indicators: road surface smoothness, driving vibration index, noise level, and in-vehicle vibration amplitude , , , , compare and obtain the maximum value therein, and then determine the driving comfort evaluation index having an influence based on the maximum value; For driving safety indicators: Compare the correlation between accident rate SGL, emergency braking frequency ZP, and vehicle handling stability index CW , , , compare and obtain the maximum value therein, and then determine the driving safety evaluation index having an impact based on the maximum value; S9.7. Set a linear correlation equation between the driving comfort evaluation index and time, and then solve the equation using the threshold of the driving comfort evaluation index to obtain the life value of the driving comfort evaluation index. ; A linear correlation equation between the driving safety evaluation index and time is set, and then the threshold of the driving safety evaluation index is used to solve the equation to obtain the life value of the driving safety evaluation index. ; Compare and ,choose and The minimum value is recorded as tmin. When the road operation time reaches tmin, the road surface is repaired to ensure that the road service status meets the requirements of comfort and safety.

[0011] Beneficial effects of the present invention: The driving performance evaluation method under the effect of road congestion based on digital twins described in the present invention combines the characteristics of road congestion and designs a model structure including input layer, convolution layer, activation layer, pooling layer, merging layer, upsampling layer, PANet layer, pooling splicing layer, SPP layer (spatial pyramid pooling), and output layer. The functions of each structural layer are as follows: The input layer receives road images captured by the vehicle-mounted camera, which contain the local ridge features of the bump and its surrounding environment information.

[0012] The convolutional layer extracts features from the image through a series of filters, such as the contours of the bump, its texture, and the difference from the normal road surface.

[0013] The activation layer introduces nonlinearity, allowing the model to capture the complex pattern differences between congested and normal roads.

[0014] The pooling layer helps the model maintain sensitivity to the features of the packed image while reducing the amount of computation by reducing the spatial size, and improves tolerance to position changes.

[0015] The merging layer is used to fuse features from different convolutional layers to enhance the model's ability to recognize the shape and size of packets.

[0016] The upsampling layer restores the spatial resolution of the enveloping features so that the location of the enveloping features can be accurately located in the output layer.

[0017] The PANet layer improves the recognition accuracy of the geometric and structural features of the package by optimizing the feature pyramid and enhancing feature transfer.

[0018] The pooling concatenation layer is used to integrate the features of different pooling operations before the SPP layer to capture multi-scale encapsulation information.

[0019] The SPP layer processes the enveloping features to preserve the key information of enveloping images of different sizes and provide robustness to the input image size.

[0020] The output layer maps the extracted features into the final judgment output, such as the probability distribution map of the congested area, to achieve accurate identification of road congestion.

[0021] The driving performance evaluation method under the effect of road bumps based on digital twins described in the present invention, the integration of digital twin technology and the rapid identification method of road bumps, can use high-precision digital models combined with real-time data to achieve instant and accurate diagnosis of road conditions. By integrating sensor data, image recognition technology and machine learning models, it can not only detect and locate road bumps in a timely manner, but also predict future damage development, thereby optimizing maintenance plans and resource allocation. This helps to improve road safety, reduce maintenance costs, and extend the service life of roads, which is of great value in improving public safety and economic efficiency.

[0022] The present invention discloses a method for evaluating driving performance under the influence of road congestion based on digital twins, which can more accurately determine the severity and impact range of road diseases, thereby formulating more effective maintenance or reconstruction plans, optimizing resource allocation, reducing maintenance costs, and improving driver safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a method for evaluating driving performance under road congestion based on digital twins described in the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the specific embodiments described are only part of the embodiments of the present invention, rather than all of the specific embodiments. The components of the specific embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations, and the present invention can also have other embodiments.

[0025] Therefore, the following detailed description of the specific embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected specific embodiments of the present invention. Based on the specific embodiments of the present invention, all other specific embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0026] In order to further understand the content, features and effects of the present invention, the following specific implementation methods are given as examples, and the attached Figure 1 The detailed instructions are as follows:

[0027] Embodiment 1: A driving performance evaluation method under road congestion based on digital twins includes the following steps: S1. Using a vehicle-mounted three-dimensional structured light camera to collect road images, and then compressing the collected road images to obtain compressed road images; Furthermore, the specific implementation method of step S1 includes the following steps: S1.1. Use a vehicle-mounted 3D structured light camera to collect road images, and control the vehicle speed within 70 km / h during the collection process; S1.2. compressing the collected road surface image by using Huffman coding to obtain a compressed road surface image; S1.3. Perform a plane projection on the compressed road surface image, and obtain a projected road surface image which is a compressed road surface image.

[0028] S2. Perform similar disease identification on the compressed road surface image obtained in step S1, extract all road surface images containing the bag and the disease type most similar to the image features of the bag, and construct a data set for rapid identification of road bag; Furthermore, the specific implementation method of step S2 includes the following steps: S2.1. Similar diseases of road bumps include longitudinal cracks, transverse cracks, cracks, rutting, potholes, spalling, oiling, subsidence, and undercutting; S2.2. Manually select the processed road surface images obtained in step S1, select all road surface images including road bumps and similar diseases, and select the corresponding disease areas, and mark the road bumps, longitudinal cracks, transverse cracks, cracks, rutting, potholes, peeling, oil spills, subsidence, and edge gnawing areas in the road surface images in turn. PHO 1. PHO 2. PHO 3. PHO 4. PHO 5. PHO 6. PHO 7. PHO 8. PHO 9. PHO 10; S2.3. Divide the pixel grayscale value 0~255 into 16 grayscale intervals, denoted as , =1,2,3,…,16; S2.4. Count the number of pixels corresponding to each gray level in the road congestion bag Then, the number of pixels corresponding to each gray level of similar diseases of road bumps, such as longitudinal cracks, transverse cracks, cracks, rutting, potholes, peeling, oiling, subsidence, and gnawing, is counted. , , , , , , , , ; S2.5. Standardize the number of pixels corresponding to each gray level to obtain the standard number of pixels, where the standard number of pixels corresponding to the road congestion is , the calculation formula is: ; The standard number of pixels corresponding to longitudinal cracks, transverse cracks, cracks, rutting, potholes, spalling, oil spills, subsidence, and edge gnawing is , , , , , , , , ; S2.6. As a benchmark, the relationship function between pavement bumps and other diseases is established in turn, and the calculation formula is: ; Among them, ii represents the serial number of other similar diseases, ii=2,3,4,5…10, and we get , , , , , , , , They represent the relationship functions between road surface bumps and longitudinal cracks, transverse cracks, cracks, rutting, potholes, spalling, oiling, subsidence, and undercutting respectively; S2.7. Comparison , , , , , , , , , and sort them in descending order, select the disease types corresponding to the top three relationship functions as the disease types most similar to the image features of road surface bumps, and then select all road surface images containing road surface bumps and the disease types most similar to the image features of road surface bumps; S2.8. By manually labeling, the road surface images obtained in step S2.8 that contain road bumps and the types of defects that are most similar to the image features of the road bumps are framed to select the road pothole area and the other three defects, and the road bumps are labeled as upheaval, and the other three defects are labeled as other1, other2, and other3, respectively, to obtain a data set for rapid identification of road bumps; S2.9. Divide the dataset for rapid identification of road congestion into a training set and a test set in a ratio of 8:2.

[0029] S3. Construct the basic model, first auxiliary model, second auxiliary model and enhanced model for rapid identification of road congestion; Furthermore, the specific implementation method of step S3 includes the following steps: S3.1. Constructing a basic model for rapid identification of road congestion: S3.1.1. The basic model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 path aggregation network layer PANet, 1 pooling splicing layer, 1 spatial pyramid pooling layer SPP, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 32; stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Convolutional layer 2: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type is global average pooling; pooling size: 5×5; Stride: 1; Pooling layer 2: Type is global average pooling; pooling size: 9×9; Stride: 1; Pooling layer 3: Type is global average pooling; pooling size: 13×13; Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolution layer 5: kernel size 3×3; number of kernels 768; stride: 2; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 768; stride: 2; batch normalization; Activation layer 6: The activation function is LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 128; Stride: 1; batch normalization; Activation layer 8: The activation function is LeakyReLU; Convolution layer 9: kernel size 3×3; number of kernels 64; stride: 1; batch normalization; Activation layer 9: The activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; number of neurons: 4; S3.1.2. Input the training set obtained in step S2 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: ; in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the iaThe true labels of samples, Based on the prediction of model A ia The probability that a sample is class 1; S3.1.3. The node output values ​​of the four output units corresponding to the basic model A are a 1 , a 2 , a 3 , a 4 , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as Specifically, for road congestion 1, for other diseases is 0; S3.1.4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ; S3.2. Constructing the first auxiliary model for rapid identification of road congestion: S3.2.1. The first auxiliary model B consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 32; Stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Convolutional layer 2: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 256; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type is global average pooling; pooling size: 5×5; Stride: 1; Pooling layer 2: Type is global average pooling; pooling size: 7×7; Stride: 1; Pooling layer 3: Type is global average pooling; pooling size: 9×9; Stride: 1; Pooling layer 4: Type is global average pooling; pooling size: 11×11; Stride: 1; Pooling layer 5: Type is global average pooling; pooling size: 13×13; Stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolution layer 5: kernel size 3×3; number of kernels 1280; stride: 2; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 1280; stride: 2; batch normalization; Activation layer 6: The activation function is LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3; number of kernels 512; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 7: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 128; Stride: 1; batch normalization; Activation layer 8: The activation function is LeakyReLU; Convolution layer 9: kernel size 3×3; number of kernels 64; stride: 1; batch normalization; Activation layer 9: The activation function is LeakyReLU; Output layer: type is Softmax; number of neurons: 4; S3.2.2. Input the training set obtained in step S2 into the first auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the first auxiliary model B, and obtain the trained first auxiliary model B; The calculation formula of the loss function of the first auxiliary model B is: ; in, is the loss function of the first auxiliary model B; is the total number of data samples in the first auxiliary model B; For the ib The true labels of samples, The first auxiliary model B predicts ib The probability that a sample is class 1; S3.2.3. The node output values ​​of the four output units corresponding to the first auxiliary model B are b 1 , b 2 , b 3 , b 4 , the node output values ​​of the four output units are initially converted to obtain the output value b i The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value b i The corresponding enhancement probability value The calculation formula is: ; S3.2.4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the first auxiliary model B. After the training is completed, the disease output value corresponding to the first auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , i =1,2,3,4, specifically, for road congestion 1, for other diseases is 0; S3.3. Constructing the second auxiliary model for rapid identification of road congestion: S3.3.1. The second auxiliary model C consists of 1 input layer, 12 convolutional layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 32; Stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Convolutional layer 2: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 128; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Convolution layer 5: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 6: The activation function is LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type is global average pooling; pooling size: 5×5; Stride: 1; Pooling layer 2: Type is global average pooling; pooling size: 7×7; Stride: 1; Pooling layer 3: Type is global average pooling; pooling size: 9×9; Stride: 1; Pooling layer 4: Type is global average pooling; pooling size: 11×11; Stride: 1; Pooling layer 5: Type is global average pooling; pooling size: 13×13; Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 7: kernel size 3×3; number of kernels 1280; stride: 2; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 1280; stride: 2; batch normalization; Activation layer 8: The activation function is LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 9: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 9: The activation function is LeakyReLU; Convolution layer 10: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 10: The activation function is LeakyReLU; Convolutional layer 11: kernel size 3×3; number of kernels 128; Stride: 1; batch normalization; Activation layer 11: The activation function is LeakyReLU; Convolution layer 12: kernel size 3×3; number of kernels 64; stride: 1; batch normalization; Activation layer 12: The activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; number of neurons: 4; S3.3.2. Input the training set obtained in step S2 into the second auxiliary model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the second auxiliary model C, and obtain the trained second auxiliary model C; The calculation formula of the loss function of the second auxiliary model C is: ; in, is the loss function of the second auxiliary model C; is the total number of data samples in the second auxiliary model C; For the ic The true labels of samples, The second auxiliary model C predicts ic The probability that a sample is class 1; S3.3.3. The node output values ​​of the four output units corresponding to the second auxiliary model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as Specifically, for road congestion 1, for other diseases is 0; S3.3.4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the second auxiliary model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the second auxiliary model C are obtained as follows: ; S3.4. Construct an enhanced model for rapid identification of road congestion: S3.4.1. The enhanced model D consists of 1 input layer, 14 convolutional layers, 14 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer. The specific network structure is the following structure connected in sequence: Input layer: image size 227×227×3; Convolutional layer 1: convolution kernel size 3×3; number of convolution kernels 32; Stride: 1; batch normalization; Activation layer 1: The activation function is LeakyReLU; Convolutional layer 2: convolution kernel size 3×3; number of convolution kernels 32; stride: 1; batch normalization; Activation layer 2: The activation function is LeakyReLU; Convolution layer 3: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; batch normalization; Activation layer 3: The activation function is LeakyReLU; Convolutional layer 4: convolution kernel size 3×3; number of convolution kernels 64; stride: 1; batch normalization; Activation layer 4: The activation function is LeakyReLU; Convolution layer 5: kernel size 3×3; number of kernels 128; stride: 1; batch normalization; Activation layer 5: The activation function is LeakyReLU; Convolutional layer 6: kernel size 3×3; number of kernels 128; Stride: 1; batch normalization; Activation layer 6: The activation function is LeakyReLU; Convolutional layer 7: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 7: The activation function is LeakyReLU; Convolutional layer 8: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 8: The activation function is LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type is global average pooling; pooling size: 5×5; Stride: 1; Pooling layer 2: Type is global average pooling; pooling size: 7×7; Stride: 1; Pooling layer 3: Type is global average pooling; pooling size: 9×9; Stride: 1; Pooling layer 4: Type is global average pooling; pooling size: 11×11; Stride: 1; Pooling layer 5: Type is global average pooling; pooling size: 13×13; Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolution layer 9: kernel size 3×3; number of kernels 1280; stride: 2; batch normalization; Activation layer 9: The activation function is LeakyReLU; Convolution layer 10: kernel size 3×3; number of kernels 1280; stride: 2; batch normalization; Activation layer 10: The activation function is LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 11: kernel size 3×3; number of kernels 512; Stride: 1; batch normalization; Activation layer 11: The activation function is LeakyReLU; Convolutional layer 12: kernel size 3×3; number of kernels 256; stride: 1; batch normalization; Activation layer 12: The activation function is LeakyReLU; Convolution layer 13: kernel size 3×3; number of kernels 128; stride: 1; batch normalization; Activation layer 13: The activation function is LeakyReLU; Convolutional layer 14: kernel size 3×3; number of kernels 64; Stride: 1; batch normalization; Activation layer 14: the activation function is LeakyReLU; Output layer: type is normalized exponential function Softmax; number of neurons: 4; S3.4.2. Input the training set obtained in step S2 into the enhanced model D, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the enhanced model D, and obtain the trained enhanced model D; The loss function of the enhanced model D is calculated as: ; in, is the loss function of the enhanced model D; is the total number of data samples in the reinforcement model D; For the id The true labels of samples, To strengthen the model D to predict the id The probability that a sample is class 1; S3.4.3. The node output values ​​of the four output units corresponding to the enhanced model D are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ; in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as Specifically, for road congestion 1, for other diseases is 0; S3.4.4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the enhanced model D. After the training is completed, the output values ​​of the four types of diseases corresponding to the enhanced model D are obtained as follows: .

[0030] S4. Construct a loss function among the basic model, the first auxiliary model, the second auxiliary model and the output layer of the enhanced model based on the rapid identification of road congestion; Furthermore, the specific implementation method of step S4 includes the following steps: S4.1. Based on and , establish the loss function between the first auxiliary model B and the output layer of the basic model A, and the calculation formula is: ; based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is: ; based on and , establish the relationship loss function between the output layer of the first auxiliary model B and the basic model A , the calculation formula is: ; in, for The weight coefficient of for The weight coefficient of S4.2. Establish the relationship loss function between the output layer of the second auxiliary model C and the first auxiliary model B , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S4.3. Establish the relationship loss function between the output layer of the second auxiliary model C and the basic model A , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function is Based on and The loss function of The calculation formula is: ; The calculation formula is: ; S4.4. Establishing the relationship loss function between the output layer of the enhanced model D and the base model A , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function of The calculation formula is: ; S4.5. Establish the relationship loss function between the output layer of the reinforcement model D and the second auxiliary model C , the expression is: ; in, for The weight coefficient of for The weight coefficient of Based on and The loss function is Based on and The loss function of The calculation formula is: ; The calculation formula is: .

[0031] S5. Constructing the inter-layer loss function of the enhanced model and the basic model for rapid identification of road congestion; then combining the loss function between the basic model, the first auxiliary model, the second auxiliary model and the output layer of the enhanced model obtained in step S4 to construct the overall loss function of the road congestion rapid identification model; Furthermore, the specific implementation method of step S5 includes the following steps: S5.1. Assuming a small model a and large models d , build a small model a The second, third, fourth and output layers in the large model are d Establish connections between the fourth, sixth, tenth layers and the output layer; S5.2. For the second layer in the small model and the large model d The fourth layer in the small model establishes relationships a The output of the second layer is recorded as , the corresponding size of width × length × number of channels is Large Model d The output of the fourth layer is recorded as , the corresponding size of width × length × number of channels is ; For size comparison, When , a convolution operation is added to the second layer output of the small model, the convolution kernel size is 1×1, and the number of convolution kernels is , the output is ,at this time, The corresponding output values ​​of each channel are , k i is the channel number, The corresponding output values ​​of each channel are ; Build on Small model a The second floor with the large modeld The loss function between the fourth layers , the calculation formula is: ; when When in the large model d In the output of the fourth layer, a convolution operation is added, the convolution kernel size is 1×1, and the number of convolution kernels is , the output is ;at this time, The corresponding output values ​​of each channel are: , The corresponding output values ​​of each channel are ; Establish Small model a The second floor with the large model d The loss function between the fourth layers , the calculation formula is: ; S5.3. For the third layer in the small model and the large model b The sixth layer in the relationship is established to obtain a small model a The third floor and the large model b Loss function between the sixth layers ; S5.4. For the fourth layer in the small model and the large model b The tenth layer in the relationship is established to obtain a small model a Fourth floor and large model b Loss function between the tenth layer ; S5.5. When the small model is the base model A, the small model a The second, third, and fourth layers in the graph are the output of pooling layer 1, pooling layer 2, and pooling layer 4 of the basic model A, respectively. When the large model is the enhanced model D, the large model d The fourth, sixth, and tenth layers in are the output of pooling layer 2, the output of pooling layer 3, and the output of pooling layer 5 of the enhanced model D respectively; S5.6. Based on the loss function between the output layer of the enhanced model D and the output layer of the base model A and the inter-layer loss function, define the total loss function between models , the calculation formula is: ; in, for The corresponding weight coefficient is, for The corresponding weight coefficient is, for The corresponding weight coefficient.

[0032] S6. Using the data set obtained in step S2 for rapid identification of road congestion, the constructed basic model, the first auxiliary model, the second auxiliary model, and the enhanced model are trained to obtain a trained basic model as a road congestion rapid identification model; Furthermore, the specific implementation method of step S6 is to use the data set obtained in step S2 based on the chain derivative rule to train the basic model, the first auxiliary model, the second auxiliary model and the enhanced model through error back propagation. When the overall loss function When is minimum, the corresponding model parameters are optimal, and the trained basic model is obtained as a road congestion rapid recognition model.

[0033] S7. The road congestion rapid identification model obtained in step S6 is improved to obtain a lightweight road congestion rapid identification model; Furthermore, the specific implementation method of step S7 includes the following steps: S7.1. For the road congestion rapid identification model obtained in step S6, set the enhancement coefficient G The values ​​are: G =1,3,5,7,9,11,15,20,30,40,50; S7.2. Based on different enhancement factors G 11 working conditions are obtained, and 11 models are obtained by training the road congestion rapid recognition model. The accuracy rates of the 11 road congestion rapid recognition models are set in order. , i=1,2,3,…,11, the calculation formula is: ; in, Represents the first i The number of samples that a model predicts as having a bag and actually has a bag when identifying the data in the test set; Represents the first i The number of samples that a model predicts as having a packet but is not actually having a packet when identifying the data in the test set; Comparison The maximum value in ; The corresponding road surface congestion rapid recognition model is the final lightweight road surface congestion rapid recognition model.

[0034] S8. Constructing a digital twin model based on the lightweight road surface fast identification model obtained in step S7; Furthermore, the specific implementation method of step S8 includes the following steps: S8.1. The lightweight road congestion rapid identification model obtained in step S7 is exported as a deployment file for independent services, including a pt file, a pth file, a json file, or a txt file; S8.2. Construct a road information model; S8.2.1. Collect information on the location, length, width, and layout of roads; S8.2.2. The data obtained in step S8.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S8.2.3. Based on step S8.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S8.2.4. Export the road information model to a format file supported by the web application, including a json file or a js file; S8.3. In the digital twin platform, import the deployment file of the lightweight road congestion rapid identification model obtained in step S7.1, and import the road information model format file obtained in step S7.2; S8.4. Use Python to construct an API interface between image data and a lightweight road bump rapid identification model for intelligent rutting identification. Use Python to construct an API interface between a lightweight road bump rapid identification model and a road information model to complete the construction of a digital twin model.

[0035] S9. Based on the digital twin model obtained in step S8, a driving performance evaluation method under road congestion based on digital twin is constructed.

[0036] Furthermore, the specific implementation method of step S9 includes the following steps: S9.1. Use a 3D structured light camera to collect road surface data and upload it to the digital twin model established in step S7. Then use the lightweight road bump rapid identification model deployed in the system to intelligently identify the road bump area in the road image, select all images containing road bumps, conduct on-site review of the bumps, and measure the area GM and height BD of the bumps. S9.2. Establish driving evaluation indicators, including driving comfort evaluation indicators and driving safety evaluation indicators: Driving comfort evaluation indicators include road surface flatness PZ, driving vibration index CZ, noise level ZY, and in-vehicle vibration amplitude ZD; Driving safety evaluation indicators include accident rate SGL, emergency braking frequency ZP, and vehicle handling stability index CW; S9.3. Select 8 time nodes t1, t2, t3, ... t8, and collect the values ​​of the corresponding driving evaluation indicators at the 8 time nodes: Road surface flatness: PZ1, PZ2, PZ3, PZ4, PZ5, PZ6, PZ7, PZ8; Driving vibration index: CZ1, CZ2, CZ3, CZ4, CZ5, CZ6, CZ7, CZ8; data is derived from the measurement of internal vehicle sensors; Noise level: ZY1, ZY2, ZY3, ZY4, ZY5, ZY6, ZY7, ZY8; In-vehicle vibration amplitude: ZD1, ZD2, ZD3, ZD4, ZD5, ZD6, ZD7, ZD8; data is derived from the measurement of sensors inside the vehicle; Accident rate: SGL1, SGL2, SGL3, SGL4, SGL5, SGL6, SGL7, SGL8; Emergency braking frequency: ZP1, ZP2, ZP3, ZP4, ZP5, ZP6, ZP7, ZP8; data is measured by sensors inside the vehicle; Vehicle handling stability index: CW1, CW2, CW3, CW4, CW5, CW6, CW7, CW8; data is derived from the measurement of internal vehicle sensors; Occupancy area: GM1, GM2, GM3, GM4, GM5, GM6, GM7, GM8, GM9, GM10; data comes from step S9.1; Packet height: BD1, BD2, BD3, BD4, BD5, BD6, BD7, BD8; data comes from step S9.1; The above data are obtained from public channels; S9.4. Based on the values ​​of each driving evaluation index collected in step S9.3, calculate the average value of each driving evaluation index, including the average value of road surface smoothness , average value of driving vibration index , average noise level , Average value of vibration amplitude inside the vehicle , average accident rate , Average emergency braking frequency , vehicle handling stability index average , Average area of ​​the package , Average value of bag height ; S9.5. Construct the correlation between the area and height of the enclosed area and each driving evaluation index , the calculation formula is as follows; ; in, Any one of the eight time nodes; ZB represents the evaluation index, including road surface smoothness, driving vibration index, noise level, in-vehicle vibration amplitude, accident rate, emergency braking frequency, and vehicle handling stability index. That is, the correlation between the enveloping area and enveloping height and road surface smoothness, driving vibration index, noise level, in-vehicle vibration amplitude, accident rate, emergency braking frequency, and vehicle handling stability index is respectively , , , , , , ; S9.6. Correlation between driving comfort evaluation indicators: road surface smoothness, driving vibration index, noise level, and in-vehicle vibration amplitude , , , , compare and obtain the maximum value therein, and then determine the driving comfort evaluation index having an influence based on the maximum value; For driving safety indicators: Compare the correlation between accident rate SGL, emergency braking frequency ZP, and vehicle handling stability index CW , , , compare and obtain the maximum value therein, and then determine the driving safety evaluation index having an impact based on the maximum value; S9.7. Set a linear correlation equation between the driving comfort evaluation index and time, and then solve the equation using the threshold of the driving comfort evaluation index to obtain the life value of the driving comfort evaluation index. ; A linear correlation equation between the driving safety evaluation index and time is set, and then the threshold of the driving safety evaluation index is used to solve the equation to obtain the life value of the driving safety evaluation index. ; Compare and ,choose and The minimum value is recorded as tmin. When the road operation time reaches tmin, the road surface is repaired to ensure that the road service status meets the requirements of comfort and safety.

[0037] The key points and intended protection points of the present invention are: (1) A method for identifying similar diseases; (2) A lightweight intelligent road congestion recognition model; (3) A basic model improvement method based on enhancement coefficient; (4) A method for evaluating driving performance under road congestion based on digital twins.

[0038] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0039] Although the present application has been described above with reference to specific embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present application. In particular, as long as there is no structural conflict, the various features in the specific embodiments disclosed in the present application may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A driving performance evaluation method under road congestion based on digital twins, It is characterized in that The steps include: S1. Using a vehicle-mounted three-dimensional structured light camera to collect road images, and then compressing the collected road images to obtain a compressed road image; S2. Perform similar disease identification on the compressed road surface image obtained in step S1, extract all road surface images containing the bag and the disease type most similar to the image features of the bag, and construct a data set for rapid identification of road bag; S3. Construct the basic model, first auxiliary model, second auxiliary model and enhanced model for rapid identification of road congestion; S4. Construct a loss function among the basic model, the first auxiliary model, the second auxiliary model and the output layer of the enhanced model based on the rapid identification of road congestion; S5. Constructing the inter-layer loss function of the enhanced model and the basic model for rapid identification of road congestion; then combining the loss function between the basic model, the first auxiliary model, the second auxiliary model and the output layer of the enhanced model obtained in step S4 to construct the overall loss function of the road congestion rapid identification model; S6. Using the data set obtained in step S2 for rapid identification of road congestion, the constructed basic model, the first auxiliary model, the second auxiliary model, and the enhanced model are trained to obtain a trained basic model as a road congestion rapid identification model; S7. The road congestion rapid identification model obtained in step S6 is improved to obtain a lightweight road congestion rapid identification model; S8. Constructing a digital twin model based on the lightweight road surface fast identification model obtained in step S7; S9. Based on the digital twin model obtained in step S8, a driving performance evaluation method under road congestion based on digital twin is constructed.

2. According to claim 1, a method for evaluating driving performance under road congestion based on digital twins, It is characterized in that The specific implementation method of step S1 includes the following steps: S1.

1. Use a vehicle-mounted 3D structured light camera to collect road images, and control the vehicle speed within 70 km / h during the collection process; S1.

2. compressing the collected road surface image by using Huffman coding to obtain a compressed road surface image; S1.

3. Perform a plane projection on the compressed road surface image, and obtain a projected road surface image which is a compressed road surface image.

3. A method for evaluating driving performance under road congestion based on digital twins according to claim 1 or 2, It is characterized in that The specific implementation method of step S3 includes the following steps: S3.

1. Constructing a basic model for rapid identification of road congestion: S3.1.

1. The basic model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 path aggregation network layer PANet, 1 pooling splicing layer, 1 spatial pyramid pooling layer SPP, and 1 output layer; S3.1.

2. Input the training set obtained in step S2 into the basic model A, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the basic model A, and obtain the trained basic model A; The calculation formula of the loss function of the basic model A is: ； in, is the loss function of the basic model A; is the total number of data samples in the basic model A; For the ia The true labels of samples, Based on the prediction of model A ia The probability that a sample is class 1; S3.1.

3. The node output values ​​of the four output units corresponding to the basic model A are a 1 , a 2 , a 3 , a 4 , the node output values ​​of the four output units are initially converted to obtain the output value The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ； in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S3.1.

4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the basic model A. After the training is completed, the output values ​​of the four types of diseases corresponding to the basic model A are obtained, which are recorded as , , , ; S3.

2. Constructing the first auxiliary model for rapid identification of road congestion: S3.2.

1. The first auxiliary model B consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; S3.2.

2. Input the training set obtained in step S2 into the first auxiliary model B, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the first auxiliary model B, and obtain the trained first auxiliary model B; The calculation formula of the loss function of the first auxiliary model B is: ； in, is the loss function of the first auxiliary model B; is the total number of data samples in the first auxiliary model B; For the ib The true labels of samples, The first auxiliary model B predicts ib The probability that a sample is class 1; S3.2.

3. The node output values ​​of the four output units corresponding to the first auxiliary model B are b 1 , b 2 , b 3 , b 4 , the node output values ​​of the four output units are initially converted to obtain the output value b i The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value b i The corresponding enhancement probability value The calculation formula is: ； S3.2.

4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the first auxiliary model B. After the training is completed, the disease output value corresponding to the first auxiliary model B is obtained as , among the 4 nodes in the output layer, the actual disease corresponding to each node is marked as , i =1,2,3,4; S3.

3. Constructing the second auxiliary model for rapid identification of road congestion: S3.3.

1. The second auxiliary model C consists of 1 input layer, 12 convolutional layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; S3.3.

2. Input the training set obtained in step S2 into the second auxiliary model C, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the second auxiliary model C, and obtain the trained second auxiliary model C; The calculation formula of the loss function of the second auxiliary model C is: ; in, is the loss function of the second auxiliary model C; is the total number of data samples in the second auxiliary model C; For the ic The true labels of samples, The second auxiliary model C predicts ic The probability that a sample is class 1; S3.3.

3. The node output values ​​of the four output units corresponding to the second auxiliary model C are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ; Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ； in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S3.3.

4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the second auxiliary model C. After the training is completed, the output values ​​of the four types of diseases corresponding to the second auxiliary model C are obtained as follows: ; S3.

4. Construct an enhanced model for rapid identification of road congestion: S3.4.

1. The enhanced model D consists of 1 input layer, 14 convolutional layers, 14 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; S3.4.

2. Input the training set obtained in step S2 into the enhanced model D, calculate the weights and biases of the neural network units through the error back propagation algorithm, complete the training of the enhanced model D, and obtain the trained enhanced model D; The loss function of the enhanced model D is calculated as: ； in, is the loss function of the enhanced model D; is the total number of data samples in the reinforcement model D; For the id The true labels of samples, To strengthen the model D to predict the id The probability that a sample is class 1; S3.4.

3. The node output values ​​of the four output units corresponding to the enhanced model D are , the node output values ​​of the four output units are initially converted to obtain The corresponding probability value The calculation formula is: ； Then the node output values ​​of the four output units are further optimized and converted to obtain the output value The corresponding enhancement probability value The calculation formula is: ； in, G is the enhancement coefficient, and then the actual disease corresponding to each node is marked as ; S3.4.

4. Adoption The training set obtained in step S2 is combined with the error chain derivation rule to train the enhanced model D. After the training is completed, the output values ​​of the four types of diseases corresponding to the enhanced model D are obtained as follows: .

4. According to claim 3, a method for evaluating driving performance under road congestion based on digital twins, It is characterized in that The specific implementation method of step S4 includes the following steps: S4.

1. Based on and , establish the loss function between the first auxiliary model B and the output layer of the basic model A, and the calculation formula is: ； based on and , construct the output value The corresponding probability value The loss function of the base model A , the calculation formula is: ； based on and , establish the relationship loss function between the output layer of the first auxiliary model B and the basic model A , the calculation formula is: ； in, for The weight coefficient of for The weight coefficient of S4.

2. Establish the relationship loss function between the output layer of the second auxiliary model C and the first auxiliary model B , the expression is: ； in, for The weight coefficient of for The weight coefficient of Based on and The loss function of Based on and The loss function of The calculation formula is: ； The calculation formula is: ； S4.

3. Establish the relationship loss function between the output layer of the second auxiliary model C and the basic model A , the expression is: ； in, for The weight coefficient of for The weight coefficient of Based on and The loss function is Based on and The loss function of The calculation formula is: ； The calculation formula is: ； S4.

4. Establishing the relationship loss function between the output layer of the enhanced model D and the base model A , the expression is: ； in, for The weight coefficient of for The weight coefficient of Based on and The loss function of The calculation formula is: ； S4.

5. Establish the relationship loss function between the output layer of the reinforcement model D and the second auxiliary model C , the expression is: ； in, for The weight coefficient of for The weight coefficient of Based on and The loss function is Based on and The loss function of The calculation formula is: ； The calculation formula is: 。 5. According to claim 4, a method for evaluating driving performance under road congestion based on digital twins, It is characterized in that The specific implementation method of step S5 includes the following steps: S5.

1. Assuming a small model a and large models d , build a small model a The second, third, fourth and output layers in the large model are d Establish connections between the fourth, sixth, tenth layers and the output layer; S5.

2. For the second layer in the small model and the large model d The fourth layer in the small model establishes relationships a The output of the second layer is recorded as , the corresponding size of width × length × number of channels is Large Model d The output of the fourth layer is recorded as , the corresponding size of width × length × number of channels is ; For size comparison, When , a convolution operation is added to the second layer output of the small model, the convolution kernel size is 1×1, and the number of convolution kernels is , the output is ,at this time, The corresponding output values ​​of each channel are , k i is the channel number, The corresponding output values ​​of each channel are ; Build on Small model a The second floor with the large model d The loss function between the fourth layers , the calculation formula is: ； when When in the large model d In the output of the fourth layer, a convolution operation is added, the convolution kernel size is 1×1, and the number of convolution kernels is , the output is ;at this time, The corresponding output values ​​of each channel are: , The corresponding output values ​​of each channel are ; Establish Small model a The second floor with the large model d The loss function between the fourth layers , the calculation formula is: ； S5.

3. For the third layer in the small model and the large model b The sixth layer in the relationship is established to obtain a small model a The third floor and the large model b Loss function between the sixth layers ; S5.

4. For the fourth layer in the small model and the large model b The tenth layer in the relationship is established to obtain a small model a Fourth floor and large model b Loss function between the tenth layer ; S5.

5. When the small model is the base model A, the small model a The second, third, and fourth layers in the graph are the output of pooling layer 1, pooling layer 2, and pooling layer 4 of the basic model A, respectively. When the large model is the enhanced model D, the large model d The fourth, sixth, and tenth layers in are the output of pooling layer 2, the output of pooling layer 3, and the output of pooling layer 5 of the enhanced model D respectively; S5.

6. Based on the loss function between the output layer of the enhanced model D and the output layer of the base model A and the inter-layer loss function, define the total loss function between models , the calculation formula is: ； in, for The corresponding weight coefficient is, for The corresponding weight coefficient is, for The corresponding weight coefficient.

6. According to claim 5, a method for evaluating driving performance under road congestion based on digital twins, It is characterized in that The specific implementation method of step S6 is to use the data set obtained in step S2 based on the chain derivative rule to train the basic model, the first auxiliary model, the second auxiliary model and the enhanced model through error back propagation. When the overall loss function When is minimum, the corresponding model parameters are optimal, and the trained basic model is obtained as a road congestion rapid recognition model.

7. According to claim 6, a method for evaluating driving performance under road congestion based on digital twins, It is characterized in that The specific implementation method of step S7 includes the following steps: S7.

1. For the road congestion rapid identification model obtained in step S6, set the enhancement coefficient G The values ​​are: G =1,3,5,7,9,11,15,20,30,40,50; S7.

2. Based on different enhancement factors G 11 working conditions are obtained, and 11 models are obtained by training the road congestion rapid recognition model. The corresponding accuracy rates of the 11 road congestion rapid recognition models are as follows: , i=1,2,3,…,11, the calculation formula is: ； in, Represents the first i The number of samples that a model predicts as having a bag and actually has a bag when identifying the data in the test set; Represents the first i The number of samples that a model predicts as having a packet but is not actually having a packet when identifying the data in the test set; Comparison The maximum value in ; The corresponding road surface congestion rapid recognition model is the final lightweight road surface congestion rapid recognition model.

8. According to claim 7, a method for evaluating driving performance under road congestion based on digital twins, It is characterized in that The specific implementation method of step S8 includes the following steps: S8.

1. The lightweight road congestion rapid identification model obtained in step S7 is exported as a deployment file for independent services, including a pt file, a pth file, a json file, or a txt file; S8.

2. Construct a road information model; S8.2.

1. Collect information on the location, length, width, and layout of roads; S8.2.

2. The data obtained in step S8.2.1 is stored in a database, and a basic three-dimensional model of the road is constructed, wherein the basic three-dimensional model of the road includes geometric information of the road; S8.2.

3. Based on step S8.2.2, material properties, design specifications, and historical maintenance data are added to the basic three-dimensional model of the road to obtain a road information model; S8.2.

4. Export the road information model to a format file supported by the web application, including a json file or a js file; S8.

3. In the digital twin platform, import the deployment file of the lightweight road congestion rapid identification model obtained in step S7.1, and import the road information model format file obtained in step S7.2; S8.

4. Use Python to construct an API interface between image data and a lightweight road bump rapid identification model for intelligent rutting identification. Use Python to construct an API interface between a lightweight road bump rapid identification model and a road information model to complete the construction of a digital twin model.

9. According to claim 8, a method for evaluating driving performance under road congestion based on digital twins, It is characterized in that The specific implementation method of step S9 includes the following steps: S9.

1. Use a 3D structured light camera to collect road surface data and upload it to the digital twin model established in step S7. Then use the lightweight road bump rapid identification model deployed in the system to intelligently identify the road bump area in the road image, select all images containing road bumps, conduct on-site review of the bumps, and measure the area GM and height BD of the bumps. S9.

2. Establish driving evaluation indicators, including driving comfort evaluation indicators and driving safety evaluation indicators: Driving comfort evaluation indicators include road surface flatness PZ, driving vibration index CZ, noise level ZY, and in-vehicle vibration amplitude ZD; Driving safety evaluation indicators include accident rate SGL, emergency braking frequency ZP, and vehicle handling stability index CW; S9.

3. Select 8 time nodes t1, t2, t3, ... t8, and collect the values ​​of the corresponding driving evaluation indicators at the 8 time nodes: S9.

4. Based on the values ​​of each driving evaluation index collected in step S9.3, calculate the average value of each driving evaluation index, including the average value of road surface smoothness , average value of driving vibration index , average noise level , Average value of vibration amplitude inside the vehicle , average accident rate , Average emergency braking frequency , vehicle handling stability index average , Average area of ​​the package , Average value of bag height ; S9.

5. Construct the correlation between the area and height of the enclosed area and each driving evaluation index , the calculation formula is as follows; ； in, Any one of the eight time nodes; ZB represents the evaluation index, including road surface smoothness, driving vibration index, noise level, in-vehicle vibration amplitude, accident rate, emergency braking frequency, and vehicle handling stability index. That is, the correlation between the enveloping area and enveloping height and road surface smoothness, driving vibration index, noise level, in-vehicle vibration amplitude, accident rate, emergency braking frequency, and vehicle handling stability index is respectively , , , , , , ; S9.

6. Correlation between driving comfort evaluation indicators: road surface smoothness, driving vibration index, noise level, and in-vehicle vibration amplitude , , , , compare and obtain the maximum value therein, and then determine the driving comfort evaluation index having an influence based on the maximum value; For driving safety indicators: Compare the correlation between accident rate SGL, emergency braking frequency ZP, and vehicle handling stability index CW , , , compare and obtain the maximum value therein, and then determine the driving safety evaluation index having an impact based on the maximum value; S9.

7. Set a linear correlation equation between the driving comfort evaluation index and time, and then solve the equation using the threshold of the driving comfort evaluation index to obtain the life value of the driving comfort evaluation index. ; A linear correlation equation between the driving safety evaluation index and time is set, and then the equation is solved by using the threshold value of the driving safety evaluation index to obtain the life value of the driving safety evaluation index; Compare and ,choose and The minimum value is recorded as tmin. When the road operation time reaches tmin, the road surface is repaired to ensure that the road service status meets the requirements of comfort and safety.

Images