A traffic congestion prediction method based on drone and vehicle driving status

Abstract

The present invention discloses a traffic congestion prediction method based on unmanned aerial vehicles and vehicle driving status, and belongs to the technical field of traffic congestion prediction. The problem that the traditional traffic congestion prediction method in the prior art is difficult to achieve traffic congestion prediction with high real-time performance covering a wide area is solved; the present invention collects images of various types of vehicles traveling on the road to generate an image matrix, constructs a data set, inputs the data set into the three established models for training, obtains the fused loss function, trains the three models according to the intersection-and-union optimization method, uses the trained basic model as the final model, obtains the final optimal model by screening the accuracy of the optimal model, and outputs the vehicle driving status and type; combines the collected morning and evening peak traffic flow statistics information to perform traffic congestion prediction, and obtains the predicted results of the road sections prone to congestion. The present invention realizes a more comprehensive traffic congestion prediction, and can be applied to predict traffic conditions.

Description

Technical Field

[0001] The present invention relates to a traffic congestion prediction method, in particular to a traffic congestion prediction method based on the driving status of an unmanned aerial vehicle and a vehicle, and belongs to the technical field of traffic congestion prediction. Background Art

[0002] Traffic congestion prediction can significantly improve traffic efficiency. Through real-time data analysis, it can predict and manage road usage, avoid traffic paralysis, and reduce commuting time. Secondly, effective congestion prediction can improve the response speed of emergency services and ensure that ambulances, fire trucks, etc. can reach their destination quickly and unimpeded in emergency situations. In addition, high emissions caused by parking and low-speed driving can be significantly reduced by optimizing traffic flow. Traffic congestion prediction helps reduce environmental pollution.

[0003] Traffic congestion prediction based on drones and vehicle driving status has unique advantages and far-reaching significance. Drones provide the ability to observe traffic flow from the air, can cover a wide area and capture traffic conditions in real time. The monitoring perspective of drones is a major supplement to traditional ground monitoring systems. Combined with vehicle driving data such as speed and location, traffic congestion can be analyzed and predicted more accurately, making traffic management more dynamic and responsive, and can adjust traffic signals in time and optimize route planning, thereby effectively reducing traffic delays and avoiding accidents. In addition, reducing congestion can also help reduce automobile exhaust emissions, play a positive role in environmental protection, and have significant economic benefits, because reducing stagnation time can improve production efficiency and reduce fuel consumption and transportation costs.

[0004] In the prior art 1, the patent document with the publication (announcement) number CN103366563B discloses a traffic congestion prediction method and a traffic congestion prediction device, which first receives the current location and destination information of the vehicle, predicts the vehicle's driving path, and then calculates the passing time of the vehicle at a specific location on the predicted path, and based on these times, counts the number of vehicles on each road section. Using these data, the device calculates the expected passing time of each vehicle at each location; in the prior art 2, the patent document with the publication (announcement) number CN113469425A discloses a deep traffic congestion prediction method, whose network server stores and processes historical traffic data, establishes road section flow vectors, converts the traffic network into a topological graph structure, and screens key road section data based on node degree information. For missing data road section, which includes a data completion step to ensure the integrity of the information. Finally, a deep learning model is used to predict congestion on the completed data set, thereby improving the accuracy of the prediction and reducing communication overhead; however, the above-mentioned traffic congestion prediction method mainly relies on historical data and vehicle location information. Although it can provide a certain degree of prediction accuracy, it fails to achieve real-time traffic congestion prediction covering a wide area due to the limitations of traditional ground monitoring for image collection. Image recognition and drone monitoring are not considered to enhance the prediction ability and data collection efficiency. Image recognition technology can analyze vehicle density and behavior in real time and provide more detailed data for the prediction model, while drone technology can perform efficient aerial monitoring in wider or difficult-to-reach areas and quickly obtain a panoramic view of the traffic situation.

[0005] Therefore, a traffic congestion prediction method based on the driving status of drones and vehicles is needed. Summary of the invention

[0006] A brief overview of the present invention is provided below in order to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.

[0007] In view of this, in order to solve the problem that traditional traffic congestion prediction methods in the prior art are difficult to achieve traffic congestion prediction covering a wide area and with high real-time performance, the present invention provides a traffic congestion prediction method based on drones and vehicle driving status.

[0008] The technical solution is as follows: A traffic congestion prediction method based on drone and vehicle driving status includes the following steps:

[0009] S1. Collect images of various vehicles on the road to generate an image matrix, enhance the image quality, and obtain a new image matrix;

[0010] S2. equally divide the image in the new image matrix and update the image pixels to obtain an image after pixel update;

[0011] S3. construct a data set based on the image after pixel update, input the data set into the established basic model, auxiliary model and enhanced model for training, fuse the loss functions of the three models to obtain the fused loss function, train the three models, and obtain the three trained models;

[0012] S4. According to the intersection-over-union optimization method, the three trained models are trained, and the trained basic model is used as the final model. The final optimal model is obtained by screening the accuracy of the optimal model, and the driving status and type of the vehicle are output;

[0013] S5. By analyzing the driving status of vehicles and combining it with the collected traffic statistics during the morning and evening peak hours, traffic congestion prediction is carried out to obtain the predicted results of road sections prone to congestion.

[0014] Furthermore, the S1 specifically includes the following steps:

[0015] S11. Use the high-definition camera on the drone to collect images ph of various vehicles traveling on the road, and the corresponding image matrix is ​​PH;

[0016] S12. Construct a 5th-order denoising kernel function N, normalize all elements in the kernel function, and obtain the normalized denoising kernel function ;

[0017] S13. According to the normalized denoising kernel function , perform denoising kernel function processing on the image to obtain a new image matrix P to achieve image quality enhancement;

[0018] In S12, the fifth-order denoising kernel function N is expressed as:

[0019] ;

[0020] ;

[0021] Where x=1, 2, 3, 4 or 5, x is the row number of the matrix, and y=1, 2, 3, 4 or 5, y is the column number of the matrix. is the standard deviation of all elements in the kernel function, is the base of natural logarithms;

[0022] Normalized denoising kernel function It is expressed as:

[0023] ;

[0024] In S13, the new image matrix P is expressed as:

[0025] .

[0026] Furthermore, the step S2 includes the following steps:

[0027] S21. According to the new image matrix P, the pixel grayscale value is divided into 16 grayscale levels, and the image in the new image matrix P is divided into n equal parts to obtain the divided image;

[0028] S22. Count the pixel frequencies corresponding to each gray level in the equally divided image , calculate the grayscale frequency cumulative value corresponding to each grayscale level in the equally divided image ;

[0029] S23.According to the gray frequency cumulative value , construct a pixel value update method and calculate the pixel update value corresponding to the gray level , get the image after pixel update;

[0030] In S21, the pixel grayscale value is 0-255, and the grayscale level is , un=1, 2, 3, ..., 16, un is the grayscale number, the image after equal division is Pnum, num=1, 2, ..., n;

[0031] In S22, each gray level in the equally divided image Pnum The corresponding gray frequency cumulative value It is expressed as:

[0032] ;

[0033] Where no is the gray level number less than or equal to un, Indicates the gray level with gray level number no;

[0034] In S23, the pixel update value corresponding to the gray level It is expressed as:

[0035] ;

[0036] in, is the minimum non-zero value among all pixel frequency accumulation values, is the total number of pixels corresponding to the equally divided image Pnum, is the maximum value of the gray level.

[0037] Furthermore, the step S3 specifically includes the following steps:

[0038] S31. Based on the pixel updated image, select all images including passenger cars, commercial vehicles, buses and special vehicles as input images;

[0039] S32. Select four types of vehicles, namely, passenger cars, commercial vehicles, buses and special vehicles, from the input image through manual annotation, construct a data set, and divide the data set into a training set and a test set according to a ratio of 8:2;

[0040] S33. According to the data set, a basic model, an auxiliary model and an enhanced model are constructed, and the loss functions of the three models are calculated and integrated to obtain a fused loss function;

[0041] In S32, passenger cars, commercial vehicles, buses and special vehicles are marked as Passenger, Commercial, Bus and Special respectively;

[0042] In the S33, a network structure of a basic model, i.e., a deep learning model A, is constructed. The deep learning model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer;

[0043] The data set is input into the deep learning model A, and the weights and biases of the neural network units of the deep learning model A are calculated through the error back propagation algorithm to complete the training of the deep learning model A and obtain the trained deep learning model A and its loss function La;

[0044] The output value of the output unit node corresponding to the deep learning model A is , respectively, are the output values ​​of the first output unit node , the output value of the second output unit node , the output value of the third output unit node and the output value of the fourth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model A are initially converted into the probability values ​​of the first preliminary output value and the first preliminary output value. , and then further optimize and transform it to obtain the first optimized output value and the enhanced probability value of the first optimized output value ;

[0045] The enhanced probability value using the first optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model A. After the training is completed, the output value of the first vehicle type is obtained The actual vehicle type label corresponding to the node of the output layer of the deep learning model A , the output value of the first vehicle type The output values ​​for the first passenger car are , the output value of the first commercial vehicle , the output value of the first bus and the output value of the first special vehicle ;

[0046] The first auxiliary model, deep learning model B, is constructed. Deep learning model B consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer.

[0047] The data set is input into the deep learning model B, and the weights and biases of the neural network units of the deep learning model B are calculated through the error back propagation algorithm, the training of the deep learning model B is completed, and the trained deep learning model B and its loss function Lb are obtained;

[0048] The output value of the output unit node corresponding to the deep learning model B is , respectively, are the output values ​​of the fifth output unit node , the output value of the sixth output unit node , the output value of the seventh output unit node and the output value of the eighth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model B are initially converted into the second preliminary output value and the probability value of the second preliminary output value , and then further optimize and transform it to obtain the second optimized output value and the enhanced probability value of the second optimized output value ;

[0049] The enhanced probability value using the second optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model B. After the training is completed, the output value of the second vehicle type is obtained The actual vehicle type label corresponding to the node of the output layer of the deep learning model B , the output value of the second vehicle type The output values ​​for the second passenger car are , output value of the second commercial vehicle , the output value of the second bus and the second special vehicle output value ;

[0050] The second auxiliary model, namely the deep learning model C, is constructed. The deep learning model C consists of 1 input layer, 12 convolutional layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer.

[0051] The data set is input into the deep learning model C, and the weights and biases of the neural network units of the deep learning model C are calculated through the error back propagation algorithm to complete the training of the deep learning model C and obtain the trained deep learning model C and its loss function Lc;

[0052] The output value of the output unit node corresponding to the deep learning model C is , respectively, are the output values ​​of the ninth output unit node , the output value of the tenth output unit node , the output value of the eleventh output unit node and the output value of the twelfth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model C are initially converted into the third preliminary output value and the probability value of the third preliminary output value , and then further optimize and transform it to obtain the third optimized output value and the enhanced probability value of the third optimized output value ;

[0053] The enhanced probability value using the third optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model C. After the training is completed, the output value of the third vehicle type is obtained The actual vehicle type label corresponding to the node of the output layer of the deep learning model B , the output value of the third vehicle type The output values ​​for the third passenger car are , Output value of third commercial vehicle , the output value of the third bus and the third special vehicle output value ;

[0054] Construct a reinforcement model, namely the deep learning model D, which consists of 1 input layer, 14 convolutional layers, 14 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer;

[0055] The data set is input into the deep learning model D, and the weights and biases of the neural network units of the deep learning model D are calculated through the error back propagation algorithm to complete the training of the deep learning model D and obtain the trained deep learning model D and its loss function Ld;

[0056] The output value of the output unit node corresponding to the deep learning model D is , respectively, are the output values ​​of the thirteenth output unit node , the output value of the fourteenth output unit node , the output value of the fifteenth output unit node and the output value of the sixteenth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model D are initially converted into the fourth preliminary output value and the probability value of the fourth preliminary output value , and then further optimize and transform it to obtain the fourth optimized output value and the enhanced probability value of the fourth optimized output value ;

[0057] The enhanced probability value using the fourth optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model D. After the training is completed, the output value of the fourth vehicle type is obtained The actual vehicle type annotations corresponding to the nodes in the output layer of the deep learning model D , output value of the fourth vehicle type The output values ​​for the fourth passenger car are , Output value of the fourth commercial vehicle , the output value of the fourth bus and the output value of the fourth special vehicle ;

[0058] The output value is based on the second vehicle type and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model B and deep learning model A , get the corresponding weight coefficient;

[0059] Probability value based on the first preliminary output value The actual vehicle type label corresponding to the node of the output layer of the deep learning model A , construct the probability value considering the first preliminary output value The loss function of the deep learning model A is , get the corresponding weight coefficient;

[0060] Based on the loss function between deep learning model B and deep learning model A and the probability value considering the first preliminary output value The loss function of the deep learning model A is , establish the relationship loss function between the output layer of deep learning model B and deep learning model A ;

[0061] According to the above operations, the relationship loss function between the output layers of deep learning model C and deep learning model B is established. , the relationship loss function between the output layer of deep learning model C and deep learning model A , the relationship loss function between the output layer of deep learning model D and deep learning model A The relationship loss function between the output layers of deep learning model D and deep learning model C .

[0062] Furthermore, the step S4 specifically includes the following steps:

[0063] S41. For each target prediction bounding box and the corresponding true bounding box of the three trained models, calculate the intersection area between the two ;

[0064] S42. For each target prediction bounding box and the corresponding true bounding box of the three trained models, calculate the union area of ​​the two ;

[0065] S43. Calculate the intersection area and the area of ​​the union Corresponding intersection-over-union ratio IoU, introducing adjustment coefficient , construct the loss function of the target position , get the optimized intersection-over-union ratio IoU;

[0066] S44. According to the optimized intersection-over-union ratio IoU, the optimized intersection-over-union ratio calculation method is used to train the model, and the optimal model with the highest accuracy is selected to obtain the final optimal model, and the vehicle driving status and type are output;

[0067] In S43, the intersection over union (IoU) is expressed as:

[0068] ;

[0069] Loss function for target location It is expressed as:

[0070] ;

[0071] Set the intersection-over-union ratio threshold , according to the intersection ratio Intersection-over-union threshold The relationship between range, when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding adjustment coefficient of the loss function ;

[0072] In S44, the corresponding overall loss function between deep learning model A, deep learning model B, deep learning model C and deep learning model D is defined. , and get 32 ​​overall loss functions In the form of, the three models are trained, and 32 deep learning models A trained with the overall loss function are obtained as the optimal model;

[0073] Overall loss function It is expressed as:

[0074] ;

[0075] in, is the relationship loss function between the output layers of deep learning model D and deep learning model C The weight coefficient is 0 or 1. is the relationship loss function between the output layers of deep learning model C and deep learning model B The weight coefficient is 0 or 1. is the relationship loss function between the output layers of deep learning model B and deep learning model A The weight coefficient is 0 or 1. is the relationship loss function between the output layer of deep learning model D and deep learning model A The weight coefficient is 0 or 1. is the relationship loss function between the output layers of deep learning model D and deep learning model C The weight coefficient of is 0 or 1;

[0076] The accuracy of the deep learning model A trained with the overall loss function is recorded as , i =1, 2, 3, ..., 32, calculate the maximum value of the accuracy of the deep learning model A trained with the overall loss function , , select the maximum value The corresponding deep learning model A is the final optimal model.

[0077] Furthermore, the S5 specifically includes the following steps:

[0078] S51. According to the output vehicle driving state, obtain the center position CP of the vehicle and record the time corresponding to the current frame image , based on the current frame image, obtain the predicted vehicle driving state at any time;

[0079] S52. Determine the road sections prone to congestion based on the predicted vehicle driving status at any time and the collected traffic statistics during the morning and evening peaks;

[0080] S53. Based on the relationship between the position of the vehicle in the vehicle driving state and the position of the congested road section, the relationship between the speed of the vehicle in the vehicle driving state and the critical value of the vehicle speed in the congested road section, and the relationship between the total number of vehicles in the congested road section and the critical value of the number of vehicles in the congested road section, the congested road section is predicted and judged to obtain the predicted congested road section result;

[0081] In the S51, the vehicle driving state includes the vehicle position and speed;

[0082] Predicted The vehicle driving status of the next frame image at the moment It is expressed as:

[0083] ;

[0084] ;

[0085] in, is the state transfer matrix, is the transposed matrix of the state transfer matrix, is the estimated vehicle driving state of the current frame, is the control input matrix, is the control vector, is the covariance matrix of the predicted next frame image, is the covariance matrix of the current frame, is the process noise covariance;

[0086] According to the predicted The vehicle driving status of the next frame image at the moment , calculate the error between the measured value of the vehicle driving state and the predicted value of the vehicle driving state , and further get the updated The vehicle driving status of the next frame image at the moment ;

[0087] The error between the measured value of the vehicle's driving state and the predicted value of the vehicle's driving state It is expressed as:

[0088] ;

[0089] in, for The vehicle driving state of the next frame of image at the moment is the measurement value, is the measurement matrix;

[0090] Updated The vehicle driving status of the next frame image at the moment It is expressed as:

[0091] ;

[0092] ;

[0093] ;

[0094] in, is the Kalman gain, is the updated prediction covariance matrix, T is the matrix transpose;

[0095] Based on the above calculation, the predicted vehicle driving state at any time is obtained, which includes the predicted vehicle position RP and the predicted speed RV, where: , , is the position of the CM-th vehicle, , is the horizontal coordinate of the position of the CM-th vehicle, is the ordinate of the position of the CMth vehicle, CM is the total number of vehicles, is the speed of the CMth vehicle;

[0096] In S52, the road sections prone to congestion are all set as square areas, a two-dimensional coordinate system is established for the images of various types of vehicles traveling on the road, and the coordinates of the four corner points of the square area are measured, wherein the coordinates of the corner point with the smallest horizontal and vertical coordinates are , the coordinates of the corner point with the largest horizontal and vertical coordinates are ;

[0097] In the above S53, The vehicle position at the moment is recorded as , , For the The position of the CMth vehicle at the time, The speed at that moment is recorded as , , is the speed of the CMth car, The moment The position of the car is , ,in, For the The moment The horizontal coordinate of the vehicle's position, For the The moment The vertical coordinate of the vehicle's position;

[0098] Compare The moment The horizontal coordinate of the vehicle's position and The moment The vertical coordinate of the vehicle's position The relationship between the location of the congested road section and the square area: and When The moment The vehicle is in the congested section area, and the vehicles that meet the above conditions are retained. If the above conditions are not met, it means that the corresponding vehicle has not entered the congested section and is not considered. All CM vehicles are traversed and the vehicles in the first All vehicles entering the congested section at any time, the total number of vehicles entering the congested section is DM, and the position matrix formed by all vehicles entering the congested section is , the speed matrix corresponding to all vehicles entering the congested road section is ;

[0099] According to the set critical value of the number of vehicles in the congested road section and the critical speed of vehicles in congested sections Identify the congested sections and compare the total number of vehicles entering the congested sections DM with the critical number of vehicles in the congested sections relationship,

[0100] when When There was no congestion at all times. When the average speed of all vehicles in the congested section is determined The critical value of vehicle speed in congested road sections relationship, if , indicating that the current section is in There is no congestion at all times. , indicating that the current section is in Congestion occurs all the time.

[0101] Furthermore, in S33, the layers in the network structure of the deep learning model A are connected in the following order:

[0102] Input layer: image size 227×227×3;

[0103] Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization;

[0104] Activation layer 1: LeakyReLU;

[0105] Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0106] Activation layer 2: LeakyReLU;

[0107] Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization;

[0108] Activation layer 3: LeakyReLU;

[0109] Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0110] Activation layer 4: LeakyReLU;

[0111] SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model;

[0112] Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1;

[0113] Pooling layer 2: Type: global average pooling, pooling size: 9×9, Stride: 1;

[0114] Pooling layer 3: Type: global average pooling, pooling size: 13×13, Stride: 1;

[0115] Pooling concatenation layer: connect the outputs of the three pooling layers;

[0116] PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations;

[0117] Convolutional layer 5: kernel size 3×3, number of kernels 768, stride: 2, batch normalization;

[0118] Activation layer 5: LeakyReLU;

[0119] Convolutional layer 6: kernel size 3×3, number of kernels 768, stride: 2, batch normalization;

[0120] Activation layer 6: LeakyReLU;

[0121] Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer;

[0122] Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer;

[0123] Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0124] Activation layer 7: LeakyReLU;

[0125] Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0126] Activation layer 8: LeakyReLU;

[0127] Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0128] Activation layer 9: LeakyReLU;

[0129] Output layer: Type: Softmax, Number of neurons: 4.

[0130] Furthermore, in S33, the layers in the network structure of the deep learning model B are connected in the following order:

[0131] Input layer: image size 227×227×3;

[0132] Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization;

[0133] Activation layer 1: LeakyReLU;

[0134] Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0135] Activation layer 2: LeakyReLU;

[0136] Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization;

[0137] Activation layer 3: LeakyReLU;

[0138] Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0139] Activation layer 4: LeakyReLU;

[0140] SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model;

[0141] Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1;

[0142] Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1;

[0143] Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1;

[0144] Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1;

[0145] Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1;

[0146] Pooling concatenation layer: connect the outputs of the five pooling layers;

[0147] PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations;

[0148] Convolution layer 5: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0149] Activation layer 5: LeakyReLU;

[0150] Convolutional layer 6: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0151] Activation layer 6: LeakyReLU;

[0152] Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer;

[0153] Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer;

[0154] Convolutional layer 7: kernel size 3×3, number of kernels 512, stride: 1, batch normalization;

[0155] Activation layer 7: LeakyReLU;

[0156] Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0157] Activation layer 7: LeakyReLU;

[0158] Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0159] Activation layer 8: LeakyReLU;

[0160] Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0161] Activation layer 9: LeakyReLU;

[0162] Output layer: Type: Softmax, Number of neurons: 4.

[0163] Furthermore, in S33, the layers in the network structure of the deep learning model C are connected in the following order:

[0164] Input layer: image size 227×227×3;

[0165] Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization;

[0166] Activation layer 1: LeakyReLU;

[0167] Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0168] Activation layer 2: LeakyReLU;

[0169] Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization;

[0170] Activation layer 3: LeakyReLU;

[0171] Convolutional layer 4: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0172] Activation layer 4: LeakyReLU;

[0173] Convolutional layer 5: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0174] Activation layer 5: LeakyReLU;

[0175] Convolutional layer 6: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0176] Activation layer 6: LeakyReLU;

[0177] SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model;

[0178] Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1;

[0179] Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1;

[0180] Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1;

[0181] Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1;

[0182] Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1;

[0183] Pooling concatenation layer: connect the outputs of the five pooling layers;

[0184] PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations;

[0185] Convolutional layer 7: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0186] Activation layer 7: LeakyReLU;

[0187] Convolutional layer 8: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0188] Activation layer 8: LeakyReLU;

[0189] Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer;

[0190] Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer;

[0191] Convolutional layer 9: kernel size 3×3, number of kernels 512, stride: 1, batch normalization;

[0192] Activation layer 9: LeakyReLU;

[0193] Convolution layer 10: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0194] Activation layer 10: LeakyReLU;

[0195] Convolutional layer 11: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0196] Activation layer 11: LeakyReLU;

[0197] Convolutional layer 12: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0198] Activation layer 12: LeakyReLU;

[0199] Output layer: Type: Softmax, Number of neurons: 4.

[0200] Furthermore, in S33, the layers in the network structure of the deep learning model D are connected in the following order:

[0201] Input layer: image size 227×227×3;

[0202] Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization;

[0203] Activation layer 1: LeakyReLU;

[0204] Convolutional layer 2: convolution kernel size 3×3, number of convolution kernels 32, stride: 1, batch normalization;

[0205] Activation layer 2: LeakyReLU;

[0206] Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 64, stride: 1, batch normalization;

[0207] Activation layer 3: LeakyReLU;

[0208] Convolutional layer 4: convolution kernel size 3×3, number of convolution kernels 64, stride: 1, batch normalization;

[0209] Activation layer 4: LeakyReLU;

[0210] Convolutional layer 5: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0211] Activation layer 5: LeakyReLU;

[0212] Convolutional layer 6: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0213] Activation layer 6: LeakyReLU;

[0214] Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0215] Activation layer 7: LeakyReLU;

[0216] Convolutional layer 8: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0217] Activation layer 8: LeakyReLU;

[0218] SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model;

[0219] Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1;

[0220] Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1;

[0221] Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1;

[0222] Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1;

[0223] Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1;

[0224] Pooling concatenation layer: connect the outputs of the five pooling layers;

[0225] PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations;

[0226] Convolutional layer 9: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0227] Activation layer 9: LeakyReLU;

[0228] Convolution layer 10: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0229] Activation layer 10: LeakyReLU;

[0230] Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer;

[0231] Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer;

[0232] Convolutional layer 11: kernel size 3×3, number of kernels 512, stride: 1, batch normalization;

[0233] Activation layer 11: LeakyReLU;

[0234] Convolutional layer 12: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0235] Activation layer 12: LeakyReLU;

[0236] Convolutional layer 13: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0237] Activation layer 13: LeakyReLU;

[0238] Convolutional layer 14: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0239] Activation layer 14: LeakyReLU;

[0240] Output layer: Type: Softmax, Number of neurons: 4.

[0241] The beneficial effects of the present invention are as follows: the present invention enhances the quality of images of various types of vehicles traveling on the road, improves the image clarity, and reduces image noise; the present invention updates the pixels of the image, and improves the contrast between the target vehicle area and the surrounding environment; the present invention optimizes the optimal basic model by integrating the intersection-and-union method, thereby realizing efficient and intelligent detection and recognition of vehicle categories, improving the detection accuracy of different types of vehicles, locating the exact position of the vehicle more quickly, and improving the adaptability and accuracy of the model to vehicle detection; the present invention distinguishes congested sections by setting a critical value of the number of vehicles in the congested section and a critical value of the speed of vehicles in the congested section, thereby realizing accurate prediction of the time and place of traffic congestion, and the images collected by the present invention are based on drone collection, covering a wider range of information. BRIEF DESCRIPTION OF THE DRAWINGS

[0242] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0243] Figure 1The figure is a flowchart of a traffic congestion prediction method based on UAV and vehicle driving status. DETAILED DESCRIPTION

[0244] In order to make the technical solutions and advantages of the embodiments of the present invention more clearly understood, the exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0245] refer to Figure 1 The present embodiment is described in detail, a traffic congestion prediction method based on the driving status of drones and vehicles, specifically comprising the following steps:

[0246] S1. Collect images of various vehicles on the road to generate an image matrix, enhance the image quality, and obtain a new image matrix;

[0247] S2. equally divide the image in the new image matrix and update the image pixels to obtain an image after pixel update;

[0248] S3. construct a data set based on the image after pixel update, input the data set into the established basic model, auxiliary model and enhanced model for training, fuse the loss functions of the three models to obtain the fused loss function, train the three models, and obtain the three trained models;

[0249] S4. According to the intersection-over-union optimization method, the three trained models are trained, and the trained basic model is used as the final model. The final optimal model is obtained by screening the accuracy of the optimal model, and the driving status and type of the vehicle are output;

[0250] S5. By analyzing the driving status of vehicles and combining it with the collected traffic statistics during the morning and evening peak hours, traffic congestion prediction is carried out to obtain the predicted results of road sections prone to congestion.

[0251] Furthermore, the S1 specifically includes the following steps:

[0252] S11. Use the high-definition camera on the drone to collect images ph of various vehicles traveling on the road, and the corresponding image matrix is ​​PH;

[0253] S12. Construct a 5th-order denoising kernel function N, normalize all elements in the kernel function, and obtain the normalized denoising kernel function ;

[0254] S13. According to the normalized denoising kernel function , perform denoising kernel function processing on the image to obtain a new image matrix P to achieve image quality enhancement;

[0255] In S12, the fifth-order denoising kernel function N is expressed as:

[0256] ;

[0257] ;

[0258] Where x=1, 2, 3, 4 or 5, x is the row number of the matrix, and y=1, 2, 3, 4 or 5, y is the column number of the matrix. is the standard deviation of all elements in the kernel function, is the base of natural logarithms;

[0259] Normalized denoising kernel function It is expressed as:

[0260] ;

[0261] In S13, the new image matrix P is expressed as:

[0262] .

[0263] Specifically, drones may be affected by wind or other environmental factors during flight, causing body vibrations, which in turn affects the stability and clarity of the image. The camera equipment of drones is limited by their size and cost, and is prone to noise under undesirable lighting conditions. Changes in the flight altitude and angle of the drone will also affect the angle and proportion of the image, increasing the complexity of image processing. Due to the combined effect of the above factors, the quality of images collected by drones is affected and the noise level is relatively high. Therefore, in order to improve the accuracy of vehicle category detection, the image quality is enhanced.

[0264] Furthermore, the step S2 includes the following steps:

[0265] S21. According to the new image matrix P, the pixel grayscale value is divided into 16 grayscale levels, and the image in the new image matrix P is divided into n equal parts to obtain the divided image;

[0266] S22. Count the pixel frequencies corresponding to each gray level in the equally divided image , calculate the grayscale frequency cumulative value corresponding to each grayscale level in the equally divided image ;

[0267] S23.According to the gray frequency cumulative value , construct a pixel value update method and calculate the pixel update value corresponding to the gray level , get the image after pixel update;

[0268] In S21, the pixel grayscale value is 0-255, and the grayscale level is , un=1, 2, 3, ..., 16, un is the grayscale number, the image after equal division is Pnum, num=1, 2, ..., n;

[0269] In S22, each gray level in the equally divided image Pnum The corresponding gray frequency cumulative value It is expressed as:

[0270] ;

[0271] Where no is the gray level number less than or equal to un, Indicates the gray level with gray level number no;

[0272] In S23, the pixel update value corresponding to the gray level It is expressed as:

[0273] ;

[0274] in, is the minimum non-zero value among all pixel frequency accumulation values, is the total number of pixels corresponding to the equally divided image Pnum, is the maximum value of the gray level.

[0275] Specifically, when drones collect vehicle images, the constantly changing lighting conditions, such as cloud cover, changes in sunlight angle, and weather changes, will affect the brightness and contrast of the image, reducing the distinction between the vehicle and the background. The flight altitude and angle changes of the drone also have an important impact on the image quality. Different shooting angles will cause perspective problems, such as obscured vehicles, further interfering with the recognition process. The technical limitations of drone cameras, especially in terms of dynamic range and color processing, will also lead to a decrease in the ability to capture details under complex lighting. Natural and man-made environmental factors, such as reflections and visual clutter, will interfere with the clarity and contrast of vehicle images. The above problems make it more difficult to identify vehicles from images collected by drones. Therefore, in order to obtain more accurate recognition results, the image pixels need to be updated.

[0276] Furthermore, the step S3 specifically includes the following steps:

[0277] S31. Based on the pixel updated image, select all images including passenger cars, commercial vehicles, buses and special vehicles as input images;

[0278] S32. Select four types of vehicles, namely, passenger cars, commercial vehicles, buses and special vehicles, from the input image through manual annotation, construct a data set, and divide the data set into a training set and a test set according to a ratio of 8:2;

[0279] S33. According to the data set, a basic model, an auxiliary model and an enhanced model are constructed, and the loss functions of the three models are calculated and integrated to obtain a fused loss function;

[0280] In S32, passenger cars, commercial vehicles, buses and special vehicles are marked as Passenger, Commercial, Bus and Special respectively;

[0281] In the S33, a network structure of a basic model, i.e., a deep learning model A, is constructed, and the deep learning model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer;

[0282] The data set is input into the deep learning model A, and the weights and biases of the neural network units of the deep learning model A are calculated through the error back propagation algorithm to complete the training of the deep learning model A and obtain the trained deep learning model A and its loss function La;

[0283] The loss function La of the trained deep learning model A is expressed as:

[0284] ;

[0285] Among them, Na is the total number of data samples in the data set of the deep learning model A, For the The true labels of samples, For deep learning model A, predict The probability that a sample is of category 1, is the natural logarithm;

[0286] The output value of the output unit node corresponding to the deep learning model A is , respectively, are the output values ​​of the first output unit node , the output value of the second output unit node , the output value of the third output unit node and the output value of the fourth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model A are initially converted into the probability values ​​of the first preliminary output value and the first preliminary output value. , and then further optimize and transform it to obtain the first optimized output value and the enhanced probability value of the first optimized output value ;

[0287] Probability value of the first preliminary output value It is expressed as:

[0288] ;

[0289] The enhanced probability value of the first optimized output value It is expressed as:

[0290] ;

[0291] in, G is the enhancement coefficient;

[0292] The enhanced probability value using the first optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model A. After the training is completed, the output value of the first vehicle type is obtained The actual vehicle type label corresponding to the node of the output layer of the deep learning model A , the output value of the first vehicle type The output values ​​for the first passenger car are , the output value of the first commercial vehicle , the output value of the first bus and the output value of the first special vehicle ;

[0293] The first auxiliary model, deep learning model B, is constructed. Deep learning model B consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer.

[0294] The data set is input into the deep learning model B, and the weights and biases of the neural network units of the deep learning model B are calculated through the error back propagation algorithm, the training of the deep learning model B is completed, and the trained deep learning model B and its loss function Lb are obtained;

[0295] The loss function Lb of the trained deep learning model B is expressed as:

[0296] ;

[0297] Among them, Nb is the total number of data samples in the data set of deep learning model B, For the The true labels of samples, For deep learning model B to predict The probability that a sample is class 1;

[0298] The output value of the output unit node corresponding to the deep learning model B is , respectively, are the output values ​​of the fifth output unit node , the output value of the sixth output unit node , the output value of the seventh output unit node and the output value of the eighth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model B are initially converted into the second preliminary output value and the probability value of the second preliminary output value , and then further optimize and transform it to obtain the second optimized output value and the enhanced probability value of the second optimized output value ;

[0299] Probability value of the second preliminary output value It is expressed as:

[0300] ;

[0301] The enhanced probability value of the second optimized output value It is expressed as:

[0302] ;

[0303] The enhanced probability value using the second optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model B. After the training is completed, the output value of the second vehicle type is obtained The actual vehicle type label corresponding to the node of the output layer of the deep learning model B , the output value of the second vehicle type The output values ​​for the second passenger car are , output value of the second commercial vehicle , the output value of the second bus and the second special vehicle output value ;

[0304] The second auxiliary model, namely the deep learning model C, is constructed. The deep learning model C consists of 1 input layer, 12 convolutional layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer.

[0305] The data set is input into the deep learning model C, and the weights and biases of the neural network units of the deep learning model C are calculated through the error back propagation algorithm to complete the training of the deep learning model C and obtain the trained deep learning model C and its loss function Lc;

[0306] The loss function Lc of the trained deep learning model C is expressed as:

[0307] ;

[0308] Among them, Nc is the total number of data samples in the data set of the deep learning model C, For the The true labels of samples, For the deep learning model C to predict The probability that a sample is class 1;

[0309] The output value of the output unit node corresponding to the deep learning model C is , respectively, are the output values ​​of the ninth output unit node , the output value of the tenth output unit node , the output value of the eleventh output unit node and the output value of the twelfth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model C are initially converted into the third preliminary output value and the probability value of the third preliminary output value , and then further optimize and transform it to obtain the third optimized output value and the enhanced probability value of the third optimized output value ;

[0310] The probability value of the third preliminary output value It is expressed as:

[0311] ;

[0312] The enhanced probability value of the third optimized output value It is expressed as:

[0313] ;

[0314] The enhanced probability value using the third optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model C. After the training is completed, the output value of the third vehicle type is obtained The actual vehicle type label corresponding to the node of the output layer of the deep learning model B , the output value of the third vehicle type The output values ​​for the third passenger car are , Output value of third commercial vehicle , the output value of the third bus and the third special vehicle output value ;

[0315] Construct a reinforcement model, namely the deep learning model D, which consists of 1 input layer, 14 convolutional layers, 14 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer;

[0316] The data set is input into the deep learning model D, and the weights and biases of the neural network units of the deep learning model D are calculated through the error back propagation algorithm to complete the training of the deep learning model D and obtain the trained deep learning model D and its loss function Lb;

[0317] The loss function Lb of the trained deep learning model D is expressed as:

[0318] ;

[0319] Among them, Nb is the total number of data samples in the data set of the deep learning model D, For the The true labels of samples, For the deep learning model D to predict The probability that a sample is class 1;

[0320] The output value of the output unit node corresponding to the deep learning model D is , respectively, are the output values ​​of the thirteenth output unit node , the output value of the fourteenth output unit node , the output value of the fifteenth output unit node and the output value of the sixteenth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model D are initially converted into the fourth preliminary output value and the probability value of the fourth preliminary output value , and then further optimize and transform it to obtain the fourth optimized output value and the enhanced probability value of the fourth optimized output value ;

[0321] Probability value of the fourth preliminary output value It is expressed as:

[0322] ;

[0323] Enhanced probability value of the fourth optimized output value It is expressed as:

[0324] ;

[0325] The enhanced probability value using the fourth optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model D. After the training is completed, the output value of the fourth vehicle type is obtained The actual vehicle type annotations corresponding to the nodes in the output layer of the deep learning model D , output value of the fourth vehicle type The output values ​​for the fourth passenger car are , Output value of the fourth commercial vehicle , the output value of the fourth bus and the output value of the fourth special vehicle ;

[0326] The output value is based on the second vehicle type and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model B and deep learning model A , get the corresponding weight coefficient;

[0327] Loss function between deep learning model B and deep learning model A It is expressed as:

[0328] ;

[0329] Probability value based on the first preliminary output value The actual vehicle type label corresponding to the node of the output layer of the deep learning model A , construct the probability value considering the first preliminary output value The loss function of the deep learning model A is , get the corresponding weight coefficient;

[0330] Consider the probability value of the first preliminary output value The loss function of the deep learning model A is It is expressed as:

[0331] ;

[0332] Based on the loss function between deep learning model B and deep learning model A and the probability value considering the first preliminary output value The loss function of the deep learning model A is , establish the relationship loss function between the output layer of deep learning model B and deep learning model A ;

[0333] The relationship loss function between the output layer of deep learning model B and deep learning model A It is expressed as:

[0334] ;

[0335] in, is the loss function between deep learning model B and deep learning model A The weight coefficient of is the probability value considering the first preliminary output value The loss function of the deep learning model A is The weight coefficient of

[0336] According to the above operations, the relationship loss function between the output layers of deep learning model C and deep learning model B is established. , the relationship loss function between the output layer of deep learning model C and deep learning model A , the relationship loss function between the output layer of deep learning model D and deep learning model A The relationship loss function between the output layers of deep learning model D and deep learning model C ;

[0337] Loss function between deep learning model C and deep learning model B It is expressed as:

[0338] ;

[0339] Consider the probability value of the second preliminary output value The loss function of the deep learning model B is It is expressed as:

[0340] ;

[0341] The relationship loss function between the output layers of deep learning model C and deep learning model B It is expressed as:

[0342] ;

[0343] in, is the loss function between deep learning model C and deep learning model B The weight coefficient of is the probability value considering the second preliminary output value The loss function of the deep learning model B is The weight coefficient of

[0344] Loss function between deep learning model C and deep learning model A It is expressed as:

[0345] ;

[0346] The relationship loss function between the output layer of deep learning model C and deep learning model A It is expressed as:

[0347] ;

[0348] in, is the loss function between deep learning model C and deep learning model A The weight coefficient of

[0349] Loss function between deep learning model D and deep learning model A It is expressed as:

[0350] ;

[0351] The relationship loss function between the output layers of deep learning model D and deep learning model C It is expressed as:

[0352] ;

[0353] in, is the loss function between deep learning model D and deep learning model A The weight coefficient of

[0354] Loss function between deep learning model D and deep learning model C It is expressed as:

[0355] ;

[0356] Consider the probability value of the third preliminary output value The loss function of the deep learning model C It is expressed as:

[0357]

[0358] The relationship loss function between the output layers of deep learning model D and deep learning model C It is expressed as:

[0359]

[0360] in, is the loss function between deep learning model D and deep learning model C The weight coefficient of

[0361] Specifically, in the vehicle images collected by drones, different types of vehicles have differences in size features, shape features, color features and logos. It is necessary to design the deep learning model structure based on the characteristics of different vehicle types. The designed model includes input layer, convolution layer, activation layer, pooling layer, merging layer, upsampling layer, PANet layer, pooling splicing layer, SPP layer and output layer. The functions of each layer in the structure are as follows:

[0362] Input layer: receives image data collected by drones and provides the network with raw pixel value input; Convolution layer: identifies basic image features of vehicles, such as edges and corners. Further deep convolution layers can identify complex features such as windows and contours; Activation layer: enables the model to handle nonlinear problems, such as distinguishing different models from various angles and conditions; Pooling layer: reduces the spatial size of data, improves computational efficiency, and makes feature detection more robust, helping the model maintain feature invariance in vehicle images of different sizes and shapes; Merging layer: merges feature maps from different paths, increases the information flow of the network, and combines features of different scales, which helps to process local details and overall structures of vehicles at the same time; Upsampling layer: enlarges the size of feature maps for restoration to higher resolution. When performing vehicle classification and detection, it helps to restore detail information and improve the accuracy of boundaries; PANet layer: enhances feature pyramids and feature fusion to improve the utilization of multi-scale features. Effectively integrate features of different scales to improve the recognition ability of small vehicles and long-distance vehicles; Pooling splicing layer: Combine multiple pooling outputs of different sizes to enhance the model's adaptability to scale, and enhance the recognition ability of vehicle size and shape through fields of view of different sizes; SPP layer: Allow the network to process input images of different sizes, ensure that the model can work effectively for vehicle images of different sizes and proportions, and enhance scale invariance; Output layer: Output key information such as vehicle category, location, and size based on the feature processing of the previous layer.

[0363] Furthermore, the step S4 specifically includes the following steps:

[0364] S41. For each target prediction bounding box and the corresponding true bounding box of the three trained models, calculate the intersection area between the two ;

[0365] S42. For each target prediction bounding box and the corresponding true bounding box of the three trained models, calculate the union area of ​​the two ;

[0366] S43. Calculate the intersection area and the area of ​​the union Corresponding intersection-over-union ratio IoU, introducing adjustment coefficient , construct the loss function of the target position , get the optimized intersection-over-union ratio IoU;

[0367] S44. According to the optimized intersection-over-union ratio IoU, the optimized intersection-over-union ratio calculation method is used to train the model, and the optimal model with the highest accuracy is selected to obtain the final optimal model, and the vehicle driving status and type are output;

[0368] In S43, the intersection over union (IoU) is expressed as:

[0369] ;

[0370] Loss function for target location It is expressed as:

[0371] ;

[0372] Set the intersection-over-union ratio threshold , according to the intersection ratio Intersection-over-union threshold The relationship between range, when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding adjustment coefficient of the loss function ;

[0373] In S44, the corresponding overall loss function between deep learning model A, deep learning model B, deep learning model C and deep learning model D is defined. , , and get 32 ​​overall loss functions In the form of, the three models are trained, and 32 deep learning models A trained with the overall loss function are obtained as the optimal model;

[0374] Overall loss function It is expressed as:

[0375] ;

[0376] in, is the relationship loss function between the output layers of deep learning model D and deep learning model C The weight coefficient is 0 or 1. is the relationship loss function between the output layers of deep learning model C and deep learning model B The weight coefficient is 0 or 1. is the relationship loss function between the output layers of deep learning model B and deep learning model A The weight coefficient is 0 or 1. is the relationship loss function between the output layer of deep learning model D and deep learning model A The weight coefficient is 0 or 1. is the relationship loss function between the output layers of deep learning model D and deep learning model C The weight coefficient of is 0 or 1;

[0377] The accuracy of the deep learning model A trained with the overall loss function is recorded as , i =1, 2, 3, ..., 32, calculate the maximum value of the accuracy of the deep learning model A trained with the overall loss function , , select the maximum value The corresponding deep learning model A is the final optimal model, which has the highest lightweight level and can be easily deployed in terminal devices such as edge devices, and can achieve fast and accurate identification of different types of vehicles;

[0378] Specifically, in the training of deep learning models, IoU is an important indicator to measure the accuracy of the model's positioning of objects in the image. It calculates the coverage area ratio between the bounding box predicted by the model and the actual bounding box. The coverage area ratio ranges from 0 (no coverage) to 1 (complete coverage). IoU is crucial for the performance adjustment of the model because it can screen out qualified predictions by setting a threshold value, and can be included in the loss function to guide the model to improve the accuracy of target detection during the learning process. In the early stage of model learning, due to the lack of sufficient training, the model often shows a low IoU value, which means that the match between the predicted box and the actual box is not high, and the optimization rate of the model is slow. When the model is trained to the later stage, although the IoU value is relatively high, the model still needs to be carefully adjusted to more accurately approximate the real bounding box. Therefore, in order to improve the model training speed and model performance, the model is optimized by the image intersection-over-union optimization method.

[0379] Furthermore, the S5 specifically includes the following steps:

[0380] S51. According to the output vehicle driving state, obtain the center position CP of the vehicle and record the time corresponding to the current frame image , based on the current frame image, obtain the predicted vehicle driving state at any time;

[0381] S52. Determine the road sections prone to congestion based on the predicted vehicle driving status at any time and the collected traffic statistics during the morning and evening peaks;

[0382] S53. Based on the relationship between the position of the vehicle in the vehicle driving state and the position of the congested road section, the relationship between the speed of the vehicle in the vehicle driving state and the critical value of the vehicle speed in the congested road section, and the relationship between the total number of vehicles in the congested road section and the critical value of the number of vehicles in the congested road section, the congested road section is predicted and judged to obtain the predicted congested road section result;

[0383] In the S51, the vehicle driving state includes the vehicle position and speed;

[0384] Predicted The vehicle driving status of the next frame image at the moment It is expressed as:

[0385] ;

[0386] ;

[0387] in, is the state transfer matrix, is the transposed matrix of the state transfer matrix, is the estimated vehicle driving state of the current frame, is the control input matrix, is the control vector, is the covariance matrix of the predicted next frame image, is the covariance matrix of the current frame, is the process noise covariance;

[0388] According to the predicted The vehicle driving status of the next frame image at the moment , calculate the error between the measured value of the vehicle driving state and the predicted value of the vehicle driving state , and further get the updated The vehicle driving status of the next frame image at the moment ;

[0389] The error between the measured value of the vehicle's driving state and the predicted value of the vehicle's driving state It is expressed as:

[0390] ;

[0391] in, for The vehicle driving state of the next frame of image at the moment is the measurement value, is the measurement matrix;

[0392] Updated The vehicle driving status of the next frame image at the moment It is expressed as:

[0393] ;

[0394] ;

[0395] ;

[0396] in, is the Kalman gain, is the updated prediction covariance matrix, T is the matrix transpose;

[0397] Based on the above calculation, the predicted vehicle driving state at any time is obtained, which includes the predicted vehicle position RP and the predicted speed RV, where: , , is the position of the CM-th vehicle, , is the horizontal coordinate of the position of the CM-th vehicle, is the ordinate of the position of the CMth vehicle, CM is the total number of vehicles, is the speed of the CMth vehicle;

[0398] In S52, the road sections prone to congestion are all set as square areas, a two-dimensional coordinate system is established for the images of various types of vehicles traveling on the road, and the coordinates of the four corner points of the square area are measured, wherein the coordinates of the corner point with the smallest horizontal and vertical coordinates are , the coordinates of the corner point with the largest horizontal and vertical coordinates are ;

[0399] In the above S53, The vehicle position at the moment is recorded as , , For the The position of the CMth vehicle at the time, The speed at that moment is recorded as , , is the speed of the CMth car, The moment The position of the car is , ,in, For the The moment The horizontal coordinate of the vehicle's position, For the The moment The vertical coordinate of the vehicle's position;

[0400] Compare The moment The horizontal coordinate of the vehicle's position and The moment The vertical coordinate of the vehicle's position The relationship between the location of the congested road section and the square area: and When The moment The vehicle is in the congested section area, and the vehicles that meet the above conditions are retained. If the above conditions are not met, it means that the corresponding vehicle has not entered the congested section and is not considered. All CM vehicles are traversed and the vehicles in the first All vehicles entering the congested section at any time, the total number of vehicles entering the congested section is DM, and the position matrix formed by all vehicles entering the congested section is , the speed matrix corresponding to all vehicles entering the congested road section is ;

[0401] According to the set critical value of the number of vehicles in the congested road section and the critical speed of vehicles in congested sections Identify the congested sections and compare the total number of vehicles entering the congested sections DM with the critical number of vehicles in the congested sections DM relationship,

[0402] when When There was no congestion at all times. When the average speed of all vehicles in the congested section is determined The critical value of vehicle speed in congested road sections relationship, if , indicating that the current section is in There is no congestion at all times. , indicating that the current section is in Congestion occurs all the time;

[0403] Average speed of all vehicles on the congested road section It is expressed as:

[0404] ;

[0405] At this point, the prediction of the road sections prone to congestion has been completed. By analyzing the collected data, it can be analyzed whether congestion will occur in the target road sections prone to congestion at a certain time in the future;

[0406] Specifically, based on the predicted vehicle position and the predicted speed , the position and speed information of all vehicles at any time can be predicted. However, for vehicle congestion prediction, it is necessary to analyze the position relationship and average speed of all vehicles. According to the prediction results, daily traffic flow can be improved by optimizing signal control, traffic guidance, emergency response and resource allocation, which also helps to optimize long-term transportation policies and infrastructure, and comprehensively improve the sustainability and efficiency of urban transportation systems.

[0407] Furthermore, in S33, the layers in the network structure of the deep learning model A are connected in the following order:

[0408] Input layer: image size 227×227×3;

[0409] Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization;

[0410] Activation layer 1: LeakyReLU;

[0411] Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0412] Activation layer 2: LeakyReLU;

[0413] Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization;

[0414] Activation layer 3: LeakyReLU;

[0415] Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0416] Activation layer 4: LeakyReLU;

[0417] SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model;

[0418] Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1;

[0419] Pooling layer 2: Type: global average pooling, pooling size: 9×9, Stride: 1;

[0420] Pooling layer 3: Type: global average pooling, pooling size: 13×13, Stride: 1;

[0421] Pooling concatenation layer: connect the outputs of the three pooling layers;

[0422] PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations;

[0423] Convolutional layer 5: kernel size 3×3, number of kernels 768, stride: 2, batch normalization;

[0424] Activation layer 5: LeakyReLU;

[0425] Convolutional layer 6: kernel size 3×3, number of kernels 768, stride: 2, batch normalization;

[0426] Activation layer 6: LeakyReLU;

[0427] Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer;

[0428] Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer;

[0429] Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0430] Activation layer 7: LeakyReLU;

[0431] Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0432] Activation layer 8: LeakyReLU;

[0433] Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0434] Activation layer 9: LeakyReLU;

[0435] Output layer: Type: Softmax, Number of neurons: 4.

[0436] Furthermore, in S33, the layers in the network structure of the deep learning model B are connected in the following order:

[0437] Input layer: image size 227×227×3;

[0438] Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization;

[0439] Activation layer 1: LeakyReLU;

[0440] Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0441] Activation layer 2: LeakyReLU;

[0442] Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization;

[0443] Activation layer 3: LeakyReLU;

[0444] Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0445] Activation layer 4: LeakyReLU;

[0446] SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model;

[0447] Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1;

[0448] Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1;

[0449] Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1;

[0450] Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1;

[0451] Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1;

[0452] Pooling concatenation layer: connect the outputs of the five pooling layers;

[0453] PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations;

[0454] Convolution layer 5: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0455] Activation layer 5: LeakyReLU;

[0456] Convolutional layer 6: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0457] Activation layer 6: LeakyReLU;

[0458] Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer;

[0459] Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer;

[0460] Convolutional layer 7: kernel size 3×3, number of kernels 512, stride: 1, batch normalization;

[0461] Activation layer 7: LeakyReLU;

[0462] Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0463] Activation layer 7: LeakyReLU;

[0464] Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0465] Activation layer 8: LeakyReLU;

[0466] Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0467] Activation layer 9: LeakyReLU;

[0468] Output layer: Type: Softmax, Number of neurons: 4.

[0469] Furthermore, in S33, the layers in the network structure of the deep learning model C are connected in the following order:

[0470] Input layer: image size 227×227×3;

[0471] Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization;

[0472] Activation layer 1: LeakyReLU;

[0473] Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0474] Activation layer 2: LeakyReLU;

[0475] Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization;

[0476] Activation layer 3: LeakyReLU;

[0477] Convolutional layer 4: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0478] Activation layer 4: LeakyReLU;

[0479] Convolutional layer 5: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0480] Activation layer 5: LeakyReLU;

[0481] Convolutional layer 6: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0482] Activation layer 6: LeakyReLU;

[0483] SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model;

[0484] Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1;

[0485] Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1;

[0486] Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1;

[0487] Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1;

[0488] Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1;

[0489] Pooling concatenation layer: connect the outputs of the five pooling layers;

[0490] PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations;

[0491] Convolutional layer 7: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0492] Activation layer 7: LeakyReLU;

[0493] Convolutional layer 8: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0494] Activation layer 8: LeakyReLU;

[0495] Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer;

[0496] Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer;

[0497] Convolutional layer 9: kernel size 3×3, number of kernels 512, stride: 1, batch normalization;

[0498] Activation layer 9: LeakyReLU;

[0499] Convolution layer 10: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0500] Activation layer 10: LeakyReLU;

[0501] Convolutional layer 11: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0502] Activation layer 11: LeakyReLU;

[0503] Convolutional layer 12: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0504] Activation layer 12: LeakyReLU;

[0505] Output layer: Type: Softmax, Number of neurons: 4.

[0506] Furthermore, in S33, the layers in the network structure of the deep learning model D are connected in the following order:

[0507] Input layer: image size 227×227×3;

[0508] Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization;

[0509] Activation layer 1: LeakyReLU;

[0510] Convolutional layer 2: convolution kernel size 3×3, number of convolution kernels 32, stride: 1, batch normalization;

[0511] Activation layer 2: LeakyReLU;

[0512] Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 64, stride: 1, batch normalization;

[0513] Activation layer 3: LeakyReLU;

[0514] Convolutional layer 4: convolution kernel size 3×3, number of convolution kernels 64, stride: 1, batch normalization;

[0515] Activation layer 4: LeakyReLU;

[0516] Convolutional layer 5: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0517] Activation layer 5: LeakyReLU;

[0518] Convolutional layer 6: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0519] Activation layer 6: LeakyReLU;

[0520] Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0521] Activation layer 7: LeakyReLU;

[0522] Convolutional layer 8: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0523] Activation layer 8: LeakyReLU;

[0524] SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model;

[0525] Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1;

[0526] Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1;

[0527] Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1;

[0528] Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1;

[0529] Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1;

[0530] Pooling concatenation layer: connect the outputs of the five pooling layers;

[0531] PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations;

[0532] Convolutional layer 9: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0533] Activation layer 9: LeakyReLU;

[0534] Convolution layer 10: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization;

[0535] Activation layer 10: LeakyReLU;

[0536] Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer;

[0537] Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer;

[0538] Convolutional layer 11: kernel size 3×3, number of kernels 512, stride: 1, batch normalization;

[0539] Activation layer 11: LeakyReLU;

[0540] Convolutional layer 12: kernel size 3×3, number of kernels 256, stride: 1, batch normalization;

[0541] Activation layer 12: LeakyReLU;

[0542] Convolutional layer 13: kernel size 3×3, number of kernels 128, stride: 1, batch normalization;

[0543] Activation layer 13: LeakyReLU;

[0544] Convolutional layer 14: kernel size 3×3, number of kernels 64, stride: 1, batch normalization;

[0545] Activation layer 14: LeakyReLU;

[0546] Output layer: Type: Softmax, Number of neurons: 4.

[0547] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative, not restrictive, with respect to the scope of the present invention, which is defined by the appended claims.

Claims

1. A traffic congestion prediction method based on drone and vehicle driving status, characterized in that: The following steps are involved: S1. Collect images of various vehicles on the road to generate an image matrix, enhance the image quality, and obtain a new image matrix; S2. equally divide the image in the new image matrix and update the image pixels to obtain an image after pixel update; S3. construct a data set based on the image after pixel update, input the data set into the established basic model, auxiliary model and enhanced model for training, fuse the loss functions of the four models to obtain the fused loss function, train the four models, and obtain four trained models; S4. According to the intersection-over-union optimization method, the four trained models are trained, and the model with the smallest overall loss function after training is taken as the optimal model. The accuracy of the optimal model is screened to obtain the final optimal model, and the driving status and type of the vehicle are output; S5. By analyzing the vehicle driving status and combining the collected traffic flow statistics during the morning and evening peak hours, traffic congestion prediction is performed to obtain the predicted results of the road sections prone to congestion; The S5 specifically includes the following steps: S51. According to the output vehicle driving state, the center position CP of the vehicle is obtained, the time ct corresponding to the current frame image is recorded, and based on the current frame image, the predicted vehicle driving state at any time is obtained; S52. Determine the road sections prone to congestion based on the predicted vehicle driving status at any time and the collected traffic statistics during the morning and evening peaks; S53. Based on the relationship between the position of the vehicle in the vehicle driving state and the position of the congested road section, the relationship between the speed of the vehicle in the vehicle driving state and the critical value of the vehicle speed in the congested road section, and the relationship between the total number of vehicles in the congested road section and the critical value of the number of vehicles in the congested road section, the congested road section is predicted and judged to obtain the predicted congested road section result; In the S51, the vehicle driving state includes the vehicle position and speed; Predicted vehicle driving status of the next frame image at time ct+1 It is expressed as: Among them, F ct+1 is the state transfer matrix, is the transposed matrix of the state transfer matrix, is the estimated vehicle driving state of the current frame, β ct+1 is the control input matrix, v ct+1 is the control vector, is the covariance matrix of the predicted next frame image, is the covariance matrix of the current frame, Q ct+1 is the process noise covariance; According to the predicted vehicle driving state of the next frame image at time ct+1 Calculate the error Er between the measured value of the vehicle driving state and the predicted value of the vehicle driving state ct+1 , and further obtain the updated vehicle driving state of the next frame image at time ct+1 The error Er between the measured value of the vehicle driving state and the predicted value of the vehicle driving state ct+1 It is expressed as: Among them, CP ct+1 is the vehicle driving state of the next frame image at time ct+1, i.e., the measured value, H ct+1 is the measurement matrix; The vehicle driving status of the next frame image at time ct+1 after update It is expressed as: Among them, K ct+1 is the Kalman gain, is the updated prediction covariance matrix, T is the matrix transpose; Based on the above calculation, the predicted vehicle driving state at any time is obtained, which includes the predicted vehicle position RP and the predicted speed RV, where RP = [rp1 rp2 ... rp CM ], RV = [rv1 rv2...rv CM ], rp CM is the position of the CMth vehicle, rp CM =(jx CM ,jy CM ), jx CM is the horizontal coordinate of the position of the CMth vehicle, jy CM is the ordinate of the position of the CMth vehicle, CM is the total number of vehicles, rv CM is the speed of the CMth vehicle; In S52, the road sections prone to congestion are all set as square areas, a two-dimensional coordinate system is established for the images of various types of vehicles traveling on the road, and the coordinates of the four corner points of the square area are measured, wherein the coordinates of the corner point with the smallest horizontal and vertical coordinates are JD min (jx min ,jy min ), the coordinates of the corner point with the largest horizontal and vertical coordinates are JD max (jx max ,jy max ); In the step S53, the vehicle position at the et time is recorded as RP et , RP et =[rp1 et rp2 et ...rp CM et ], rp CM et is the position of the CMth vehicle at the et-th time, and the speed at the et-th time is recorded as RV et , RV et =[rv1 et rv2 et ...rv CM et ], rv CM et is the speed of the CMth car, and the position of the eith car at the etth time is rp ei et , rp ei et =(jx ei et ,jy ei et ), where jx ei et is the horizontal coordinate of the position of the ei-th vehicle at the et-th time, jy ei et is the ordinate of the position of the ei-th vehicle at the et-th time; Compare the horizontal coordinate jx of the position of the ei-th vehicle at the et-th time ei et and the ordinate jy of the position of the ei-th vehicle at the et-th time ei et The relationship between the position of the congested road section and the square area: when jx min ≤jx ei et ≤jx max Andjy min ≤jy ei et ≤jy max When , it indicates that the ei-th vehicle at the et-th time is in the congested section area. Vehicles that meet the above conditions are retained. When the above conditions are not met, it indicates that the corresponding vehicle has not entered the congested section and is not considered. All CM vehicles are traversed and all vehicles that enter the congested section at the et-th time are screened out. The total number of vehicles entering the congested section is recorded as DM, and the position matrix formed by all vehicles entering the congested section is YRP et =[yrp1 et yrp2 et ...yrp DM et ], the speed matrix corresponding to all vehicles entering the congested section is YRV et =[yrv1 et yrv2 et ...yrv DM et ]; According to the set critical value of the number of vehicles in the congested road section DM tr and the critical value of vehicle speed in congested road section yrv tr Identify the congested sections and compare the total number of vehicles entering the congested sections DM with the critical number of vehicles in the congested sections DM tr When DM<DM tr When DM ≥ DM tr When the average speed of all vehicles in the congested section is determined, yrv pj et and the critical value of vehicle speed in congested road section yrv tr If yrv pj et >yrv tr , indicating that there is no congestion on the current road section at time et. pj et ≤yrv tr , indicating that the current road section is congested at time et.

2. The method for predicting traffic congestion based on the driving status of drones and vehicles according to claim 1, characterized in that: The S1 specifically includes the following steps: S11. Use the high-definition camera on the drone to collect images ph of various vehicles traveling on the road, and the corresponding image matrix is ​​PH; S12. Construct a 5th-order denoising kernel function N, and normalize all elements in the kernel function to obtain the normalized denoising kernel function N G ; S13. According to the normalized denoising kernel function N G , perform denoising kernel function processing on the image to obtain a new image matrix P to achieve image quality enhancement; In S12, the fifth-order denoising kernel function N is expressed as: N=[N xy ] Where x = 1, 2, 3, 4 or 5, x is the row number of the matrix, y = 1, 2, 3, 4 or 5, y is the column number of the matrix, σ is the standard deviation of all elements in the kernel function, and e is the base of the natural logarithm; Normalized denoising kernel function N G It is expressed as: In S13, the new image matrix P is expressed as: P=PH×N G 。 3. The traffic congestion prediction method based on the driving status of drones and vehicles according to claim 2 is characterized in that: The S2 comprises the following steps: S21. According to the new image matrix P, the pixel grayscale value is divided into 16 grayscale levels, and the image in the new image matrix P is divided into n equal parts to obtain the divided image; S22. Count the pixel frequency FR corresponding to each gray level in the equally divided image nu , calculate the grayscale frequency cumulative value TR corresponding to each grayscale level in the equally divided image nu ; S23. Based on the gray frequency cumulative value TR nu , construct a pixel value update method and calculate the pixel update value corresponding to the gray level Get the image after pixel update; In S21, the pixel gray value is 0-255, and the gray level is Q nu , nu=1,2,3,…,16, nu is the grayscale number, the image after equal division is Pnum, num=1,2,…,n; In S22, each gray level Q in the equally divided image Pnum nu The corresponding grayscale frequency cumulative value TR nu It is expressed as: Where no is the gray level number less than or equal to nu, Q no Indicates the gray level with gray level number no; In S23, the pixel update value corresponding to the gray level It is expressed as: in, is the minimum non-zero value among all pixel frequency accumulation values, P pix is the total number of pixels corresponding to the equally divided image Pnum, is the maximum value of the gray level.

4. The method for predicting traffic congestion based on the driving status of drones and vehicles according to claim 3, characterized in that: The S3 specifically includes the following steps: S31. Based on the pixel updated image, select all images including passenger cars, commercial vehicles, buses and special vehicles as input images; S32. Select four types of vehicles, namely, passenger cars, commercial vehicles, buses and special vehicles, from the input image through manual annotation, construct a data set, and divide the data set into a training set and a test set according to a ratio of 8:2; S33. According to the data set, a basic model, an auxiliary model and an enhanced model are constructed, and the loss functions of the four models are calculated and integrated to obtain a fused loss function; In S32, passenger cars, commercial vehicles, buses and special vehicles are marked as Passenger, Commercial, Bus and Special respectively; In the S33, a network structure of a basic model, i.e., a deep learning model A, is constructed, and the deep learning model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; The layers in the network structure of deep learning model A are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 13×13, Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 5: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4; The data set is input into the deep learning model A, and the weights and biases of the neural network units of the deep learning model A are calculated through the error back propagation algorithm to complete the training of the deep learning model A and obtain the trained deep learning model A and its loss function La; The output value of the output unit node corresponding to the deep learning model A is a i , respectively, the output value a1 of the first output unit node, the output value a2 of the second output unit node, the output value a3 of the third output unit node, and the output value a4 of the fourth output unit node. According to the exponential function exp, the output values ​​of the four output unit nodes corresponding to the deep learning model A are initially converted into the probability value f of the first preliminary output value. ai , and then further optimize and transform it to obtain the first optimized output value and the enhanced probability value g of the first optimized output value ai ; The enhanced probability value g of the first optimized output value is adopted ai And the data set, combined with the error chain derivative rule, continue to train the deep learning model A. After the training is completed, the output value j of the first vehicle type is obtained ai The actual vehicle type label h corresponding to the node of the output layer of the deep learning model A ai , the output value j of the first vehicle type ai are the output values ​​j of the first passenger car respectively a1 , the output value j of the first commercial vehicle a2 , the output value of the first bus j a3 and the first special vehicle output value j a4 ; The first auxiliary model, deep learning model B, is constructed. Deep learning model B consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer. The layers in the network structure of deep learning model B are connected in the following order: Input layer: image size 227×227×3; Convolution layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; activation layer 1: LeakyReLU; Convolutional layer 2: convolution kernel size 3×3, number of convolution kernels 64, stride: 1, batch normalization; activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; activation layer 3: LeakyReLU; Convolutional layer 4: convolution kernel size 3×3, number of convolution kernels 256, stride: 1, batch normalization; activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet: Aggregate features at different levels and implement feature pyramid through upsampling and downsampling operations; Convolution layer 5: convolution kernel size 3×3, number of convolution kernels 1280, Stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolution layer 6: convolution kernel size 3×3, number of convolution kernels 1280, stride: 2, batch normalization; activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: merges the output feature map of the downsampling layer with the output feature map of the upsampling layer; Convolution layer 7: convolution kernel size 3×3, number of convolution kernels 512, stride: 1, batch normalization; activation layer 7: LeakyReLU; Convolution layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; activation layer 7: LeakyReLU; Convolution layer 8: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; activation layer 8: LeakyReLU; Convolution layer 9: convolution kernel size 3×3, number of convolution kernels 64, stride: 1, batch normalization; activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4; The data set is input into the deep learning model B, and the weights and biases of the neural network units of the deep learning model B are calculated through the error back propagation algorithm, the training of the deep learning model B is completed, and the trained deep learning model B and its loss function Lb are obtained; The output value of the output unit node corresponding to the deep learning model B is b i , respectively, the output value b1 of the fifth output unit node, the output value b2 of the sixth output unit node, the output value b3 of the seventh output unit node, and the output value b4 of the eighth output unit node. According to the exponential function exp, the output values ​​of the four output unit nodes corresponding to the deep learning model B are initially converted into the second preliminary output value and the probability value f of the second preliminary output value. bi , and then further optimize and transform it to obtain the second optimized output value and the enhanced probability value g of the second optimized output value bi ; The enhanced probability value g of the second optimized output value is adopted bi And the data set, combined with the error chain derivative rule, continue to train the deep learning model B. After the training is completed, the output value j of the second vehicle type is obtained bi The actual vehicle type label h corresponding to the node of the output layer of the deep learning model B bi , the output value j of the second vehicle type bi are the output values ​​j of the second passenger car respectively b1 , the output value j of the second commercial vehicle b2 , the output value j of the second bus b3 and the second special vehicle output value j b4 ; The second auxiliary model, namely the deep learning model C, is constructed. The deep learning model C consists of 1 input layer, 12 convolutional layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer. The layers in the network structure of the deep learning model C are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 4: LeakyReLU; Convolutional layer 5: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 6: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 7: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 8: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 9: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Convolution layer 10: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 10: LeakyReLU; Convolutional layer 11: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 11: LeakyReLU; Convolutional layer 12: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 12: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4; The data set is input into the deep learning model C, and the weights and biases of the neural network units of the deep learning model C are calculated through the error back propagation algorithm to complete the training of the deep learning model C and obtain the trained deep learning model C and its loss function Lc; The output value of the output unit node corresponding to the deep learning model C is c i , are the output value c1 of the ninth output unit node, the output value c2 of the tenth output unit node, the output value c3 of the eleventh output unit node, and the output value c4 of the twelfth output unit node, respectively. According to the exponential function exp, the output values ​​of the four output unit nodes corresponding to the deep learning model C are initially converted into the third preliminary output value and the probability value f of the third preliminary output value. ci , and then further optimize and transform it to obtain the third optimized output value and the enhanced probability value g of the third optimized output value ci ; The enhanced probability value g of the third optimized output value is adopted ci And the data set, combined with the error chain derivative rule, continue to train the deep learning model C. After the training is completed, the output value j of the third vehicle type is obtained ci The actual vehicle type label h corresponding to the node of the output layer of the deep learning model B bi , the third vehicle type output value j ci are the output values ​​j of the third passenger car respectively c1 , the output value j of the third commercial vehicle c2 , the output value of the third bus j c3 and the third special vehicle output value j c4 ; Construct a reinforcement model, namely the deep learning model D, which consists of 1 input layer, 14 convolutional layers, 14 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; The layers in the network structure of the deep learning model D are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: convolution kernel size 3×3, number of convolution kernels 32, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 64, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: convolution kernel size 3×3, number of convolution kernels 64, stride: 1, batch normalization; Activation layer 4: LeakyReLU; Convolutional layer 5: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 6: LeakyReLU; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 8: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet: Aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 9: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 9: LeakyReLU; Convolution layer 10: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 10: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 11: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 11: LeakyReLU; Convolutional layer 12: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 12: LeakyReLU; Convolutional layer 13: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 13: LeakyReLU; Convolutional layer 14: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 14: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4; The data set is input into the deep learning model D, and the weights and biases of the neural network units of the deep learning model D are calculated through the error back propagation algorithm to complete the training of the deep learning model D and obtain the trained deep learning model D and its loss function Ld; The output value of the output unit node corresponding to the deep learning model D is d i , respectively, the output value d1 of the thirteenth output unit node, the output value d2 of the fourteenth output unit node, the output value d3 of the fifteenth output unit node, and the output value d4 of the sixteenth output unit node. According to the exponential function exp, the output values ​​of the four output unit nodes corresponding to the deep learning model D are initially converted into the fourth preliminary output value and the probability value f of the fourth preliminary output value. di , and then further optimize and transform it to obtain the fourth optimized output value and the enhanced probability value g of the fourth optimized output value di ; The enhanced probability value g of the fourth optimized output value is adopted di And the data set, combined with the error chain derivative rule, continue to train the deep learning model D. After the training is completed, the output value j of the fourth vehicle type is obtained di The actual vehicle type label h corresponding to the node of the output layer of the deep learning model D di , the fourth vehicle type output value j di are the output values ​​j of the fourth passenger car respectively d1 , the output value j of the fourth commercial vehicle d2 、The output value of the fourth bus j d3 and the fourth special vehicle output value j d4 ; According to the second vehicle type, the output value is j bi and the enhanced probability value g of the first optimized output value ai , construct the loss function L between deep learning model B and deep learning model A BA , get the corresponding weight coefficient; The probability value f based on the first preliminary output value ai The actual vehicle type label h corresponding to the node of the output layer of the deep learning model A ai , construct the probability value f considering the first preliminary output value ai The loss function L of the deep learning model A is AA , get the corresponding weight coefficient; Based on the loss function L between deep learning model B and deep learning model A BA and considering the probability value f of the first preliminary output value ai The loss function L of the deep learning model A is AA , establish the relationship loss function L between the output layer of deep learning model B and deep learning model A BAA ; According to the above operations, the relationship loss function L between the output layers of the deep learning model C and the deep learning model B is established. CBB , the relationship loss function L between the output layer of the deep learning model C and the deep learning model A CAA , the relationship loss function L between the output layer of the deep learning model D and the deep learning model A DAA The relationship loss function L between the output layers of the deep learning model D and the deep learning model C DCC .

5. The method for predicting traffic congestion based on the driving status of drones and vehicles according to claim 4, characterized in that: The S4 specifically includes the following steps: S41. For each target prediction bounding box and the corresponding true bounding box of the four trained models, calculate the intersection area AR between the two j ; S42. For each target predicted bounding box and the corresponding true bounding box of the four trained models, calculate the union area AR of the two b ; S43. Calculate the intersection area AR j and the area of ​​the union AR b The corresponding intersection-over-union ratio IoU introduces the adjustment coefficient g t , construct the loss function L of the target position w , get the optimized intersection-over-union ratio IoU; S44. According to the optimized intersection-over-union ratio IoU, the optimized intersection-over-union ratio calculation method is used for model training. When the overall loss function L is defined total When it is minimum, the corresponding model parameters are optimal, and the overall loss function L total The smallest model is taken as the optimal model, and the optimal model with the highest accuracy is selected to obtain the final optimal model, and the vehicle driving status and type are output; In S43, the intersection over union (IoU) is expressed as: The loss function L of the target position w It is expressed as: Set the Intersection over Union (IoU) threshold CR , according to the intersection over union ratio IoU and the intersection over union ratio threshold IoU CR The relationship between the adjustment factor g t In the range of, when IoU>IoU CR When the corresponding target position loss function L w The adjustment factor g t <1, when IoU = IoU CR When the corresponding target position loss function L w The adjustment factor g t =1, when IoU<IoU CR When the corresponding loss function adjustment coefficient g t >1; In S44, the overall loss function L corresponding to the deep learning model A, the deep learning model B, the deep learning model C and the deep learning model D is defined. total , 2 5 =32, and we get 32 ​​overall loss functions L total In the form of, four models are trained, and 32 deep learning models A trained with the overall loss function are obtained as the optimal model; The overall loss function L total It is expressed as: L total =β1L DCC +β2L CBB +β3L BAA +β4L DAA +β5L CAA Among them, β1 is the relationship loss function L between the output layers of the deep learning model D and the deep learning model C DCC The weight coefficient is 0 or 1, and β2 is the relationship loss function L between the output layers of the deep learning model C and the deep learning model B. CBB The weight coefficient is 0 or 1, and β3 is the relationship loss function L between the output layer of deep learning model B and deep learning model A. BAA The weight coefficient is 0 or 1, and β4 is the relationship loss function L between the output layer of the deep learning model D and the deep learning model A. DAA The weight coefficient is 0 or 1, and β5 is the relationship loss function L between the output layers of the deep learning model D and the deep learning model C. DCC The weight coefficient of is 0 or 1; The accuracy of the deep learning model A trained with the overall loss function is recorded as ACa i , i = 1, 2, 3, ..., 32, calculate the maximum value ACa of the accuracy corresponding to the deep learning model A trained with the overall loss function max , ACa max =max[ACa i ], select the maximum value ACa max The corresponding deep learning model A is the final optimal model.

Images