Vehicle dynamic weighing and multi-parameter detection method and system based on roadside camera

Abstract

The present invention proposes a vehicle dynamic weighing and multi-parameter detection method and system based on a roadside camera, which belongs to the field of dynamic weighing, including S1, installing a vehicle dynamic weighing system on the monitored road, and obtaining vehicle dynamic weighing parameters according to the vehicle dynamic weighing system; S2, building and training a traffic detection network that locks the vehicle's stable characteristics; S3, obtaining the real-time dynamic picture of the vehicle driving captured by the roadside camera on the monitored road, and inputting the real-time dynamic picture of the vehicle driving into the traffic detection network to lock the vehicle characteristics and obtain vehicle characteristic information; S4, fusing the vehicle characteristic information with the vehicle dynamic weighing parameters to obtain vehicle parameters. This application combines the parameters obtained by the vehicle dynamic weighing system and the vehicle characteristic information extracted by the traffic detection network to obtain more comprehensive and accurate vehicle parameters, realize real-time monitoring and analysis of vehicle dynamic parameters, and improve the accuracy and reliability of vehicle status assessment.

Description

Technical Field

[0001] The present invention relates to the field of dynamic weighing technology, and in particular to a vehicle dynamic weighing and multi-parameter detection method and system based on a roadside camera. Background Art

[0002] With the booming development of the transportation industry, traffic volume continues to rise, accompanied by the increasingly prominent problems of vehicle overloading and over-limit transportation. When vehicles exceed the load limit, they will cause serious damage to the road pavement and bridge structure, resulting in a significant increase in the incidence of diseases in these infrastructures, thereby greatly shortening their service life. Traditional traffic video surveillance systems perform well in capturing the appearance characteristics and kinematic information of vehicles. They can identify the size, speed and trajectory of the vehicle, but they cannot directly measure the weight of the vehicle. From the perspective of the roadside camera, the local features of the vehicle change dramatically when it moves from far to near, and the captured stable features are easily lost.

[0003] In the existing FBG array vehicle dynamic weighing system, it is impossible to weigh multiple vehicles simultaneously in complex traffic scenarios. Due to insufficient vehicle recognition capabilities, it is impossible to accurately match vehicle characteristics with vehicle weight information, resulting in the inability to fully monitor road traffic and reduce traffic management efficiency.

[0004] Therefore, finding a method that can weigh multiple vehicles simultaneously and improve the accuracy of dynamic weighing of vehicles is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] In view of this, the present invention proposes a vehicle dynamic weighing and multi-parameter detection method and system based on a roadside camera, which can combine the parameters obtained by the vehicle dynamic weighing system and the vehicle feature information extracted by the traffic detection network to obtain more comprehensive and accurate vehicle parameters, realize real-time monitoring and analysis of vehicle dynamic parameters, and improve the accuracy and reliability of vehicle status assessment.

[0006] The technical solution of the present invention is achieved in this way:

[0007] On the one hand, the present invention provides a vehicle dynamic weighing and multi-parameter detection method based on a roadside camera, comprising the following steps:

[0008] S1. Install a vehicle dynamic weighing system on the monitored road and obtain vehicle dynamic weighing parameters according to the vehicle dynamic weighing system;

[0009] S2. Build and train a traffic detection network that locks vehicle stability features; the traffic detection network includes a feature extraction module, a feature fusion module, and a feature locking module, wherein the feature locking module detects and analyzes the fusion features and performs vehicle feature locking;

[0010] S3, obtaining a real-time dynamic picture of the vehicle driving captured by a roadside camera on the monitored road, inputting the real-time dynamic picture of the vehicle driving into a traffic detection network to lock the vehicle characteristics, and obtaining the vehicle characteristic information;

[0011] S4. Vehicle characteristic information is integrated with vehicle dynamic weighing parameters to obtain vehicle parameters; the vehicle parameters include vehicle type, axle, vehicle length, color characteristics, and vehicle weight data.

[0012] On the basis of the above technical solution, preferably, the traffic detection network is:

[0013] The feature extraction module includes a first convolution layer, a first feature pooling unit and four feature extraction units, wherein the four feature extraction units are respectively a first feature extraction unit, a second feature extraction unit, a third feature extraction unit and a fourth feature extraction unit, wherein the four feature extraction units have the same structure but different numbers of channels;

[0014] The feature fusion module includes a second feature pooling unit and four feature fusion units, wherein the four feature fusion units are a first feature fusion unit, a second feature fusion unit, a third feature fusion unit and a fourth feature fusion unit;

[0015] The feature locking module includes a detection unit, a parsing unit and a locking unit, which detects the fused features and determines whether they meet the parsing requirements. If they meet the parsing requirements, the vehicle features are locked.

[0016] On the basis of the above technical solution, preferably, the feature extraction unit includes five convolutional layers and a first aggregation layer, the five convolutional layers are respectively a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer, wherein the second convolutional layer includes a convolution kernel with a size of 3×3 and a step size of 2, a ReLU activation function layer and a batch normalization layer, the fourth convolutional layer and the fifth convolutional layer have different sizes, and the sixth convolutional layer has the same structure as the third convolutional layer.

[0017] On the basis of the above technical solution, preferably, the first feature pooling unit includes a seventh convolutional layer, two maximum pooling layers and a second aggregation layer, wherein the convolution kernel of the seventh convolutional layer is 1x1, the step size is 1, the two maximum pooling layers include a first maximum pooling layer and a second maximum pooling layer, the first maximum pooling layer and the second aggregation layer are jump-connected, and the pooling operation of the first feature pooling unit is:

[0018] The feature map of the feature extraction unit is input into the seventh convolutional layer for convolution operation to obtain the feature map y1;

[0019] Input the feature map y1 to the first maximum pooling layer for the first pooling operation to obtain the feature map y2;

[0020] Input the feature map y2 to the second maximum pooling layer for a second pooling operation to obtain the feature map y3;

[0021] The feature map y2 and the feature map y3 are input into the second aggregation layer for aggregation to obtain the feature map of the first feature pooling unit.

[0022] On the basis of the above technical solution, preferably, the second feature pooling unit includes an eighth convolutional layer, two maximum pooling layers and a fourth aggregation layer, wherein the convolution kernel of the eighth convolutional layer is 1x1, the step size is 1, the two maximum pooling layers include a third maximum pooling layer and a fourth maximum pooling layer, a jump connection is formed between the eighth convolutional layer and the third aggregation layer, a jump connection is formed between the third maximum pooling layer and the fourth aggregation layer, and a jump connection is formed between the fourth maximum pooling layer and the third aggregation layer, and the pooling operation of the second feature pooling unit is:

[0023] The feature map of the first feature pooling unit is input into the eighth convolutional layer for convolution operation to obtain the feature map x1;

[0024] Input feature map x1 to the third maximum pooling layer for the first pooling operation to obtain feature map x2;

[0025] Input feature map x2 to the fourth maximum pooling layer for a second pooling operation to obtain feature map x3;

[0026] The feature map x1, the feature map x2 and the two feature maps x3 are input into the fourth aggregation layer for aggregation to obtain the feature map of the second feature pooling unit.

[0027] More preferably, the training process includes:

[0028] S21, obtaining a vehicle driving dynamic video sequence captured by a roadside camera on the monitored road, extracting frames from the vehicle driving dynamic video sequence to obtain a video image sequence, and inputting the video image sequence into a traffic detection network in sequence;

[0029] S22, using the feature extraction module to extract features from the video image sequence respectively, to obtain five feature sequences of different scales {F i}, i = 1, 2, 3, 4, 5;

[0030] S23, using the feature fusion module to fusion the five feature sequences of different scales {F i} respectively perform feature fusion to obtain the fused feature sequence {P j}, j = 1, 2, 3, 4;

[0031] S24, using the feature locking module to fusion feature sequence {P j} to parse and detect, the detection unit performs fusion feature sequence {Pj} Perform vehicle feature detection respectively to obtain a vehicle feature sequence. If the vehicle features in the vehicle feature sequence meet the parsing requirements of the parsing module, the vehicle features are locked to obtain vehicle feature information.

[0032] On the basis of the above technical solution, preferably, step S22 specifically includes:

[0033] S221, using the first convolutional layer to process any video image in the video image sequence to obtain a feature map F1;

[0034] S222, inputting the feature map F1 into the first feature extraction unit for first feature extraction to obtain a feature map F2;

[0035] S223, inputting the feature map F2 into the second feature extraction unit for a second feature extraction to obtain a feature map F2';

[0036] S224, inputting the feature map F2' into the first feature pooling unit for pooling to obtain a feature map F3;

[0037] S225, inputting the feature map F3 into a third feature extraction unit for performing a third feature extraction to obtain a feature map F4;

[0038] S226, inputting the feature map F4 into a fourth feature extraction unit for performing a fourth feature extraction to obtain a feature map F5;

[0039] S227, repeating steps S221-S226 until all the video images in the video image sequence are feature extracted, and obtaining five feature sequences of different scales {F i}.

[0040] On the basis of the above technical solution, preferably, step S23 specifically includes:

[0041] S231, input any feature map in the feature map sequence {F5} into the second feature pooling unit for pooling operation, and obtain a feature map F P ;

[0042] S232, feature map F4 and feature map F P Input to the fourth feature fusion unit for the first feature fusion to obtain feature map P1;

[0043] S233, inputting the feature map P1 and the feature map F3 into a third feature fusion unit for a second feature fusion to obtain a feature map P2;

[0044] S234, inputting the feature map P2 and the feature map F2 into the second feature fusion unit for third feature fusion to obtain the feature map P3;

[0045] S235, inputting the feature map P3 and the feature map F1 into the first feature fusion unit for fourth feature fusion to obtain a feature map P4;

[0046] S236, repeat steps S231-S235 until all feature graphs in the feature graph sequence {F5} are subjected to feature fusion to obtain a fused feature sequence {P j}.

[0047] On the basis of the above technical solution, preferably, the training further includes:

[0048] L VFL Function as the classification loss function, with L DFL The function is used as the regression loss function to construct the joint loss function of the traffic detection network. The calculation formula of the joint loss function is as follows:

[0049] Loss = α × L VFL +β×L DFL ;

[0050] Among them, α and β represent the weight coefficients of each loss function;

[0051] The calculation formula of the classification loss function is as follows:

[0052]

[0053] Among them, p represents the predicted probability, q represents the true category label, is the loss weight of the image foreground and background, and τ is the weight of each sample data;

[0054] The calculation formula of the regression loss function is as follows:

[0055] L DFL (S i ,S i+1 )=-((g i+1 -g)log(S i )+(gg i )log(S i+1 ));

[0056]

[0057] Among them, S i represents the predicted probability of the category, g represents the true value, and g i and g i+1 are the two predicted values ​​on the left and right of the true value g, and i represents the index of the bounding box.

[0058] On the other hand, the present invention provides a vehicle dynamic weighing and multi-parameter detection system based on a roadside camera, which adopts the vehicle dynamic weighing and multi-parameter detection method described above, including:

[0059] A weighing parameter acquisition module is configured to install a vehicle dynamic weighing system on the monitored road and acquire vehicle dynamic weighing parameters according to the vehicle dynamic weighing system;

[0060] A network construction and training module configured to construct and train a traffic detection network that locks on stable features of vehicles;

[0061] A vehicle feature acquisition module is configured to acquire real-time dynamic images of vehicles traveling captured by roadside cameras on the monitored roads, input the real-time dynamic images of vehicles traveling into a traffic detection network to lock vehicle features, and obtain vehicle feature information;

[0062] The multi-parameter detection module is configured to obtain vehicle parameters by fusing vehicle characteristic information with vehicle dynamic weighing parameters.

[0063] The vehicle dynamic weighing and multi-parameter detection method of the present invention has the following beneficial effects compared with the prior art:

[0064] (1) By inputting the vehicle driving dynamic images into the traffic detection network, real-time vehicle feature locking and parameter analysis are realized to obtain vehicle feature information. By combining the parameters obtained by the vehicle dynamic weighing system and the vehicle feature information extracted by the traffic detection network, more comprehensive and accurate vehicle parameters are obtained, realizing real-time monitoring and analysis of vehicle dynamic parameters, and improving the accuracy and reliability of vehicle status assessment;

[0065] (2) The multi-dimensional feature information of the vehicle image is effectively extracted by using the feature extraction module. At the same time, the feature extraction units of four different channels can capture the features of different aspects of the vehicle. The multi-dimensional feature information output by the feature extraction module is effectively fused through the feature fusion module. The fused vehicle feature information is detected and judged by the feature locking module, thereby achieving effective locking of the vehicle feature information. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 It is a flow chart of the vehicle dynamic weighing and multi-parameter detection method of the present invention;

[0068] Figure 2 A block diagram of the vehicle dynamic weighing and multi-parameter detection method of the present invention;

[0069] Figure 3 A block diagram of a feature extraction unit of a vehicle dynamic weighing and multi-parameter detection method of the present invention;

[0070] Figure 4 A block diagram of a feature fusion unit of a vehicle dynamic weighing and multi-parameter detection method of the present invention;

[0071] Figure 5 A block diagram of a first feature pooling unit of a vehicle dynamic weighing and multi-parameter detection method of the present invention;

[0072] Figure 6 It is a block diagram of the second feature pooling unit of the vehicle dynamic weighing and multi-parameter detection method of the present invention. DETAILED DESCRIPTION

[0073] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] like Figure 1 As shown, the present invention provides a vehicle dynamic weighing and multi-parameter detection method based on a roadside camera, comprising the following steps:

[0075] S1. Install a vehicle dynamic weighing system on the monitored road and obtain vehicle dynamic weighing parameters according to the vehicle dynamic weighing system.

[0076] Specifically, step S1 specifically includes:

[0077] Install FBG sensors on the monitoring road;

[0078] The wavelength displacement of each FBG sensor is read through the demodulation device, and dynamic sensing and static sensing are performed using phase demodulation to calculate the dynamic weighing parameters of the vehicle.

[0079] It is understandable that when a vehicle passes by the FBG sensor, the vehicle tire will exert pressure on the FBG sensor, causing the FBG sensor to deform, thereby causing the grating spacing to change, and the reflection wavelength to change accordingly.

[0080] The FBG sensor is used to achieve non-contact, high-precision vehicle weighing measurement, and the dynamic information of the vehicle passing by is obtained by demodulating the reflected wavelength displacement. Phase demodulation is used to accurately measure the dynamic load information of the vehicle passing by, and static load data is obtained. The dynamic weighing parameters of the vehicle, such as axle weight, total weight, etc., are calculated through the dynamic load information and static load data.

[0081] S2. Construct and train a traffic detection network that locks vehicle stability features; the traffic detection network includes a feature extraction module, a feature fusion module and a feature locking module, wherein the feature locking module detects and analyzes the fusion features and locks the vehicle features.

[0082] In an embodiment of the present application, step S2 specifically includes:

[0083] S21, obtaining a vehicle driving dynamic picture from the perspective of a roadside camera on the monitored road, and extracting frames of the vehicle driving dynamic picture to obtain a video frame data set image;

[0084] S22, calibrating and dividing the video frame data set images to obtain training set data, validation set data and test set data, wherein the data division ratio is training set data: validation set data: test set data = 7:2:1;

[0085] S23. Construct a traffic detection network, use the training set data for training, use the validation set data to evaluate the performance of the traffic detection network and adjust hyperparameters during the training process, and use the test set data to evaluate the generalization ability and performance of the trained traffic detection network; wherein, when training the traffic detection network, construct a joint loss function according to the weights of each loss function, and the joint loss function includes spatial mapping loss, classification loss and regression loss.

[0086] It can be understood that roadside cameras are used to record and monitor the dynamic images of vehicles on the road, and the dynamic images of vehicles are extracted and processed as data set images for subsequent vehicle identification, classification and axle identification.

[0087] In one embodiment of the present application, video frames are extracted at a time interval of 0.5 seconds, that is, 2 frames of images can be obtained per second, which can capture subtle changes in the movement of the vehicle, thereby obtaining a data set image with more data volume and richer information.

[0088] In one embodiment of the present application, a data set image is imported into the Labelme tool, and a truck, a car, and a wheel (axle) are marked with a rectangular frame. After each image is labeled, a json file is generated, and then a program is written in Python to realize the function of batch conversion of json files into txt files. The shuffle function in the random library of Python is used to randomly shuffle the data set images and txt files, and the shuffled data set is divided into a training set, a validation set, and a test set, wherein the training set is 2800, the validation set is 800, and the test set is 400, and the division ratio is 7:2:1.

[0089] In one embodiment of the present application, L VFL Function as the classification loss function, with L DFL The function is used as the regression loss function to construct the joint loss function of the traffic detection network. The calculation formula of the joint loss function is as follows:

[0090] Loss = α × L VFL +β×L DFL

[0091] Among them, α and β represent the weight coefficients of each loss function.

[0092] Furthermore, the calculation formula of the classification loss function is as follows:

[0093]

[0094] Among them, p represents the predicted probability, q represents the true category label, for the positive samples in training, q is the intersection over union (IoU) between the generated predicted box and the true box, for the negative samples in training, the training target q of all categories is 0, is the loss weight of the image foreground and background, and τ is the weight of each sample data.

[0095] Furthermore, the calculation formula of the regression loss function is as follows:

[0096] L DFL (S i ,S i+1 )=-((g i+1 -g)log(S i )+(gg i )log(S i+1 ))

[0097]

[0098] Among them, S i represents the predicted probability of the category, g represents the true value, and g i and g i+1are the two predicted values ​​on the left and right of the true value g, that is, g i <g<g i+1 , i is the index of the bounding box.

[0099] By continuously optimizing network parameters and adjusting hyperparameters to tune the traffic detection network, the predicted value of the traffic detection network is made closer to the true value, so as to achieve the best effect of the traffic detection network.

[0100] Specifically, in one embodiment of the present application, the traffic detection network is:

[0101] The feature extraction module includes a first convolution layer, a first feature pooling unit and four feature extraction units, wherein the four feature extraction units are respectively a first feature extraction unit, a second feature extraction unit, a third feature extraction unit and a fourth feature extraction unit, wherein the four feature extraction units have the same structure but different numbers of channels;

[0102] The feature fusion module includes a second feature pooling unit and four feature fusion units, wherein the four feature fusion units are a first feature fusion unit, a second feature fusion unit, a third feature fusion unit and a fourth feature fusion unit;

[0103] The feature locking module includes a detection unit, a parsing unit and a locking unit, which detects the fused features and determines whether they meet the parsing requirements. If they meet the parsing requirements, the vehicle features are locked.

[0104] The feature extraction module is used to effectively extract the multi-dimensional feature information of the vehicle image. At the same time, the feature extraction units of four different channels can capture the features of different aspects of the vehicle, such as appearance, size, number of axles, etc. The multi-dimensional feature information output by the feature extraction module is effectively fused through the feature fusion module, and the feature locking module is used to detect and judge the fused vehicle feature information to achieve effective locking of the vehicle feature information.

[0105] In one embodiment of the present application, the first convolution layer includes a convolution kernel with a size of 3×3 and a stride of 2, a SiLU activation function layer, and a batch normalization (BN) layer.

[0106] It can be understood that when the initial image is input into the first convolutional layer, the size, height and width of the initial image are halved, and the number of channels of the output image of the first convolutional layer is equal to the number of channels of the first convolutional layer.

[0107] In one embodiment of the present application, the size of the input image is 640 in height, 640 in width, and the number of channels is 3, i.e., 640×640×3. The number of channels of the first convolutional layer is C0. After the input image is processed by the first convolutional layer, the output image is 320×320×C0.

[0108] In one embodiment of the present application, the input image is 640×640×3, the number of channels of the first feature extraction unit is C1, the number of channels of the second feature extraction unit is C2, the number of channels of the third feature extraction unit is C3, and the number of channels of the fourth feature extraction unit is C4. The images output by the four feature extraction units are 160×160×C1, 80×80×C2, 40×40×C3, and 20×20×C4, respectively.

[0109] Furthermore, if Figure 3 As shown, the feature extraction unit includes five convolutional layers and a first aggregation layer. The five convolutional layers are respectively a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer, wherein the second convolutional layer includes a convolution kernel with a size of 3×3 and a step size of 2, a ReLU activation function layer and a batch normalization layer, the fourth convolutional layer and the fifth convolutional layer have different sizes, and the sixth convolutional layer has the same structure as the third convolutional layer.

[0110] Among them, the structures of the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer can be set according to actual usage requirements, and this application does not make specific settings for this.

[0111] The multi-level convolution structure of 5 convolutional layers is used to effectively extract the multi-scale feature information of the vehicle image. An aggregation layer is added after the 5 convolutional layers to effectively aggregate the extracted feature information, enhance the representativeness of the features, and fully capture the rich feature information of the vehicle image.

[0112] Furthermore, if Figure 4 As shown, the feature fusion unit includes a third aggregation layer, an upsampling layer and a third pooling unit, wherein the structure and pooling operation of the third pooling unit are the same as those of the first feature pooling unit.

[0113] Specifically, a specific embodiment is used to illustrate:

[0114] The two feature maps are input into the third aggregation layer for aggregation, the result of the third aggregation layer is input into the upsampling layer for upsampling, and the result of the upsampling layer is input into the third pooling unit for pooling operation to obtain fused features.

[0115] It can be understood that the upsampling layer adjusts the scale of the feature map, doubling the height and width of the feature map of the third aggregation layer to provide a suitable input scale for subsequent feature processing.

[0116] Furthermore, if Figure 5As shown, the first feature pooling unit includes a seventh convolutional layer, two maximum pooling layers and a second aggregation layer, wherein the convolution kernel of the seventh convolutional layer is 1x1, the step size is 1, the two maximum pooling layers include a first maximum pooling layer and a second maximum pooling layer, and the first maximum pooling layer is skipped with the second aggregation layer. The pooling operation of the first feature pooling unit is:

[0117] The feature map of the feature extraction unit is input into the seventh convolutional layer for convolution operation to obtain the feature map y1;

[0118] Input the feature map y1 to the first maximum pooling layer for the first pooling operation to obtain the feature map y2;

[0119] Input the feature map y2 to the second maximum pooling layer for a second pooling operation to obtain the feature map y3;

[0120] The feature map y2 and the feature map y3 are input into the second aggregation layer for aggregation to obtain the feature map of the first feature pooling unit.

[0121] By using two maximum pooling layers for continuous pooling operations, the size and information content of the feature map are effectively reduced. A 1x1 convolution layer is added before the pooling operation to adjust the number of channels and compress the features of the feature map, thereby improving the quality of feature representation. The feature maps y2 and y3 after the two pooling operations are aggregated and feature information of different scales is combined to obtain a richer and more representative feature representation. This enables efficient multi-scale feature extraction and fusion of the input feature map, effectively reducing the dimension and information redundancy of the feature map and retaining important vehicle feature information.

[0122] In one embodiment of the present application, the pooling kernel size of the maximum pooling layer in the first feature pooling unit is k=5, the step size is 1, and the padding is 2.

[0123] Furthermore, if Figure 6 As shown, the second feature pooling unit includes an eighth convolutional layer, two maximum pooling layers and a fourth aggregation layer, wherein the convolution kernel of the eighth convolutional layer is 1x1, the step size is 1, the two maximum pooling layers include a third maximum pooling layer and a fourth maximum pooling layer, a jump connection between the eighth convolutional layer and the third aggregation layer, a jump connection between the third maximum pooling layer and the fourth aggregation layer, and a jump connection between the fourth maximum pooling layer and the third aggregation layer. The pooling operation of the second feature pooling unit is:

[0124] The feature map of the first feature pooling unit is input into the eighth convolutional layer for convolution operation to obtain the feature map x1;

[0125] Input feature map x1 to the third maximum pooling layer for the first pooling operation to obtain feature map x2;

[0126] Input feature map x2 to the fourth maximum pooling layer for a second pooling operation to obtain feature map x3;

[0127] The feature map x1, the feature map x2 and the two feature maps x3 are input into the fourth aggregation layer for aggregation to obtain the feature map of the second feature pooling unit.

[0128] By aggregating and fusing the feature maps x1, x2 and two feature maps x3 output by different pooling layers, feature information of multiple scales is combined to further enhance the capability of the feature fusion module.

[0129] In one embodiment of the present application, the pooling kernel size of the maximum pooling layer in the second feature pooling unit is k=5, the step size is 1, and the padding is 2.

[0130] In one embodiment of the present application, the training process includes:

[0131] S21, obtaining a vehicle driving dynamic video sequence captured by a roadside camera on the monitored road, extracting frames from the vehicle driving dynamic video sequence to obtain a video image sequence, and inputting the video image sequence into a traffic detection network in sequence;

[0132] S22, using the feature extraction module to extract features from the video image sequence respectively, to obtain five feature sequences of different scales {F i}, i = 1, 2, 3, 4, 5;

[0133] S23, using the feature fusion module to fusion the five feature sequences of different scales {F i} respectively perform feature fusion to obtain the fused feature sequence {P j}, j = 1, 2, 3, 4;

[0134] S24, using the feature locking module to fusion feature sequence {P j} to parse and detect, the detection unit performs fusion feature sequence {P j} Perform vehicle feature detection respectively to obtain a vehicle feature sequence. If the vehicle features in the vehicle feature sequence meet the parsing requirements of the parsing module, the vehicle features are locked to obtain vehicle feature information.

[0135] The feature extraction module extracts features of five different scales from the video image, which can capture the feature information of the vehicle at different levels. The feature fusion module is used to fuse the features of five different scales, integrate the advantages of features of different scales, and enhance the ability of feature representation. The feature locking module is used to detect and analyze the fused features, accurately lock the feature information of the vehicle, and improve the accuracy of vehicle dynamic monitoring.

[0136] Furthermore, step S22 specifically includes:

[0137] S221, using the first convolutional layer to process any video image in the video image sequence to obtain a feature map F1;

[0138] S222, inputting the feature map F1 into the first feature extraction unit for first feature extraction to obtain a feature map F2;

[0139] S223, inputting the feature map F2 into the second feature extraction unit for a second feature extraction to obtain a feature map F2';

[0140] S224, inputting the feature map F2' into the first feature pooling unit for pooling to obtain a feature map F3;

[0141] S225, inputting the feature map F3 into a third feature extraction unit for performing a third feature extraction to obtain a feature map F4;

[0142] S226, inputting the feature map F4 into a fourth feature extraction unit for performing a fourth feature extraction to obtain a feature map F5;

[0143] S227, repeating steps S221-S226 until all the video images in the video image sequence are feature extracted, and obtaining five feature sequences of different scales {F i}.

[0144] Through multiple feature extractions, features of different semantic levels are extracted layer by layer from the video image. The pooling operation is used to reduce the size of the feature map and extract higher-order features while reducing the computational complexity. The pooled feature map is then used for feature extraction to better capture the dynamic information and structural characteristics of the vehicle.

[0145] Further, step S23 specifically includes:

[0146] S231, input any feature map in the feature map sequence {F5} into the second feature pooling unit for pooling operation, and obtain a feature map F P ;

[0147] S232, feature map F4 and feature map F P Input to the fourth feature fusion unit for the first feature fusion to obtain feature map P1;

[0148] S233, inputting the feature map P1 and the feature map F3 into a third feature fusion unit for a second feature fusion to obtain a feature map P2;

[0149] S234, inputting the feature map P2 and the feature map F2 into the second feature fusion unit for third feature fusion to obtain the feature map P3;

[0150] S235, inputting the feature map P3 and the feature map F1 into the first feature fusion unit for fourth feature fusion to obtain a feature map P4;

[0151] S236, repeat steps S231-S235 until all feature graphs in the feature graph sequence {F5} are subjected to feature fusion to obtain a fused feature sequence {P j}.

[0152] By gradually fusing feature maps of different levels using four feature fusion units, the fusion process gradually integrates feature information of different semantic levels from high-order features to low-order features, and fully utilizes the advantages of features at each level. In each feature fusion unit, the two input feature maps are fused to integrate their channel information. Through gradual feature fusion, the semantic information of the feature maps is gradually enhanced, and the spatial resolution is gradually reduced, so as to better capture visual information of different scales and improve the accuracy of detection and estimation.

[0153] S3. Acquire the real-time dynamic images of vehicles traveling captured by the roadside cameras on the monitored roads, input the real-time dynamic images of vehicles traveling into the traffic detection network to lock vehicle characteristics, and obtain vehicle characteristic information.

[0154] In one embodiment of the present application, vehicle feature locking specifically includes:

[0155] The detection unit of the feature locking module is used to fusion feature P j Detection is performed to obtain the vehicle feature sequence R, where R = {r1, r2, ..., r n}, vehicle characteristics include but are not limited to vehicle model, axle, vehicle length, color, speed, etc.;

[0156] The vehicle feature sequence R is parsed using a parsing unit, and corresponding feature subsequences R1, R2, ..., R are created according to the elements of the vehicle feature sequence R. n ;

[0157] If the feature subsequence is stable within the M frames, the locking unit is used to lock the vehicle feature to obtain the vehicle feature information.

[0158] Specifically, if the feature subsequence R1 is stable within the M frames, the vehicle model feature is locked;

[0159] If the vehicle type feature is locked as a car, when the axle feature subsequence R2 is stable in the M1 frame, the axle feature is locked; if the vehicle type feature is locked as a bus, when the axle feature subsequence R2 is stable in the M2 frame, the axle feature is locked; if the vehicle type feature is locked as a truck, when the axle feature subsequence R2 is stable in the M3 frame, the axle feature is locked;

[0160] If the vehicle type feature is locked as a car, when the vehicle length feature subsequence R3 is stable in the M4 frame, the vehicle length feature is locked; if the vehicle type feature is locked as a bus, when the vehicle length feature subsequence R3 is stable in the M5 frame, the vehicle length feature is locked; if the vehicle type feature is locked as a truck, when the vehicle length feature subsequence R3 is stable in the M6 ​​frame, the vehicle length feature is locked;

[0161] If the vehicle model feature is locked as a car, the color feature is locked when the color feature subsequence R4 is stable in the M7 frame; if the vehicle model feature is locked as a bus, the color feature is locked when the color feature subsequence R4 is stable in the M8 frame; if the vehicle model feature is locked as a truck, the color feature is locked when the color feature subsequence R4 is stable in the M9 frame, where M and M1~M9 are natural numbers.

[0162] S4. Vehicle characteristic information is integrated with vehicle dynamic weighing parameters to obtain vehicle parameters; the vehicle parameters include vehicle type, axle, vehicle length, color characteristics, and vehicle weight data.

[0163] Specifically, the vehicle dynamic weighing parameters are formed into a sequence Q according to time, Q = {Q1, Q2, ..., Q m}, where Q1, Q2, ..., Q m It represents the record of dynamic weighing parameters of different vehicles over time, and integrates vehicle characteristic information with vehicle dynamic weighing parameters according to time to obtain vehicle parameters.

[0164] The present application obtains vehicle driving dynamic images captured in real time by roadside cameras, inputs the vehicle driving dynamic images into a traffic detection network to realize real-time vehicle feature locking and parameter analysis, obtains vehicle feature information, combines the parameters obtained by the vehicle dynamic weighing system and the vehicle feature information extracted by the traffic detection network, obtains more comprehensive and accurate vehicle parameters, realizes real-time monitoring and analysis of vehicle dynamic parameters, and improves the accuracy and reliability of vehicle status assessment.

[0165] The present invention provides a vehicle dynamic weighing and multi-parameter detection system based on a roadside camera, which adopts the vehicle dynamic weighing and multi-parameter detection method described above, including:

[0166] A weighing parameter acquisition module is configured to install a vehicle dynamic weighing system on the monitored road and acquire vehicle dynamic weighing parameters according to the vehicle dynamic weighing system;

[0167] A network construction and training module configured to construct and train a traffic detection network that locks on stable features of vehicles;

[0168] A vehicle feature acquisition module is configured to acquire real-time dynamic images of vehicles traveling captured by roadside cameras on the monitored roads, input the real-time dynamic images of vehicles traveling into a traffic detection network to lock vehicle features, and obtain vehicle feature information;

[0169] The multi-parameter detection module is configured to obtain vehicle parameters by fusing vehicle characteristic information with vehicle dynamic weighing parameters.

[0170] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle dynamic weighing and multi-parameter detection method based on a roadside camera, characterized in that: The following steps are involved: S1. Install a vehicle dynamic weighing system on the monitored road and obtain vehicle dynamic weighing parameters according to the vehicle dynamic weighing system; S2. Build and train a traffic detection network that locks vehicle stability features; the traffic detection network includes a feature extraction module, a feature fusion module, and a feature locking module. The feature locking module detects and analyzes the fusion features and locks the vehicle features; S3, obtaining a real-time dynamic picture of the vehicle driving captured by a roadside camera on the monitored road, inputting the real-time dynamic picture of the vehicle driving into a traffic detection network to lock the vehicle characteristics, and obtaining the vehicle characteristic information; S4. The vehicle characteristic information is integrated with the vehicle dynamic weighing parameters to obtain vehicle parameters; the vehicle parameters include vehicle type, axle, vehicle length, color characteristics, and vehicle weight data; The traffic detection network is: The feature extraction module includes a first convolution layer, a first feature pooling unit and four feature extraction units, wherein the four feature extraction units are respectively a first feature extraction unit, a second feature extraction unit, a third feature extraction unit and a fourth feature extraction unit, wherein the four feature extraction units have the same structure but different numbers of channels; The feature fusion module includes a second feature pooling unit and four feature fusion units, wherein the four feature fusion units are a first feature fusion unit, a second feature fusion unit, a third feature fusion unit and a fourth feature fusion unit; The feature locking module includes a detection unit, a parsing unit and a locking unit, which detects the fusion feature and determines whether it meets the parsing requirements. If it meets the parsing requirements, the vehicle feature is locked; The training process includes: S21, obtaining a vehicle driving dynamic video sequence captured by a roadside camera on the monitored road, extracting frames from the vehicle driving dynamic video sequence to obtain a video image sequence, and inputting the video image sequence into a traffic detection network in sequence; S22, using the feature extraction module to extract features from the video image sequence respectively, to obtain five feature sequences of different scales {F i }, i = 1, 2, 3, 4, 5; S23, using the feature fusion module to fusion the five feature sequences of different scales {F i } respectively perform feature fusion to obtain the fused feature sequence {P j }, j = 1, 2, 3, 4; S24, using the feature locking module to fusion feature sequence {P j } to parse and detect, the detection unit performs fusion feature sequence {P j } Perform vehicle feature detection respectively to obtain a vehicle feature sequence. If the vehicle features in the vehicle feature sequence meet the parsing requirements of the parsing module, the vehicle features are locked to obtain vehicle feature information.

2. A vehicle dynamic weighing and multi-parameter detection method based on a roadside camera as claimed in claim 1, characterized in that: The feature extraction unit includes five convolutional layers and a first aggregation layer. The five convolutional layers are respectively a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer, wherein the second convolutional layer includes a convolution kernel with a size of 3×3 and a step size of 2, a ReLU activation function layer and a batch normalization layer, the fourth convolutional layer and the fifth convolutional layer have different sizes, and the sixth convolutional layer has the same structure as the third convolutional layer.

3. A vehicle dynamic weighing and multi-parameter detection method based on a roadside camera as claimed in claim 2, characterized in that: The first feature pooling unit includes a seventh convolutional layer, two maximum pooling layers and a second aggregation layer, wherein the convolution kernel of the seventh convolutional layer is 1x1, the step size is 1, the two maximum pooling layers include a first maximum pooling layer and a second maximum pooling layer, and the first maximum pooling layer is skip-connected to the second aggregation layer. The pooling operation of the first feature pooling unit is: The feature map of the feature extraction unit is input into the seventh convolutional layer for convolution operation to obtain the feature map y1; Input the feature map y1 to the first maximum pooling layer for the first pooling operation to obtain the feature map y2; Input the feature map y2 to the second maximum pooling layer for a second pooling operation to obtain the feature map y3; The feature map y2 and the feature map y3 are input into the second aggregation layer for aggregation to obtain the feature map of the first feature pooling unit.

4. A vehicle dynamic weighing and multi-parameter detection method based on a roadside camera as claimed in claim 3, characterized in that: The second feature pooling unit includes an eighth convolutional layer, two maximum pooling layers and a fourth aggregation layer, wherein the convolution kernel of the eighth convolutional layer is 1x1, the step size is 1, the two maximum pooling layers include a third maximum pooling layer and a fourth maximum pooling layer, a jump connection is formed between the eighth convolutional layer and the third aggregation layer, a jump connection is formed between the third maximum pooling layer and the fourth aggregation layer, and a jump connection is formed between the fourth maximum pooling layer and the third aggregation layer. The pooling operation of the second feature pooling unit is: The feature map of the first feature pooling unit is input into the eighth convolutional layer for convolution operation to obtain the feature map x1; Input feature map x1 to the third maximum pooling layer for the first pooling operation to obtain feature map x2; Input feature map x2 to the fourth maximum pooling layer for a second pooling operation to obtain feature map x3; The feature map x1, the feature map x2 and the two feature maps x3 are input into the fourth aggregation layer for aggregation to obtain the feature map of the second feature pooling unit.

5. The vehicle dynamic weighing and multi-parameter detection method based on a roadside camera as claimed in claim 1, characterized in that: Step S22 specifically includes: S221, using the first convolutional layer to process any video image in the video image sequence to obtain a feature map F1; S222, inputting the feature map F1 into the first feature extraction unit for first feature extraction to obtain a feature map F2; S223, inputting the feature map F2 into the second feature extraction unit for a second feature extraction to obtain a feature map F2'; S224, inputting the feature map F2' into the first feature pooling unit for pooling to obtain a feature map F3; S225, inputting the feature map F3 into a third feature extraction unit for performing a third feature extraction to obtain a feature map F4; S226, inputting the feature map F4 into a fourth feature extraction unit for performing a fourth feature extraction to obtain a feature map F5; S227, repeating steps S221-S226 until all the video images in the video image sequence are feature extracted, and obtaining five feature sequences of different scales {F i }.

6. A vehicle dynamic weighing and multi-parameter detection method based on a roadside camera as claimed in claim 5, characterized in that: Step S23 specifically includes: S231, input any feature map in the feature map sequence {F5} into the second feature pooling unit for pooling operation, and obtain a feature map F P ; S232, feature map F4 and feature map F P Input to the fourth feature fusion unit for the first feature fusion to obtain feature map P1; S233, inputting the feature map P1 and the feature map F3 into a third feature fusion unit for a second feature fusion to obtain a feature map P2; S234, inputting the feature map P2 and the feature map F2 into the second feature fusion unit for third feature fusion to obtain the feature map P3; S235, inputting the feature map P3 and the feature map F1 into the first feature fusion unit for fourth feature fusion to obtain a feature map P4; S236, repeat steps S231-S235 until all feature graphs in the feature graph sequence {F5} are subjected to feature fusion to obtain a fused feature sequence {P j }.

7. The vehicle dynamic weighing and multi-parameter detection method based on a roadside camera as claimed in claim 1, characterized in that: The training also includes: L VFL Function as the classification loss function, with L DFL The function is used as the regression loss function to construct the joint loss function of the traffic detection network. The calculation formula of the joint loss function is as follows: Loss=α×L VFL +β×L DFL ; Among them, α and β represent the weight coefficients of each loss function; The calculation formula of the classification loss function is as follows: Among them, p represents the predicted probability, q represents the true category label, is the loss weight of the image foreground and background, and τ is the weight of each sample data; The calculation formula of the regression loss function is as follows: L DFL (S i ,S i+1 )=-((g i+1 -g)log(S i )+(g-g i )log(S i+1 )); Among them, S i represents the predicted probability of the category, g represents the true value, and g i and g i+1 are the two predicted values ​​on the left and right of the true value g, and i represents the index of the bounding box.

8. A vehicle dynamic weighing and multi-parameter detection system based on a roadside camera, characterized in that: The vehicle dynamic weighing and multi-parameter detection method according to any one of claims 1 to 7 comprises: A weighing parameter acquisition module is configured to install a vehicle dynamic weighing system on the monitored road and acquire vehicle dynamic weighing parameters according to the vehicle dynamic weighing system; A network construction and training module configured to construct and train a traffic detection network that locks on stable features of vehicles; A vehicle feature acquisition module is configured to acquire real-time dynamic images of vehicles traveling captured by roadside cameras on the monitored roads, input the real-time dynamic images of vehicles traveling into a traffic detection network to lock vehicle features, and obtain vehicle feature information; The multi-parameter detection module is configured to obtain vehicle parameters by fusing vehicle characteristic information with vehicle dynamic weighing parameters.

Images