A vehicle axle load distribution recognition method based on computer vision technology

Abstract

The application discloses a vehicle axle load distribution recognition method based on computer vision technology, comprising the following steps: S1, a YOLOv4 model is used to construct a visual recognizer, and the YOLOv4 model is optimized; S2, a vehicle axle load distribution prediction model based on a generative adversarial network is constructed; S3, a training set of the visual recognizer is constructed, the visual recognizer is trained, and the performance of the visual recognizer is tested; S4, a data set is constructed, the vehicle axle load distribution prediction model is trained, and optimal model parameters are obtained; and S5, the trained visual recognizer and the vehicle axle load distribution prediction model are used for vehicle axle load distribution prediction and recognition in an actual scene. According to the application, computer vision technology is used to recognize information such as the number of vehicle axles, real-time structural responses of a bridge are acquired through a bridge sensor, and the axle load probability distribution of a vehicle is predicted through a generative adversarial network.

Description

Technical Field

[0001] This invention belongs to the field of vehicle load recognition technology, specifically relating to a method for identifying vehicle axle load distribution based on computer vision technology. Background Technology

[0002] Bridge Weigh-in-Motion (BWIM) technology is a vehicle load identification technique that uses sensors installed on railway bridges to measure the bridge response under vehicle loads, and then calculates vehicle weight and axle information based on the bridge response. By acquiring real-time bridge responses and combining them with structural influence line theory, vehicle loads on the bridge can be estimated using the bridge structure's response to traffic loads. Bridge Weigh-in-Motion has broad application prospects in practical bridge operation and management. Obtaining actual vehicle loads on bridges is of great significance for traffic data statistics and safety assessments of bridges in service.

[0003] Currently, most BWIM systems on the market are based on the Moses algorithm, which obtains vehicle weight information by optimizing the difference between the measured and predicted bridge responses. Its advantages include low system parameters, simple implementation, and low cost, making it the preferred choice for most BWIM systems. However, simply having a high-efficiency and low-cost algorithm is far from sufficient. Numerous studies and practical applications have shown that this technology also has the following limitations: First, BWIM system sensors are installed on the bridge structure, without direct contact with vehicles. Conventional sensors can only acquire limited traffic information, obtaining only real-time structural responses. They cannot directly detect vehicle axle counts, speeds, and travel trajectories, which are crucial for vehicle load identification. Second, even if information such as vehicle axle counts, speeds, and travel paths can be obtained, it is difficult to predict the weight of a single axle of an individual vehicle, resulting in significant identification errors. Summary of the Invention

[0004] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and to propose a vehicle axle load distribution recognition method based on computer vision technology.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for identifying vehicle axle load distribution based on computer vision technology includes the following steps:

[0007] S1. Construct a visual recognizer using the YOLOv4 model and optimize the YOLOv4 model.

[0008] S2. Construct a vehicle axle load distribution prediction model based on generative adversarial networks;

[0009] S3. Construct a training set for the visual recognizer, train the visual recognizer, and test its performance.

[0010] S4. Construct a dataset, train the vehicle axle load distribution prediction model, and obtain the optimal model parameters;

[0011] S5. Apply the trained visual recognizer and vehicle axle load distribution prediction model to predict and recognize vehicle axle load distribution in real-world scenarios.

[0012] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0013] 1. This invention utilizes computer vision technology to accurately obtain real-time vehicle information. Through deep learning-based computer vision technology, even in complex traffic conditions, it can still accurately and in real-time obtain the number of axles, speed, type, and spatial position of vehicles on the bridge from traffic monitoring videos. The identified axle spatiotemporal trajectories are fused with the bridge's strain data, thus providing the necessary known conditions for a bridge dynamic weighing algorithm based on the inverse problem of influence line theory. This information is input into the vehicle axle load prediction network as an important input, and finally, the probability distribution of axle loads and the total vehicle weight are solved. This invention can accurately identify vehicle weights in complex traffic conditions, solving the problem of significant accuracy degradation in existing bridge dynamic weighing technologies under complex traffic conditions.

[0014] 2. In predicting real-time vehicle axle load, this invention employs a generative adversarial network (GAN), which leverages its powerful learning capabilities to establish a mapping relationship between vehicle axle load, vehicle position, and bridge strain. By minimizing the loss function through constraint equations, the prediction model with optimal parameters can significantly improve the accuracy of previous vehicle axle load identification methods.

[0015] 3. The method of the present invention can also be used to identify multi-granularity traffic information such as the number, type, number of axles, real-time location, movement trajectory, and driving speed of vehicles on the bridge, as well as structural state parameters such as the influence surface of the bridge structure. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention;

[0017] Figure 2 This is the optimization flowchart for the YOLOv4 model;

[0018] Figure 3 This is a schematic diagram of a generative adversarial network data flow framework;

[0019] Figure 4 This is a schematic diagram of a generative adversarial network framework;

[0020] Figure 5 This is a schematic diagram of the scaling experiment in the embodiment;

[0021] Figure 6 This is a schematic diagram of the time history of the bridge structure response in a simulation experiment;

[0022] Figure 7 This is a schematic diagram of the bridge response time history generated by a generative adversarial network simulation;

[0023] Figure 8 This is the predicted axle load distribution diagram of vehicle Experiment 1 in the embodiment;

[0024] Figure 9 This is the predicted axle load distribution diagram of vehicle Experiment 2 in the embodiment;

[0025] Figure 10 This is the predicted axle load distribution diagram for vehicle Experiment 3 in the embodiment. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0027] Example

[0028] like Figure 1 As shown, the present invention provides a vehicle axle load distribution recognition method based on computer vision technology, comprising the following steps:

[0029] S1. Construct a visual recognizer using the YOLOv4 model and optimize the YOLOv4 model.

[0030] A visual recognizer is constructed using the YOLOv4 model, consisting of five parts: input, backbone, neck, head, and output. The input includes training on datasets using Mosaic data augmentation, cmBN, and SAT self-adversarial training. The backbone uses Darknet53 with CSP. The neck network fuses feature information from feature maps of different sizes. The head network is trained using the CIOU_Loss loss function. The output provides the object detection and recognition results.

[0031] The YOLOv4 model uses SPP (Spatial Pyramid Pooling) and PAN (Path Aggregation Network) for parsing.

[0032] like Figure 2 As shown, channel pruning is used to optimize the YOLOv4 model, specifically as follows:

[0033] Using the torch_pruning library, the convolutional layers of the backbone network are pruned as needed. The process includes determining the pruning rate and selecting weights. After pruning is complete, the original weights should be backed up. The optimized weights and the original weights will be compared and used to further optimize the network.

[0034] S2. Construct a vehicle axle load distribution prediction model based on generative adversarial networks;

[0035] The input to the vehicle axle load distribution prediction model includes the observed bridge structural response ε. p And vehicle information T(n, v, a) identified by a vision recognition device;

[0036] The output of the vehicle axle load distribution prediction model is the generated bridge structural response data ε. G And the generated vehicle axle load distribution prediction data P G ;

[0037] Where, ε p The actual bridge structural response observation data is used to train the discriminator D. Simultaneously, during the continuous training and iterative optimization of the discriminator D, the generator G is trained, ultimately completing the training of the generative adversarial network. T(n, v, a) contains the number of axles n, the vehicle's speed v, and its lane number a. T(n, v, a) is used to generate the vehicle axle load distribution prediction data P. G The input generator G uses vehicle information as a label optimization generator; ε G As a generator P G The intermediate data input is used to observe its comparison with the actual structural response data ε. p The difference ensures the generator's performance; P G The final data output by the generative adversarial network is the estimated complete probability distribution of vehicle axle load; P G The generation process requires the establishment of constraint equations, P G and ε G A variable z from a certain distribution passes through a function and f θ (z) mapping.

[0038] like Figure 3 and Figure 4 As shown, the construction of the vehicle axle load distribution prediction model includes building a generator G and a discriminator D, establishing constraint equations, training the generator, and training the discriminator.

[0039] The generator G and discriminator D are constructed using a feedforward neural network (FNN).

[0040] The feedforward neural network (FNN) that constructs the generator G and discriminator D consists of an input layer, n hidden layers, and an output layer. There is no feedback in the entire network, and the signal propagates unidirectionally from the input layer to the output layer. The number of hidden layers n is set as needed. During training, gradient descent is used to train the model parameters.

[0041] Establish the constraint equations as follows:

[0042] Based on influence line theory, the relationship between vehicle axle load and bridge structural response is established using the following formula:

[0043]

[0044] Where n is the number of axles in the vehicle, I i Let P be the strain influence line at the location of the i-th axle of the vehicle. i Let the weight of the i-th axle be denoted as 'i'.

[0045] Establish the equations for the vehicle during time period t:

[0046]

[0047] Where k is the number of time steps, I is the influence line matrix, and P is the vector of vehicle axle load;

[0048] Create a generator class ε G With P G Constraint equations:

[0049] r constrain =ε G -I(t)P G

[0050]

[0051] During the training of a generative model, it is necessary to minimize the loss function. This ensures that the generated P G The result is consistent with the actual vehicle axle load P;

[0052] Due to vehicle axle load P G The value must be greater than 0. The ReLU function is used to eliminate the generated negative number P. G Impact:

[0053]

[0054] Finally, a loss function based on the above two constraint equations is obtained to train the generative model:

[0055]

[0056]

[0057] Generator training specifically involves:

[0058] To train the generator, the distribution P of the generated structural response is minimized. θ (t) and the distribution of the measured bridge response Q θ KL divergence between (t):

[0059]

[0060] Based on Bayes' criterion, it can be simplified to:

[0061]

[0062] Based on the constraint equations established above, the constraint equations for the generative model are as follows:

[0063]

[0064] Here, α is an adjustable coefficient that determines the degree of influence of the constraint equations on the generator training process.

[0065] Discriminator training specifically involves:

[0066] The loss function is established by minimizing the binary cross-entropy of the discriminator, i.e.:

[0067] L D (γ)=-E[log(D γ (t,f θ (z)))]-E[log(1-D γ (t,ε P ))]

[0068] Ultimately, the training objective of the vehicle axle load distribution prediction model is:

[0069]

[0070]

[0071] S3. Construct a training set for the visual recognizer, train the visual recognizer, and test its performance, specifically as follows:

[0072] For the training samples of the visual recognition device, the vehicles passing through the bridge in real time in the collected monitoring video are marked by manual labeling, indicating their lane, measuring their actual speed and the number of axles of the vehicles, and forming the training set.

[0073] Set a detection error threshold, train the visual recognizer using the labeled training set, and obtain a set of network coefficients that meet the requirements. The specific steps are as follows:

[0074] Prepare the YOLOv4 model and dataset, which includes manually labeled and unlabeled images; determine the number of training epochs, and based on the equipment configuration used, determine the number of images and the number of working cores; input the preset error threshold; after completing the above steps and determining the parameters, start training; after training is complete, import the resulting weight file into the YOLOv4 model.

[0075] To perform performance testing on the visual recognizer, the trained visual recognizer is used to perform target recognition tests. The specific steps are as follows:

[0076] Test sets are prepared by manually labeling vehicle operation information. A trained visual recognition device is then used to identify the test sets. The recognition results are compared with the manually labeled information. If the recognition is completed within the allowable error range, the performance is considered reliable. Vehicle operation information includes the number of axles, speed, type, and spatial location.

[0077] S4. Construct a dataset and train the vehicle axle load distribution prediction model to obtain the optimal model parameters, specifically:

[0078] Vehicles are driven under various load conditions to collect real bridge strain data, which is then used as the training set for the discriminator in the GAN, minimizing the loss function L. D (γ) is then used to obtain the optimal parameters. In this embodiment, four load conditions as shown in Table 1 are used for training.

[0079]

[0080] Table 1

[0081] S5. Apply the trained visual recognizer and vehicle axle load distribution prediction model to predict and recognize vehicle axle load distribution in real-world scenarios.

[0082] To verify the method of this embodiment, a scaled-down bridge model test was first conducted. The setup of the scaled-down test is as follows: Figure 5 As shown. This scaled-down bridge model is designed with reference to a three-span reinforced concrete box girder bridge of a highway. The similarity ratio is 1:20. The model has a span length of 1m+1m+1m, a top plate width of 0.6m, a bottom plate width of 0.4m, a beam height of 0.07m, and 3 webs.

[0083] The vehicle model is a remote-controlled three-axis simulation truck, with a similarity ratio of 1:14 to the real truck. The truck model is 12cm wide, with wheelbases of 18cm and 8cm from front to rear. The front axle has a mass of 1.47kg, and the sum of the masses of the two rear axles is 1.17kg.

[0084] Three groups of test teams with different speeds were set up and passed through the middle lane in sequence, as shown in Table 2 below:

[0085] Experiment number Axle 1 (kg) Axle 2 (kg) Speed ​​(m / s) Lane 1 1.47 1.17 0.05 2 2 1.47 1.17 0.06 2 3 1.47 1.17 0.07 2

[0086] Table 2

[0087] Three DH1205 type sensors were installed at the bottom of the mid-span beam of the model, with a sampling frequency of 50Hz. The structural response of the bridge model collected by the three strain gauge sensors is as follows: Figure 6 As shown, the bridge response time history generated by the generative adversarial network simulation is as follows: Figure 7 As shown in the figure. The probability distribution of vehicle axle load obtained through prediction calculation is as follows. Figure 8 , Figure 9 as well as Figure 10 As shown in the figure, the predicted vehicle axle load distribution results show that Experiment 1 has a smaller variance and higher confidence level compared to Experiments 2 and 3, indicating that the method in this embodiment provides better prediction results under lower driving speed conditions.

[0088] It should also be noted that, in this specification, terms such as "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle axle load distribution recognition method based on computer vision technology, characterized in that, The method comprises the following steps: S1, a visual recognizer is constructed using a YOLOv4 model, and the YOLOv4 model is optimized; S2, constructing a vehicle axle load distribution prediction model based on a generative adversarial network; the input of the vehicle axle load distribution prediction model includes the observed bridge structure response ε p and the vehicle information T(n, v, a) identified by the visual identifier; The output of the vehicle axle load distribution prediction model is generated bridge structure response data ε G and generated vehicle axle load distribution prediction data P G ; wherein ε p As the real bridge structure response observation data, it is used to train the discriminator D, while the generator G is trained in the process of continuous training iteration optimization of the discriminator D, and finally the training of the generative adversarial network is completed; T(n, v, a) contains the axle number n of the vehicle, the vehicle travel speed v and the lane label a where the vehicle is located, and T(n, v, a) is input into the generator G when the vehicle axle load distribution prediction data P G is generated, and the vehicle information is used as the label to optimize the generator; ε G is used as the intermediate data input to observe the difference between P G and the real structure response data ε p to ensure the performance of the generator; P G is the final data output by the generative adversarial network, which is the estimated probability distribution of the vehicle axle load; P G The generation process needs to combine the established constraint equation, P G and ε G are mapped from a variable z of a certain distribution through functions and f θ (z); The construction of the vehicle axle load distribution prediction model comprises constructing a generator G and a discriminator D, establishing a constraint equation, generator training, and discriminator training; The generator G and the discriminator D are constructed using a feedforward neural network FNN; The feedforward neural network FNN of the generator G and the discriminator D is composed of an input layer, n hidden layers, and an output layer, and there is no feedback in the entire network, and the signal is unidirectionally propagated from the input layer to the output layer; the number n of hidden layers is set as required, and gradient descent is used to train the model parameters in the training process; S3, a training set of the visual recognizer is constructed, the visual recognizer is trained, and the performance thereof is tested; S4, a data set is constructed, the vehicle axle load distribution prediction model is trained, and the best model parameters are obtained; S5, the trained visual recognizer and the vehicle axle load distribution prediction model are used for vehicle axle load distribution prediction and identification in an actual scene. 2.The vehicle axle load distribution recognition method based on computer vision technology according to claim 1, characterized in that, The visual recognizer is constructed using the YOLOv4 model, and specifically comprises five parts: an input end, a backbone network, a neck network, a head network, and an output end; the input end comprises Mosaic data enhancement, cmBN, and SAT self-adversarial training processing data sets; the backbone network adopts darknet53 with CSP; the neck network is used for fusing feature information of feature maps of different sizes; the head network adopts a loss function CIOU_Loss for training; and the output end outputs a target detection and identification result; The YOLOv4 model adopts SPP plus PAN for analysis. 3.The vehicle axle load distribution recognition method based on computer vision technology according to claim 1, characterized in that, In step S1, the YOLOv4 model is optimized by using channel pruning, and specifically: torch_pruning is used to prune the convolutional layers of the backbone network according to requirements, and the pruning rate and weight are selected during the operation process; after the pruning is completed, the original weight should be backed up, and the optimized weight and the original weight are compared to continue optimizing the network.

4. The vehicle axle load distribution recognition method based on computer vision technology according to claim 1, characterized in that, The constraint equation is established, and specifically: According to the influence line theory, the following formula is used to establish the relationship between the vehicle axle load and the bridge structure response: where n is the number of vehicle axles, I i is the strain influence line at the location of the i-th vehicle axle, P i is the weight of the i-th vehicle axle; The equation of the vehicle in the time period t is established as: Wherein, k is the time step, I is the influence line matrix, and P is the vector of the vehicle axle load; Establishment of the generating class ε G With the constraint equation: G of the generating class ε r constrain = ε G -I(t)P G In the training process of the generation model, it is required to minimize the loss function This ensures that the generated P G The result is consistent with the actual vehicle axle load P; Since the vehicle axle load P G must be a value greater than 0, the ReLU function is used to eliminate the generated negative P G effect: Finally, the loss function based on the above two constraint equations is obtained to train the generation model:

5. The vehicle axle load distribution recognition method based on computer vision technology according to claim 4, characterized in that, The generator training is specifically: To train the generator, the KL divergence between the distribution P θ (t) of the generated structural response and the distribution Q θ (t) of the measured bridge response is minimized: According to the Bayesian criterion, it is simplified as: Combined with the constraint equation established above, the constraint equation of the generation model is: Wherein, a is an adjustable coefficient, which determines the influence degree of the constraint equation in the generator training process.

6. The vehicle axle load distribution recognition method based on computer vision technology according to claim 4, characterized in that, The discriminator training is specifically: The loss function is established by minimizing the binary cross entropy of the discriminator, that is: Finally, the training target of the vehicle axle load distribution prediction model is:

7. The computer vision technology-based vehicle axle load distribution identification method according to claim 1, characterized in that, Step S3 is specifically: For the training samples of the visual recognizer, the real-time passing vehicles on the bridge in the collected monitoring video are labeled by manual labeling, the lanes where the vehicles are located are marked, the actual speeds of the vehicles are measured, and the number of vehicle axles is measured to form a training set; Set a detection error threshold, train the visual recognizer with the labeled training set, and obtain a set of network coefficients that meet the requirements, the specific operation steps are: Prepare the YOLOv4 model and the data set, wherein the data set includes images that have been manually labeled and images that have not been labeled; determine the training rounds, determine the number of pictures and the number of cores according to the device configuration, input the preset error threshold, after the above steps are completed and the parameters are determined, start training, and the weight file obtained after the training is imported into the YOLOv4 model; Test the performance of the visual recognizer, use the trained visual recognizer to test the target recognition, the specific steps are: Prepare the test set by manually labeling the vehicle running information, use the trained visual recognizer to identify the test set, compare the identification result with the manually labeled information error, if the identification is completed within the error allowed range, it is determined that the performance is reliable; the vehicle running information includes the number of shafts, speed, type and spatial position. 8.The vehicle axle load distribution recognition method based on computer vision technology according to claim 6, characterized in that, Step S4 is specifically: The vehicle is arranged to travel under a plurality of different load conditions to collect real bridge strains, which are used as the training set for the discriminator in the GAN, minimizing the loss function L D (g) the optimal parameters are obtained afterwards.

Images