A method of monitoring the position of a vehicle axle load

Abstract

The application relates to a vehicle axle load position monitoring method, comprising the following steps: detecting a vehicle in a real-time monitoring picture based on a target detection model; extracting geometric features of the vehicle, establishing an indirect inference method of the axle load position, and establishing a vehicle axle load key point data set and a key point detection model based on the indirect inference method; training the key point detection model by using the vehicle axle load key point data set; based on the detection result of the key point detection model, transforming axle load coordinates in an image coordinate system into world coordinates in a world coordinate system through a homography matrix to obtain the axle load coordinates in the world coordinate system, and performing optimal estimation on a motion track of the vehicle axle load by using Kalman filtering and a kinematic bicycle model of the vehicle, so that real-time spatial distribution monitoring of the vehicle axle load is realized. Compared with the prior art, the application has the advantages that the positions of all axle load action points can be obtained according to a monocular image, the real-time spatial distribution monitoring precision of the axle load is high, and the like.

Description

Technical Field

[0001] This invention relates to the field of vehicle load monitoring, and in particular to a method for monitoring the position of vehicle axle loads. Background Technology

[0002] For bridges in operation, vehicle loads are the most critical load. Unlike other types of loads, vehicle loads exhibit significant uncertainty and randomness, thus greatly impacting the safety and durability of bridges during their operational period. With the continuous development of the logistics and transportation industry, vehicle overloading on bridges is becoming increasingly common, accelerating bridge performance degradation and even leading to serious damage (such as the overturning of main beams). Therefore, it is necessary to establish a vehicle load monitoring system to detect the spatiotemporal distribution of vehicle loads on bridges. Among these methods, computer vision-based detection is gaining increasing popularity due to its advantages such as low cost, real-time performance, and ease of maintenance.

[0003] A literature review of existing technologies revealed that research on vehicle load detection based on computer vision primarily focuses on detecting the vehicles themselves to accurately determine the real-time location of vehicle loads. CN112444311A discloses a method for monitoring the spatiotemporal load of vehicles on bridges. This method constructs a panoramic view of vehicles on the bridge deck using a vision system, obtains vehicle detection frames using a vehicle detection model, and then acquires or updates vehicle trajectory information through a pre-set improved Kalman filter model to monitor the spatiotemporal distribution of vehicle loads. However, its detection of vehicle loads is relatively coarse, generally treating the vehicle load as a concentrated load, and cannot effectively detect the real-time spatial distribution of vehicle axle loads. Summary of the Invention

[0004] The purpose of this invention is to provide a method for monitoring the position of vehicle axle loads, and to effectively monitor the real-time spatial distribution of vehicle axle loads.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A method for monitoring the axle load position of a vehicle includes the following steps:

[0007] Based on the target detection model, vehicles are detected in real-time monitoring footage;

[0008] Extract the geometric features of the vehicle and establish an indirect inference method for axle load position. Based on the indirect inference method, establish a vehicle axle load key point dataset and key point detection model.

[0009] A key point detection model was trained using a vehicle axle load key point dataset.

[0010] Based on the detection results of the key point detection model, the axle load coordinates in the image coordinate system are transformed to the world coordinate system through the homography matrix to obtain the axle load coordinates in the world coordinate system. Then, Kalman filtering and the vehicle's kinematic bicycle model are used to make the optimal estimation of the vehicle axle load motion trajectory, so as to realize the real-time spatial distribution monitoring of vehicle axle load.

[0011] The indirect inference method for axle load position is based on the detected axle load position and window position on one side of the vehicle. It is based on the assumption that the distance between the tires on both sides of the vehicle is equal to the width of the vehicle, and the assumption that the contact points between the vehicle tires and the ground form a rectangle. The method calculates the axle load position that is obscured by the vehicle body based on the rectangular features, thereby obtaining the position of all axle load action points through monocular images.

[0012] The key point detection model is based on HourglassNet. HourglassNet performs preliminary feature extraction on the image through several convolutional layers, and then performs secondary feature extraction on the image through multiple encoder-decoder structures. Each encoder-decoder structure is an Hourglass module. By stacking multiple Hourglass modules, a heatmap is generated to detect key points in the image. The output of the previous Hourglass module is directly used as the input of the next Hourglass module.

[0013] The basic structure of the Hourglass module includes two branches. One branch extracts deep features from the input array through multiple pooling operations and gradually restores the resolution to the original size through upsampling. The other branch only extracts features without changing the array size. When the Hourglass module outputs, it adds the arrays from the two branches, enabling the neural network to combine information from multiple scales in the image.

[0014] The keypoint detection model is trained using intermediate supervision, and the loss function of the model is set as the sigmoid cross-entropy between the feature map output by each Hourglass module and the labeled data.

[0015] Assuming homography matrix The coordinates of point i in the image coordinate system are The coordinates in the real coordinate system are If n is the number of reference points, then:

[0016]

[0017] By solving the above equation, the least squares solution of H is obtained, thus determining the transformation relationship between the image coordinate system and the world coordinate system as follows:

[0018]

[0019] The kinematic bicycle model is used to track and correct the trajectory of the axle load application point. The kinematic bicycle model uses four variables to describe the current state of the vehicle: center point coordinates (x, y) and velocity (v), and vehicle body attitude (θ). The variables describing the real-time position of the axle load application point of a car include: the length (h) and width (w) of the rectangle, the coordinates (x, y) of the center of the rectangle, and vehicle body attitude (θ).

[0020] According to the principles of rigid body motion, the kinematic equations of the bicycle model are:

[0021]

[0022] in, The angle between the direction of the velocity at the center point and the x-axis;

[0023] The projection of velocity onto the coordinate axes is defined as v. x and v y Then there is The kinematic equations of the bicycle model are equivalent to:

[0024]

[0025] Based on the equivalent kinematic equations of the bicycle model, five parameters (h, w, x, y, θ) are needed to determine the real-time state of the axle load for each vehicle. Since the dimensions of the rectangle (h, w) do not change over time, and the calculation of the vehicle attitude parameter θ depends on the value of h, the optimal estimation of the vehicle axle load trajectory specifically includes the following steps:

[0026] The optimal estimate (x) of the center point coordinates and its corresponding velocity is determined based on Kalman filtering. opt ,y opt ,v x,opt ,v y,opt ), and achieve vehicle tracking;

[0027] The optimal estimate of the rectangle size (h) is calculated based on the tracking and detection results. opt ,w opt );

[0028] The optimal estimate θ of the bicycle's attitude is calculated based on the equivalent kinematic equations of the bicycle model. opt .

[0029] The system state S at time t t Let S be the position and velocity of the center point M. t =(x t ,y t ,v x,t ,v y,t ) TThe measurement state is the position of the center point M, i.e., M t =(x t ,y t ) T ;

[0030] According to the system state equation in Kalman filtering, the state transition and measurement equations for the center point of the vehicle axle load rectangle are as follows:

[0031]

[0032] Among them, u t This is process noise.

[0033] Based on the principle of Kalman filtering, the optimal estimates of the coordinates of the center point M and its corresponding velocity are obtained through three steps: initialization, prediction, and update, thus achieving vehicle tracking.

[0034] a) Initialization:

[0035] When the center point M is detected for the first time, the system state S is initialized to determine S1, where the position is initialized to the measured value and the speed is initialized according to the design speed of the urban road.

[0036] b) Prediction step:

[0037]

[0038] Where Q is u t The covariance matrix, P t The covariance matrix of the predicted values;

[0039] Based on the prediction step, the predicted value of the coordinates of point M in the current frame is calculated using the state information of each vehicle center point M obtained in the previous frame, and the distance between the predicted value and the measured value of each M point coordinate is calculated. If the distance is less than the pre-configured threshold, it is matched as the same vehicle.

[0040] c) Update step:

[0041]

[0042] Where R is the variance of the observation noise;

[0043] According to the update step, after each prediction step is completed, the vehicle state is updated to achieve vehicle tracking.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] (1) The present invention has made reasonable abstraction of the geometry of the vehicle, making it possible to detect the spatiotemporal distribution of axle load based on monocular vision. Key point information can be obtained by using only one camera to take pictures, which is more practical.

[0046] (2) This invention uses Kalman filtering and a vehicle kinematic bicycle model to make optimal estimation of the motion trajectory of vehicle axle load, realizing real-time spatial distribution monitoring of vehicle axle load. The monitoring accuracy is high and the monitoring results are more referential. Attached Figure Description

[0047] Figure 1 This is a flowchart of the method of the present invention;

[0048] Figure 2 The detection results of the target detection model;

[0049] Figure 3 A schematic diagram illustrating the indirect method for inferring axle load location;

[0050] Figure 4 A diagram illustrating the standardization and annotation of a keypoint dataset;

[0051] Figure 5 Here is a structural diagram of the Hourglass module;

[0052] Figure 6 Here is a diagram of the HourglasNet structure;

[0053] Figure 7 The test results are for HourglasNet;

[0054] Figure 8 A schematic diagram of a kinematic bicycle model;

[0055] Figure 9 A flowchart illustrating the process of optimally estimating the motion trajectory of axle loads of a vehicle using Kalman filtering and a kinematic bicycle model of the vehicle.

[0056] Figure 10 This is a detection area in one embodiment;

[0057] Figure 11 This is the trajectory estimation result of the center point of the axle load rectangle in one embodiment;

[0058] Figure 12 This is a result of the spatial distribution of vehicle axle load in one embodiment;

[0059] Figure 13 This is a schematic diagram of the trajectory of the axle load matrix and its center point within the monitoring area in one embodiment. Detailed Implementation

[0060] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0061] This embodiment provides a method for monitoring the axle load position of a vehicle, such as... Figure 1 As shown, it includes the following steps:

[0062] 1) Based on the target detection model, detect vehicles in the real-time monitoring screen.

[0063] This embodiment uses a YOLO-v5 network and trains it using the public dataset DETRAC to obtain a vehicle target detection model, such as... Figure 2 As shown.

[0064] 2) Extract the geometric features of the vehicle and establish an indirect inference method for axle load position. Based on the indirect inference method, establish a vehicle axle load key point dataset and key point detection model.

[0065] 21) Extract the geometric features of the vehicle and establish an indirect inference method for axle load position and a key point dataset.

[0066] Due to the occlusion of the vehicle body, only the position of one side's tires can be directly obtained from a single image; the position of the other side's tires cannot be directly obtained from the image. However, the shape of a vehicle is highly standardized; the distance between the tires on both sides can be considered equal to the width of the vehicle, and the contact points between the tires and the ground form a rectangle. Therefore, by obtaining the axle load application point position on one side of the vehicle and the vehicle width, the positions of all wheel load application points can be calculated, and the vehicle width can be considered equal to the distance between the rearview mirrors. In other words, the indirect inference method for axle load position, based on the detected axle load position and window position on one side of the vehicle, the assumption that the distance between the tires on both sides of the vehicle is equal to the vehicle width, and the assumption that the contact points between the tires and the ground form a rectangle, calculates the axle load position obscured by the vehicle body based on the rectangular features, thus achieving the position of all axle load application points through a monocular image.

[0067] like Figure 3 As shown in the diagram, the points of contact between the vehicle and the ground are A, B, C, and D. Therefore, quadrilateral ABCD is a rectangle. Furthermore, line segment BC is parallel to and equal in length to line segment W1W2. The points whose coordinates can be directly obtained from the diagram are A:(x... A ,y A B:(x) B ,y B ), W1:(x W1 ,y W1 W2:(x) W2 ,yW2 Based on the above geometric relationships, the coordinates of C and D can be obtained: C:(x A +x W2 -x W1 ,y A +y W2 -y W1 ), D:(x B +x W2 -x W1 ,y B +y W2 -y W1 Therefore, by detecting the axle load position and window position on one side of the vehicle, the positions of all axle load application points can be obtained through monocular images.

[0068] This embodiment uses videos from the publicly available DETRAC dataset and then utilizes the YOLO-v5 model trained in step 1) for detection. Vehicle images detected in the videos are cropped out, and the positions of points A, B, W1, and W2 in the images are labeled, resulting in the labeling of key points for 1638 different vehicles. Since the image sizes in the original dataset vary, image standardization is necessary. Assuming the original image size is (m, n), it is first scaled to a width of 256, i.e., the size becomes (m / n × 256, 256). Then, gray is filled underneath the image, transforming it into a 256 × 256 square. Simultaneously, the labeled data is also transformed accordingly, such as... Figure 4 As shown.

[0069] Through annotation, a dataset containing 1638 size-normalized images and their corresponding keypoints was obtained. 60% (983 images) were used for training the keypoint detection model, 20% (328 images) were used for model validation, and 20% (327 images) were used for testing the keypoint detection model.

[0070] 22) Establish a key point detection model

[0071] The keypoint detection model is based on HourglassNet. HourglassNet performs preliminary feature extraction on the image through several convolutional layers, and then performs secondary feature extraction on the image through multiple encoder-decoder structures. Each encoder-decoder structure is an Hourglass module. By stacking multiple Hourglass modules, a heatmap is generated to detect keypoints in the image. The output of the previous Hourglass module is directly used as the input of the next Hourglass module.

[0072] The basic structure of the Hourglass module includes two branches. One branch extracts deep features from the input array through multiple pooling operations and gradually restores the resolution to the original size through upsampling. The other branch only extracts features without changing the array size. When the Hourglass module outputs, it adds the arrays from the two branches, enabling the neural network to combine information from multiple scales in the image.

[0073] The Hourglass module in this embodiment is recursively composed of four basic structures, integrating multi-scale information at input resolutions of 1 / 2, 1 / 4, 1 / 8, and 1 / 16, such as... Figure 5 As shown.

[0074] The structure of HourglasNet in this embodiment is as follows: Figure 6 As shown, it consists of a coding layer and four Hourglass modules.

[0075] 3) Train a key point detection model using the vehicle axle load key point dataset.

[0076] Because HourglassNet is quite deep, intermediate supervision is used during training to prevent gradient vanishing. The model's loss function is the sigmoid cross-entropy between the feature map output by each Hourglass module and the labeled data. Furthermore, in this embodiment, each Hourglass module outputs a 64×64 feature map; therefore, the loss function is:

[0077]

[0078] Where i and j are the number of rows and columns of the feature map, g i,j It is labeled data. This is the output of the k-th Hourglass module.

[0079] Train the keypoint detection model using the keypoint dataset generated in step 21), and test it on the test set, such as... Figure 7 As shown. From Figure 7 As can be seen, the HourglassNet key point detection model can accurately identify the location of key points on a vehicle.

[0080] 4) Based on the detection results of the key point detection model, the axle load coordinates in the image coordinate system are transformed to the world coordinate system through the homography matrix to obtain the axle load coordinates in the world coordinate system. Then, the motion trajectory of the vehicle axle load is optimally estimated by using Kalman filtering and the vehicle's kinematic bicycle model, so as to realize the real-time spatial distribution monitoring of the vehicle axle load.

[0081] 41) Coordinate Transformation

[0082] Since the coordinates of the directly detected key points are in the image coordinate system, a coordinate system transformation is required, which needs to be determined through on-site calibration. Based on the principle of homography transformation, as long as several non-collinear points (≥4) are marked on-site, the homography matrix can be calculated, thereby transforming the coordinates in the image coordinate system into coordinates in the real coordinate system.

[0083] Assuming homography matrix The coordinates of point i in the image coordinate system are The coordinates in the real coordinate system are If n is the number of reference points, then:

[0084]

[0085] By solving the above equation, the least squares solution of H is obtained, thus determining the transformation relationship between the image coordinate system and the world coordinate system as follows:

[0086]

[0087] 42) Establish a kinematic bicycle model

[0088] The motion of a vehicle on a road surface is the motion of a rigid body in a two-dimensional plane. This embodiment uses a kinematic bicycle model to describe the vehicle's motion and tracks and corrects the trajectory of the axle load application point based on the kinematic bicycle model.

[0089] The kinematic bicycle model makes the following three assumptions about the motion of the car:

[0090] 1. Only consider the motion of the car in a two-dimensional plane, and ignore its motion in the vertical direction;

[0091] 2. The speeds of the two front wheels of a car are equal in magnitude and direction, and the same is true for the rear wheels. Therefore, the front and rear wheels can each be described by a single wheel.

[0092] 3. Similar to bicycles, the turning angle of a car body is also controlled by the front wheel.

[0093] The basic assumptions of the kinematic bicycle model are as follows: Figure 8 As shown in (a).

[0094] like Figure 8As shown in (b), the kinematic bicycle model is used to track and correct the trajectory of the axle load application point. The kinematic bicycle model uses four variables to describe the current state of the vehicle: the center point coordinates (x, y) and velocity (v), and the vehicle's attitude (θ). The variables describing the real-time position of the axle load application point of a car include: the length (h) and width (w) of the rectangle, the coordinates (x, y) of the rectangle's center, and the vehicle's attitude (θ), such as... Figure 8 As shown in (c).

[0095] According to the principles of rigid body motion, the kinematic equations of the bicycle model are:

[0096]

[0097] in, The angle between the direction of the velocity at the center point and the x-axis;

[0098] Since velocity is a vector, which is not conducive to subsequent calculations, the projection of velocity onto the coordinate axes is defined as v. x and v y Then there is The kinematic equations of the bicycle model are equivalent to:

[0099]

[0100] 43) Combine Kalman filtering and a vehicle kinematic bicycle model to perform optimal estimation of the vehicle axle load trajectory.

[0101] Although keypoint detection models can detect key points of vehicles in images and thus infer the real-time location of axle load application points, this method still suffers from measurement errors and cannot track vehicles. Therefore, it is not suitable to directly use the detection results for monitoring the spatiotemporal distribution of axle loads. To eliminate measurement errors and enable vehicle tracking, this invention employs Kalman filtering to further process the detection results.

[0102] Based on the equivalent kinematic equations of the bicycle model, five parameters (h, w, x, y, θ) are needed to determine the real-time state of the axle load for each vehicle. Since the dimensions of the rectangle (h, w) do not change over time, the calculation of the vehicle attitude parameter θ depends on the value of h. Therefore, the process for optimally estimating the motion trajectory of the vehicle axle load is as follows: Figure 9 As shown, the specific steps include:

[0103] 431) Determine the optimal estimate (x) of the center point coordinates and its corresponding velocity based on Kalman filtering. opt ,y opt ,v x,opt ,v y,opt ), and achieve vehicle tracking;

[0104] This embodiment will focus on the center point of the axle load rectangle (i.e. Figure 8 (b) tracks and optimizes the motion state of point M), but the key point detection model can only measure the real-time state of the vehicle in a single frame image and cannot measure the current speed. Therefore, the system state S at time t is not calculated. t Let S be the position and velocity of the center point M. t =(x t ,y t ,v x,t ,v y,t ) T The measurement state is the position of the center point M, i.e., M t =(x t ,y t ) T ;

[0105] According to the system state equation in Kalman filtering, the state transition and measurement equations for the center point of the vehicle axle load rectangle are as follows:

[0106]

[0107] Among them, u t This is process noise.

[0108] Based on the principle of Kalman filtering, the optimal estimates of the coordinates of the center point M and its corresponding velocity are obtained through three steps: initialization, prediction, and update, thus achieving vehicle tracking.

[0109] a) Initialization:

[0110] When the center point M is detected for the first time, the system state S is initialized to determine S1, where the position is initialized to the measured value and the speed is initialized according to the design speed of the urban road.

[0111] b) Prediction step:

[0112]

[0113] Where Q is u t The covariance matrix, P t The covariance matrix of the predicted values;

[0114] Based on the prediction step, the predicted value of the coordinates of point M in the current frame is calculated using the state information of each vehicle center point M obtained in the previous frame, and the distance between the predicted value and the measured value of each M point coordinate is calculated. If the distance is less than the pre-configured threshold, it is matched as the same vehicle.

[0115] c) Update step:

[0116]

[0117] Where R is the variance of the observation noise;

[0118] According to the update step, after each prediction step is completed, the vehicle state is updated to achieve vehicle tracking.

[0119] 432) Calculate the optimal estimate of the rectangle size (h) based on the tracking and detection results. opt ,w opt );

[0120] For each vehicle, the dimensions (h, w) of the rectangle formed by its axle contact points remain constant during vehicle movement. 431) Vehicle tracking and trajectory optimization at point M are achieved using Kalman filtering. This step will utilize the tracking results to statistically analyze the detection results of the same vehicle in different frames, and determine the maximum likelihood estimates (h and w). opt ,w opt ).

[0121] Assuming vehicle k passes through the detection area in frames m to n of the video, a total of n-m+1 detection results for the vehicle can be obtained based on the detection results of each frame. These data are statistically analyzed to obtain frequency distribution tables for h and w. The midpoint of the interval with the highest frequency is used as the maximum likelihood estimate for calculating the real-time axle load position.

[0122] 433) Calculate the optimal estimate θ of the bicycle's attitude based on the equivalent kinematic equations of the bicycle model. opt .

[0123] Based on the equivalent kinematic equations of the bicycle model:

[0124]

[0125] The vehicle's (h,w,x,y,v) at each moment x ,v y The optimal estimate of ) has been determined, while θ t+1 It can be derived from (h,v) x ,v y ) and θ t Therefore, by simply initializing θ1, the value of θ at each time step can be calculated. In summary, the spatial distribution of axle loads of the vehicle at each time step has been completely determined.

[0126] This embodiment provides a specific test case to further illustrate the feasibility and effectiveness of the method described in this invention.

[0127] The field test was conducted on a road in a certain location, acquiring traffic monitoring video over a period of time via cameras. Through marking and calculation, the homography matrix from the image coordinate system to the road surface coordinate system was obtained as follows:

[0128]

[0129] Therefore, to ensure accuracy, this embodiment selects a region of 22.50m × 55.00m as the detection area for the spatiotemporal distribution of axle load, such as... Figure 10 As shown.

[0130] A 1000-frame (16-second) video segment was selected as an example. In this video, eight vehicles passed through the detection area, denoted as vehicle 1, vehicle 2, ..., vehicle 8. Based on the vehicle axle load position monitoring method described above, the optimal estimate of the axle load rectangle center trajectory is obtained as follows: Figure 11 As shown in the figure, the trajectory of the center of the axle load rectangle after Kalman filtering optimization is smoother and more consistent with the actual situation.

[0131] After obtaining the trajectory of the center point of the axle load rectangle, the dimensions h×w of the axle load rectangle and its motion attitude angle θ at each time moment can be obtained using the maximum likelihood estimation method and the vehicle's kinematic bicycle model. Based on the optimized values ​​of the above parameters (x,y,h,w,θ), the optimal estimate of the state of each frame (x,y,h,w,θ) can be obtained. Figure 12 The results of the detection of vehicle axle load spatial distribution at certain times are shown. As can be seen from the figure, the position of each vehicle's axle load within the monitoring area can be accurately calculated.

[0132] Figure 13 The figure shows the trajectory of the axle load rectangles of vehicles 1 to 8 and their center points within the monitoring area. As can be seen from the figure, the trajectories of the axle load application points are relatively smooth, and all axle load application points move along the lane, which is consistent with reality. Therefore, it can be considered that the method proposed in this invention can accurately perceive the spatiotemporal distribution of vehicles.

[0133] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for monitoring the axle load position of a vehicle, characterized in that, Includes the following steps: Based on the target detection model, vehicles are detected in real-time monitoring footage; Extract the geometric features of the vehicle and establish an indirect inference method for axle load position. Based on the indirect inference method, establish a vehicle axle load key point dataset and key point detection model. A key point detection model was trained using a vehicle axle load key point dataset. Based on the detection results of the key point detection model, the axle load coordinates in the image coordinate system are transformed to the world coordinate system through the homography matrix to obtain the axle load coordinates in the world coordinate system. Then, Kalman filtering and the vehicle's kinematic bicycle model are used to make the optimal estimation of the vehicle axle load motion trajectory, so as to realize the real-time spatial distribution monitoring of vehicle axle load. The indirect inference method for axle load position is based on the detected axle load position and window position on one side of the vehicle. It is based on the assumption that the distance between the tires on both sides of the vehicle is equal to the width of the vehicle, and the assumption that the contact points between the vehicle tires and the ground form a rectangle. The method calculates the axle load position that is obscured by the vehicle body based on the rectangular features, thereby obtaining the position of all axle load action points through monocular images.

2. The method for monitoring vehicle axle load position according to claim 1, characterized in that, The key point detection model is based on HourglassNet. HourglassNet performs preliminary feature extraction on the image through several convolutional layers, and then performs secondary feature extraction on the image through multiple encoder-decoder structures. Each encoder-decoder structure is an Hourglass module. By stacking multiple Hourglass modules, a heatmap is generated to detect key points in the image. The output of the previous Hourglass module is directly used as the input of the next Hourglass module.

3. The method for monitoring vehicle axle load position according to claim 2, characterized in that, The basic structure of the Hourglass module includes two branches. One branch extracts deep features from the input array through multiple pooling operations and gradually restores the resolution to the original size through upsampling. The other branch only extracts features without changing the array size. When the Hourglass module outputs, it adds the arrays from the two branches, enabling the neural network to combine information from multiple scales in the image.

4. The method for monitoring vehicle axle load position according to claim 2, characterized in that, The keypoint detection model is trained using intermediate supervision, and the loss function of the model is set as the sigmoid cross-entropy between the feature map output by each Hourglass module and the labeled data.

5. The method for monitoring vehicle axle load position according to claim 1, characterized in that, Assuming homography matrix ,point i The coordinates in the image coordinate system are The coordinates in the real coordinate system are , i =1, 2, 3, …, n , n Given the number of reference points, then: By solving the above equation, we obtain... H The least squares solution is used to determine the transformation relationship between the image coordinate system and the world coordinate system: 。 6. The method for monitoring vehicle axle load position according to claim 1, characterized in that, The kinematic bicycle model is used to track and correct the trajectory of the axle load application point. The kinematic bicycle model uses four variables to describe the current state of the vehicle: center point coordinates ( x , y ) and speed ( v ), the posture of the vehicle body ( θ ); Variables describing the real-time location of the axle load application point of a vehicle include: the length of the rectangle ( h ) and width ( w ), coordinates of the center of the rectangle ( x , y ), vehicle body attitude angle ( θ ); According to the principles of rigid body motion, the kinematic equations of the bicycle model are: in, φ The direction of velocity at the center point and x The included angle of the axis; The projection of velocity along the coordinate axes is defined as v x and v y Then there is v x = v cos φ , v y = v ·sin φ, φ= arctan( v y / v x Then the kinematic equations of the bicycle model are equivalent to: 。 7. The method for monitoring vehicle axle load position according to claim 6, characterized in that, Based on the equivalent kinematic equations of the bicycle model, to determine the real-time state of the axle load for each bicycle, it is necessary to ( h , w , x , y , θ Five parameters, due to the size parameters of the rectangle ( h , w The vehicle body attitude parameters will not change over time. θ computational dependency h The optimal estimation of the vehicle axle load trajectory involves the following steps: The optimal estimate of the center point coordinates and its corresponding velocity is determined based on Kalman filtering. x opt , y opt , v x,opt , v y,opt ), and achieve vehicle tracking; The optimal estimate of the rectangle size is calculated based on the tracking and detection results. h opt , w opt ); Calculate the optimal estimate of the bicycle's attitude based on the equivalent kinematic equations of the bicycle model. θ opt .

8. The method for monitoring vehicle axle load position according to claim 7, characterized in that, t System state at time 1 S t Center point M Position and velocity, i.e. S t =( x t , y t , v x,t , v y,t ) T The measurement status is the center point. M The position, that is M t =( x t , y t ) T ; According to the system state equation in Kalman filtering, the state transition and measurement equations for the center point of the vehicle axle load rectangle are as follows: in, u t This is process noise.

9. A method for monitoring vehicle axle load position according to claim 8, characterized in that, Based on the principle of Kalman filtering, the center point is obtained through three steps: initialization, prediction, and update. M The optimal estimates of the coordinates and corresponding velocities are used to track vehicles. a) Initialization: When the center point M When first detected, the system state S Perform initialization processing and determine S 1. In this case, the position is initialized to the measured value, and the speed is initialized according to the design speed of the urban road. b) Prediction step: in, Q yes u t The covariance matrix, P t The covariance matrix of the predicted values; Based on the prediction step, the center points of each vehicle obtained from the previous frame are used. M Calculate the state information of the current frame M Predict the coordinates of the points and calculate each... M If the distance between the predicted and measured coordinates of a point is less than a pre-configured threshold, then the point is matched as the same vehicle. c) Update step: in, R The variance of the observed noise; According to the update step, after each prediction step is completed, the vehicle state is updated to achieve vehicle tracking.

Images