A heavy vehicle weight distribution recognition method based on a lightweight network

Abstract

The application discloses a heavy vehicle weight distribution identification method based on a light network, triggers a camera to collect vehicle flow video, and inputs the vehicle flow video frame by frame into a trigger network to determine whether the current frame image contains a yellow license plate; vehicle flow video collected by a linkage camera is input into a vehicle identification network and a license plate identification network respectively to obtain all heavy vehicles in the vehicle flow video and corresponding yellow license plates; vehicle tracking is performed by matching the features of the identified vehicle subgraphs to obtain the complete trajectory of the vehicle under the field of view of the linkage camera; the vehicle trajectory corresponding to the license plate matched successfully and the toll station vehicle weight data are connected through full character matching of the license plate to obtain the heavy vehicle weight distribution. The application removes meaningless video collection through the network RepVGG, greatly reduces the calculation task of the heavy vehicle weight distribution identification method, and identifies the vehicle and the license plate through the light target identification network YOLOv5s-p, greatly improving the operation efficiency of the algorithm.

Description

Technical Field

[0001] This invention relates to vehicle weight identification technology, and in particular to a method for identifying the weight distribution of heavy vehicles based on lightweight networks. Background Technology

[0002] With the development of transportation infrastructure, more and more bridges are exceeding 10 years of service life. To avoid catastrophic accidents caused by bridge aging, real-time monitoring of bridges is crucial. Since vehicle load is the main variable load borne by bridges, vehicle load monitoring is also an important component of bridge structural monitoring systems. In fact, bridge operators place particular emphasis on monitoring the distribution of heavy vehicle loads.

[0003] Identifying vehicle weight and its distribution on bridges has become a research hotspot. These studies mainly involve two steps in vehicle weight distribution identification: the first step is to detect and track vehicles from traffic flow videos to obtain their distribution; the second step is to connect the vehicle weight data collected by the bridge's dynamic weighing system to the corresponding vehicles using various matching methods. Current vehicle detection methods are primarily based on traditional image processing algorithms (such as background subtraction and optical flow) and deep learning algorithms (such as the Faster R-CNN object recognition network). Current vehicle weight connection methods are mainly based on temporal (or spatial) synchronization and vehicle feature matching. These methods estimate vehicle weight through vehicle features or install a dynamic weighing system at the bridgehead. The former is based on statistical features and does not represent the actual vehicle load, while the latter requires closing traffic, making installation time-consuming and labor-intensive. Summary of the Invention

[0004] Purpose of the invention: To address the above problems, the purpose of this invention is to provide a method for identifying the weight distribution of heavy vehicles based on lightweight networks.

[0005] Technical solution: The present invention provides a method for identifying the weight distribution of heavy vehicles based on lightweight networks, comprising:

[0006] The system uses a trigger camera to capture traffic flow video and inputs it frame by frame into a trigger network. The trigger network then determines whether the current frame contains a yellow license plate. If it does, the system activates the linked camera to start capturing traffic flow video.

[0007] The traffic flow video captured by the linked camera is input into the vehicle recognition network and the license plate recognition network respectively to obtain all heavy vehicles and their corresponding yellow license plates in the traffic flow video;

[0008] Vehicle tracking is performed by matching the identified vehicle sub-image features to obtain the complete trajectory of the vehicle under the view of the linked camera;

[0009] By matching the full characters of the license plate, the trajectory of the vehicle corresponding to the successfully matched license plate is connected with the vehicle weight data of the toll station to obtain the weight distribution of heavy vehicles.

[0010] Furthermore, this method for identifying the weight distribution of heavy vehicles also includes:

[0011] Construct a first image dataset, labeled as containing complete yellow license plates and two other classes. Train the RepVGG network using the first image dataset, and use the trained RepVGG network as the trigger network.

[0012] Furthermore, this method for identifying the weight distribution of heavy vehicles also includes:

[0013] Construct a second image dataset, label the bounding boxes of all heavy vehicles, train the first network YOLOv5s-p using the second image dataset, and use the trained first network YOLOv5s-p as the vehicle recognition network.

[0014] Furthermore, the heavy vehicle weight distribution recognition method also includes constructing a third image dataset, labeling the license plates of all heavy vehicles, training a second network YOLOv5s-p using the third image dataset, and using the trained second network YOLOv5s-p as the license plate recognition network.

[0015] Beneficial effects: Compared with the prior art, the significant advantages of this invention are:

[0016] 1. This invention first uses the RepVGG network to remove meaningless video captures, which greatly reduces the computational task of the vehicle weight distribution recognition method; then, it uses the lightweight target recognition network YOLOv5s-p to recognize vehicles and license plates, which greatly improves the algorithm's running efficiency.

[0017] 2. This invention obtains a more accurate vehicle weight distribution than constant speed assumption or interval interpolation through video input; and achieves a more robust tracking effect than position prediction through vehicle feature matching and tracking.

[0018] 3. This invention only requires the installation of a camera on the side of the bridge, which is low in cost, simple to install, and easy to maintain, and has broad application prospects on various high-speed bridges and highways. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method for rapid identification of vehicle weight distribution in this embodiment;

[0020] Figure 2 This is a schematic diagram of the triggering network structure and a flowchart of the license plate color classification in the embodiment;

[0021] Figure 3This is a schematic diagram of the lightweight network YOLOv5s-p and a flowchart of its recognition process in this embodiment;

[0022] Figure 4 This is a flowchart of the vehicle tracking method in the embodiment;

[0023] Figure 5 This is a schematic diagram of the license plate full character matching process in the embodiment. (a) is a schematic diagram of the calculation of the correlation matrix during the vehicle tracking process; (b) is a vehicle trajectory diagram in video VID1 and VID2.

[0024] Figure 6 This is the result of matching all characters of the license plate in the example;

[0025] Figure 7 This is a diagram showing the identified vehicle weight distribution in the embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments.

[0027] like Figure 1 The heavy vehicle weight distribution identification method based on lightweight networks described in this embodiment includes the following steps:

[0028] Step 1: Construct the first image dataset, label the images in the first image dataset as containing complete yellow license plates and two other classes, train the RepVGG network using the first image dataset, and use the trained RepVGG network as the trigger network.

[0029] Step 2: Construct a second image dataset, label the bounding boxes of all heavy vehicles in the images in the second image dataset, train the first network YOLOv5s-p using the second image dataset, and use the trained first network YOLOv5s-p as the vehicle recognition network.

[0030] Specifically, the aforementioned YOLOv5s-p network is generated by replacing all the basic units of the lightweight network ShuffleNet with the backbone network of YOLOv5s. Then, the network parameters are further reduced by performing a region-separated calculation on every unit in the backbone network to generate the final lightweight network YOLOv5s-p.

[0031] Step 3: Construct a third image dataset, label the license plates of all heavy vehicles, train the second network YOLOv5s-p using the third image dataset, and use the trained second network YOLOv5s-p as the license plate recognition network.

[0032] The second network YOLOv5s-p has the same structure as the first network YOLOv5s-p, but the training dataset is different. The trained first network YOLOv5s-p is used to identify heavy vehicles appearing in traffic flow videos, and the second network YOLOv5s-p is used to identify the corresponding yellow license plates.

[0033] Step 4: Use the trigger camera to capture traffic flow video and input it frame by frame into the trigger network. Use the trigger network to determine whether the current frame contains a yellow license plate. If it contains a yellow license plate, activate the linkage camera to start capturing traffic flow video. Input the traffic flow video captured by the linkage camera into the vehicle recognition network and the license plate recognition network respectively to obtain all heavy vehicles and their corresponding yellow license plates in the traffic flow video.

[0034] Specifically, when the trigger network determines that the current frame does not contain a yellow license plate, the trigger camera continuously captures traffic flow video, and the trigger network continues to make judgments without activating the linked camera to capture video. When the trigger network determines that the current frame contains a yellow license plate, a video capture command is sent to the linked camera, activating the linked camera to begin capturing traffic flow video. The duration of video capture is fixed and must be longer than the time it takes for a vehicle to pass through the linked camera's field of view.

[0035] Step 5: Use the identified vehicle sub-image feature matching to track the vehicle and obtain the complete trajectory of the vehicle under the field of view of the linked camera. Then, through full character matching of the license plate, connect the vehicle trajectory corresponding to the successfully matched license plate with the vehicle weight data of the toll station to obtain the weight distribution of heavy vehicles.

[0036] Specifically, vehicle sub-images are extracted from the vehicles identified by the vehicle recognition network and license plate recognition network according to the bounding boxes. The vehicle sub-images are then normalized to a uniform size. Based on the features of the normalized vehicle sub-images, the correlation coefficient between vehicle sub-images in all consecutive frames is calculated using the image correlation coefficient, and a correlation matrix is ​​formed. By searching for the peak value in the correlation matrix row by row, the same vehicles in consecutive frames are tracked to obtain the trajectory of all vehicles.

[0037] The system sequentially matches yellow license plates identified from traffic flow video with license plates marked with toll station weights using a full character matching method. The number of successfully matched characters determines whether they belong to the same vehicle. If they are the same vehicle, the corresponding toll station weight data is linked to the vehicle's trajectory to obtain the vehicle's gravity trajectory.

[0038] To further illustrate the implementation process of the vehicle weight distribution identification method based on lightweight networks described in this invention, the following uses a bridge example to illustrate the specific implementation steps of this method. Figure 1 The diagram shown is a flowchart of the heavy vehicle weight distribution identification process.

[0039] Step 1: Camera installation.

[0040] A camera is installed at the mid-span of one side of the bridge as a trigger camera, and another camera is installed at the end as a linkage camera. The installation height is 4 meters and the angle is about 25 degrees. The camera resolution is 4096 pixels × 2160 pixels, the vehicle detection area resolution is 3000 pixels × 1760 pixels, and the license plate detection area resolution is 320 pixels × 320 pixels.

[0041] Step 2: Manual creation of the dataset and network training.

[0042] The first image dataset consists of 12,000 license plate region sub-images selected from the trigger cameras, and the RepVGG network is trained for 20 epochs. The second image dataset consists of 750 vehicle region sub-images selected from the linked cameras, and the first network YOLOv5s-p is trained for 500 epochs. The third image dataset consists of 1,200 license plate region sub-images selected from the linked cameras, and the second network YOLOv5s-p is trained for 500 epochs.

[0043] Step 3: Trigger the judgment.

[0044] The trained RepVGG network is used to classify videos triggered by camera captures, determining whether the current frame contains a yellow license plate, which then decides whether to activate the linked camera. The flowchart is as follows. Figure 2 As shown, specifically, a trigger camera continuously captures traffic flow video and inputs it frame by frame into a trigger network. The trigger network determines whether the current frame contains a yellow license plate. If it does, the linked camera is activated to start capturing traffic flow video and perform vehicle weight identification separately; otherwise, the trigger camera continues to capture traffic flow video, and the trigger network continues to make judgments. The structure of the RepVGG network is as follows. Figure 2 As shown on the right, the license plate recognition region sub-image is input into the trained network RepVGG, and the probability that the current frame contains a yellow license plate is output. In one example, if the output is ≥0.5, the current frame is considered to contain a yellow license plate, and if the output is <0.5, the current frame is considered not to contain a yellow license plate.

[0045] Step 4: Vehicle and license plate recognition.

[0046] Two trained YOLOv5s-p networks were used to identify vehicles and license plates in traffic flow videos captured by linked cameras, respectively, to obtain vehicle location information, vehicle sub-image features, and all characters of the license plate, such as... Figure 3As shown, the traffic flow captured by the linked camera is input into the YOLOv5s-p network, which outputs the vehicle bounding boxes and classifications. The YOLOv5s-p network is generated by replacing all the basic units of the lightweight network ShuffleNet with the backbone of the YOLOv5s network. Then, the backbone network is calculated in a region-separated manner every other unit to generate the final lightweight network YOLOv5s-p.

[0047] Step 5: Information fusion.

[0048] Vehicle sub-images are extracted from the vehicle recognition network and license plate recognition network based on their bounding boxes. These sub-images are then uniformly transformed to 76 pixels × 76 pixels. Figure 4 The correlation coefficients between all vehicle sub-images in two consecutive frames are calculated to form a correlation matrix. The peak value of each row of the correlation matrix is ​​found. If the peak value is greater than 0.9, the two vehicle sub-images corresponding to the peak value contain the same vehicle, and one vehicle tracking is completed. Figure 5 This example demonstrates the tracking results for a specific video instance. After vehicle tracking, a vehicle trajectory is obtained. Then, a full character match is performed between the license plate corresponding to the vehicle trajectory and the license plate marked on the tollbooth's vehicle weight marker. The number of successfully matched characters is calculated. If the number of successful matches is greater than 6, the two license plates are considered a successful match, and all their vehicle trajectory and weight data are concatenated accordingly. In another example, such as... Figure 6 As shown, if the number of successfully matched characters is 3, which is less than 6, then the two license plates are considered unmatched.

[0049] For this bridge, the vehicle weight distribution identified using the method of this embodiment is as follows: Figure 7 As shown, the triggering network can save about 70% of the computational tasks, and YOLOv5s-p improves the computational speed by about 30% compared to YOLOv5s, verifying the effectiveness and efficiency of the method of the present invention in identifying the weight distribution of heavy vehicles.

[0050] The present invention provides a method for rapid identification of heavy vehicle weight distribution. Based on the RepVGG classification network, a triggering device controls a bridge-side camera to collect only traffic flow videos containing heavy vehicles. The lightweight target recognition network YOLOv5s-p quickly identifies heavy vehicles and yellow license plates in the traffic flow video. The heavy vehicle trajectory is obtained by vehicle feature matching, and the toll station vehicle weight data is connected by license plate matching. By combining the vehicle trajectory and vehicle weight data, the heavy vehicle weight distribution at any time can be obtained.

Claims

1. A method for identifying the weight distribution of heavy vehicles based on lightweight networks, characterized in that, include: The system uses a trigger camera to capture traffic flow video and inputs it frame by frame into a trigger network. The trigger network then determines whether the current frame contains a yellow license plate. If it does, the system activates the linked camera to start capturing traffic flow video. The traffic flow video captured by the linked camera is input into the vehicle recognition network and the license plate recognition network respectively to obtain all heavy vehicles and their corresponding yellow license plates in the traffic flow video; Vehicle tracking is performed by matching the features of the identified vehicle sub-images to obtain the complete trajectory of the vehicle under the view of the linked camera. Specifically, the vehicle sub-images identified by the vehicle recognition network and the license plate recognition network are cropped according to the bounding box. The vehicle sub-images are normalized to a uniform size. Then, based on the features of the normalized vehicle sub-images, the correlation coefficient between the vehicle sub-images in all consecutive frames is calculated using the image correlation coefficient, and a correlation matrix is ​​formed. By finding the peak value in the correlation matrix row by row, the same vehicle in two consecutive frames is tracked to obtain the trajectory of all vehicles. By matching the full characters of license plates, the vehicle trajectories corresponding to the successfully matched license plates are connected with the vehicle weight data at toll stations to obtain the weight distribution of heavy vehicles. Specifically, the full character matching method is used to sequentially match the yellow license plates identified from the traffic flow video with the license plates marked with vehicle weight at toll stations. The number of successfully matched characters is used to determine whether they belong to the same vehicle. If they are the same vehicle, the corresponding toll station vehicle weight data is connected to the vehicle's trajectory to obtain the vehicle's gravity trajectory.

2. The method for identifying the weight distribution of loaded vehicles according to claim 1, characterized in that, Also includes: Construct a first image dataset, labeled as containing complete yellow license plates and two other classes. Train the RepVGG network using the first image dataset, and use the trained RepVGG network as the trigger network.

3. The method for identifying the weight distribution of loaded vehicles according to claim 1, characterized in that, Also includes: Construct a second image dataset, label the bounding boxes of all heavy vehicles, train the first network YOLOv5s-p using the second image dataset, and use the trained first network YOLOv5s-p as the vehicle recognition network.

4. The method for identifying the weight distribution of loaded vehicles according to claim 1, characterized in that, Also includes: A third image dataset is constructed to label the license plates of all heavy vehicles. The second network YOLOv5s-p is trained using the third image dataset and used as the license plate recognition network.

Images