A dynamic weighing method, system, device, and medium integrating machine vision.

Abstract

A dynamic weighing method, system, device, and medium integrating machine vision are disclosed, comprising: setting up a calibration vehicle and collecting bridge responses when the calibration vehicle passes over a bridge; collecting images of the calibration vehicle and using these images as training samples; after training, capturing vehicles appearing in the monitoring system to obtain the length of the calibration vehicle; setting up virtual axles on the calibration vehicle and obtaining the location interval of the real axles based on the bridge responses; refining the location interval of the real axles, selecting an element in each location interval to construct a vehicle matrix, iterating to obtain several sets of axle load results, substituting them into an error formula to obtain the axle load and wheelbase of the real axles. This invention utilizes a video monitoring system installed on a bridge and performs vehicle recognition through machine learning methods. After obtaining the vehicle length and speed, the virtual axle method is used to determine the real axle data, thereby calculating the axle load and total weight.

Description

Technical Field

[0001] This invention relates to the field of vehicle weighing methods, specifically a dynamic weighing method, system, device, and medium that integrates machine vision. Background Technology

[0002] In recent years, bridge collapses caused by overloaded vehicles, resulting in loss of life and property, have been alarming. Studies have shown that overloading can be mitigated in areas with well-developed weighing systems. Weighing systems are divided into static and dynamic systems. While static weighing systems can accurately identify axle load and total weight, they require traffic interruption and are expensive to maintain. Dynamic weighing systems, on the other hand, can weigh vehicles while they are in motion, and their higher weighing efficiency has made them an effective tool for controlling overloaded vehicles.

[0003] Dynamic weighing systems are further divided into road surface dynamic weighing systems and bridge-type dynamic weighing systems. Road surface dynamic weighing systems require a weighing platform equipped with sensors to be pre-embedded under the road surface. When a vehicle passes over the platform, the sensor system converts the gravity signal into other signals, which are then processed to obtain the vehicle's weight information. However, this method requires road excavation, and the sensors are in direct contact with the wheels, resulting in high maintenance costs. Bridge-type dynamic weighing systems, on the other hand, use the bridge structure as the weighing platform. Sensors installed on the bottom of the beam acquire the dynamic response and calculate the vehicle's weight. This system can capture the complete dynamic response of a vehicle crossing the bridge, and it eliminates the need for road excavation. The sensors are not in direct contact with the wheels, extending the system's lifespan. Traditional bridge-type dynamic weighing methods require installing axle detection sensors on the structure to identify the number of axles and vehicle speed, increasing the system's cost. Furthermore, traditional bridge-type dynamic weighing systems cannot directly assess the operating conditions of vehicles on the bridge, affecting the accuracy of the weighing results. Summary of the Invention

[0004] The purpose of this invention is to provide a dynamic weighing method, system, device and medium that integrates machine vision to solve the problems of high cost and low weighing accuracy of traditional weighing methods.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a dynamic weighing method incorporating machine vision, comprising:

[0007] Set up calibration vehicles and collect bridge responses when the calibration vehicles pass over the bridge.

[0008] Collect images of the calibrated vehicles and use these images as training samples. After training, capture the vehicles appearing in the monitoring to obtain the length of the calibrated vehicles.

[0009] Virtual axles are set on the calibrated vehicle, and the location range of the real axles is obtained based on the bridge response.

[0010] The actual axle location range is refined, and an element is selected in each range to construct a vehicle matrix. After iteration, several sets of axle load results are obtained. Substituting these results into the error formula, the actual axle load and wheelbase of the axle are obtained.

[0011] Optionally, the step of setting up a calibration vehicle and collecting the bridge response when the calibration vehicle crosses the bridge specifically involves:

[0012] Given that the axle load of each axle of the calibrated vehicle is P i The distance from each axis to the previous axis is D. i The calibration vehicle travels along a preset calibration path at a specified speed, and the corresponding bridge response is collected as R(k); it can be expressed by the following formula:

[0013]

[0014] Where N is the number of axles of the vehicle. The vertical index value of the influence line corresponding to the i-th axle at time step k, C i For the corresponding wheelbase D i The number of sampling points in the acquisition system; C i It can be expressed by the following formula:

[0015]

[0016] Where f is the sampling frequency and v is the vehicle speed.

[0017] Optionally, a matrix L is constructed based on the vehicle information as shown below, where m represents the number of sampling points where the vehicle is entirely on the bridge; (m+C N-1 () indicates the number of sampling points from when the vehicle starts to enter the bridge until it completely exits the bridge;

[0018]

[0019] The matrix is ​​solved using the Tikhonov regularization method, which is expressed as follows:

[0020]

[0021] λ is a non-negative regularization parameter, and T represents the regularization matrix.

[0022] Optionally, camera devices installed at both ends of the bridge are used to monitor the vehicles on the bridge. A large number of pictures containing vehicles are intercepted from the monitoring videos. The vehicles in the pictures are labeled according to different vehicle models using labeling tools, and these pictures are used as training samples. The YOLO V4 algorithm is used to perform machine learning on the vehicle pictures, and the vehicles appearing in the monitoring videos are captured after the training is completed. 0000069Optionally, virtual axles are set on the calibrated vehicle, and the position interval where the real axle is located is obtained based on the bridge response. Specifically: According to the vehicle length recognized by machine vision, virtual axles are evenly arranged on the vehicle at a certain interval d. The dynamic response R(k) collected when the vehicle passes through the bridge is obtained, and the virtual axle weights are calculated using the non - negative least squares method. The virtual axles with a result of 0 are removed, and the position interval where the real axle is located is obtained.

[0024] Optionally, the position interval where the real axle is located is refined: The lower limit value AS of the axle wheelbase is given according to the vehicle model recognized by machine vision min , the virtual axles with the wheelbase between adjacent virtual axles less than AS min are divided into the same axle group, and the real axle is within the axle group interval; for each axle group, the axle group interval is further divided with σ (σ < d) as the step size to fill virtual axles, and the filled axle group is denoted as {g1, g2,..., g N}.

[0025] Optionally, obtaining the axle weight and wheelbase of the real axle is specifically as follows:

[0026] One element is taken from each of the finely divided axle groups to form an alternative combination of possible positions of the real axle, and there are possible combinations of real axle positions; a set of axle weight results is obtained for each combination of axle positions, that is, there are N sets of results; the error function E is calculated for all combinations, and its formula is as follows: E = ||R - I·P||2; the combination with the smallest error value is found, and the corresponding result is the axle weight and wheelbase of the real axle.

[0027] In the second aspect of the present invention, a dynamic weighing system integrating machine vision is provided, which is characterized by including:

[0028] A bridge response acquisition module, which is used to set a calibrated vehicle and collect the bridge response when the calibrated vehicle passes through the bridge;

[0029] A vehicle picture acquisition module, which is used to collect pictures of the calibrated vehicle, use these pictures as training samples, capture the vehicles appearing in the monitoring after the training is completed, and obtain the length of the calibrated vehicle;

[0030] The location range acquisition module is used to set up virtual axles on the calibrated vehicle and obtain the location range of the real axles based on the bridge response.

[0031] The output module is used to refine the actual axle location range. Within each location range, an element is selected to construct a vehicle matrix. After iteration, several sets of axle load results are obtained. Substituting these results into the error formula, the actual axle load and wheelbase of the axle are obtained.

[0032] In a third aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a dynamic weighing method incorporating machine vision.

[0033] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a dynamic weighing method incorporating machine vision.

[0034] Compared with the prior art, the present invention has the following technical effects:

[0035] 1. This invention can obtain vehicle length and speed based on existing video surveillance on bridges, avoiding the cumbersome steps of installing axle detectors required by existing methods and reducing cost investment.

[0036] 2. This invention monitors traffic conditions on bridges using traffic surveillance video, enabling direct assessment of vehicle conditions, including the number of vehicles, their speed, and precise location. This avoids the coarse judgments of speed and location found in existing technologies, thus effectively improving algorithm accuracy.

[0037] 3. This invention uses response sensors installed beneath the bridge deck to collect data for dynamic weighing of the bridge. This avoids the inconveniences associated with installing sensors on the bridge deck, including traffic disruptions, high costs, and high maintenance costs.

[0038] This invention utilizes a video surveillance system installed on a bridge to identify vehicles using machine learning methods. After obtaining vehicle length and speed, a virtual axle method is used to determine the actual axle data, thereby calculating axle load and total weight. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the steps of a dynamic weighing method integrating machine vision according to the present invention;

[0040] Figure 2 This is a spatial architecture diagram of the YOLO V4 algorithm used in machine learning in the method of this invention.

[0041] Figure 3This is a diagram illustrating the effect of machine vision vehicle recognition in the method of this invention.

[0042] Figure 4 This is a schematic diagram of the arrangement of virtual axes;

[0043] Figure 5 To implement the structural strain influence line calibration process in the case study;

[0044] Figure 6 To implement the strain influence line of the bridge structure in the case study;

[0045] Figure 7 The results of virtual axis recognition in the implementation case. Detailed Implementation

[0046] The present invention will be further described below with reference to the accompanying drawings:

[0047] Please see Figures 1 to 7 A novel dynamic weighing method integrating machine vision includes the following steps:

[0048] S1: Given that the axle load of each axle of the calibrated vehicle is P i The distance from each axis to the previous axis is D. i The calibration vehicle travels along a preset calibration path at a specified speed, and the corresponding bridge response is collected as R(k). This can be expressed by the following formula:

[0049]

[0050] Where N is the number of axles of the vehicle. The vertical index value of the influence line corresponding to the i-th axle at time step k, C i For the corresponding wheelbase D i The number of sampling points in the acquisition system. (C) i It can be expressed by the following formula:

[0051]

[0052] Where f is the sampling frequency and v is the vehicle speed.

[0053] Based on the vehicle information, matrix L is constructed as follows: This matrix has a clear physical meaning, where m represents the number of sampling points where the vehicle is completely on the bridge; (m+C N-1 () indicates the number of sampling points from when the vehicle starts to enter the bridge until it completely exits the bridge.

[0054]

[0055] S2: Solve for the influence line coefficient matrix using the Tikhonov regularization method. The expression for this method is as follows:

[0056]

[0057] Let λ be a non - negative regularization parameter, and T represent the regularization matrix. The determination of the optimal regularization parameter adopts the L - curve method. Its basic idea is to plot the semi - norm log||TI|| of the regularized solution and the residual norm log||LI - R|| of the regularized solution for different regularization coefficients in a double - logarithmic form. The shape of the curve is "L" - shaped, and the optimal regularization parameter λ is selected as the inflection point of the curve, and then the influence line coefficient matrix is solved.

[0058] S3: Use the camera devices installed at both ends of the bridge to monitor the vehicles on the bridge.截取大量包含车辆的图片,利用标注工具对图片中的车辆按照不同车型进行标注,并将这些图片作为训练样本。

[0059] S4: Adopt the YOLO V4 algorithm to perform machine learning on the vehicle pictures. The algorithm's spatial architecture is as Figure 3 . After the training is completed, the vehicles appearing in the monitoring video can be captured.

[0060] S5: According to the vehicle length recognized by machine vision, evenly arrange virtual axles on the vehicle at a certain interval d, as shown in the figure. Obtain the dynamic response R(k) collected when the vehicle passes through the bridge, use the non - negative least - squares method to calculate the virtual axle weights, remove the virtual axles with a result of 0, and obtain the position interval where the real axles are located.

[0061] S6: Given the lower limit value ASmin of the axle spacing according to the vehicle type recognized by machine vision, divide the virtual axles with an adjacent virtual axle spacing less than ASmin into the same axle group, and the real axles are within the axle group interval. For each axle group, divide the axle group interval again with σ (σ < d) as the step size to fill in virtual axles, and the filled axle group is denoted as {g1, g2,..., g N}}.

[0062] S7: Take one element from each of the finely - divided axle groups, and a candidate combination of possible positions of the real axles can be formed. Then there are possible combinations of real axle positions. Each combination of axle positions will obtain a set of axle weight solution results, that is, there are N sets of results in total. Calculate the error function E for all combinations, and its formula is as follows: E = ||R - I·P||2. Find the smallest set of error values, and the corresponding result at this time is the axle weight and axle spacing of the real axle.

[0063] Embodiment: Take a 3 - 16m reinforced concrete continuous slab bridge as an example to further elaborate on the present invention in detail.

[0064] S1: First, arrange strain sensing devices at the mid - span position of the middle span of the structure and camera devices at both ends of the bridge. It should be noted that in the translation of the part "截取大量包含车辆的图片,利用标注工具对图片中的车辆按照不同车型进行标注,并将这些图片作为训练样本。" in step S5, the Chinese part is not complete in the original text. The above translation is based on the overall understanding and translation of the context. If there are any inaccuracies, please adjust according to the correct original content.

[0065] S2: Then, have a calibration vehicle with known axle load and wheelbase drive at a constant speed of 10 km / h along the center line of the lane. The calibration process is as follows: Figure 5 As shown.

[0066] S3: After obtaining the dynamic response, the Tikhonov regularization method is used to calculate the structural strain influence lines, and the results are as follows. Figure 6 As shown.

[0067] S4: Machine learning is used to perform machine vision on vehicle images captured from surveillance video to identify vehicles on the bridge. The recognition results are as follows: Figure 4 .

[0068] S5: A six-axle trailer with axle loads of 7.95t, 7.95t, 7.95t, 7t, 7t, and 9t is driven onto the bridge. Machine vision identifies the vehicle length as 15m and the vehicle speed as 20km / h. 60 virtual axles are evenly spaced at 0.25m intervals. The minimum wheelbase is set to 1m.

[0069] S6: Using the non-negative least squares method to solve for the virtual axis weight, after removing the values ​​with a weight of 0, we can obtain the following: Figure 7 The following are several axle groups. The approximate positions of axle two to axle five are [1.25m, 1.5m]; [2.5m, 2.75m]; [9.75m, 10m]; [11.25m, 11.5m].

[0070] S7: Refine the interval spacing of the actual axle to 0.05m, and select an element within each position interval to construct a vehicle matrix. Iterate repeatedly to obtain several sets of axle load results. Substitute these into the error formula E, and find the set of axle load results with the smallest E. Calculate the wheelbase based on the number of sampling points and sampling frequency between different axles.

[0071] In another embodiment of the present invention, a dynamic weighing system integrating machine vision is provided, which can be used to implement the above-mentioned dynamic weighing method integrating machine vision. Specifically, the system includes:

[0072] The bridge response acquisition module is used to set up a calibration vehicle and collect the bridge response when the calibration vehicle passes over the bridge.

[0073] The vehicle image acquisition module is used to acquire images of the calibrated vehicles and use these images as training samples. After training, the module captures vehicles appearing in the monitoring and obtains the length of the calibrated vehicles.

[0074] The location range acquisition module is used to set up virtual axles on the calibrated vehicle and obtain the location range of the real axles based on the bridge response.

[0075] The output module is used to refine the actual axle location range. Within each location range, an element is selected to construct a vehicle matrix. After iteration, several sets of axle load results are obtained. Substituting these results into the error formula, the actual axle load and wheelbase of the axle are obtained.

[0076] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0077] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used in the operation of a dynamic weighing method integrating machine vision.

[0078] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the dynamic weighing method integrating machine vision in the above embodiments.

[0079] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A dynamic weighing method integrating machine vision, characterized in that, include: Set up calibration vehicles and collect bridge responses when the calibration vehicles pass over the bridge. Collect images of the calibrated vehicles and use these images as training samples. After training, capture the vehicles appearing in the monitoring to obtain the length of the calibrated vehicles. Virtual axles are set on the calibrated vehicle, and the location range of the real axles is obtained based on the bridge response. The actual axle location range is refined, and an element is selected in each range to construct a vehicle matrix. After iteration, several sets of axle load results are obtained. Substituting these results into the error formula, the actual axle load and wheelbase of the axle are obtained.

2. The dynamic weighing method integrating machine vision according to claim 1, characterized in that, The process of setting up calibration vehicles and collecting bridge responses when the calibration vehicles cross bridges specifically involves: Given that the axle load of each axle of the calibrated vehicle is P i The distance from each axis to the previous axis is D. i The calibration vehicle travels along a preset calibration path at a specified speed, and the corresponding bridge response is collected as R(k); it can be expressed by the following formula: Where N is the number of axles of the vehicle. The vertical index value of the influence line corresponding to the i-th axle at time step k, C i For the corresponding wheelbase D i The number of sampling points in the acquisition system; C i It can be expressed by the following formula: Where f is the sampling frequency and v is the vehicle speed.

3. The dynamic weighing method integrating machine vision according to claim 2, characterized in that, The matrix L is constructed based on the vehicle information as shown below, where m represents the number of sampling points where the vehicle is entirely on the bridge; (m+C N-1 () indicates the number of sampling points from when the vehicle starts to enter the bridge until it completely exits the bridge; The matrix is ​​solved using the Tikhonov regularization method, which is expressed as follows: λ is a non-negative regularization parameter, and T represents the regularization matrix.

4. The dynamic weighing method integrating machine vision according to claim 1, characterized in that, Cameras installed at both ends of the bridge are used to monitor vehicles on the bridge. A large number of images containing vehicles are extracted from the surveillance video. The vehicles in the images are labeled according to different vehicle models using a labeling tool. These images are used as training samples. The YOLO V4 algorithm is used to perform machine learning on the vehicle images. After training, vehicles appearing in the surveillance video are captured.

5. The dynamic weighing method integrating machine vision according to claim 1, characterized in that, The process of setting virtual axles on the calibrated vehicle and obtaining the location range of the real axles based on the bridge response is as follows: based on the vehicle length identified by machine vision, virtual axles are evenly arranged on the vehicle at a certain interval d, the bridge response R(k) collected by the vehicle passing through the bridge is obtained, the axle weight of the virtual axles is calculated using the non-negative least squares method, and the virtual axles with a result of 0 are removed to obtain the location range of the real axles.

6. The dynamic weighing method integrating machine vision according to claim 1, characterized in that, Refining the position interval where the real axle is located: Given the lower limit value AS of the axle wheelbase according to the vehicle type recognized by machine vision min , virtual axles with an adjacent virtual axle wheelbase less than AS min are divided into the same axle group, and the real axle is within the axle group interval; for each axle group, the axle group interval is again divided with σ and σ < d as the step lengths to fill virtual axles, and the filled axle group is denoted as {g1, g2, …, g N}; d is the preset spacing when virtual axles are evenly arranged on the vehicle, and N is the number of axles of the calibrated vehicle.

7. The dynamic weighing method integrating machine vision according to claim 1, characterized in that, The specific steps for obtaining the actual axle load and wheelbase are as follows: By taking one element from each meticulously divided axle group to form a possible combination of the actual axle positions, there are a total of There are N possible combinations of actual axle positions; for each combination, a set of axle load results is obtained, resulting in a total of N sets of results; the error function E for all combinations is calculated using the following formula: E = ||RI·P|| 2 Find the set of error values ​​with the smallest value; the corresponding result is the actual axle load and wheelbase of the axle. R represents the collected bridge dynamic response vector or measured value; P represents the axle load of each axle of the vehicle to be solved; g i The i-th axis group is obtained after detailed division and virtual axis filling with a step size of σ; N is the number of axles of the calibrated vehicle.

8. A dynamic weighing system integrating machine vision, characterized in that, include: The bridge response acquisition module is used to set up a calibration vehicle and collect the bridge response when the calibration vehicle passes over the bridge. The vehicle image acquisition module is used to acquire images of the calibrated vehicles and use these images as training samples. After training, the module captures vehicles appearing in the monitoring and obtains the length of the calibrated vehicles. The location range acquisition module is used to set up virtual axles on the calibrated vehicle and obtain the location range of the real axles based on the bridge response. The output module is used to refine the actual axle location range. Within each location range, an element is selected to construct a vehicle matrix. After iteration, several sets of axle load results are obtained. Substituting these results into the error formula, the actual axle load and wheelbase of the axle are obtained.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the dynamic weighing method integrating machine vision as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the dynamic weighing method incorporating machine vision as described in any one of claims 1 to 7.

Images