Non-contact vehicle load identification method based on perspective distortion compensation

Abstract

The application discloses a non-contact vehicle load identification method based on perspective distortion compensation, which comprises the following steps: calibrating the camera internal parameter for non-contact vehicle load identification; calculating the angle between the tire side and the camera image plane, and the two coordinates of the rim center which are subjected to perspective distortion in the camera coordinate system; identifying the end points of the tire-ground contact line in the image, and calculating the vertical projection of the rim center point on the tire-ground contact line; according to the projection relationship of the space vector on the image plane, combining the calibrated camera internal parameter, the angle between the tire side and the camera image plane and the two coordinates subjected to perspective distortion obtained in step two, the length of the actual tire-ground contact line is calculated; and based on this, vehicle load identification is carried out. The application can realize automatic compensation of perspective distortion in non-contact vehicle load identification without adding any markers.

Description

Technical Field

[0001] This invention belongs to the field of vehicle load recognition, specifically relating to a non-contact vehicle load recognition method based on perspective distortion compensation. Background Technology

[0002] Non-contact vehicle load identification technology has gained increasing attention due to its ability to significantly reduce the installation and maintenance costs of dynamic weighing systems. However, in real-world highway scenarios, the direction of vehicle travel is not always parallel to the camera's image plane; furthermore, the trajectories of different vehicles on the same road segment are unlikely to be completely identical. Therefore, images acquired by non-contact vehicle load identification systems inevitably suffer from perspective distortion, negatively impacting the accuracy of vehicle weight estimation. However, due to the randomness of vehicle travel directions on highways, it is difficult to automatically correct these perspective distortions using a uniform algorithm. Currently, the most common method is to manually select four corner points and then use an algorithm for image correction. However, this method is labor-intensive and inefficient, making it difficult to handle the massive amounts of data generated by the current high traffic volume. While algorithms for automatically selecting four corner points have been developed in some fields, these methods typically require pre-fixed markers (such as a checkerboard pattern) on the object being measured. Therefore, unless all vehicles are required to install checkerboard-like markers on each wheel according to a uniform standard, such methods will be difficult to adapt to non-contact vehicle load identification. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes a non-contact vehicle load identification method based on perspective distortion compensation. The specific technical solution is as follows:

[0004] A non-contact vehicle load identification method based on perspective distortion compensation, the method comprising:

[0005] Step 1: Calibrate the camera intrinsic parameters for non-contact vehicle load identification, including the camera principal distance f and principal point coordinates (c). x ,c y ) and the horizontal pixel pitch s on the camera sensor x and vertical pixel spacing s y The camera needs to be installed on the roadside with its image plane perpendicular to the road surface; the camera coordinate system is centered at the projection center O. c A rectangular coordinate system with the origin at the origin, and a line parallel to the image plane and the road surface as the x-axis of the camera coordinate system. c The axis is the line perpendicular to the road surface, and the line is y. c The axis, the direction perpendicular to the image plane is z. c axis; x c The axis is positive with the direction of vehicle movement, y c The axis is positive in the direction away from the road surface, z cThe axis is positive in the direction away from the image plane;

[0006] Step 2: Calculate in real time the angle θ between the tire sidewall and the camera image plane, and the distance along the x-axis of the rim center C in the camera coordinate system. c Coordinates p and z along the axes c The coordinate d on the axis;

[0007] Step 3: Identify the endpoints of the tire-ground contact line in the image and calculate the vertical projection of the rim center point onto the tire-ground contact line; based on the projection relationship of the spatial vector on the image plane, combined with the camera intrinsic parameters calibrated in Step 1 and θ, p, and d obtained in Step 2, calculate the actual length of the tire-ground contact line.

[0008] Step 4: Identify vehicle load based on the actual tire-ground contact length after perspective distortion compensation.

[0009] Furthermore, step two specifically includes the following sub-steps:

[0010] S2.1: Obtain the tire model markings using character recognition technology, and then query the actual rim diameter D. rim ;

[0011] S2.2: Identify the maximum row coordinate A′ of the point on the rim edge in the image. θ (r) and minimum row coordinate B′ θ (r), and calculate the coordinates d of the rim center C in the camera coordinate system in the direction perpendicular to the image plane based on the following formula:

[0012]

[0013] S2.3: Identify the minimum column coordinates D′ of points on the rim edge in the image. θ (c) Maximum column coordinate E′ θ (c) and column coordinates C of the rim center point θ Based on the following equations, the coordinates p of the rim center C in the camera coordinate system along the direction parallel to the image plane and the road surface, and the angle θ between the tire sidewall and the camera image plane are obtained by solving simultaneously:

[0014]

[0015]

[0016] Furthermore, step three specifically includes the following sub-steps:

[0017] S3.1: Identify the endpoints D′ of the tire-ground contact line in the image. L and E′ L Here D′ LE′ represents the endpoint with the smaller column coordinate. L Represents the endpoint with the larger column coordinate;

[0018] S3.2: Calculate the vertical projection C of the rim center point onto the tire-ground contact line. L ′;

[0019] S3.3: Calculate the vectors according to the following formulas respectively. In y c Length of axial component vector In x c O c z c Length of the component on the plane vector In x c O c z c Length of the component on the plane And finally calculate the compensated tire-ground contact line length L:

[0020]

[0021]

[0022]

[0023]

[0024] Among them, D L and E L This represents the endpoint of the actual tire-ground contact line.

[0025] Furthermore, identify the maximum row coordinate A′ of the point on the rim edge in the image. θ (r), minimum row coordinate B′ θ (r), minimum column coordinate D′ θ (c) Maximum column coordinate E′ θ (c) includes the following methods:

[0026] (1) Identify wheel rims in images based on target detection algorithms, and perform binary segmentation and morphological processing on the regions within the detection boxes;

[0027] (2) Rotate a fan-shaped window with the center of the detection box as the vertex around the center of the detection box in a step τ of units in arcs. If the pixel at the center of the detection box is white, record the coordinates of the black pixel closest to the center of the detection box in each step window and fit these recorded coordinates into an ellipse. If the pixel at the center of the detection box is black, record the coordinates of the white pixel closest to the center of the detection box in each step window and fit these recorded coordinates into an ellipse.

[0028] (3) Solve for the maximum row coordinates A′ of the points on the ellipse based on the fitted ellipse equation. θ (r), minimum row coordinate B′ θ (r), minimum column coordinate D′ θ (c) Maximum column coordinate E′ θ (c)

[0029] Furthermore, step four specifically includes the following sub-steps:

[0030] S4.1: Based on the tire model identifier obtained in S2.1, query the nominal section width B of the tire. s and tire outer diameter D o ;

[0031] S4.2: Calculate the vertical contact force F between the wheel and the ground based on the following formula:

[0032]

[0033] Where, p in It refers to the tire inflation pressure, which is measured by the vehicle's onboard tire pressure monitoring system and wirelessly transmitted to a roadside contactless vehicle load identification system; λ e It is the equivalent Poisson ratio, λ e =0.33; ξ is a correction factor determined by the tire type, specifically: ξ=1 for radial tires of passenger cars and light trucks, ξ=1.1 for radial tires of buses and heavy trucks, ξ=1.1 for bevel tires of passenger cars and light trucks, and ξ=1.2 for bevel tires of buses and heavy trucks.

[0034] S4.3: Repeat steps S2.1 to S4.2 to obtain the vertical contact force between each wheel of the vehicle under test and the ground, and obtain the total load of the vehicle under test by summing them.

[0035] Furthermore, before the ellipse fitting, outliers among the fitting points are removed based on the distance of each fitting point from the center of the detection box.

[0036] Furthermore, the camera is selected from any one of a regular camera, an infrared camera, or a night vision device.

[0037] The beneficial effects of this invention are as follows:

[0038] (1) The technical solution provided by the present invention does not require the extraction of corner points by manual means or by using algorithms, nor does it require the addition of any markers on the object being tested.

[0039] (2) The technical solution provided by this invention has lower computational load and faster operation speed, making it more suitable for application in actual road traffic scenarios. Existing perspective correction techniques for images typically require correcting the position of every pixel in the image. This process often involves filling in unknown grayscale values ​​in the corrected image using techniques such as nearest neighbor interpolation and bilinear interpolation. The large amount of computation involved in this process does not contribute to non-contact load recognition, and pixel interpolation may reduce the accuracy of tire edge recognition. In contrast, the technical solution of this invention, based on imaging geometry principles, only corrects the coordinates of a few key points, such as the endpoints of the tire-ground contact line segment and the vertical projection of the rim center point onto the tire-ground contact line, without requiring pixel interpolation, thus greatly reducing redundancy. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of an optical model for a non-contact vehicle load recognition method based on perspective distortion compensation.

[0041] Figure 2 The optical model for perspective distortion compensation method in y c O c z c Projection onto a plane.

[0042] Figure 3 The optical model for perspective distortion compensation method in x c O c z c Projection onto a plane.

[0043] Figure 4 This is a general flowchart of a preferred embodiment of the present invention.

[0044] Figure 5 This is a schematic diagram of a preferred embodiment for identifying extreme points on the edge of a wheel rim from an image. Detailed Implementation

[0045] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become clearer. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0046] The present invention provides a non-contact vehicle load identification method based on perspective distortion compensation, which is based on a single-view optical model for non-contact vehicle load identification that takes into account perspective distortion (e.g., Figure 1This optical model is based on three assumptions:

[0047] i. The imaging geometry model conforms to the pinhole camera model;

[0048] ii. The wheel (measuring plane) and the image plane are perpendicular to the ground;

[0049] iii. The rim edge is a circle on the measuring plane.

[0050] The camera can be selected from any one of the following: a regular webcam, an infrared camera, or a night vision device.

[0051] like Figure 1 As shown, this embodiment uses three coordinate systems in the optical model: the image coordinate system (ICS), the camera coordinate system (CCS), and the world coordinate system (WCS). Point P in the WCS... w =(x w ,y w ,z w ) T Point P in the CCS can be mapped through a rigid transformation. c =(x c ,y c ,z c ) T And point P in CCS c =(x c ,y c ,z c ) T This can be further mapped to the point (c,r) in ICS. T .

[0052] To make the content of this invention clearer, in this embodiment, the optical model described above is projected onto the y-axis of the camera coordinate system. c O c z c Plane (e.g.) Figure 2 ) and x c O c z c On a plane (e.g.) Figure 3 ).

[0053] Specifically, such as Figure 2 As shown, if the measurement plane is parallel to the image plane, the edge of the rim is at the y-axis of the CCS. c O c z c The projection onto the plane is a line segment A0B0; if there is an angle θ between the measurement plane and the image plane, the edge of the rim is on the CCS at y c O c z c The projection onto the plane is an ellipse; such as Figure 3As shown, if the measurement plane is parallel to the image plane, the edge of the rim is at the x-axis of the CCS. c O c z c The projection onto the plane is line segment D0E0; if there is an angle θ between the measurement plane and the image plane, the edge of the rim is on the x-axis of the CCS. c O c z c The projection on the plane is line segment D. θ E θ ; Figure 2 and Figure 3 In this context, the subscript 0 indicates that the measurement plane and the image plane are parallel, while the subscript θ indicates that the measurement plane and the image plane have an angle θ between them, where θ∈(-π / 2,π / 2); the apostrophe in the upper right corner of the letter indicates that the point is a projection onto the image plane.

[0054] In this embodiment, the preferred non-contact vehicle load identification method based on perspective distortion compensation (such as...) Figure 4 ),include:

[0055] Step S1: During the installation or maintenance phase of the non-contact vehicle load recognition system, the camera's intrinsic parameters are pre-calibrated. These camera intrinsic parameters include the camera principal distance f and the principal point coordinates (c). x ,c y ) and the horizontal pixel pitch s on the camera sensor x and vertical pixel spacing s y .

[0056] In this embodiment, an industrial camera was used and mounted on the roadside using a tripod equipped with a level and tilt meter. During installation, the camera's image plane was adjusted to be perpendicular to the road surface.

[0057] Step S2: Calculate the angle θ between the tire sidewall and the camera image plane, and the distance from the rim center C to the x-axis in the camera coordinate system. c Coordinates p and z along the axes c The coordinate d on the axis.

[0058] In this embodiment, the camera coordinate system is based on the projection center O. c A rectangular coordinate system with the origin at the origin, and a line parallel to the image plane and the road surface as the x-axis of the camera coordinate system. c The axis, the straight line perpendicular to the road surface is y c The axis, the direction perpendicular to the image plane is z. c axis, x c The axis is positive with the direction of vehicle movement, y c The axis is positive in the direction away from the road surface, z c The axis is positive in the direction away from the image plane.

[0059] As one implementation method, step S2 specifically includes the following sub-steps:

[0060] S2.1: Obtain the tire model markings using character recognition technology, and then determine the actual rim diameter D by looking up a table. rim ;

[0061] S2.2: Identify the maximum row coordinate A′ of the point on the rim edge in the image. θ (r) and minimum row coordinate B′ θ (r), and calculate the coordinates d of the rim center C in the camera coordinate system in the direction perpendicular to the image plane based on the following formula:

[0062]

[0063] S2.3: Identify the minimum column coordinates D′ of points on the rim edge in the image. θ (c) Maximum column coordinate E′ θ (c) and column coordinates C of the rim center point θ Based on the following equations, the coordinates p of the rim center C in the camera coordinate system along the direction parallel to the image plane and the road surface, and the angle θ between the tire sidewall and the camera image plane are obtained by solving simultaneously:

[0064]

[0065]

[0066] Additionally, the center C of the wheel rim lies along the y-axis in the camera coordinate system. c The coordinate h along the axial direction can also be calculated. Specifically, the row coordinate C′(r) of the center point of the wheel rim in the identified image is used to calculate h using the following formula:

[0067]

[0068] On the other hand, the maximum row coordinate A′ of the point on the rim edge in the image is identified. θ (r), minimum row coordinate B′ θ (r), minimum column coordinate D′ θ (c) Maximum column coordinate E′ θ (c) methods include (such as) Figure 5 As shown):

[0069] (1) The target detection algorithm based on the YOLOv5s model identifies the wheel rim in the image and performs binary segmentation and morphological processing on the region within the wheel rim detection box.

[0070] (2) Rotate a fan-shaped window with the center of the detection box as its vertex around the center of the detection box in a step τ of units of arc. If the pixel at the center of the detection box is white, record the coordinates of the nearest black pixel to the center of the detection box in each step window, and fit these recorded coordinates into an ellipse. If the pixel at the center of the detection box is black, record the coordinates of the nearest white pixel to the center of the detection box in each step window, and fit these recorded coordinates into an ellipse. Figure 5 In the illustrated embodiment, the stride τ is π / 18, and the center pixel of the detection box is white.

[0071] (3) Solve for the maximum row coordinates A′ of the points on the ellipse based on the fitted ellipse equation. θ (r), minimum row coordinate B′ θ (r), minimum column coordinate D′ θ (c) Maximum column coordinate E′ θ (c)

[0072] To reduce computational load and ensure accurate ellipse fitting, outliers can be removed from the fitted points before fitting, based on the distance of each fitted point from the center of the detection box.

[0073] Step S3: Identify the endpoints of the tire-ground contact line in the image and calculate the vertical projection of the rim center point onto the tire-ground contact line; based on the projection relationship of the spatial vector on the image plane, combined with the camera intrinsic parameters calibrated in Step 1 and θ, p, and d obtained in Step 2, calculate the endpoints of the actual tire-ground contact line.

[0074] In one implementation, step S3 includes the following sub-steps:

[0075] S3.1: Identify the endpoints D′ of the tire-ground contact line in the image. L and E′ L Here D′ L E′ represents the endpoint with the smaller column coordinate. L Represents the endpoint with the larger column coordinate.

[0076] S3.2: Calculate the vertical projection C of the rim center point onto the tire-ground contact line. L ′.

[0077] S3.3: Calculate the vectors according to the following formulas respectively. In y c Length of axial component vector In x c O c z c Length of the component on the plane vector In xc O c z c Length of the component on the plane And finally calculate the actual tire-ground contact line length L after compensation:

[0078]

[0079]

[0080]

[0081]

[0082] Among them, D L and E L This represents the endpoint of the actual tire-ground contact line.

[0083] Step S4: Identify vehicle load based on the actual tire-ground contact length after perspective distortion compensation.

[0084] In one implementation, step S4 includes the following sub-steps:

[0085] S4.1: Based on the tire model identifier obtained in S2.1, query the nominal section width B of the tire. s and tire outer diameter D o ;

[0086] S4.2: Calculate the vertical contact force F between the wheel and the ground based on the following formula:

[0087]

[0088] Where, p in It refers to the tire inflation pressure, which is measured by the vehicle's onboard tire pressure monitoring system and wirelessly transmitted to a roadside contactless vehicle load identification system; λ e It is the equivalent Poisson ratio, λ e =0.33; ξ is a correction factor determined by the tire type, specifically: ξ=1 for radial tires of passenger cars and light trucks, ξ=1.1 for radial tires of buses and heavy trucks, ξ=1.1 for beveled tires of passenger cars and light trucks, and ξ=1.2 for beveled tires of buses and heavy trucks.

[0089] S4.3: Repeat steps S2.1 to S4.2 to obtain the vertical contact force between each wheel of the vehicle under test and the ground, and obtain the total load of the vehicle under test by summing them.

[0090] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A non-contact vehicle load identification method based on perspective distortion compensation, characterized in that, The method includes: Step 1: Calibrate the camera intrinsic parameters for non-contact vehicle load identification, including the camera principal distance f and principal point coordinates (c). x c y ) and the horizontal pixel pitch s on the camera sensor x and vertical pixel spacing s y The camera needs to be installed on the roadside with its image plane perpendicular to the road surface; the camera coordinate system is centered at the projection center O. c A rectangular coordinate system with the origin at the origin, and a line parallel to the image plane and the road surface as the x-axis of the camera coordinate system. c The axis, the straight line perpendicular to the road surface is y c The axis, the direction perpendicular to the image plane is z. c axis; x c The axis is positive with the direction of vehicle movement, y c The axis is positive in the direction away from the road surface, z c The axis is positive in the direction away from the image plane; Step 2: Calculate in real time the angle θ between the tire sidewall and the camera image plane, and the distance along the x-axis of the rim center C in the camera coordinate system. c Coordinates p and z along the axes c The coordinate d on the axis; Step 3: Identify the endpoints of the tire-ground contact line in the image and calculate the vertical projection of the rim center point onto the tire-ground contact line; based on the projection relationship of the spatial vector on the image plane, combined with the camera intrinsic parameters calibrated in Step 1 and θ, p, and d obtained in Step 2, calculate the actual length of the tire-ground contact line. Step 4: Identify vehicle load based on the actual tire-ground contact length after perspective distortion compensation.

2. The non-contact vehicle load identification method based on perspective distortion compensation according to claim 1, characterized in that, Step two specifically includes the following sub-steps: S2.1: Obtain the tire model markings using character recognition technology, and then query the actual rim diameter D. rim ; S2.2: Identify the maximum row coordinate A′ of the point on the rim edge in the image. θ (r) and minimum row coordinate B′ θ (r), and calculate the coordinates d of the rim center C in the camera coordinate system in the direction perpendicular to the image plane based on the following formula: S2.3: Identify the minimum column coordinates D′ of points on the rim edge in the image. θ (c) Maximum column coordinate E′ θ (c) and the column coordinates C′ of the rim center point θ (c) Based on the following equations, the coordinates p of the rim center C in the camera coordinate system along the direction parallel to the image plane and the road surface, and the angle θ between the tire sidewall and the camera image plane are obtained:

3. The non-contact vehicle load identification method based on perspective distortion compensation according to claim 1, characterized in that, Step three specifically includes the following sub-steps: S3.1: Identify the endpoints D′ of the tire-ground contact line in the image. L and E′ L Here D′ L E′ represents the endpoint with the smaller column coordinate. L Represents the endpoint with the larger column coordinate; S3.2: Calculate the vertical projection C′ of the rim center point onto the tire-ground contact line. L ; S3.3: Calculate the vectors according to the following formulas respectively. In y c Length of axial component vector In x c O c z c Length of the component on the plane vector In x c O c z c Length of the component on the plane And finally calculate the compensated tire-ground contact line length L: Among them, D L and E L This represents the endpoint of the actual tire-ground contact line.

4. The non-contact vehicle load identification method based on perspective distortion compensation according to claim 2, characterized in that, Identify the maximum row coordinate A′ of a point on the rim edge in the image. θ (r), minimum row coordinate B′ θ (r), minimum column coordinate D′ θ (c) Maximum column coordinate E′ θ (c) includes the following methods: (1) Identify wheel rims in images based on target detection algorithms, and perform binary segmentation and morphological processing on the regions within the detection boxes; (2) Rotate a fan-shaped window with the center of the detection box as the vertex around the center of the detection box in a step τ of units in arcs. If the pixel at the center of the detection box is white, record the coordinates of the black pixel closest to the center of the detection box in each step window and fit these recorded coordinates into an ellipse. If the pixel at the center of the detection box is black, record the coordinates of the white pixel closest to the center of the detection box in each step window and fit these recorded coordinates into an ellipse. (3) Solve for the maximum row coordinates A′ of the points on the ellipse based on the fitted ellipse equation. θ (r), minimum row coordinate B′ θ (r), minimum column coordinate D′ θ (c) Maximum column coordinate E′ θ (c) 5. The non-contact vehicle load identification method based on perspective distortion compensation according to claim 2, characterized in that, Step four specifically includes the following sub-steps: S4.1: Based on the tire model identifier obtained in S2.1, query the nominal section width B of the tire. s and tire outer diameter D o ; S4.2: Calculate the vertical contact force F between the wheel and the ground based on the following formula: Where, p in It refers to the tire inflation pressure, which is measured by the vehicle's onboard tire pressure monitoring system and wirelessly transmitted to a roadside contactless vehicle load identification system; λ e It is the equivalent Poisson ratio, λ e =0.33; ξ is a correction factor determined by the tire type, specifically: ξ=1 for radial tires of passenger cars and light trucks, ξ=1.1 for radial tires of buses and heavy trucks, ξ=1.1 for bevel tires of passenger cars and light trucks, and ξ=1.2 for bevel tires of buses and heavy trucks. S4.3: Repeat steps S2.1 to S4.2 to obtain the vertical contact force between each wheel of the vehicle under test and the ground, and obtain the total load of the vehicle under test by summing them.

6. The non-contact vehicle load identification method based on perspective distortion compensation according to claim 4, characterized in that, Before the ellipse fitting, outliers among the fitting points are removed based on the distance of each fitting point from the center of the detection box.

7. The non-contact vehicle load identification method based on perspective distortion compensation according to claim 1, characterized in that, The camera is selected from any one of the following: a regular webcam, an infrared camera, or a night vision device.

Images