Commercial vehicle-based tractor and trailer included angle monitoring device and visual identification calculation method

Abstract

The invention provides a commercial vehicle-based tractor and trailer included angle monitoring device and a visual identification calculation method. Spherical markers with different colors are mounted on the left and right symmetrical sides of the front end face of a trailer with the central axis of a vehicle as the symmetry axis, and the colors of the spherical markers are collected through a camera mounted on the rear side of a tractor cab cabin; the size of pixels is calculated through segmentation, graying processing and color recognition, the distance between the current spherical marker and the camera is calculated through pixel change and distance calibration data, and the included angle between the current central axis of the trailer and the central axis of the tractor is calculated according to the distances between the spherical markers on the two sides and the camera. The characteristic that the spherical marker can not deform under the condition of any included angle of the vehicle is utilized, the image recognition precision is improved, and the trailer with irregular side faces can also be precisely recognized. According to the invention, the problems of reliability, calculation accuracy and cost in long-time use of traditional tractor and trailer included angle identification are solved.

Description

Technical Field

[0001] The present invention belongs to the field of automobile electrical technology, and in particular relates to a monitoring device and a visual recognition calculation method for an angle between a tractor and a trailer of a commercial vehicle. Background Art

[0002] As a common type of commercial vehicle, a tractor truck is generally composed of a tractor and a trailer, which are connected by a saddle and a rotating pin. When turning, the tractor and the trailer form an angle less than 180°, which is called the tractor-trailer angle. In the vehicle's 360° surround view system, it is necessary to perform real-time splicing of images around the vehicle and display the vehicle model. Therefore, it is necessary to monitor the angle between the tractor and the trailer in real time to avoid monitoring blind spots. In addition, in the field of autonomous driving, real-time monitoring of the angle between the tractor and the trailer is also essential when calculating the vehicle's trajectory, turning control, and other processes.

[0003] In traditional methods, the angle between the tractor and the trailer is generally recognized through electronic sensors. There are also emerging solutions for visual recognition through cameras. The calibration and calibration process of electronic sensors is very complicated, and the accuracy of the angle calculation is very dependent on the accuracy of the sensor. However, high-precision sensors are often expensive, which increases the cost and complexity of the system. When the edge of the trailer is recognized by the camera, a lot of image processing is required to achieve the purpose of extracting edge features, and the recognition of the edge is only applicable to trailers with vertical edges. For trailers without vertical edges and irregular sides (such as tank trailers), this solution cannot recognize. When using a QR code image as a marker to monitor the angle between the tractor and the trailer, the marker is a two-dimensional image of the plane. When the angle between the tractor and the trailer is large, the distortion and occlusion of the plane are very obvious, making it difficult to use the recognized image information, resulting in inaccurate calculated angle between the tractor and the trailer. Summary of the invention

[0004] In order to solve the calculation accuracy and cost problems of traditional tractor-trailer angle recognition, the present invention provides a monitoring device for the angle between a tractor and a trailer based on a commercial vehicle and a visual recognition calculation method. The spherical marker will not deform under any vehicle angle to improve the image recognition accuracy, and trailers with irregular sides can also be accurately identified. The system structure involved in the present invention is simple and the algorithm is accurate and fast.

[0005] The present invention is achieved by adopting the following technical solutions: On the one hand, the present invention provides a monitoring device for the angle between a tractor and a trailer of a commercial vehicle, the monitoring device comprising a tractor, a trailer, a camera, a spherical marker and a controller; the camera is installed on the rear side of the tractor cab; two spherical markers of different colors are arranged on the left and right symmetrical sides of the front end surface of the trailer with the vehicle center axis as the symmetry axis.

[0006] Preferably, the controller receives image information of spherical markers on both sides of the front of the trailer collected by the camera, performs image processing and calculation, and calculates the angle between the current center axis of the trailer and the center axis of the tractor.

[0007] On the other hand, the present invention also provides a visual recognition and calculation method for the angle between a tractor and a trailer based on a commercial vehicle, including the monitoring device described above, which collects the color of the spherical marker through a camera installed on the rear side of the tractor cab, and calculates the pixel size through segmentation, edge feature recognition, and color recognition. The distance between the current spherical marker and the camera is calculated through the calibration data of pixel change and distance, and the angle between the current center axis of the trailer and the center axis of the tractor is calculated based on the distance from the spherical markers on both sides to the camera.

[0008] Furthermore, the visual recognition calculation method specifically includes the following steps: Step S1: The camera collects image information of the front of the trailer; Step S2: performing image segmentation on the spherical markers in the image; Step S3: using image recognition technology to perform edge feature recognition on the spherical markers in the segmented image information; Step S4: obtaining the pixel area where the spherical marker is located according to the identified edge features; Step S5: identifying the color of the spherical marker on each side according to the image information of the pixel area where the spherical marker is located obtained in step S4; Step S6: judging the position of the spherical marker, i.e., the left side or the right side, according to the color of the identified spherical marker; Step S7: Calculate the pixel size of the spherical marker in the image according to the pixel area where the spherical marker is located obtained in step S4; Step S8: according to the pixel size of the spherical marker and the parameter data of the calibrated spherical marker size, the distance from the current spherical marker to the camera is obtained through calculation; Step S9: According to the distances from the spherical markers on both sides to the camera, the angle between the current center axis of the trailer and the center axis of the tractor is obtained through calculation.

[0009] Preferably, in step S3, the edge feature recognition includes inputting the image information of the front of the trailer into a pre-established object detection model to predict the edge information of the spherical marker in the image of the front of the trailer.

[0010] Preferably, in step S6, the HSV color model is used to convert the original image color space into HSV in OpenCV, and then the color range is defined, and the color ratio detected is used to determine whether the color of the current target spherical marker is the color of the spherical marker on the left or right.

[0011] Preferably, step S8, calculate the distance L from the current spherical marker to the optical center of the camera EF , L EF The calculation formula is: L EF =L EJ ×L GH / L IK Among them, point E is the optical center of the camera, points I and K are the edge points of the spherical marker in the image, point J is the center of points I and K, points H and G are the edge points of the spherical marker, and point F is the center of the spherical marker; L IK is the imaging width of the spherical marker in the image, L GH is the actual width of the spherical marker, L EJ is the focal length, L EF is the distance from the optical center of the camera to the spherical marker.

[0012] Preferably, in step S9, the angle θ between the tractor and the trailer is calculated as follows: θ=180°-∠a, where cos∠a=(L AD ²+L AE ²-L DE ²) / (2×L AD ×L AE ), Among them, point A is the rotation axis of the trailer, points B and C are the centers of the two spherical markers, point D is the midpoint between the two spherical markers, and point E is the optical center of the camera; L AD L is the distance from the trailer's rotation axis to the midpoint between the two spherical markers; AE L is the distance from the trailer's rotation axis to the camera's optical center; DE L is the distance from the midpoint between the two spherical markers to the optical center of the camera; AD , L AE and L BC For fixed distance, it can be calibrated in advance, L DE The calculation method is also through trigonometric function according to L BE , L CE and L BC Determine; the calculation formula is: ; In a third aspect, the present invention also provides the application of the monitoring device for the angle between the tractor and the trailer and the visual recognition calculation method thereof in a tractor and a semi-trailer tractor.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention utilizes the characteristic of spherical markers that will not deform at any angle of the vehicle to improve the accuracy of image recognition, and can also accurately identify trailers with irregular sides. The system involved in this patent has a simple structure and an accurate and fast algorithm; it solves the reliability, calculation accuracy and cost issues of traditional tractor and trailer angle recognition for long-term use. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Attached Figure 1 It is a schematic diagram of the angle between the tractor and the trailer; Attached Figure 2 This is a schematic diagram of the installation of a spherical marker on a trailer. In the figure, 1 is the tractor; 2 is the trailer; 3 is the camera; and 4 is the spherical marker.

[0015] Attached Figure 3 This is the flow chart for monitoring the angle between the tractor and trailer.

[0016] Attached Figure 4 Schematic diagram for calculating the distance from the current spherical marker to the camera. In the figure, point E is the optical center of the camera; points I and K are the edge points of the spherical marker in the image; point J is the midpoint of points I and K; points H and G are the edge points of the spherical marker, and point F is the center of the spherical marker.

[0017] Attached Figure 5 This is a schematic diagram of calculating the angle between the tractor and the trailer based on the distance. In the figure, point A is the rotation axis of the trailer, points B and C are the centers of two spherical markers, point D is the midpoint between the two spherical markers, and point E is the optical center of the camera. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0019] Glossary: ​​Camera Optical Center means Camera lens The optical center ofThe center point of the camera imaging system. The optical center of the camera is the center position of the optical axis of the lens optical system and the projection center. All light is projected onto the imaging plane through this point to form an image; it is equivalent to the position of the pinhole in pinhole imaging.

[0020] Example 1 A monitoring device for the angle between a tractor and a trailer based on a commercial vehicle, characterized in that the monitoring device comprises a tractor 1, a trailer 2, a camera 3, a spherical marker 4 and a controller; the camera 3 is installed at the rear side of the cab compartment of the tractor 1; two spherical markers 4 of different colors are arranged on both sides of the front face of the trailer 2, more precisely, on the front face of the trailer 2, on both sides symmetrically with the vehicle center axis as the symmetry axis. The controller receives the image information of the spherical markers on both sides of the front of the trailer collected by the camera 3, performs image processing and calculation, and calculates the angle between the current center axis of the trailer 2 and the center axis of the tractor 1.

[0021] The patent of the present invention is applied to tractors and semi-trailer tractors. The camera 3 is located at the rear of the cab, on the central axis of the vehicle, and its height is within 1 / 3 of the cab. The optical axis of the camera 3 is tilted downward relative to the horizontal plane, with an angle of 30-45° with the horizontal plane. The spherical marker 4 is symmetrically installed on both sides of the central axis of the trailer vehicle, with a height within the range of 2.5-3.5m, and the height should be lower than the camera 3, and the height difference with the camera 3 is less than 1m.

[0022] The spherical marker 4 is distinguished by color according to the installation position on the left and right sides. The color of the spherical marker 4 is bright color, which is used to distinguish it from the surrounding environment of the vehicle body (such as the color of the sky, the color of the road, etc.); the colors of the two spherical markers 4 can be any two colors. The surface of the spherical marker 4 is matte, and the roughness should be greater than 1.5μm; Specifically, the color range (RGB color gamut) of the two spherical markers can be set to the following colors: Red range: R (170-255); G (0-50); B (0-50); Yellow range: R (0-50); G (220-255); B (220-255); Blue range: R (0-50); G (0-50); B (170-255); The above is only an example, but the patent does not limit the combination of the above listed colors. As long as the two spherical markers 4 can be clearly distinguished in the camera image, it will be fine.

[0023] Example 2 A method for visual recognition and calculation of the angle between a tractor and a trailer based on a commercial vehicle includes the monitoring device described in Example 1, wherein the color of a spherical marker 4 is collected by a camera 3 installed on the rear side of a cab compartment of the tractor 1, and the pixel size is calculated by segmentation, edge feature recognition, and color recognition, and the distance between the current spherical marker 4 and the camera 3 is calculated by calibration data of pixel change and distance, and the angle between the current central axis of the tractor 1 and the central axis of the trailer 2 is calculated based on the distance from the spherical markers 4 on both sides to the camera 3.

[0024] The specific steps include: Step S1: The camera collects image information of the front of the trailer; Step S2: performing image segmentation on the spherical markers in the image; Step S3: Apply image recognition technology to perform edge feature recognition on the spherical markers in the segmented image information; specifically, in step S3, the edge feature recognition includes inputting the image information of the front of the trailer into a pre-established object detection model to predict the edge information of the spherical markers in the image of the front of the trailer.

[0025] Step S4: obtaining the pixel area where the spherical marker is located according to the identified edge features; Step S5: identifying the color of the spherical marker on each side according to the image information of the pixel area where the spherical marker is located obtained in step S4; Step S6: According to the color of the identified spherical marker, determine the position of the spherical marker, that is, the left or right side; specifically, in step S6, use the HSV color model to convert the original image color space into HSV in OpenCV, and then define the color range, and determine whether the color of the current target spherical marker is the color of the spherical marker on the left or right side through the detected color ratio.

[0026] Step S7: Calculate the pixel size of the spherical marker in the image according to the pixel area where the spherical marker is located obtained in step S4; Step S8: according to the pixel size of the spherical marker and the parameter data of the calibrated spherical marker size, the distance from the current spherical marker to the camera is obtained through calculation; Specifically, step S8, calculate the distance L from the current spherical marker to the optical center of the camera EF , L EF The calculation formula is: L EF =L EJ ×L GH / L IK like Figure 4As shown, point E is the optical center of the camera, points I and K are the edge points of the marker in the image, point J is the midpoint of points I and K, points H and G are the edge points of the spherical marker, and point F is the center of the spherical marker; L IK is the imaging width of the marker in the image, L GH is the actual width of the spherical marker, L EJ is the focal length, L EF is the distance from the optical center of the camera to the spherical marker.

[0027] Step S9: According to the distances from the spherical markers on both sides to the camera, the angle between the current central axis of the tractor 1 and the central axis of the trailer 2 is calculated.

[0028] Specifically, in step S9, the angle θ between the tractor and the trailer is calculated as follows: θ=180°-∠a, where cos∠a=(L AD ²+L AE ²-L DE ²) / (2×L AD ×L AE ); like Figure 5 As shown, point A is the rotation axis of the trailer, points B and C are the centers of the two spherical markers, point D is the midpoint between the two spherical markers, and point E is the optical center of the camera; L AD L is the distance from the trailer's rotation axis to the midpoint between the two spherical markers; AE L is the distance from the trailer's rotation axis to the camera's optical center; DE L is the distance from the midpoint between the two spherical markers to the optical center of the camera; AD , L AE and L BC For fixed distance, it can be calibrated in advance, L DE The calculation method is also through trigonometric function according to L BE , L CE and L BC Determine; the calculation formula is: ; The above description is only an example of the implementation of the present invention and does not limit the present invention in any form. The protection scope of the present invention shall be based on the claims and is not limited by the above specific embodiments. Any simple modification or equivalent changes and modifications made to the above implementation according to the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A monitoring device for the angle between a tractor and a trailer of a commercial vehicle, characterized in that: The monitoring device comprises a tractor (1), a trailer (2), a camera (3), a spherical marker (4) and a controller; the camera (3) is mounted on the rear side of the cab of the tractor (1); and two spherical markers (4) of different colors are arranged on the front end surface of the trailer (2) on two symmetrical sides with the vehicle centerline as the axis of symmetry.

2. The monitoring device according to claim 1, characterized in that: The controller receives image information of spherical markers on both sides of the front of the trailer collected by the camera (3), performs image processing and calculation, and calculates the angle between the current center axis of the trailer (2) and the center axis of the tractor (1).

3. A visual recognition calculation method for the angle between a tractor and a trailer of a commercial vehicle, characterized in that: The monitoring device comprises the monitoring device as claimed in claim 1, which collects the color of the spherical marker (4) through a camera (3) installed on the rear side of the cab compartment of the tractor (1), and calculates the pixel size through segmentation, edge feature recognition, and color recognition. The distance between the current spherical marker (4) and the camera (3) is calculated through the calibration data of pixel change and distance, and the angle between the center axis of the current trailer (2) and the center axis of the tractor (1) is calculated based on the distance from the spherical markers (4) on both sides to the camera (3).

4. The visual recognition calculation method according to claim 3, characterized in that: The specific steps include: Step S1: The camera collects image information of the front of the trailer; Step S2: performing image segmentation on the spherical markers in the image; Step S3: using image recognition technology to perform edge feature recognition on the spherical markers in the segmented image information; Step S4: obtaining the pixel area where the spherical marker is located according to the identified edge features; Step S5: identifying the color of the spherical marker on each side according to the image information of the pixel area where the spherical marker is located obtained in step S4; Step S6: judging the position of the spherical marker, i.e., the left side or the right side, according to the color of the identified spherical marker; Step S7: Calculate the pixel size of the spherical marker in the image according to the pixel area where the spherical marker is located obtained in step S4; Step S8: according to the pixel size of the spherical marker and the parameter data of the calibrated spherical marker size, the distance from the current spherical marker to the camera is obtained through calculation; Step S9: According to the distances from the spherical markers on both sides to the camera, the angle between the current trailer head plane and the tractor plane is calculated.

5. The visual recognition calculation method according to claim 3, characterized in that: In step S3, the edge feature recognition includes inputting the image information of the front of the trailer into a pre-established object detection model to predict the edge information of the spherical marker in the image of the front of the trailer.

6. The visual recognition calculation method according to claim 3, characterized in that: In step S6, the HSV color model is used to convert the original image color space into HSV in OpenCV, and then the color range is defined. The color ratio detected is used to determine whether the color of the current target spherical marker is the color of the spherical marker on the left or right.

7. The visual recognition calculation method according to claim 3, characterized in that: Step S8, calculate the distance L from the current spherical marker to the optical center of the camera EF , L EF The calculation formula is: L EF =L EJ ×L GH / L IK Among them, point E is the optical center of the camera, point I and point K are the edge points of the spherical marker in the image, point J is the midpoint of point I and point K, point H and point G are the edge points of the spherical marker, and point F is the center of the spherical marker; L IK is the imaging width of the spherical marker in the image, L GH is the actual width of the spherical marker, L EJ is the focal length, L EF is the distance from the optical center of the camera to the spherical marker.

8. The visual recognition calculation method according to claim 7, characterized in that: Step S9: The angle θ between the tractor and the trailer is calculated as follows: θ=180°-∠a, where cos∠a=(L AD ²+L AE ²-L DE ²) / (2×L AD ×L AE ), Among them, point A is the rotation axis of the trailer, points B and C are the centers of the two spherical markers, point D is the midpoint between the two spherical markers, and point E is the optical center of the camera; L AD The distance from the trailer's rotation axis to the midpoint between the two spherical markers; L AE L is the distance from the trailer's rotation axis to the camera's optical center; DE L is the distance from the midpoint between the two spherical markers to the optical center of the camera; AD , L AE and L BC For fixed distance, it can be calibrated in advance, L DE The calculation method is also through trigonometric function according to L BE , L CE and L BC Determine; the calculation formula is: 。 9. Application of the monitoring device for the angle between a tractor and a trailer and the visual recognition calculation method thereof according to any one of claims 1 to 8 in a tractor and a semi-trailer tractor.

Images