Method and system for calculating a vehicle trailer angle
By capturing images of the trailer from different angles and calculating the yaw angle between the trailer and the tractor, the robustness problem when the image quality is poor is solved, and the effect of accurately calculating the yaw angle in the automatic trailer system is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-01
- Publication Date
- 2026-03-03
AI Technical Summary
Existing technologies struggle to provide robust trailer-to-tractor angle calculations when image quality is poor.
By using a vehicle camera to capture at least two images of the trailer from different angles, identifying feature points on the trailer, calculating the yaw angle between the trailer and the fixed point of the tractor, and using geometric methods and angle estimation, combined with the median or average value, the yaw angle is calculated to reduce the impact of noise.
Even under conditions of poor image quality or noise, it can accurately calculate the trailer yaw angle, improving the robustness and reliability of the calculation, and is suitable for automatic trailer reversing systems.
Smart Images
Figure CN115335863B_ABST
Abstract
Description
Technical Field
[0001] This invention generally relates to the field of vehicle assistance systems. More specifically, this invention relates to a method and system for calculating the yaw angle of a trailer connected to a tractor based on image information provided by a vehicle camera device. Background Technology
[0002] Methods for calculating the angle of the trailer relative to the tractor based on image information provided by the vehicle's camera device are known.
[0003] Specifically, known methods have low computational complexity, but cannot provide robust angular information when image quality is poor. Summary of the Invention
[0004] The objective of this invention is to provide a robust and reliable method for calculating trailer yaw angles without requiring prior knowledge of the tow bar position. This objective is achieved through the features of the independent claims. Preferred embodiments are given in the dependent claims. Unless otherwise explicitly stated, embodiments of this invention can be freely combined with each other.
[0005] According to one aspect, the present invention relates to a method for determining the yaw angle of a trailer relative to the longitudinal axis of a tractor. The method includes the following steps:
[0006] First, at least first and second images of the trailer are captured using a camera device. The method for capturing the first and second images is such that the orientation of the trailer relative to the vehicle is different in at least the two images.
[0007] After capturing the images, at least a first feature and a second feature of the trailer that must be visible in both the first and second images are determined. Furthermore, the first feature is located at a different position on the trailer than the second feature. For example, the first feature could be a first feature clearly visible at a first position, and the second feature could be a second feature clearly visible at a second position.
[0008] A first angle estimate is calculated based on the determined first and second features. The first angle estimate characterizes the sway angle between the first feature on the first image and the first feature on the second image relative to a fixed point on the tractor unit. In other words, the first angle estimate refers to a sway angle between a first line constrained by the position of the first feature on the first image and the fixed point position, and a second line constrained by the position of the first feature on the second image and the fixed point position. The sway angle opens in the direction from the vehicle toward the trailer.
[0009] Furthermore, a second angle estimate is calculated. This second angle estimate characterizes the sway angle between a second feature on the first image and a second feature on the second image relative to a fixed point on the tractor. In other words, the second angle estimate refers to the sway angle between a first line constrained by the position of the second feature on the first image and the fixed point position, and a second line between the position of the second feature on the second image and the fixed point position. This sway angle opens in the direction from the vehicle toward the trailer.
[0010] It is worth noting that the term "first / second feature position on the first / second image" does not refer to a point on the image, but rather to a specific location in the environment surrounding the tractor, where the corresponding trailer features are located at the specific point in time when the image was captured.
[0011] Finally, based on the first and second angle estimates, the trailer's yaw angle is calculated.
[0012] The method is advantageous because it uses two or more images and calculates the yaw angle using two or more trailer features. Even if the detection of trailer features is affected by high noise or the image quality is poor, the determination of the yaw angle is extremely reliable and robust.
[0013] Other methods for pinpointing the location of a fixed point often require a triangulation of the feature's position to produce accurate angles. This makes the method highly susceptible to noise or inaccuracies when tracking features. If these features are inaccurate, the method may become mathematically unstable or produce no results at all.
[0014] According to one embodiment, in the first or second image, the yaw angle of the trailer relative to the vehicle is zero. Therefore, this image can serve as a "zero-attitude image," i.e., a reference for accurate alignment of the vehicle's longitudinal axis with the trailer's longitudinal axis. However, another yaw angle value can also be used as a reference. If the other yaw angle is unknown, the system can calculate the change in the trailer's angle, rather than the absolute angle of the trailer.
[0015] According to one embodiment, the fixed point is the location of a camera device or a tow bar. Since using a camera device as a fixed point is technically simple and feasible because it captures images, it is technically straightforward. However, using a tow bar as a fixed point may be more accurate. Therefore, the information contained in the images captured by the camera device can be transformed to reduce the loss of accuracy by using the location of a fixed point, such as a tow bar, to allow for corresponding adjustments to the lighting. However, if the tow bar is relatively close to the camera device and relatively far from the trailer feature, the proposed method can calculate the trailer angle without adjusting the tow bar position, which is sufficiently accurate for an automated trailer reversing system. If the tow bar is close to the camera device (e.g., less than 0.3 meters horizontally) and 2 meters or more horizontally from the trailer feature, the proposed method may lead to improved results.
[0016] According to one embodiment, calculating the first and second angle estimates includes determining a ray between the fixed point and the first and second features on the first and second images. The ray refers to a line between the fixed point and the first and second features. Based on the ray, the current swing angle can be determined with reduced computational effort, for example, based on geometric methods.
[0017] According to one embodiment, the camera device calibration information is used to convert the positions of the first and / or second features into light. For example, when the camera device calibration information is used to determine the position of the camera device, the position of a specific feature on the image can be transmitted in the position information based on or associated with the camera device position.
[0018] According to one embodiment, in addition to the first and second features, at least one more trailer feature is used for yaw angle calculation. Using three or more features further improves the robustness and reliability of yaw angle determination.
[0019] According to one implementation, the yaw angle is calculated by establishing the median of at least two angle estimates. This results in a very stable yaw angle determination.
[0020] According to other implementations, the yaw angle is calculated by establishing the average of at least two angle estimates or by using statistical methods applied to the angle estimates.
[0021] According to one embodiment, the method further includes the step of determining an angle window. The angle window may include an upper and lower limit for the yaw angle. Furthermore, a set of features is determined, wherein features in the feature set result in an angle estimate that falls within the angle window. Preferably, only features included in the determined feature set are used for future yaw angle calculations. In other words, information previously used to determine the yaw angle is used to determine two or more features of the trailer that result in an angle estimate that is quite close to the determined yaw angle (i.e., within the angle window), without tracking those features that cause the angle estimate to deviate significantly from the determined yaw angle (i.e., beyond the angle window). The computational complexity and accuracy requirements for angle estimation are thus significantly reduced.
[0022] According to one implementation, the calculated yaw angle value is increased by a certain percentage to compensate for underestimation. For example, the calculated yaw angle may be increased proportionally by 5% to 15%, especially by 10%, to compensate for underestimation in the calculation result.
[0023] According to one embodiment, the camera device is a rear-view camera device for a vehicle. Based on the rear-view camera device, images of the trailer can be captured with less technical effort.
[0024] According to another aspect, a system for determining the yaw angle of a trailer relative to the longitudinal axis of a tractor is disclosed. The system includes a camera device for capturing images of the trailer and a processing entity for processing the captured images. Furthermore, the system is configured to perform the following steps:
[0025] - Use a camera device to detect at least first and second images of the trailer, wherein the trailer's orientation relative to the vehicle is different in at least two images.
[0026] - Identify at least a first feature and a second feature of the trailer visible in the first and second images, wherein the first feature and the second feature are set at different locations on the trailer;
[0027] - Calculate a first angle estimate, wherein the first angle estimate characterizes the sway angle between a first feature on a first image and a first feature on a second image relative to a fixed point of the tractor on a horizontal plane;
[0028] - Calculate a first angle estimate, wherein the second angle estimate characterizes the swing angle between a second feature on a first image and a second feature on a second image relative to the fixed point of the tractor on a horizontal plane;
[0029] - Calculate the yaw angle based on the first and second angle estimates.
[0030] Any of the features described as embodiments of the method described above may also be system features in the system described in this patent application published herein.
[0031] According to another embodiment, a vehicle including the system described in any of the above embodiments is disclosed.
[0032] The term "vehicle" as used in this invention may refer to automobiles, trucks, buses, rail vehicles or any other means of transportation.
[0033] The term "yaw angle" as used in this patent application published herein may refer to the yaw angle between the longitudinal axis of the vehicle and the longitudinal axis of the trailer.
[0034] The term “median” as used in this patent application published herein may refer to a value that separates the higher half of a data sample or probability distribution from the lower half.
[0035] The terms “substantially” or “approximately” as used in this invention refer to deviations from the exact value by + / -10%, preferably + / -5%, and / or variations that are not significant to function and / or to traffic rules. Attached Figure Description
[0036] Different aspects of the invention, including its particular features and advantages, will be more readily understood from the following detailed description and accompanying drawings, wherein:
[0037] Figure 1 An exemplary top view of a vehicle towing a trailer is shown;
[0038] Figure 2 A schematic diagram illustrating angle estimation of first and second features detected at different swing angles between the trailer and the tractor based on images from a camera device;
[0039] Figure 3 A schematic block diagram illustrating the steps of a method for determining the yaw angle of a trailer relative to the longitudinal axis of a tractor is shown. Detailed Implementation
[0040] The invention will now be described in more detail with reference to the accompanying drawings, which illustrate exemplary embodiments. The embodiments in the drawings relate to preferred embodiments, and all elements and features described in connection with the embodiments can be used, where possible, in conjunction with any other embodiments and features discussed herein, particularly in relation to any other embodiments further discussed above. However, the invention should not be construed as limited to the embodiments described herein. Throughout the following description, similar reference numerals are used to denote similar elements, parts, items, or features, where applicable.
[0041] The features of the invention disclosed in the description, claims, embodiments and / or drawings can be used individually or in any combination to implement the invention.
[0042] Figure 1The diagram shows a top view of vehicle 1 towing trailer 2. Vehicle 1 includes a longitudinal axis LAV passing through its center. Similarly, trailer 2 has a longitudinal axis LAT passing through its center. Trailer 2 is connected to vehicle 1 via a trailer hitch assembly including a drawbar 4.
[0043] In certain driving situations, the vehicle's longitudinal axis LAV and the trailer's longitudinal axis LAT may not be parallel or coincident; instead, the two longitudinal axes may define a yaw angle YA. In other words, the yaw angle YA defines the angular deviation of the trailer 2's longitudinal axis LAT relative to the vehicle 1's longitudinal axis LAV. The yaw angle YA can be measured on a horizontal plane that includes both the trailer 2's longitudinal axis LAT and the vehicle 1's longitudinal axis LAV.
[0044] Understanding the yaw angle YA is also beneficial, for example, in trailer assist systems.
[0045] To determine the yaw angle YA, multiple images of at least a portion of the trailer 2 are captured using a camera device 3. The camera device 3 may be, for example, a vehicle rearview camera. It can also be used to capture images of the vehicle's surroundings while reversing.
[0046] Figure 2 The diagram shows the angular relationship between the first and second features F1 and F2 of the trailer at different points in time, at which the trailer 2 has different yaw angles relative to the tractor 1.
[0047] The camera device 3 can capture two or more images of the trailer 2 at different times with different angular positions relative to the vehicle 1. For example, a series of images can be captured.
[0048] In this example, the second image may show the orientation of trailer 2 relative to the vehicle when the yaw angle YA = 0 degrees. However, according to other embodiments, the yaw angle YA can be any other known reference yaw angle and can be used to determine the current yaw angle.
[0049] Features on the trailer are located and matched using feature detection and matching algorithms. For example, Harris Corner Detector, Scale Invariant Feature Transform (SIFT), Speed-Up Robust Features (SURF), Binary Robust Invariant Scalable Keypoints (BRISK), Binary Robust Independent Basic Features (BRIEF), Oriented Fast Rotating Briefing (ORB), or other suitable feature detection and matching algorithms can be used.
[0050] The feature detection and matching algorithm can detect image features that are on or off the trailer. Several different methods can be used to separate trailer features from non-trailer features. For example, when traveling in a straight line, trailer features can be separated from non-trailer features by finding features that remain in the same position over time. Alternatively, the motion of background features can be modeled using known vehicle motion over time. This can be extracted from CAN (Controller Area Network) data regarding speed and steering. Features that do not conform to the basic matrix epipolar constraints can be considered trailer features.
[0051] Multiple different features can be identified in the images captured by camera device 3. Figure 2 This describes features F1 and F2 identified at different angular positions relative to a fixed point on vehicle 1. Thus, the upper pair of first and second features F1, F2 (associated with the solid line connecting features F1 and F2 to the camera device 3) are identified in the first image, while the lower pair of first and second features F1, F2 (associated with the dashed line connecting features F1 and F2 to the camera device 3) are identified in the second image at a different time point. To determine the light connecting features F1 and F2 to the camera device 3, the feature positions in the image coordinates can be converted into light rays using the calibration information of the camera device 3. In other words, to associate the camera device position with the feature positions, the feature positions in the image are associated with a fixed point position on the vehicle based on the calibration information of the camera device 3.
[0052] After determining the fixed point in the first and second images and the ray R between at least two features, the swing angle of the first and second features is determined. Figure 2 In the diagram, α1 represents the yaw angle of the first feature F1 between the two captured images, and α2 represents the yaw angle of the second feature F2 between the images. Preferably, two or more features of the trailer are determined and tracked across multiple images. Furthermore, it is preferable to capture two or more images at different time points to improve the yaw angle estimation results.
[0053] After determining the swing angles α1 and α2, the yaw angle YA can be calculated based on the swing angles α1 and α2.
[0054] According to the first embodiment, the yaw angle YA can be calculated as the median of the set swing angles α1 and α2. According to another embodiment, the yaw angle YA can be determined by calculating the arithmetic mean of the established swing angles α1 and α2. According to yet another embodiment, the yaw angle YA can be determined using a random method. Thus, for example, the variance of each characteristic angle can be measured, and only features with low variance can be used to calculate the median.
[0055] The yaw angle can be further refined using a Kalman filter or a dynamic model based on vehicle speed and steering information. Speed and steering can be derived, for example, from CAN (Controller Area Network) data or using visual methods with processed image data.
[0056] One advantage of using the median is its high robustness. Even under poor lighting conditions, the median can generate reliable and consistent angle estimates even if only one feature is tracked. The median is also highly robust to outliers that may occur in cases of poor feature tracking or particularly cluttered images.
[0057] It appears that not all features visible in the captured images are suitable for calculating the yaw angle YA. To reduce computational complexity and improve robustness, features that provide a yaw angle very close to the actual yaw angle are selected and further used to determine the yaw angle. For feature selection, only those features providing yaw angles α1 and α2 within a specific window around the actual yaw angle are tracked in future images. This window can be defined, for example, by upper and lower limits, which define the angle window around the actual yaw angle. For example, the window can cover a range of 2 to 10 degrees, particularly a range between 3 and 5 degrees. In the last two or more steps of determining the yaw angle, all features within this window that cause the yaw angle are further tracked in subsequently captured images.
[0058] In addition, if the calculated yaw angle YA is underestimated too much compared to the actual yaw angle YA, the calculated yaw angle YA can be increased by a certain share or percentage to compensate for the underestimation.
[0059] Figure 3 The diagram illustrates the steps of a method for determining the yaw angle YA of trailer 2 relative to the longitudinal axis LAV of tractor 1.
[0060] First, take first and second images of the trailer (S10).
[0061] After the images are captured, the trailer features visible in the first and second images are determined (S11).
[0062] After determining the features, the first and second angle estimates are calculated based on the determined first and second features (S12, S13).
[0063] Finally, the yaw angle is calculated based on the first and second angle estimates (S14).
[0064] It should be noted that the description and accompanying drawings are merely illustrative of the principles of the invention. Those skilled in the art will be able to implement various arrangements that are not explicitly described or shown herein but embody the principles of the invention.
[0065] List of reference numerals
[0066] 1 vehicle
[0067] 2 trailers
[0068] 3. Camera device
[0069] 4. Towing rod
[0070] α1 First Angle Estimation
[0071] α2 Second Angle Estimation
[0072] F1 First Feature
[0073] F2 Second Feature
[0074] LAT trailer longitudinal axis
[0075] LAV vehicle longitudinal axis
[0076] R light rays
[0077] YA Yaw angle
Claims
1. A method for determining the yaw angle of the trailer (2) relative to the longitudinal axis of the tractor (1), wherein, The method includes the following steps: - Using a camera device (3) to detect at least first and second images of the trailer (2), wherein the orientation of the trailer (2) relative to the vehicle (1) is different in at least two images (S10); - Determine at least first and second features of the trailer (2) that are visible in the first and second images, wherein the first and second features are located at different positions on the trailer (2) (S11); - Calculate a first angle estimate, wherein the first angle estimate characterizes the swing angle (S12) between a first feature on a first image and a first feature on a second image on a horizontal plane relative to a fixed point of the tractor (1); - Calculate a second angle estimate, wherein the second angle estimate characterizes the swing angle (S13) between the second feature on the first image and the second feature on the second image relative to the fixed point of the tractor (1) on the horizontal plane; - Calculate the yaw angle based on the first and second angle estimates (S14).
2. The method according to claim 1, wherein, In the first or second image, the yaw angle of the trailer (2) relative to the vehicle (1) is zero, or any known yaw angle that can be used as a reference angle.
3. The method according to claim 1 or 2, wherein, The fixed point is the position of the camera device (3) or the position of the traction rod (4).
4. The method according to claim 1 or 2, wherein, The calculation of the first and second angle estimates includes determining the light rays between the fixed point and the first and second features on the first and second images.
5. The method according to claim 4, wherein, Camera calibration information is used to convert the position of the first and / or second feature into light.
6. The method according to claim 1 or 2, wherein, In addition to the first and second features, at least one other feature of the trailer (2) is used for the calculation of the yaw angle.
7. The method according to claim 1 or 2, wherein, The yaw angle is calculated by using the median value constructed based on at least two angle estimates.
8. The method according to claim 1 or 2, wherein, The yaw angle is calculated by averaging at least two angle estimates or by using a statistical method applied to the angle estimates.
9. The method according to claim 1 or 2, further comprising the step of determining an angle window, wherein, The angle window includes an upper and lower limit around the yaw angle, a set of features is determined that leads to the angle estimation within the angle window, and the determined set of features is used for future yaw angle calculations.
10. The method according to claim 1 or 2, wherein, The calculated yaw angle value is increased by a specific share or percentage to compensate for the underestimation.
11. The method according to claim 1 or 2, wherein, The camera device (3) is a rear-view camera device of the vehicle (1).
12. A system for determining the yaw angle of the trailer (2) relative to the longitudinal axis of the tractor (1), wherein, The system includes a camera device (3) for capturing images of the trailer (2) and a processing entity for processing the captured images. Furthermore, the system is configured to perform the following steps: - Use a camera device (3) to capture at least first and second images of the trailer (2), wherein the orientation of the trailer (2) relative to the vehicle (1) is different in at least two images; - Determine at least first and second features of the trailer (2) visible in the first and second images, wherein the first and second features are located at different positions on the trailer (2); - Calculate a first angle estimate, wherein the first angle estimate characterizes the swing angle between a first feature on a first image on a horizontal plane and a first feature on a second image relative to a fixed point of the tractor (1); - Calculate a second angle estimate, wherein the second angle estimate characterizes the swing angle between a second feature on the first image and a second feature on the second image relative to the fixed point of the tractor (1) on the horizontal plane; - Calculate the yaw angle based on the first and second angle estimates.
13. A vehicle, wherein, The vehicle includes the system according to claim 12.
Citation Information
Patent Citations
Apparatus for determining an angle of a trailer attached to a vehicle
EP3552926A1