Method and system for calculating a vehicle trailer angle

Abstract

The invention relates to a method for determining a yaw angle (YA) of a trailer (2) relative to a longitudinal axis (LAV) of a towing vehicle (1) on the basis of at least one feature (F1, F2) contained in a plurality of captured images, using correction information.

Description

Technical Field

[0001] This invention generally relates to the field of vehicle assistance systems. More specifically, this invention relates to a method and system for calculating the yaw angle of a trailer connected to a tractor based on image information provided by a vehicle camera device. Background Technology

[0002] Methods for calculating the angle of the trailer relative to the tractor based on image information provided by the vehicle's camera device are known.

[0003] Specifically, known methods provide reliable approximations of trailer yaw angles without considering the trailer pivot point position. However, the accuracy of these yaw angle approximations is particularly low at large yaw angles. Summary of the Invention

[0004] The objective of this invention is to provide a method for calculating trailer yaw angle with high robustness and accuracy. This task is achieved through the features of the independent claims. Preferred embodiments are given in the dependent claims. Unless otherwise expressly stated, embodiments of this invention can be freely combined with each other.

[0005] According to one aspect, the present invention relates to a method for determining the yaw angle of a trailer relative to the longitudinal axis of a tractor. The tractor includes a tow bar. The trailer can be connected to a vehicle based on the tow bar. The method includes the following steps:

[0006] First, at least first and second images of the trailer are captured using a camera device. The method for capturing the first and second images is such that the orientation of the trailer relative to the vehicle is different in at least the two images.

[0007] After taking the images, at least a first feature of the trailer is determined. The first feature must be visible in both the first and second images.

[0008] A first angle estimate is calculated based on the determined first feature. This first angle estimate characterizes the angle between the first feature in the first image and the first feature in the second image, relative to the position of the tractor-trailer camera device on a horizontal plane. In other words, the first angle estimate refers to the angle constrained between a first line extending between the position of the first feature in the first image and the position of the camera device, and the second line extending between the position of the first feature in the second image and the position of the camera device. The angle opens in the direction from the vehicle toward the trailer.

[0009] In the following text, the method of using at least one angle estimation based on at least one feature detected on at least two images is referred to as the "basic algorithm".

[0010] It is worth noting that the term "position of the first feature on the first / second image" refers to the two-dimensional planar image coordinates of the image feature, or the corresponding ray (e.g., given as a three-dimensional solid unit vector or azimuth / elevation angle). The two-dimensional planar image coordinates can be converted into rays using camera calibration information.

[0011] In addition, one or more angle coefficients are provided. These angle coefficients provide scaling information for the angle estimate. In other words, the angle estimate can be modified based on these angle coefficients to reduce inaccuracies.

[0012] Finally, the trailer's yaw angle is calculated based on the first angle estimate and one or more angle coefficients.

[0013] The method described is advantageous because, by using two or more images and calculating the yaw angle using at least one trailer feature, the determined yaw angle is extremely reliable and robust, even if the trailer feature detection is affected by high noise or poor image quality. The angle estimation can be scaled using one or more angle coefficients to improve accuracy.

[0014] According to one embodiment, at least first and second features of the trailer are determined. The first feature is located at a different trailer position than the second feature. For example, the first feature may be a prominent first feature at a first trailer position, and the second feature may be a prominent second feature at a second trailer position. Furthermore, at least a second angle estimate is calculated. The second angle estimate characterizes the angle between the second feature on a first image and the second feature on a second image relative to the position of the tractor-mounted camera on a horizontal plane. In other words, the second angle relates to the angle between a first line constrained between the second feature position on the first image and the camera position, and a second line between the second feature position on the second image and the camera position. The angle opens from the direction of the vehicle toward the trailer.

[0015] Higher reliability can be achieved by using two or more trailer features because the effects of noise and mismatch can be reduced.

[0016] According to one embodiment, one or more angle coefficients are established based on geometric information. The geometric information may take into account the distance between the camera device and the tow bar, as well as the distance between the camera device and one or more trailer features. Therefore, the lack of accuracy caused by unknown geometric relationships between vehicle features (especially the local relationship between the camera device and the tow bar) and unknown geometric relationships between the vehicle and the trailer can be mitigated.

[0017] According to one embodiment, the angle coefficient is the distance between the camera device and the tow bar divided by the distance between the camera device and the trailer feature. The farther the feature is from the camera device, the closer the coefficient is to zero.

[0018] According to one embodiment, the one or more angle coefficients are calculated by dividing the distance between the camera device and the tow bar by the distance between the camera device and one or more trailer features. Calculating the quotient of these distances further improves the accuracy of determining the yaw angle.

[0019] According to one embodiment, the angle coefficient is a single coefficient customized for the corresponding vehicle. The angle coefficient can be preset for specific vehicle characteristics (especially the local relationship between the camera and the tow bar) and used without considering the specific trailer. In other words, the single coefficient may be a coefficient "suitable for all trailers." Therefore, high accuracy can be achieved with limited computational workload. The single coefficient can be used for each angle estimate, or for setting the median or average based on at least two angle estimates. In other words, either at least one angle estimate is scaled directly based on the angle coefficient, or the average or median of multiple angle estimates is calculated and scaled using the single angle coefficient.

[0020] According to one embodiment, the angle coefficient is dynamically adjusted and adapted for a specific trailer. In other words, the angle coefficient is not set as a coefficient "suitable for all trailers," but is dynamically adjusted and adapted according to the trailer currently in use. This approach can further improve the accuracy of determining the yaw angle.

[0021] According to one embodiment, the dynamic adjustment and adaptation of the angle coefficients is implemented by using a reference algorithm to determine yaw angle reference information and by adjusting the adaptation angle coefficients by comparing the two or more angle estimates with the yaw angle reference information provided by the reference algorithm. At least in certain cases, the reference algorithm can provide more accurate yaw angle reference information than at least one angle estimate provided by the basic algorithm, and is used to dynamically set the one or more angle coefficients. In other words, the result of the reference algorithm is used to scale the angle estimates provided by the basic algorithm.

[0022] According to one embodiment, the reference algorithm is configured to perform the following steps:

[0023] A ray between the camera device and the first feature determined on the first image is drawn and projected onto a horizontal plane to obtain the position of the first projected feature. Similarly, a ray between the camera device and the first feature determined on the second image is drawn and projected onto the horizontal plane to obtain the position of the second projected feature.

[0024] - Based on the established first feature, a first vertical bisector is established between the first projection feature position and the second projection feature position. More specifically, the first vertical bisector may be a vertical line passing through the center of the line connecting the first projection feature position and the second projection feature position.

[0025] -After setting the first vertical bisector, set the first intersection point of the first vertical bisector with the reference axis or another vertical bisector.

[0026] Finally, yaw angle reference information is provided based on the first reference angle estimate, wherein the first reference angle estimate refers to the angle between a first line from the first projection feature position to the first intersection point and a second line from the second projection feature position to the first intersection point on the horizontal plane.

[0027] It is worth noting that the term "first feature position on the first image" refers to the two-dimensional planar image coordinates of the image feature, or the corresponding ray (e.g., given as a three-dimensional solid unit vector or azimuth / elevation angle). A two-dimensional planar image coordinate can be converted into rays using camera calibration information.

[0028] The first reference angle estimate can be made from the direction of the tractor toward the trailer.

[0029] According to one embodiment, based on the aforementioned reference algorithm, multiple yaw angle reference information points are developed for different yaw angles, and an angle coefficient is set by averaging the multiple yaw angle reference information points. Other statistical methods can also be used, for example, variance can be measured to remove outliers. This reduces the noise impact on the reference algorithm.

[0030] According to one embodiment, in the first or second image, the yaw angle of the trailer relative to the vehicle is zero. Therefore, this image can be used as a "zero-attitude image," i.e., as a reference for accurate alignment of the vehicle's longitudinal axis with the trailer's longitudinal axis. However, if the other yaw angle is known, another yaw angle value can also be used as a reference.

[0031] According to one embodiment, in the basic algorithm, calculating the first angle estimate includes setting a ray between the camera device position and the first feature in the first and second images. The ray refers to a straight line between the camera device position and the first feature. Based on this ray, the computational workload for determining the current swing angle can be reduced, for example, based on geometric methods.

[0032] According to one embodiment, particularly in the basic algorithm, camera device calibration information is used to convert the position of the first feature into light. For example, when the camera device position is known using camera device calibration information, the position of a specific feature on the image can be transmitted in position information based on or associated with the camera device position.

[0033] According to one embodiment, camera device calibration information is used to transform the position of the first feature from a local domain of the image to a local domain of the vehicle or the position of the vehicle's camera device. If, for example, the position of the camera device is known using the camera device calibration information, the position of a specific feature on the image can be transmitted to the position information based on or associated with the position of a camera device included in or fixed to the vehicle.

[0034] According to one embodiment, in addition to the first feature, at least one other trailer feature is used in the calculation of the yaw angle. Using two or more features further improves the robustness and reliability of determining the yaw angle.

[0035] According to one implementation, the yaw angle is calculated by establishing the median of at least two angle estimates. This results in a very stable yaw angle determination.

[0036] According to other embodiments, the yaw angle is calculated by measuring the average of at least two angle estimates or by using statistical methods applied to the angle estimates.

[0037] According to one embodiment, to provide yaw angle reference information, at least two reference angle estimates are established based on at least two different trailer characteristics, and the yaw angle reference information is calculated by establishing the median of the at least two reference angle estimates. This allows for very stable determination of the yaw angle reference information.

[0038] According to other implementations, yaw angle reference information is calculated by establishing the average of at least two reference angle estimates or by using statistical methods applied to the angle estimates.

[0039] According to one embodiment, the method further includes the step of determining an angle window. The angle window may include an upper and lower limit around the yaw angle. Furthermore, a set of features is determined, wherein the features in the set of features result in an angle estimate that falls within the angle window. Preferably, only the features included in the determined set of features are used for future yaw angle calculations. In other words, previously determined yaw angle information is used to determine two or more trailer features that result in an angle estimate that is quite close to the determined yaw angle (i.e., within the angle window), and those features that cause the angle estimate to deviate significantly from the determined yaw angle (i.e., beyond the angle window) are not tracked. The computational complexity and accuracy requirements for angle estimation can also be significantly reduced in this manner.

[0040] According to one embodiment, in the reference algorithm, if the camera device and the vehicle's tow bar are arranged in a vertically oriented plane including the longitudinal axis of the tractor, then the reference axis is the longitudinal axis of the tractor.

[0041] According to another embodiment, if the camera device and / or the tow bar has a lateral offset relative to the longitudinal axis of the tractor, the reference axis in the reference algorithm is a straight line between the camera device and the tow bar. This allows for compensation for the lateral offset between the camera device and the tow bar.

[0042] According to one embodiment, the camera device is a rear-view camera device for a vehicle. Based on the rear-view camera device, images of the trailer can be captured with less technical effort.

[0043] According to another implementation, the angle coefficients can be calculated independently for each feature based on both a reference algorithm and a base algorithm for that feature. These coefficients are independent of the trailer angle; therefore, even if the reference algorithm is currently inaccurate or unavailable (due to mathematical instability, such as the error of dividing by zero at low yaw angles), the angle coefficients can still be used to scale the base algorithm, even if the angle coefficients were calculated at a different trailer angle.

[0044] Each image feature is then scaled against the base angle estimate using an angle coefficient to produce an accurate angle estimate. The estimated trailer yaw angle can be output from the yaw angle estimate of a single feature using the mean, median, or other statistical measures.

[0045] According to another aspect, a system for determining the yaw angle of a trailer relative to the longitudinal axis of a tractor is disclosed. The system includes a camera device for capturing images of the trailer and a processing entity for processing the captured images. Furthermore, the system is configured to perform the following steps:

[0046] - Use a camera device to detect at least one first and second images of the trailer, wherein the orientation of the trailer relative to the vehicle is different in at least two images.

[0047] - Identify at least one first feature of the trailer that is visible in the first and second images;

[0048] - Calculate a first angle estimate, wherein the first angle estimate characterizes the sway angle between a first feature on a first image and a first feature on a second image on a horizontal plane relative to the position of the tractor camera device;

[0049] - Provide one or more angle coefficients, wherein the one or more angle coefficients provide scaling information for the first angle estimate; and

[0050] - The yaw angle is calculated based on the first angle estimate and one or more angle coefficients.

[0051] Any of the features described as embodiments of the method described above may also be used as system features in the system described in this patent application published herein.

[0052] According to another embodiment, a vehicle including the system described in any of the above embodiments is disclosed.

[0053] The term "vehicle" as used in this invention may refer to automobiles, trucks, buses, rail vehicles or any other means of transportation.

[0054] The term "yaw angle" as used in this patent application published herein may refer to the yaw angle between the longitudinal axis of the vehicle and the longitudinal axis of the trailer.

[0055] The term “median” as used in this patent application published herein may refer to the value that separates the higher half of a data sample or probability distribution from the lower half.

[0056] The terms “substantially” or “approximately” as used in this invention refer to deviations from the exact value by + / -10%, preferably + / -5%, and / or variations that are not significant to function and / or to traffic rules. Attached Figure Description

[0057] Different aspects of the invention, including its particular features and advantages, will be more readily understood from the following detailed description and accompanying drawings, wherein:

[0058] Figure 1 An exemplary top view of a vehicle towing a trailer is shown;

[0059] Figure 2 A schematic diagram showing angle estimation of first and second features detected at different swing angles between a trailer and a tractor unit based on images from a camera device;

[0060] Figure 3 A schematic diagram is shown of a reference angle estimation based on a first feature detected at different swing angles between a trailer and a tractor unit using images from a camera device, according to a reference algorithm.

[0061] Figure 4 A schematic diagram illustrating reference angle estimation of first and second features detected at different sway angles between the trailer and tractor based on images from a camera device using a reference algorithm; and

[0062] Figure 5 A schematic block diagram of the steps involved in determining the yaw angle of the trailer relative to the longitudinal axis of the tractor is shown. Detailed Implementation

[0063] The invention will now be described in more detail with reference to the accompanying drawings, which illustrate exemplary embodiments. The embodiments in the drawings relate to preferred embodiments, and all elements and features described in connection with the embodiments can be used, where possible, in conjunction with any other embodiments and features discussed herein, particularly in relation to any other embodiments further discussed above. However, the invention should not be construed as limited to the embodiments described herein. Throughout the following description, similar reference numerals are used to denote similar elements, parts, items, or features, where applicable.

[0064] The features of the invention disclosed in the description, claims, embodiments and / or drawings can be used individually or in any combination to implement the invention.

[0065] Figure 1 The diagram shows a top view of vehicle 1 towing trailer 2. Vehicle 1 includes a longitudinal axis LAV passing through its center. Similarly, trailer 2 has a longitudinal axis LAT passing through its center. Trailer 2 is connected to vehicle 1 via a trailer hitch assembly including a drawbar 4.

[0066] In certain driving situations, the longitudinal axis LAV of vehicle 1 and the longitudinal axis LAT of trailer 2 may not be parallel or coincident; instead, the two longitudinal axes may form a yaw angle YA. In other words, the yaw angle YA defines the angular deviation of the longitudinal axis LAT of trailer 2 relative to the longitudinal axis LAV of vehicle 1. The yaw angle YA can be measured on a horizontal plane that includes the longitudinal axis LAT of trailer 2 and the longitudinal axis LAV of vehicle 1.

[0067] Understanding the yaw angle YA is also beneficial, for example, in trailer assist systems.

[0068] To determine the yaw angle YA, multiple images of at least a portion of the trailer 2 are captured using a camera device 3. The camera device 3 can be, for example, a rearview camera of the vehicle 1, or it can be used to capture images of the environment surrounding the vehicle 1 while it is reversing. One of the captured images may refer to a known angular setting of the trailer 2 relative to the tractor 1. This image can be used as a reference for calculating the yaw angle YA. In this known angular setting of the trailer 2 relative to the tractor 1, the yaw angle YA can be 0 degrees or any other angular value.

[0069] Figure 2 A schematic diagram illustrating the angular relationship between the first and second features F1 and F2 of trailer 2 is shown, wherein features F1 and F2 are identified at different time points and at different angular positions relative to a fixed point of vehicle 1.

[0070] The camera device 3 can capture two or more images of the trailer 2 at different times with different angular positions relative to the vehicle 1. For example, it can capture a series of images. The series of images may include three or more images, especially five or more images.

[0071] In this example, the second image may show the orientation of trailer 2 relative to the vehicle when the yaw angle YA = 0 degrees. However, according to other embodiments, the yaw angle YA can be any other known reference yaw angle and can be used to determine the current yaw angle.

[0072] Multiple different features can be identified in the images captured by camera device 3. Figure 2 Features F1 and F2 are described, which are identified at different angular positions relative to the vehicle 1, camera device 3, or reference axis. The first feature F1 is represented by a square, and the second feature F2 by a triangle. It is worth noting that two or more features and two or more images can be used for yaw angle estimation. Alternatively, only one feature can be used to estimate the yaw angle.

[0073] Thus, the first and second features above (associated with the solid light rays of the camera device 3 and connecting features F1 and F2) are identified in the first image, while the second and first features below (associated with the dashed light rays of the camera device 3 and connecting features F1 and F2) are identified in the second image at another point in time.

[0074] Features on trailer 2 can be located and matched using feature detection and matching algorithms. For example, Harris Corner Detector, Scale Invariant Feature Transform (SIFT), Speed-Up Robust Features (SURF), Binary Robust Invariant Scalable Keypoints (BRISK), Binary Robust Independent Basic Features (BRIEF), Oriented Fast Rotating Briefing (ORB), or another suitable feature detection and matching algorithm can be used.

[0075] The feature detection and matching algorithm can detect image features that are on or off the trailer. Several different methods can be used to separate trailer features from non-trailer features. For example, when traveling in a straight line, trailer features can be separated from non-trailer features by finding features that remain in the same position over time. Alternatively, the motion of background features can be modeled using known vehicle motion over time. This can be extracted from CAN (Controller Area Network) data regarding speed and steering. Features that do not conform to the basic matrix epipolar constraints can be considered trailer features.

[0076] To determine the angle estimates α1 and α2, a ray R connecting features F1 and F2 to the camera device 3 is used. To associate features F1 and F2 of the captured image with the position of the camera device 3, the feature positions in the image coordinates are converted into the spatial domain of the camera device using the calibration information of the camera device 3, thereby providing a ray R that associates the positions of each corresponding feature F1, F2 with the camera device position. In other words, to associate the camera device position with the feature positions, the feature positions on the image are converted into the local domain of vehicle 1 or the local domain of the camera device of vehicle 1 based on the calibration information of the camera device 3, respectively.

[0077] After determining the ray R between the position of the camera device and one or more features in the first and second images, the swing angle of the first feature F1 and the second feature F2 is determined. Figure 2 In the diagram, α1 represents the angle estimate of the yaw angle of the first feature F1 between the two captured images, and α2 represents the angle estimate of the yaw angle of the second feature F2 between the images. Depending on the implementation, only one or more trailer features are determined, and tracking is performed on multiple images. Furthermore, it is preferable to capture two or more images at different time points to improve the yaw angle estimation results.

[0078] As described above, one of the captured images can provide a reference image, wherein the angular position of the trailer 2 relative to the vehicle 1 is known. In this known angular setting of the trailer 2 relative to the tractor 1, the yaw angle YA can be 0 degrees or any other angular value. Therefore, the yaw angle YA can be calculated based on the at least one angle estimate α1, α2. Referring again... Figure 2 For example, the angle setting of the ray R represented by the dashed line may be known because the trailer 2 has a known reference direction relative to the vehicle 1 when taking the image with reference to the ray R.

[0079] The method described above exhibits excellent robustness, meaning it can provide angle estimates even under conditions of poor image quality and low angle estimation accuracy. It appears that in most cases, at least one angle estimate, α1 or α2, is lower than the actual yaw angle YA.

[0080] To improve accuracy, the method uses one or more angle coefficients to scale or modify the angle estimate to provide an angle estimate that is very close to the actual yaw angle.

[0081] The angle coefficient can be determined based on geometric information characterizing the geometry between vehicle 1 and trailer 2. More specifically, the angle coefficient can be calculated based on a set of distance information, wherein the set of distance information includes the distance between the camera device 3 of vehicle 1 and the tow bar 4, and the distance between the camera device 3 of vehicle 1 and at least one feature F1, F2 of trailer 2. The angle coefficient can be particularly determined by dividing the first distance between the camera device 3 and the tow bar 4 and the second distance between the camera device 3 of vehicle 1 and the specific features F1, F2 of trailer 2 (i.e.,...). )Calculate it.

[0082] There are several different possibilities for estimating the scaling angle using one or more angle coefficients:

[0083] First, a single angle coefficient can be used for multiple features of trailer 2. The single angle coefficient can be predetermined for each corresponding vehicle 1. The angle coefficient can take into account the distance of camera device 3 relative to tow bar 4. The single angle coefficient can be stored in the storage device of vehicle 1, or it can be implemented in the software of vehicle 1.

[0084] After calculating two or more angle estimates α1 and α2, an average value can be calculated based on the angle estimates α1 and α2. The average value can be, for example, the median of the unfolded angle estimates α1 and α2. According to another embodiment, the yaw angle YA can be determined by calculating the arithmetic mean of the unfolded angle estimates α1 and α2. According to another embodiment, the yaw angle YA can be determined by using a random method based on the angle estimates α1 and α2, such as the RANSAC algorithm (RANSAC: random sample consensus) or the least squares method.

[0085] A second possibility for providing one or more angle coefficients is to dynamically set one or more angle coefficients based on a reference algorithm. More specifically, one angle coefficient can be set for a specific trailer 2, or multiple angle coefficients can be set, where each angle coefficient is associated with specific features F1 and F2 of trailer 2. In other words, angle coefficients can be dynamically set on a per-trailer basis or on a per-feature basis.

[0086] Using the angle estimation α1 and α2 described above, the angle coefficients are dynamically set. That is, for specific features, the swing angle between features F1 and F2 on the first image and the same features F1 and F2 on the second image is determined.

[0087] To determine the angle factor by which the average or median of a single angle estimate α1, α2, or multiple angle estimates α1, α2 are scaled along the actual yaw angle YA direction, the average or median of the single angle estimate α1, α2, or multiple angle estimates α1, α2 is compared with the result of a reference algorithm. This reference algorithm provides yaw angle reference information with higher accuracy than the angle estimation of the basic algorithm. The angle factor is selected such that the angle estimate is equal to or constitutes the majority of the yaw angle reference information. Specifically, the reference method can be configured to take into account the position of the tow bar 4 around which the trailer 2 swings.

[0088] The following text is based on Figure 3 An exemplary reference method is disclosed. (Compared to...) Figure 2 resemblance, Figure 3 It also shows a schematic diagram illustrating the relationship between the first and second features F1 and F2 of the trailer 2 at different time points with different yaw angles YA relative to the tractor 1.

[0089] It is worth mentioning that the reference algorithm uses the same image as the one used to calculate the angle estimate to ensure that the angle coefficient is set based on the same trailer position or trailer yaw angle.

[0090] Figure 3 The reference yaw angle is determined based on the aforementioned reference algorithm in more detail.

[0091] and Figure 2 resemblance, Figure 3 Similarly, in the first image, the first projection feature position PFP1a, which is set based on the first feature F1 (associated with the solid line ray connecting the first projection feature position PFP1a to the camera device 3), is determined, and at another different time point, in the second image, the second projection feature position PFP1b, which is set based on the second feature F2 (associated with the solid line ray connecting the second projection feature position PFP1b to the camera device 3), is determined.

[0092] Figure 3 The method for determining the reference yaw angle based on the aforementioned reference algorithm is described in more detail. The positional changes of the first and second projection feature positions PFP1a and PFP1b between the first and second images are used to determine at least one reference angle estimate β1.

[0093] After feature recognition of each corresponding image, the first feature F1 of the first and second images is projected onto a common horizontal plane. More specifically, the ray between the imaging device 3 and the first feature F1 determined on the first image is projected onto the horizontal plane, thereby obtaining the first projected feature position PFP1a. Furthermore, the ray between the imaging device 3 and the first feature F1 determined on the second image is projected onto the same horizontal plane, thereby obtaining the second projected feature position PFP1b. It is worth noting that the projection is performed in the vertical direction, thus only changing the elevation angle of the light rays, without changing the azimuth angle.

[0094] After determining the first and second projection feature positions PFP1a and PFP1b, a first vertical bisector B1 is set based on the first and second projection feature positions PFP1a and PFP1b. For example... Figure 3 As shown, the first vertical bisector B1 is a line perpendicular to the line connecting the first and second projected feature positions PFP1a and PFP1b. Furthermore, the first vertical bisector B1 passes through the center of the connecting line. The first vertical bisector B1 intersects a reference axis, which in this embodiment is the vehicle's longitudinal axis LAV. The intersection point of the first vertical bisector B1 and the reference axis (marked by IP1) provides the point of rotation around which the trailer rotates. More specifically, the intersection point provides the position of the tow bar 4.

[0095] The first reference angle estimate β1 is calculated based on the first vertical bisector B1. The first reference angle estimate β1 refers to the angle formed between the first line L1 connecting the first projected feature position PFP1a and the intersection of the first vertical bisector B1 and the reference axis, and the second line L2 connecting the second projected feature position PFP1b and the intersection of the first vertical bisector B1 and the reference axis. The intersection point indicates the position of the tow bar 4. More specifically, the first reference angle estimate β1 characterizes the angle of swing around the first intersection point IP1 (i.e., the position of the tow bar 4) between the projected position of the first feature F1 in the first image on the horizontal plane and the projected position of the first feature F1 in the second image on the horizontal plane.

[0096] The first reference angle estimate β1 represents the yaw angle YA of trailer 2 around its actual rotation point.

[0097] Figure 4 What is being shown is with Figure 3 Similarly, an implementation method uses first and second features F1 and F2 of trailer 2, captured at different time points (at which time trailer 2 has a different yaw angle YA relative to tractor 1), to determine yaw angle reference information. The feature settings are similar to... Figure 2 The descriptions are similar to those in the text.

[0098] Multiple different features can be identified in the images captured by camera device 3. For example... Figure 4 As shown, the features are identified at different angular positions relative to the camera device 3 on vehicle 1. The first feature is represented by a square, and the second feature is represented by a triangle.

[0099] Figure 4 In the first image, the upper pair of first and second features (represented by PFP1a and PFP2a, and associated with the solid line light of the connecting feature and the camera device 3) are identified, while the lower pair of first and second features F1 and F2 (represented by PFP1b and PFP2b, and associated with the dashed line light of the connecting feature and the camera device 3) are identified at different time points in the second image.

[0100] The method of determining yaw angle reference information and Figure 3 The implementation method shown is similar. The main difference lies in setting two reference angle estimates β1 and β2, and developing yaw angle reference information for the trailer based on these two estimates. More specifically, as described above... Figure 3 The first vertical bisector B1 is set and the first reference angle estimate β1 is obtained.

[0101] Furthermore, by setting a third projection feature position PFP2a and a fourth projection feature position PFP2b, a second vertical bisector B2 is set to obtain a second intersection point IP2. The third projection feature position PFP2a and the fourth projection feature position PFP2b are then connected to the second intersection point IP2 to obtain a second reference angle estimate β2. The third projection feature position PFP2a is obtained by projecting the second feature F2 from the first image onto the horizontal plane, and the fourth projection feature position PFP2b is obtained by projecting the second feature F2 from the second image onto the horizontal plane. The second intersection point IP2 can be the intersection of the second vertical bisector B2 and the reference axis of the vehicle's longitudinal axis LAV in this embodiment. The second reference angle estimate β2 is the angle between the first line connecting the third projection feature position PFP2a and the intersection point IP2 and the second line connecting the fourth projection feature position PFP2b and the intersection point IP2.

[0102] In this embodiment, the reference axis is the longitudinal axis LAV of the tractor 1, because the camera device 3 and the tow bar 4 are located on the longitudinal axis LAV of the vehicle 1. In other embodiments, if the camera device 3 or the tow bar 4 has a lateral offset relative to the longitudinal axis LAV of the vehicle 1, or if the lateral offsets of the camera device 3 and the tow bar 4 relative to the longitudinal axis LAV of the vehicle 1 are different, the reference axis may be formed by a straight line connecting the camera device 3 and the tow bar 4.

[0103] Ideally, the first reference angle estimate β1 and the second reference angle estimate β2 should be equal (β1 = β2) and should represent the yaw angle YA. However, due to noise and mismatch, the first and second reference angle estimates β1 and β2 may differ.

[0104] It is worth mentioning that two or more features of trailer 2 can be identified and tracked on multiple images. Furthermore, it is preferable to take two or more images at different time points to improve the results of yaw angle reference information estimation. Therefore, two or more reference angle estimates β1 and β2 can be set to improve the quality of determining the yaw angle reference information.

[0105] Statistical measures can be used to determine yaw angle reference information based on first and second reference angle estimates β1 and β2, which have different values. According to a first embodiment, to determine yaw angle reference information, the median of two or more reference angle estimates β1 and β2 can be used. According to other embodiments, statistical methods can be used to determine yaw angle reference information based on two or more reference angle estimates β1 and β2. The statistical method can be, for example, the RANSAC algorithm (RANSAC: random sample consensus) or the least squares algorithm.

[0106] It is worth noting that the above reference algorithm is merely an example of a method that can be used to scale / adjust the adaptation angle coefficient. However, other reference algorithms can also be used to scale / adjust the adaptation angle coefficient.

[0107] Based on the at least one reference angle estimate β1, β2, yaw angle reference information can be developed. On this basis, the method for selecting the angle coefficient value can be to adjust the angle estimates α1, α2 toward the yaw angle reference information.

[0108] To eliminate the inherent noise that may exist in the above-mentioned reference method, multiple reference angle coefficients can be developed for different yaw angles of the trailer 2. These reference angle coefficients can be values ​​independent of the yaw angle. Therefore, by averaging multiple angle coefficients, for example by shifting or by taking an exponential average, the noise effect can be mitigated.

[0109] As a third example, multiple angle coefficients can be set, where each angle coefficient corresponds to a specific feature F1, F2 of trailer 2. In other words, not only can a single angle coefficient be used for all trailer features F1, F2, but a first angle coefficient can be used for the first feature F1 and a second angle coefficient can be used for the second feature F2.

[0110] The multiple angle coefficients can be dynamically determined using the reference algorithm described above.

[0111] According to an example, the following formula can determine the yaw angle YA based on the angle estimates α1, α2 and the angle coefficient.

[0112] YA = Angle estimation + sin -1 (Angle coefficient * sin(angle estimate))

[0113] By using angle estimation α1 and reference angle estimation β1, and using the yaw angle reference information obtained based on reference angle estimation β1, the above equations can be rearranged to determine the angle coefficients.

[0114] If the reference angle estimate β1 and the reference information for each yaw angle are unavailable, for example due to noisy feature matching, the angle estimate α1, which is robust to relatively noisy feature matching, can be used, and an accurate angle estimate can be established using previously calculated angle coefficients.

[0115] It is worth noting that the above formula is only an example, and the content disclosed in this patent document is not limited to using the formula described herein. Therefore, other formulas can also be used to calculate the yaw angle YA based on the angle estimates α1, α2, and the angle coefficient.

[0116] Ideally, when establishing multiple angle estimates α1 and α2, the first angle estimate α1 should be equal to the second angle estimate α2 (α1 = α2). However, due to noise and mismatch, the values ​​of the first and second angle estimates α1 and α2 may differ. It is worth noting that to improve the quality of yaw angle determination, more than two angle estimates can be established.

[0117] To determine the yaw angle YA based on first and second angle estimates α1 and α2 with different values, statistical measures can be used. According to a first embodiment, to determine the yaw angle YA, the median of two or more angle estimates α1 and α2 can be used. According to other embodiments, statistical methods can be used to determine the yaw angle YA based on two or more angle estimates α1 and α2. The statistical method can be, for example, the RANSAC algorithm (RANSAC: random sample consensus) or the least squares method.

[0118] It appears that not all features visible in the captured images are suitable for calculating the yaw angle YA. To reduce computational complexity and improve robustness, features whose provided yaw angles are very close to the actual yaw angle are selected and further used to determine the yaw angle YA. To select features, only those features providing yaw angles α1 and α2 within a specific window around the actual yaw angle are tracked in subsequent images. This window can be defined, for example, by an upper and lower bound, which define the angle window around the actual yaw angle. For example, the window can cover a range of 2 to 10 degrees, particularly 3 to 5 degrees. In the last two or more steps of determining the yaw angle, all features within this window that cause the yaw angle are further tracked in subsequently captured images.

[0119] If trailer-specific features need to be tracked across multiple images due to the movement of trailer 2, samples of these features can be set on an arc-shaped section. The center of the arc-shaped section indicates the position of the tow bar 4. Thus, by tracking specific trailer features across multiple images, the position of the tow bar 4 can be determined.

[0120] To reduce noise, determining the position of the tow bar 4 can be achieved by tracking multiple trailer features across multiple images over a period of time. Each trailer feature can correspond to an arcuate portion with a specific center estimate. The actual position of the tow bar 4 can be determined by applying statistical methods to these multiple center estimates. These statistical methods can be, for example, the RANSAC (Random Sample Consensus) algorithm or the least squares method.

[0121] Figure 5 The diagram illustrates the steps of a method for determining the yaw angle YA of trailer 2 relative to the longitudinal axis LAV of tractor 1.

[0122] First, take first and second images of the trailer (S10).

[0123] After the images are captured, at least one visible trailer feature is determined in the first and second images (S11).

[0124] After determining the at least one feature, at least a first angle estimate is calculated (S12).

[0125] In addition, one or more angle coefficients (S13) may be provided. The one or more angle coefficients provide scaling information for the at least one angle estimate.

[0126] Finally, the yaw angle is calculated based on the first angle estimate and based on one or more angle coefficients, which provide correction information for the angle estimate (S14).

[0127] It should be noted that the description and accompanying drawings are merely illustrative of the principles of the invention. Those skilled in the art will be able to implement various arrangements that are not explicitly described or shown herein but embody the principles of the invention.

[0128] List of reference numerals

[0129] 1 vehicle

[0130] 2 trailers

[0131] 3. Camera device

[0132] 4. Towing rod

[0133] α1 First Angle Estimation

[0134] α2 Second Angle Estimation

[0135] β1 First reference angle estimation

[0136] β2 Second Reference Angle Estimation

[0137] B1 First perpendicular bisector

[0138] B2 Second perpendicular bisector

[0139] The first feature projection position in the first image of PFP1a

[0140] The first feature projection location in the second image of PFP1b

[0141] Second feature projection location in the first image of PFP2a

[0142] The second feature projection location in the second image of PFP2b

[0143] F1 First Feature

[0144] F2 Second Feature

[0145] IP1 First intersection point

[0146] IP2 Second Intersection

[0147] LAT trailer longitudinal axis

[0148] LAV vehicle longitudinal axis

[0149] R light rays

[0150] YA Yaw angle

Claims

1. A method for determining the yaw angle of the trailer (2) relative to the longitudinal axis of the tractor (1) with the tow bar (4), wherein, The method includes the following steps: Using a camera device (3) to detect at least one first and second images of the trailer (2), wherein the orientation of the trailer (2) relative to the tractor (1) is different in at least two images; Identify at least one first feature of the trailer (2) that is visible in the first and second images; Calculate a first angle estimate, which represents the positional angle between a first feature on a first image and a first feature on a second image relative to the camera device (3) of the tractor (1) on the horizontal plane; Provide one or more angle coefficients, wherein the one or more angle coefficients provide scaling information for the first angle estimate; and The yaw angle is calculated based on the first angle estimate and based on one or more angle coefficients. The one or more angle coefficients are determined by dividing the distance between the camera device (3) and the tow bar (4) by the distance between at least one feature of the camera device (3) and the trailer (2).

2. The method according to claim 1, wherein, The angle coefficient is a single coefficient tailored to each corresponding vehicle.

3. The method according to claim 2, wherein, The single coefficient is applied to the first angle estimate or to the median or average value set based on at least two angle estimates.

4. The method according to any one of claims 1 to 3, wherein, The angle coefficient is dynamically adjusted and adapted for the corresponding trailer (2).

5. The method according to claim 4, wherein, The dynamic adjustment and adaptation of the angle coefficient is performed by using a reference algorithm to determine yaw angle reference information and by comparing the first angle estimate with the yaw angle reference information provided by the reference algorithm to adjust the adaptation angle coefficient.

6. The method according to claim 5, wherein, The reference algorithm configuration is used for: The ray between the camera device (3) and the first feature determined on the first image is projected onto the horizontal plane to obtain the position of the first projected feature, and the ray between the camera device (3) and the first feature determined on the second image is projected onto the horizontal plane to obtain the position of the second projected feature. A first vertical bisector is set between the first projection feature position and the second projection feature position; Determine the first intersection point of the first perpendicular bisector with the reference axis or another perpendicular bisector; as well as Yaw angle reference information is provided based on a first reference angle estimate, wherein the first reference angle estimate refers to the angle between a first line from a first projection feature position to a first intersection point and a second line from a second projection feature position to the first intersection point on a horizontal plane.

7. The method according to claim 5 or 6, wherein, Based on the aforementioned reference algorithm, multiple yaw angle reference information for different yaw angles is developed, and the angle coefficient is determined by the average value of the multiple yaw angle reference information.

8. The method according to any one of claims 1 to 3, wherein, For each feature, provide or set different angle coefficients.

9. The method according to any one of claims 1 to 3, wherein, The step of calculating the first angle estimate includes determining the light between the position of the camera device (3) and the at least one feature in the first and second images.

10. The method according to claim 9, wherein, The camera device calibration information is used to convert the position of the at least one feature from the local domain of the image to the local domain of the tractor (1) in order to determine the light.

11. The method according to any one of claims 1 to 3, wherein, In addition to the first feature, one or more other features of the trailer (2) are used to calculate the yaw angle.

12. A system for determining the yaw angle of the trailer (2) relative to the longitudinal axis of the tractor (1), wherein, The system includes a camera device (3) for capturing images of the trailer (2) and a processing entity for processing the captured images. Furthermore, the system is configured to perform the following steps: Using a camera device (3) to detect at least one first and second images of the trailer (2), wherein the orientation of the trailer (2) relative to the tractor (1) is different in at least two images; Identify at least one first feature of the trailer (2) that is visible in the first and second images; Calculate a first angle estimate, which represents the swing angle between the first feature on the first image and the first feature on the second image on the horizontal plane relative to the position of the tractor (1) and the camera device (3); Provide one or more angle coefficients, wherein the one or more angle coefficients provide scaling information for the first angle estimate; and Yaw angle is calculated based on a first angle estimate and one or more angle coefficients. The one or more angle coefficients are determined by dividing the distance between the camera device (3) and the tow bar (4) by the distance between at least one feature of the camera device (3) and the trailer (2).

Images