Method for calculating the position of a trailer hitch point

Abstract

This document describes a method for calculating the position of a trailer hitch point. A first image and a second image of the trailer are captured, and based on the captured first image and second image, the trailer rotation and trailer translation relative to a known trailer zero pose are calculated. Based on the calculated trailer rotation and trailer translation, the trailer rotation axes in the first image and the second image are calculated. The intersection point of the first rotation axis and the second rotation axis is calculated to determine a ray to the trailer hitch point position. The trailer hitch point position is calculated using the ray to the trailer hitch point position and a geometric condition.

Description

Technical Field

[0001] The present invention relates to a method for calculating a vector of a trailer hitch point position of a vehicle, an in-vehicle computing device for calculating a vector of a trailer hitch point position, and a vehicle including such a computing device, a computer program, and a computer-readable medium. Background Art

[0002] Trailer hitch assist systems are used to support a driver in connecting a trailer to a vehicle. Trailer hitch assist systems require information about the direction of the trailer and the position of the trailer hitch point installed on the vehicle. Trailer reverse assist systems are used to support a driver in reversing the vehicle with a trailer connected. A trailer reverse assist system may require the position information of the trailer hitch point relative to the position of a camera device, i.e., a vector from the camera device to the trailer hitch point. Such systems may not need to know the distance of the trailer hitch point along the vector, because such systems only need to calculate the rotation of the trailer, and the rotation is constant relative to the distance.

[0003] Although the position of the trailer hitch point can be manually measured, this is neither accurate nor cost-effective and time-consuming, especially when no hitch connector is installed during vehicle production or the hitch connector can be removed from the vehicle manually or electronically.

[0004] During trailer movement, the vehicle rotates about a point on itself, and this point is the trailer hitch point. It may be connected by a trailer ball or some other means. Summary of the Invention

[0005] It may be necessary to determine the position of the trailer hitch point electronically, or a vector of the hitch point relative to a camera device.

[0006] This task is solved by the subject matter of the appended independent claims. Embodiments are provided by the dependent claims and the following description and drawings.

[0007] According to a first aspect of the present invention, there is provided a method for calculating a vector of a trailer hitch point position of a vehicle, wherein the method includes the following steps

[0008] - Using an image capturing device, capturing a first image of the trailer at a first position and a second image of the trailer at a second position (step S1),

[0009] - Based on the captured first image and the captured second image, calculating the rotation and translation of the trailer between the first pose and the second pose (step S2),

[0010] - Based on the calculated trailer rotation and trailer translation, calculating a first rotation axis (Euler axis) of the trailer between the first image and the second image (step S3),

[0011] - Repeat steps S1 to S3 using a third and a fourth image of the trailer, thereby calculating a second axis of rotation of the trailer between the third image and the fourth image (steps S1* to S3*),

[0012] - Determine an intersection point of the first axis of rotation and the second axis of rotation to determine a vector of the trailer hitch point position (step S4), and

[0013] - Optionally, the trailer hitch point position can be calculated using the trailer hitch point position vector and a geometric condition (step S5).

[0014] Steps S1 to S3 can be iterated at least once or multiple times. When comparing with a third image, the iteration can reuse the first image or use a new pair of images, such as calculating the second axis of rotation using a third image and a fourth image.

[0015] In other words, when calculating the trailer hitch point position, mainly two images of the trailer are compared first, thereby generating a first axis of rotation. The first image can be an image of the trailer in a known direction, for example, in a direction assuming no rotation or no translation relative to the longitudinal axis of the vehicle. The reason for the translation of the trailer is that due to the relative positions of the image capturing device and the trailer hitch point, when the trailer rotates around the hitch point, it seemingly undergoes both a translation and a rotation.

[0016] The images can be captured by an imaging device mounted on the vehicle, where the trailer is within the field of view of the imaging device. In the content disclosed in the present invention, the images are designated as zero pose images. In principle, since rotation and translation are relative, any direction and translation can be defined as zero pose. To calculate an axis of rotation, the second image must show the rotation of the trailer relative to the first image. For example, when the trailer is connected to the vehicle, such images can be captured during driving. This can be more easily understood with reference to Figure 1 which is easier to understand.

[0017] Repeat the process using a third and a fourth image, where the third image and a fourth image show a rotation and a translation relative to the first image. The axis around which the trailer rotates between the first image and the third image and the fourth image should be different from the axis around which the trailer rotates between the first image and the second image. That is to say, if the rotation of the trailer between the first image and the second image only includes a deflection angle, the rotation of the trailer between the first image and the third and fourth images should include an inclination component or a rolling component. A second axis of rotation different from the first axis of rotation can be determined based on the rotation and the translation. The common feature of the two axes of rotation is that they both pass through the trailer hitch point, where the trailer hitch point is a fixed point and is equipped with a towing bar for towing the trailer. Therefore, the two axes of rotation can intersect, and the intersection point is the trailer hitch point.

[0018] Since the above calculations are performed using a single imaging device (monocular imaging device), the distances and ratios are unknown. In other words, the distances between the axis of rotation and the trailer hitch point and the imaging device are unknown. To calculate the intersection point of the axis of rotation, the axis of rotation is projected onto the image plane (a plane at z = 1, where the z-axis extends downward along the optical axis). Regardless of the distance from the imaging device, the axis of rotation will project onto the same line on the image plane. Subsequently, the two-dimensional intersection point of the axis of rotation line on the image plane is calculated, thereby obtaining the vector or ray of the trailer hitch point position.

[0019] Additional geometric conditions are required to calculate the position of the trailer hitch point.

[0020] It should be noted that it may not be necessary to calculate the position of the trailer hitch point. Many trailer reverse assist algorithms only need to calculate the angle of the trailer relative to the longitudinal axis of the vehicle. For this purpose, only the vector of the trailer hitch point position needs to be known, and an arbitrary distance (such as a unit distance) to the trailer hitch point can be assumed.

[0021] Therefore, the present invention relates to using an on-vehicle imaging device on a vehicle to determine the rotation and translation that occur when a trailer is towed by the vehicle, and determining the vector of the trailer hitch point based on these measurement results. In addition, the present invention can use the vector to determine the position of the trailer hitch point without manual measurement. Once the trailer hitch point vector or position is calculated, it can be stored and applied to trailer reverse assist or trailer hitch assist to help the driver hitch their vehicle to a trailer.

[0022] In other words, the present invention calculates a vector of the trailer hitch point by analyzing the movement (rotation and translation) of a trailer during being towed by a vehicle.

[0023] Algorithms for calculating the movement of a trailer when the position of the trailer hitch point is unknown (such as an 8-point fundamental matrix) have a greater error than algorithms when the position of the trailer hitch point is known. However, over time, the present invention will identify the position of the trailer hitch point, and then the system can switch to a more accurate method for calculating the trailer angle, which requires the input of the position or vector of the trailer hitch point.

[0024] Translation is also calculated in addition to rotation, because although the trailer only rotates around the trailer hitch point, due to the offset between the position of the imaging device and the trailer hitch point, the trailer appears to translate relative to the imaging device.

[0025] Generally speaking, "geometric conditions" can be any information that allows the trailer hitch point vector to intersect a line or plane, thereby enabling the calculation of the distance to the trailer hitch point and the position of the trailer hitch point.

[0026] Some embodiments will be elaborated in detail below.

[0027] According to one embodiment, the geometric conditions are embodied as a known external parameter, such as the positions of the imaging devices that capture the first, second, third, and fourth images, and the trailer hitch point is located at the center of the vehicle towing the trailer. The known external imaging device positions provide a reference position, and the distance to the center position of the trailer hitch point on the vehicle can be known. With this information, a vector or ray can, for example, intersect a central plane to obtain the position of the trailer hitch point.

[0028] According to another example, an imaging device installed at a different but known position can be used to determine other axes of rotation. In this case, a second ray can be determined, and since the positions of the imaging device and the intersection point are known, the position of the trailer hitch point can also be determined.

[0029] According to one embodiment, step S2 for calculating the rotation and translation of the trailer further includes comparing the first image of the trailer captured with the second image of the trailer captured (step S6), and identifying the position of the features on the trailer in the first captured image relative to its position in the second captured image (step S7).

[0030] The first image can be a zero pose image, which can be pre-captured or captured during driving and stored electronically. In this case, if the image is captured during driving, the image can be generated from a series of captured images, where, for example, the average value is calculated as the zero pose, and the generated zero pose image is stored in the memory. The zero pose can be stored, for example, as mathematical information or other information, or provided as a stored image, where the trailer is shown in the zero pose.

[0031] The captured images of the trailer can be compared using feature matching on the trailer to determine rotation and translation. These feature matches are created with a feature recognition and matching algorithm, such as Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), or Binary Robust Independent Elementary Features (BRIEF). The features in the trailer image can be distinguished from the features in the background image using a learning algorithm employed during a direct calibration drive. During this drive, the static features are recognized as trailer features. Alternative methods can also be used to determine the features on the trailer, such as finding a predetermined pattern on the trailer and tracking the pattern features during trailer movement. As an alternative, information about the appearance of the trailer can be stored and used to find the features on the trailer.

[0032] In this case, if the first image is the zero pose image captured during the calibration drive, only the features need to be stored, and thus it can be considered that the "zero pose image" only includes trailer features.

[0033] According to one embodiment, step S2 for calculating the trailer rotation and trailer translation further includes projecting the calculated first and second rotation axes onto the image plane (step S8). By projecting the rotation axes onto an image plane, lines that will intersect at a single point in the image plane can be obtained. The image plane is a virtual plane onto which a three-dimensional object is projected using an ideal pinhole camera model. The intersection point of the rotation axes in the image plane will define a vector from the optical center of the camera through the intersection point on the image plane to the estimated position of the trailer hitch point.

[0034] It can also be regarded as two intersecting planes, each plane containing the rotation axis and the camera position. The intersection point of these planes is a line or a vector from the camera center to the trailer hitch point position.

[0035] Using another geometric condition as described above, the trailer hitch point position can be calculated.

[0036] According to one embodiment, step S2 for calculating the trailer rotation and translation further includes using the calculated trailer hitch point vector or position as an input for calculating the angle of a trailer (step S9). Such methods for calculating the trailer angle may be methods known to those skilled in the art. Finally, the necessary information about the trailer direction obtained is provided electronically to a trailer hitch assist system or a trailer reverse assist system.

[0037] According to a second aspect, there is provided a vehicle computing device for calculating a trailer hitch point position vector of a vehicle to which a trailer is connected, wherein the computing device is configured to

[0038] - Based on a first image of the trailer in a first captured pose and a second image of the trailer in a second captured pose, calculate the rotation and translation of the trailer (step S2),

[0039] - Based on the calculated trailer rotation and trailer translation, calculate a first rotation axis of the trailer between the first pose and the second pose (step S3),

[0040] - Repeat steps S2 and S3 using another pair of images of the trailer, thereby calculating a second rotation axis of the trailer (steps S2* and S3*),

[0041] - Determine the intersection point of the first rotation axis and the second rotation axis to determine a ray of the trailer hitch point position (step S4), and

[0042] - Use the ray to the trailer hitch point position and a geometric condition to calculate the trailer hitch point position (step S5).

[0043] The computing device may include one or more processors, a memory, and an interface for connecting a camera device. In addition, the computing device may be equipped with an interface for connecting a trailer hitch assist system or a trailer reverse assist system, particularly for providing information about the trailer hitch point vector or position and the orientation of the trailer relative to the system, or for obtaining requests for measuring such information. The computing device is used to calculate the trailer hitch point vector, position, and orientation according to the above method.

[0044] According to a third aspect, there is provided a vehicle with a camera device for taking images of a trailer and a computing device.

[0045] According to a fourth aspect, there is provided a program unit that, when run on a processor, will instruct the processor to execute the above method.

[0046] According to a fifth aspect, there is provided a computer-readable medium that can store the program unit. Description of the Drawings

[0047] Embodiments:

[0048] Reference to the accompanying drawings and the following description helps to better understand the features, aspects, and advantages of various aspects of the present invention.

[0049] Figure 1 Shown is a diagram of a first image or a zero pose image and a second image used in the method for calculating the trailer hitch point position according to an embodiment of the present invention.

[0050] Figure 2 Shown is a diagram of a rotating shaft according to an embodiment of the present invention.

[0051] Figure 3 Shown is a flowchart of the method according to an embodiment of the present invention. Detailed Description

[0052] Figure 1 Shown is a diagram of a first image 110 and a second image 120 used in the method for calculating the trailer hitch point position according to an embodiment of the present invention. The first image 110 includes a trailer 111 that has Figure 1 front features 112 as shown and a drawbar 113. Similarly, a second image 120 includes the same trailer 121, but it is rotated around the trailer hitch point, and the front features after rotation and translation 122 are as shown Figure 1As shown, and includes a rotating drawbar 123. The image feature 112 on the trailer 111 can be recognized through a learning process. In an exemplary embodiment, the learning process may include detecting and tracking the image feature in the image 110 during straight-ahead driving. After a predetermined time, it is assumed that the trailer 111 is at a zero angle behind the vehicle. Subsequently, the recognized image feature 112 on the trailer can be considered to have a fixed orientation relative to the image 110, that is, these features appear static in the image 110, while other features such as ground features and other objects on the image 110 are dynamic in the image. Finally, the image feature 112 at the zero pose will be stored.

[0053] As described below, in an exemplary embodiment, the trailer angle relative to the first pose or the zero pose can be calculated. On a second image 120, during the movement to the current trailer pose, the trailer image feature 122 recognized in the first pose is tracked. External and internal camera calibrations are used to determine the camera rays of the respective image features 122 of the current pose and the first pose, where the external and internal camera calibration values take into account the fixed geometry of the camera on the vehicle and the internal geometric parameters of the camera optics. Next, the image feature transformation matrix between the first pose and the current pose is calculated using the camera rays. An example of a suitable transformation matrix is the so-called "essential matrix". This can be calculated using various algorithms familiar to those skilled in the art, such as the eight-point algorithm. When more than 8 features are used, the Random Sample Consensus algorithm (RanSaC) can be used to reject outliers. Finally, the rotation and translation values are extracted from the essential matrix or the transformation matrix.

[0054] Figure 2 A vehicle 201 with a camera device 202 and a trailer hitch point 203 is shown in a side view. A trailer is in a first pose 208 and a second pose 207. The pose may include rotations around the trailer hitch point, including yaw angle, pitch angle, and roll angle. The trailer yaw, pitch, and roll rotations between the first pose and the second pose can also be represented using the axis-angle representation described in Euler's rotation theorem. This describes the trailer rotation as a single rotation around a fixed rotation axis or Euler axis, rather than a rotation composed of yaw, pitch, and roll, for example. The rotation axis 205 can be determined, for example, by calculating invariant points 206 in 3D (three-dimensional) space - which can be the output of the above transformation - around which the trailer rotates between the first pose and the current pose. The invariant points 206 lie on the rotation axis 205. The rotation axis intersects at the trailer hitch point 203, where, in the absence of other conditions, the distances between the trailer hitch point 203 and the camera device 202, the camera device and the rotation axis 205, and the feature 209 are all unknown, so the position of the trailer hitch point is also unknown at this time point. When projected onto the image, these axes will appear as lines.

[0055] It is also possible to analyze and evaluate multiple images instead of analyzing and evaluating two images. Thus, over time, more rotation axes with slightly different directions accumulate. For example, if the trailer tilt angle changes, the axis is horizontal. If the trailer deflects, the axis is vertical.

[0056] Define a plane containing the rotation axis and the position of the imaging device, then these axes will intersect at a single point in the image plane. Then, the intersection point of these lines will determine a ray to the position of the trailer hitch point.

[0057] If there is a ray from the imaging device to the position of the trailer hitch point, the exact position of the trailer hitch point can be calculated given other geometric assumptions. For example, if the position of the external imaging device is known and the trailer hitch point is located at y = 0, then the ray of the trailer hitch point can be intersected with the plane y = 0, which is a vertical plane perpendicular to the rear side of the vehicle, for example, located at the center of the rear side.

[0058] As an alternative, an arbitrary scale can be selected. For example, it can be assumed that the distance from the trailer hitch point along the ray to the imaging device is 1 meter.

[0059] Figure 3 It is a flowchart of a method for calculating the position of a trailer hitch point. In a first step S1, a first image and a second image of the trailer are taken. In a second step S2, based on the first image and the second image obtained by shooting, a rotation and a translation of the trailer between the first image and the second image are calculated respectively. In a third step S3, based on the calculated trailer rotation and trailer translation, a first rotation axis of the trailer between the first image and the second image is calculated. Steps S1 to S3 are repeated at least once, but preferably multiple times, using a third image and a fourth image or multiple images of the trailer, thereby calculating a second rotation axis or more rotation axes of the trailer in the third image or more images respectively. In a subsequent step S4, an intersection point of the calculated rotation axes is determined to determine a ray to the position of the trailer hitch point. In the final step S5, the position of the trailer hitch point is calculated using the ray to the position of the trailer hitch point and a geometric condition.

[0060] Steps S1 to S3 can be repeated using any pair of images that capture the trailer image in two different poses, and thus can be used to calculate a rotation axis that intersects the position of the trailer hitch point.

[0061] Therefore, by tracking the movement of the trailer, the position of the trailer hitch point installed on a vehicle can be determined. An image is captured by a camera device also installed on the vehicle to perform the tracking. This allows the trailer hitch point position to be determined electronically without manual measurement. The trailer hitch point position can be stored and provided to a trailer hitch assist system, which may have a more accurate trailer angle calculation algorithm that requires the input of the trailer hitch point position.

[0062] The vector of the trailer hitch point can also be used in other algorithms to determine the trailer angle. Since its ratio is constant, such algorithms do not need to know the position of the trailer hitch point.

[0063] The vector of the trailer hitch point can also be drawn on a display device so that the user can determine whether the trailer hitch point has been correctly identified. When projecting on the drawn image, the system does not need to know the position of the trailer hitch point, only the trailer hitch point vector is sufficient.

[0064] Since the system calculates the trailer hitch point vector or position based on the movement of the trailer, the system can also be used when the captured image does not contain the trailer hitch point.

[0065] The system is also used to calculate the two pivot positions of a trailer, such as a trailer with two axles, where the front axle rotates on a pivot.

Claims

1. A method for calculating the position vector of a trailer hitch point of a vehicle (201), wherein, The vehicle is connected to a trailer (207, 208) via the trailer hitch point (203), and the method includes the following steps: Using an image capturing device (202), capturing a first image of the trailer in a first pose and a second image of the trailer in a second pose, wherein the image capturing device (202) is mounted on the vehicle and the trailer (207, 208) is within the field of view of the image capturing device (202), Calculating the trailer rotation and trailer translation between a first position and a second position relative to the image capturing device, Based on the calculated trailer rotation and trailer translation, calculating a first axis of rotation of the trailer between the first image and the second image, Using the image capturing device (202) to capture a third image of the trailer in a third pose and a fourth image of the trailer in a fourth pose, wherein the third image and the fourth image show another rotation and another translation relative to the first image, and calculating a second axis of rotation of the trailer between the third image and the fourth image, different from the first axis of rotation, based on the another rotation and another translation, Determining an intersection point of the first axis of rotation and the second axis of rotation to determine a vector of the trailer hitch point position, wherein the vector of the trailer hitch point position is defined as the intersection line of a first plane and a second plane, the first plane containing the first axis of rotation and the position of the image capturing device, and the second plane containing the second axis of rotation and the position of the image capturing device.

2. The method according to claim 1, wherein, The method further includes Using the calculated vector of the trailer hitch point position as an input to a method for calculating the trailer angle.

3. The method according to claim 1 or 2, wherein The method further includes Calculating the trailer hitch point position using the vector to the trailer hitch point position and a geometric condition.

4. The method according to claim 3, wherein, The geometric condition is embodied as the known external position of the image capturing device that captured the first image and the second image, and the trailer hitch point is located at the center of a vehicle (201) towing the trailer (207, 208).

5. The method according to claim 1 or 2, wherein The method further includes: Comparing the captured first image of the trailer with the captured second image of the trailer, and Identifying the position of the trailer feature in the captured first image relative to its position in the captured second image.

6. The method according to claim 1 or 2, wherein The method further includes Projecting the calculated first axis of rotation and second axis of rotation onto an image.

7. A computing device (210) of a vehicle (201) for calculating the position of a trailer hitch point, wherein, The calculating device is configured to Based on a first image of the trailer in a first pose and a second image of the trailer in a second pose, both captured, calculate the trailer rotation and trailer translation, wherein the first image and the second image are captured using an image capturing device (202) mounted on the vehicle and the trailer (207, 208) is within the field of view of the image capturing device (202), Based on the calculated trailer rotation and trailer translation, calculate a first axis of rotation of the trailer between the first pose and the second pose, Calculate a second axis of rotation of the trailer using a third image of the trailer in a third pose and a fourth image of the trailer in a fourth pose captured by the image capturing device (202), wherein the third image and the fourth image show another rotation and another translation relative to the first image, and calculate a second axis of rotation of the trailer between the third image and the fourth image that is different from the first axis of rotation based on the another rotation and the another translation. Determine an intersection point of the first axis of rotation and the second axis of rotation to determine a vector of the trailer hitch point position, wherein the vector of the trailer hitch point position is defined as an intersection line of a first plane and a second plane, the first plane contains the first axis of rotation and the position of the image capturing device, and the second plane contains the second axis of rotation and the position of the image capturing device.

8. The computing device according to claim 7, wherein, The calculating device is further configured to calculate the trailer hitch point position using the vector to the trailer hitch point position and a geometric condition.

9. A vehicle (201) comprising an image capturing device (202) for capturing images of a trailer (207, 208) and a calculating device according to claim 7 or 8.

10. A program unit which, when run on a processor, instructs the processor to perform the method according to claim 1.

11. A computer-readable medium for storing the program unit according to claim 10.

Images