Method and control unit for determining the orientation of a trailer
Patent Information
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- ZF FRIEDRICHSHAFEN AG
- Filing Date
- 2018-04-25
- Publication Date
- 2026-07-09
AI Technical Summary
Existing systems fail to efficiently detect and determine the orientation of a trailer relative to a towing vehicle, which can affect the operation of the towing vehicle.
A method using stereo image pairs captured by cameras on the towing vehicle to calculate the orientation of the trailer based on measured distances, employing stereoscopy and stereophotogrammetry to determine the trailer's position and surface, allowing for non-contact measurement.
Enables efficient and reliable determination of the trailer's orientation to the towing vehicle without requiring power supply on the trailer, facilitating accurate alignment and positioning, especially in autonomous systems.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Technical area
[0001] The invention relates to a method and a control unit for determining the orientation of a trailer to a towing vehicle, as well as a towing vehicle with such a control unit. State of the art
[0002] It is known to equip vehicles with a camera that captures an image of the vehicle's surroundings. This image can then be displayed to the driver. For example, an electronic mirror can be implemented with such a camera, allowing the camera image displayed to the driver to replace the reflection in a traditional side mirror. With such an electronic mirror, the driver can detect objects in the vehicle's vicinity even without direct visual contact.
[0003] However, the vicinity of a towing vehicle may contain not only third-party objects, such as other vehicles or people, but also a trailer attached to the towing vehicle, which can affect the operation of the towing vehicle. Summary of the invention
[0004] The invention therefore provides solutions for efficiently detecting a trailer attached to a towing vehicle in order to optimize the operation of the towing vehicle.
[0005] Such a solution for a method for determining the orientation of a trailer to a towing vehicle comprises the following process steps: capturing a stereo image pair of the trailer with a first camera and a second camera, which are arranged on the towing vehicle; determining at least one distance between the towing vehicle and the trailer based on the captured stereo image pair; and calculating the orientation of the trailer to the towing vehicle based on the determined distance.
[0006] The towing vehicle can be any vehicle designed to attach and tow a trailer, for example, a truck, a semi-trailer truck, a passenger car, a tractor, a tractor-trailer, or a van. The towing vehicle can be self-propelled or a driver-operated vehicle, and in particular, it can be an autonomous towing vehicle or transport vehicle.
[0007] The trailer can be any vehicle that has a loading platform. In particular, the trailer can be unpowered and towed behind the towing vehicle. The towing vehicle and the trailer can form a vehicle combination, whereby the towing vehicle can be the powered or motorized part and the trailer the unpowered or non-motorized part of the combination.
[0008] The stereo image pair can consist of two digital images of the pendant, whereby at least one area of the pendant and / or an identical object connected to the pendant is depicted at least partially identically in both images. The images forming the stereo image pair can be captured with two cameras that have approximately parallel axes and are taken from different camera positions. The stereo image pair can serve as the basis for three-dimensional photogrammetric image analysis using methods of stereoscopy and stereophotogrammetry to determine and measure the position, orientation, and surface of the pendant.
[0009] The first and second cameras can be identical or different in construction, in particular having the same, different or variable fields of view, resolutions and detection ranges, and the cameras can be calibrated.
[0010] The distance between the towing vehicle and the trailer can be any distance between two natural points or two points defined in a coordinate system. The coordinate system can be a vehicle coordinate system in which the camera positions and orientations for stereoscopic measurement in a stereo image pair are known. Alternatively, the distance between the towing vehicle and the trailer can also be known.
[0011] The orientation of the trailer relative to the towing vehicle can be any relative position of the trailer to the towing vehicle, where the relative position can be any angle or orientation between a reference line of the towing vehicle and a reference line of the trailer. The reference lines can be, in particular, vehicle axles or drive axles. The relative position of the trailer to the towing vehicle can also include the relative position of the trailer with respect to the towing vehicle. The orientation of the trailer can also be determined indirectly by determining the orientation of an object fixed to the trailer, for example, a container, whereby a stereo image pair of the object can be acquired and, based on a distance between the towing vehicle and the object, the orientation of the trailer to the towing vehicle, which may correspond to the orientation of the object to the towing vehicle, can be calculated.
[0012] A key aspect of the invention is that the relative orientation of a trailer to a towing vehicle can be determined using cameras on the towing vehicle by means of a non-contact, photogrammetric measurement of at least one distance between the towing vehicle and the trailer. The trailer can thus be a purely passive object being measured. This is advantageous because trailers typically lack a power supply for distance-determining sensors. Furthermore, the orientations of different trailers relative to the same towing vehicle can be determined.
[0013] The invention is based on the concept that a single photogrammetric distance measurement based on the captured stereo image pair can be sufficient to determine the relative orientation of the trailer to the towing vehicle if a distance between the trailer and the towing vehicle is already known. This could, for example, be a distance between a trailer coupling on the towing vehicle and a corresponding pivot point or reference point on the trailer.
[0014] In one embodiment, the first and second cameras are arranged as rear-view cameras on the towing vehicle, and at least a portion of the trailer is depicted in each of the first and second images of the stereo image pair. The rear-view cameras can be arranged on the side of the towing vehicle and / or at its rear. For determining the relative position of the trailer to the towing vehicle by means of stereoscopic measurement in the stereo image pair, it may be sufficient that an identical portion of the trailer is depicted in both images forming the stereo image pair, since it may be sufficient to determine a distance to a single measuring point of the trailer, for example, to an edge of the trailer, using the stereo image pair.The fields of view of the rear-view cameras can thus be aligned with a matching area of the trailer, which is particularly suitable for selecting measurement points, for example due to a high contrast in the matching area.
[0015] Another embodiment involves selecting the first and second cameras for capturing the stereo image pair from at least three cameras, wherein the at least three cameras are mounted on the towing vehicle. The at least three cameras can be rear-view cameras. More than one stereo image pair can be available with the at least three cameras, each of which can depict an identical area of the trailer. At least one camera pair can be selected from the available cameras to use a stereo image pair obtained with that camera pair as the basis for distance determination. For example, with three cameras, up to two stereo image pairs can be selected, and with four cameras, up to six stereo image pairs can be selected, in which the identical area of the trailer is depicted. The identical area can be redundantly depicted in several stereo image pairs.It is also possible for different identical areas of the trailer to be depicted in one or more stereo image pairs. Using multiple cameras is particularly advantageous because a single camera, due to dirt or obstruction of at least part of its field of view, may fail to capture a usable image of the trailer for distance determination. In such cases, a redundant camera arrangement has the beneficial effect of allowing the acquisition of a stereo image pair with clean or unobstructed cameras and the calculation of the trailer's orientation, even if one camera fails. A further advantage is that, due to varying illumination or different angles of incidence on the trailer, a stereo image pair can be selected that depicts an area of the trailer with sufficient brightness or favorable lighting for stereo image analysis.
[0016] Another embodiment involves defining a measuring point on the trailer captured in the stereo image pair, stereoscopically measuring the distance between the towing vehicle and the measuring point defined on the trailer in the stereo image pair, and calculating the orientation of the trailer relative to the towing vehicle using a trigonometric relationship, taking the measured distance into account. The distance between the towing vehicle and the trailer to a first point on the trailer can be known, and this point can be a point on the trailer whose relative position remains essentially constant when the trailer is pivoted relative to the towing vehicle.To calculate the trailer's orientation using trigonometric relationships, it may be necessary to determine the coordinates of a second point on the trailer within a coordinate system defined relative to the towing vehicle. The coordinates of the first point, with a known distance, may already be known within this defined coordinate system. The coordinates of the second point can be determined using stereophotogrammetric image analysis of a stereo image pair. This involves transforming the coordinates of the measurement point into the defined coordinate system.
[0017] Another embodiment involves defining a first measuring point on the trailer captured in the stereo image pair, defining a second measuring point on the trailer captured in the stereo image pair, stereoscopically measuring a first distance between the towing vehicle and the first measuring point defined on the trailer in the stereo image pair, stereoscopically measuring a second distance between the towing vehicle and the second measuring point defined on the trailer in the stereo image pair, and calculating the orientation of the trailer relative to the towing vehicle using a trigonometric relationship, taking into account the measured first distance and the measured second distance. The distance between the towing vehicle and the trailer may not be known.To calculate the trailer's orientation using trigonometric relationships, it may be necessary to determine the coordinates of two points on the trailer as measuring points within a coordinate system defined with respect to the towing vehicle or the trailer. At least the coordinates of the first or second point may already be known within this defined coordinate system. The coordinates of the first and second points can each be determined using stereophotogrammetric image analysis of the stereo image pair. This involves transforming the coordinates of the measuring points into the defined coordinate system.
[0018] In another embodiment, the calculated orientation has an angle between a longitudinal axis of the trailer and a longitudinal axis of the towing vehicle. If the longitudinal axes of the trailer and the towing vehicle are not parallel, they can define an angle that describes a pivoting of the trailer relative to the towing vehicle. Such a pivoting can describe the relative orientation of the trailer to the towing vehicle. The angle between the longitudinal axis of the trailer and the longitudinal axis of the towing vehicle can correspond to an angle between a respective transverse axis, drive axis, or wheel axle of the trailer and the towing vehicle.
[0019] Another embodiment involves determining the distance between the towing vehicle and the trailer using a measuring marker located on the trailer. The measuring marker can define a measuring point on the trailer, which can be measured stereoscopically using a stereo image pair. For this to work, the measuring marker must be visible in both images of the stereo image pair. The measuring marker can be attached to the trailer to enable distance measurement.
[0020] Another solution is a control unit for determining the orientation of a trailer to a towing vehicle, which is designed to capture a stereo image pair of the trailer with a first camera and a second camera located on the towing vehicle, to determine a distance between the towing vehicle and the trailer based on the captured stereo image pair, and to calculate the orientation of the trailer to the towing vehicle based on the determined distance.
[0021] Another solution is a towing vehicle equipped with such a control unit. The towing vehicle can, in principle, be any known off-highway vehicle. An off-highway vehicle can be any work vehicle that is not intended, or not exclusively intended, for road use. The off-highway vehicle can be a passenger-driven or a self-driving vehicle.
[0022] In one embodiment, the towing vehicle is designed as an autonomous container tug, with a container chassis attached to the towing vehicle as a trailer for transporting a container. The autonomous tug can be part of an autonomous container transport system, for example, in a container port or other container handling facility. Automatic determination of the orientation of a container chassis on an autonomous container tug is particularly advantageous, as precise positioning of the container chassis may be necessary for loading with a container.
[0023] If the position and orientation of the container tug and a loading crane, which can only move a container translationally, are known or measurably determinable within the autonomous container transport system, the orientation of the container chassis within the transport system can be indirectly determined by determining the relative orientation of the chassis to the tug and the position of the chassis within the transport system using at least one distance determined in the stereo image pair. This allows for precise positioning of the container with the loading crane onto the chassis, ensuring that the container can be unloaded with pinpoint accuracy. List of characters Fig. Figure 1 shows a flowchart of process steps of an exemplary embodiment of a method for determining the orientation of a trailer to a towing vehicle. Fig. Figure 2 shows a top view of the towing vehicle and the trailer to further explain a stereo image pair for calculating the orientation of the trailer to the towing vehicle. Fig. Figure 3 shows a top view of the towing vehicle and the trailer to further illustrate the acquisition of another stereo image pair for calculating the orientation of the trailer to the towing vehicle. Fig. Figure 4 shows a top view of the towing vehicle and the trailer to illustrate the acquisition of a stereo image pair for calculating the orientation of the trailer to the towing vehicle. Detailed description of embodiments
[0024] In Fig. 1 are individual process steps S1 , S2 , S3 , S4 , S5 , S6 a method for determining the orientation of a trailer 2 to a towing vehicle 4 depicted.
[0025] In a first step S1 will be a first camera 12 , 14 , 16 and a second camera 14 , 16 , 18 from four attached to a towing vehicle 4 arranged cameras 12 , 14 , 16 , 18 selected.
[0026] In a second step S2 At least one stereo image pair (not shown) of a trailer will be displayed. 2 , which is attached to the towing vehicle 4 is arranged with the step S1 selected first camera 12 , 14 , 16 and second camera 14 , 16 , 18 captured, with the first camera 12 , 14 , 16 a first image (not shown) and the second camera 14 , 16 , 18 takes a second picture (not shown).
[0027] In a third step S3 Two measuring points will be used 32 ,34 on the trailer 2 determined which in the step S2 The captured stereo image pair was recorded. The step S3 shows two substeps S3a , S3b up. In the lower step S3a A first measuring point will be established 32 and in the lower step S3b A second measuring point will be established 34 in the step S2 captured stereo image pair defined, with the first measurement point 32 on one edge of the front 3 of the trailer 2 lies and the second measuring point 34 in a central area on the front 3 of the trailer 2 lies.
[0028] In a fourth step S4 Two distances will be established between the towing vehicle 4 and the trailer 2 stereoscopically in the step S2 measured the captured stereo image pair, with step S4 two understeps S4a , S4b exhibits. In step S4a A first distance (not shown) is established between the towing vehicle 4 and the one on the trailer 2 defined first measuring point 32 The stereo image pair is measured stereoscopically. In step S4b A second gap (not shown) is created between the towing vehicle 4 and the one on the trailer 2 defined second measuring point 34 measured stereoscopically in the stereo image pair.
[0029] In a fifth step S5 Two distances between the towing vehicle will be determined 4 and followers 2 based on the step S2 captured stereo image pair and the one in step S4 The measurements were determined using stereoscopic methods. The coordinates of the two measuring points are derived from the two distances. 32 , 34 in one on the towing vehicle 4 defined coordinate system 30 certainly.
[0030] In a sixth step S6 will be an angle 22 between a drive axle 5 of the towing vehicle 4 and a front 3 of the trailer 2 as alignment between the towing vehicle 4 and the trailer 2 trigonometrically based on the in step S5 derived coordinates of the measuring points 32 , 34 calculated.
[0031] In the Fig. 2, Fig. 3 and Fig. 4 is the towing vehicle 4 with the trailer 2 shown, which has a fixed attachment to the towing vehicle 4 arranged connecting part 10 on the towing vehicle 4 It is pivotally mounted. The trailer 2 is equipped with a trailer hitch 7 with the towing vehicle 4 tied together.
[0032] The towing vehicle 4 has wheels 6and a drive (not shown) for moving the towing vehicle 4 The wheels 6 are on the drive axle 5 of the towing vehicle 4 arranged. The trailer also has wheels. 6 and a transport area 9 .
[0033] On one of the back 8 of the towing vehicle 4 The adjacent area contains the four cameras. 12 , 14 , 16 , 18 arranged, each of which has a field of view 13 , 15 , 17 , 19 exhibiting at least one area of the trailer 2 in a first position 24 and in a second layer 26 of the trailer 2 capture. In the fields of vision 13 , 15 , 17 , 19 At least one measuring point of the two measuring points will be used in each case. 32 , 34 recorded.
[0034] The first layer 24and the second layer 26 of the trailer 2 This is caused by the trailer pivoting. 2 relative to the towing vehicle 4 around the trailer hitch 34 , whereby the giving away of the front 3 the angle 22 defined, which can also be defined at other locations with the same angle magnitude.
[0035] Calculating the angle 22 is now based on individual measurement points determined by coordinates 32 , 34 based on the Fig. 2 to Fig. 4 explained.
[0036] Fig. Section 2 explains how to determine the coordinates. x1 , y1 of the measuring point 34 . According to step S1 The first camera will be the camera 14 and as a second camera the camera 16 selected which two are on the back 8 of the towing vehicle 4are positioned as a rearview camera in a central area. According to step S2 A stereo image pair of the trailer will be created. 2 , with the cameras 14 , 16 captured by the camera 14 a first image and the second camera 16 takes a second picture. According to step S3 and understep S3b First, the second measuring point will be determined. 34 defined in the stereo image pair, with the second measurement point 34 center on the front 3 of the trailer 2 The measuring point is located. 34 It lies within an overlap area of the field of vision. 15 the camera 14 and the field of vision 17 the camera 16 or in an overlap area of the two images. A distance between the towing vehicle 4 and the one on the trailer 2 defined second measuring point 34 is stereoscopically according to step S4, understep S4a and step S5 measured and determined coordinates x1 , y1 of the second measuring point 34 in the coordinate system 30 determined. The coordinates x1 , y1 of the second measuring point 34 are determined based on the specified distance using methods of stereophotogrammetry.
[0037] Fig. Section 3 explains another method for determining the coordinates. x1 , y1 of the measuring point 34 . According to step S1 The first camera will be the camera 12 and as a second camera the camera 18 selected which two are adjacent to the back 8 side of the towing vehicle 4 are arranged as a rearview camera. According to step S2 A stereo image pair of the trailer will be created. 2 , with the cameras 12 , 18 captured, with the camera 12a first image and the second camera 18 takes a second picture. According to step S3 and understep S3b First, the second measuring point will be determined. 34 defined in the stereo image pair. The measuring point 34 It lies within an overlap area of the field of vision. 13 the camera 12 and the field of vision 19 the camera 18 or in an overlap area of the two images. The coordinates x1 , y1 of the second measuring point 34 will be like Fig. 2 explained based on the specified distance determined using methods of stereophotogrammetry.
[0038] Fig. Section 4 explains how to determine coordinates. x2 , y2 and x3 , y3 of the measuring point 32 regarding two layers 24 , 26 of the trailer 2 , where the coordinates x2 , y2on the first layer 24 and the coordinates x3 , y3 on the second layer 26 refer to. For determining the coordinates x2 , y2 and x3 , y3 in both positions 24 , 26 will be according to step S1 as the first camera the camera 12 and as a second camera the camera 14 selected which are both located in a lateral area of the back 8 of the towing vehicle 4 are arranged as rearview cameras.
[0039] According to step S2 A stereo image pair of the trailer will be created. 2 , which is in the first position 24 is located, with the cameras 12 , 14 captured, with the camera 12 a first image of the first layer 24 of the trailer 2 and the second camera 14 a second image of the first layer 24 of the trailer 2 records. According to stepS3 and understep S3a will be the first measuring point 32 in the stereo image pair of the first layer 24 of the trailer 2 The measuring point was determined. 34 It lies within an overlap area of the field of vision. 13 the camera 12 and the field of vision 15 the camera 14 or in an overlap area of the two images of the first layer 24 of the trailer 2 The coordinates x2 , y2 of the first measuring point 32 the first layer 24 of the trailer 2 will be like Fig. 2 explained based on the specified distance determined using methods of stereophotogrammetry.
[0040] According to step S2 Another stereo image pair of the trailer will be created. 2 , which is in the second layer 26 is located, with the cameras 12 , 14 captured, with the camera12 a first image of the second layer 26 of the trailer 2 and the second camera 14 a second image of the second layer 26 of the trailer 2 records. According to step S3 and understep S3a will be the first measuring point 32 in the stereo image pair of the second layer 26 of the trailer 2 The measuring point was determined. 34 It lies within an overlap area of the field of vision. 13 the camera 12 and the field of vision 15 the camera 14 or in an overlap area of the two images of the second layer 26 of the trailer 2 The coordinates x3 , y3 of the first measuring point 32 the first layer 24 of the trailer 2 will be like Fig. 2 explained based on the specified distance determined using methods of stereophotogrammetry.
[0041] Based on the specified coordinates x1 , y1 ; x2 , y2 and x3 , y3 will be according to step S6 the in the Fig. 2 to Fig. 4 angles shown 22 as a description of the trailer's orientation 4 to the towing vehicle 2 trigonometrically calculated using the following relationship in formula 1, where α is the angle 22 defined: α = arcsin ( y 1 − y 3 ) ( x 1 − x 1 ) α determines the angle 22 between the drive axle 5 of the towing vehicle 4 and the front 3 of the trailer 2 , whereby other trigonometric relationships based on other coordinates and based on a tangent term or a cosine term can optionally be applied.
[0042] With the process steps S1 , S2 , S3 , S4 , S5 , S6 Is it therefore possible to determine the angle?22 as a definition of the trailer's orientation 2 to the towing vehicle 4 efficient and reliable thanks to a system with four cameras 12 , 14 , 16 , 18 To determine the implemented redundant photogrammetric measuring system without contact from the towing vehicle. Reference symbol list 2 trailers 3 Front 4 Towing vehicle 5 drive axle 6 wheels 7 Towbar 8 Back 9 Transport area 10 Connecting part 12 first camera 13 first field of vision 14 second camera 15 second field of vision 16 third camera 17 third field of vision 18 fourth camera 19 fourth field of vision 22 angles 24 first layer 26 second layer 30 Coordinate system 32 first measuring point 34 second measuring point S1 Select camera pair S2 Capture stereo image pair S3a Determining first measuring point S3b Determining the second measuring point S4a Measuring first distance S4b Measuring second distance S5 Distance Determination S6 Calculation Alignment
Claims
[1] Method for determining the orientation of a trailer (2) to a towing vehicle (4), comprising the steps Capturing (S2) a stereo image pair of the trailer (2) with a first camera (12) and a second camera (14), wherein the first camera (12) and the second camera (14) are arranged on the towing vehicle (4), Determine (S5) at least one distance between the towing vehicle (4) and the trailer (2) based on the captured stereo image pair and Calculate (S6) the orientation of the trailer (2) to the towing vehicle (4) based on the determined distance. [2] Method according to claim 1, wherein the first camera (12) and the second camera (14) are arranged as rear-view cameras on the towing vehicle (4) and wherein in a first image and in a second image of the stereo image pair at least one area of the pendant (2) is depicted. [3] Method according to claim 1 or 2, with Selecting (S1) the first camera (12) and the second camera (14) to capture (S2) the stereo image pair from at least three cameras (12, 14, 16, 18), wherein the at least three cameras (12, 14, 16, 18) are arranged on the towing vehicle (4). [4] Method according to any one of the preceding claims, with Defining (S3a) a measuring point (32) on the pendant (2) captured in the stereo image pair, stereoscopic measurement (S4a) of the distance between the towing vehicle (4) and the measuring point (32) fixed on the trailer (2) in the stereo image pair and Calculate (S6) the orientation of the trailer (2) to the towing vehicle (4) using a trigonometric relationship taking into account the measured distance. [5] Method according to any one of the preceding claims, with Defining (S3a) a first measurement point on the pendant (2) captured in the stereo image pair, Defining (S3b) a second measurement point on the pendant (2) captured in the stereo image pair, stereoscopic measurement (S4a) of a first distance between the towing vehicle (4) and the first measuring point on the trailer (2) in the stereo image pair, stereoscopic measurement (S4b) of a second distance between the towing vehicle (4) and the second measuring point on the trailer (2) in the stereo image pair and calculation (S6) of the orientation of the trailer (2) to the towing vehicle (4) using a trigonometric relationship taking into account the measured first distance and the measured second distance. [6] Method according to one of the preceding claims, wherein the calculated orientation has an angle (22) between a longitudinal axis (3) of the trailer (2) and a longitudinal axis (5) of the towing vehicle (4). [7] Method according to one of the preceding claims, comprising determining (S5) the distance between the towing vehicle (4) and the trailer (2) by means of a measuring mark provided on the trailer (2). [8] Control unit for determining the orientation of a trailer (2) to a towing vehicle (4), which is configured to perform a method according to one of the preceding claims. [9] Towing vehicle (4) with a control unit according to claim 8. [10] Towing vehicle (4) according to claim 9, which is trained as an autonomous tug for containers and to which a container chassis can be attached as a trailer for transporting a container (2).