METHOD FOR DETERMINING OBJECT INFORMATION ABOUT AN OBJECT IN A VEHICLE ENVIRONMENT, CONTROL UNIT AND VEHICLE

Description

[0001] The invention relates to a method for determining object information of an object in an environment of a multi-part vehicle, as well as a control unit and a vehicle for carrying out the method.

[0002] It is known from the prior art how, using a single camera, the structure of a scene can be determined in 3D by photogrammetric methods through the forward or backward movement of a vehicle on which the camera is mounted (so-called Structure from Motion (SfM)). Furthermore, it is known from the prior art that the determination of the baseline between two camera positions, which is either known or estimated for triangulation to determine depth, can be supported by evaluating odometry data from the vehicle.

[0003] US2018 / 0204072A1 further specifies the installation of cameras on a trailer of a vehicle combination. It also includes vehicle dynamics sensors that output odometry data relating to vehicle movement, such as vehicle speed. The camera data output is compared with the odometry data, which is used to compensate for vehicle movement when processing the camera data to create images. Camera data from different cameras can also be combined in this process.

[0004] DE 10 2005 009 814 B4 describes the processing of camera data together with odometry data output by wheel speed sensors to determine the yaw rate. DE 60 009 000 T2 further describes image processing, taking into account the vehicle's odometry data, to assist the driver with parking. DE 10 2015 105 248 A1 describes the processing of an image from a first camera together with an image from a second camera in conjunction with odometry data, whereby the cameras can be arranged on a trailer and a towing vehicle of a multi-section vehicle. The images captured by the various cameras and output as camera data are combined. This creates a combined image of the surroundings, taking into account, for example, the angle of rotation when cornering, which characterizes the relative positions of the cameras.A bird's-eye view can be overlaid on the entire multi-part vehicle to show the surroundings, for example to enable parking assistance.

[0005] WO 2016 / 164118 specifies an omnidirectional camera that captures object points in the vehicle's environment and outputs corresponding camera data. A control unit in the vehicle processes the camera data, incorporating odometry data. This odometry data, for example from wheel speed sensors, position sensors, or a steering angle sensor, is received via the vehicle's data bus. The camera detects the object points of interest in the vehicle's environment, and the control unit uses the odometry data to determine the distance to the object associated with each captured object point. This is achieved by capturing multiple images from different viewpoints with overlapping fields of view.By tracking object points, depth information of the scene can be estimated using triangulation and bundle adjustment. The camera data is also displayed as images on a screen for the driver. These images, along with the determined distance, facilitate maneuvering a car as a towing vehicle to hitch up a trailer. While other objects such as the ground, pedestrians, etc., can be detected, this requires sufficient vehicle movement, as only then can different camera viewpoints be adjusted.

[0006] Document DE102009039111A1 describes how to determine the articulation angle of a vehicle combination using two GPS receivers.

[0007] Document DE102017111530A1 discloses a method for displaying the swivel angle of a trailer to the driver on a screen from a bird's-eye view. This is achieved using image data from multiple cameras in both the towing vehicle and the trailer.

[0008] From the document XP010645870, Fintzel K., Bendahan R., Vestri C., Bougnoux S., Yamamoto S., Kakinami T. Proc. IEEE Intelligent Vehicle Symposium SYMPOSIUM, 2003, 2003-06-09 - 2003-06-11, 2003-06-09, pages 174 - 179, it is known to use the Structure-from-Motion method for the detection of objects in the vicinity of vehicles.

[0009] A disadvantage of this technology is that, for example, detecting people lying on the ground when the vehicle is stationary or moving at very low, unresolvable speeds is not possible. Similarly, other objects in the vehicle's vicinity cannot be detected using "Structure from Motion" if the vehicle is stationary or moving very slowly. Therefore, when stationary, a single camera cannot achieve spatial mapping of the vehicle's surroundings or an object, meaning that neither automated object classification nor distance determination is possible.

[0010] However, the safety requirements for driverless industrial vehicles, such as those specified in ISO 3691-4, necessitate the detection of people lying on the ground. This means that they must be identified before departure, i.e., when stationary or at very low speeds. Furthermore, distance measurements, object classifications, or the determination of other spatial object information cannot be performed with existing systems at traffic lights or in parking situations.

[0011] The object of the invention is to provide a method for determining object information of an object, whereby a spatial view of the vehicle's surroundings is enabled as accurately and reliably as possible using only one camera arranged on a trailer. A further object is to provide a control unit and a vehicle.

[0012] This task is solved by a method, a control unit, and a vehicle according to the independent claims. The dependent claims specify preferred embodiments.

[0013] Accordingly, a generic method for determining object information about an object in the environment of a multi-part vehicle consisting of at least one towing vehicle and at least one trailer is provided, wherein at least one trailer camera is arranged on the trailer, with at least the following steps: Capturing the surroundings with at least one trailer camera from a first position and, depending on this, creating a first image consisting of first pixels; changing the position of the at least one trailer camera; capturing the surroundings with at least one trailer camera from a second position and, depending on this, creating a second image consisting of second pixels, whereby the first position differs from the second position due to the intermediate change in the position of the trailer camera;Determining object information for an object in the captured environment by: selecting at least one first pixel in the first image and at least one second pixel in the second image, wherein the first pixel and the second pixel are selected such that they are assigned to the same object point of the object in the captured environment, and determining object coordinates of the assigned object point from the first image coordinates of the at least one first pixel and the second image coordinates of the at least one second pixel by triangulation, assuming a baseline length between the two camera positions, wherein the baseline length between the two positions is determined as a function of odometry data of the vehicle, the odometry data characterizing an adjustment of the trailer camera between the two positions.

[0014] According to the invention, the odometry data includes and / or depends on an articulation angle between the at least one trailer and the at least one towing vehicle. This advantageously improves the accuracy of determining the base length through odometry, as the articulation angle is also used. This angle can provide further information regarding the trailer's movement that was not available from previously used odometry data, such as steering angle or wheel speed signals.

[0015] Therefore, the rotational speed of the wheels cannot reliably indicate whether the trailer is pivoting, for example, during maneuvering or parking, and thus cannot accurately determine the position of the trailer or the trailer camera between the two viewpoints. This is primarily because the trailer wheels rub or slip on the road surface during steering movements, making it impossible to reliably predict rotation. The steering angle of the towing vehicle also does not reliably indicate the actual movement of the trailer. In this respect, odometry, taking the articulation angle [TK1] into account, can provide more accurate results regarding the base length, thereby enabling more reliable depth information about an object obtained through structure-from-motion.

[0016] Preferably, it can be provided that the bending angle is determined via an active articulation angle sensor at a coupling point between the at least one towing vehicle and the at least one trailer, for example a kingpin or trailer coupling, and / or depending on images captured by a camera on the towing vehicle and / or on the trailer.

[0017] This allows for variable determination of the buckling angle, whereby an active buckling angle sensor enables a very simple, reliable and cost-effective determination of the buckling angle.

[0018] Preferably, the odometry data can also be generated depending on an adjustment path, wherein the at least one trailer camera is adjusted by controlling an active actuator system on the trailer by the adjustment path, without changing the vehicle's driving state. Driving state is understood to mean the vehicle's overall state of motion, i.e., for example, stationary or traveling at a specific vehicle speed. The active actuator system does not change this motion; the adjustment path is therefore not linked to the vehicle's movement. Thus, the active actuator system differs from a drive system or a braking system, which directly affect the overall state of motion of the vehicle.

[0019] The advantage of this approach is that depth information and object information can be determined with just one trailer-mounted camera, regardless of the vehicle's driving state. This allows for the determination of depth information, or, to a lesser extent, the 3D position and object coordinates of the respective object point, even when stationary or at such low speeds that odometry data is insufficient to reliably predict movement between the two viewpoints. This requires only controlled operation of the active actuator system, which is independent of vehicle movement. Furthermore, the camera's adjustment via the active actuator system can also be used in conjunction with vehicle movement, enabling the trailer camera to be moved in additional directions for applications such as structure-from-motion analysis.

[0020] This means that the actuator system for adjusting the trailer camera between positions is not limited to stationary or low-speed vehicle operation. Additional adjustments can also be made while driving using the active actuator system. This allows for more flexible object information acquisition under different driving conditions and situations.

[0021] The adjustment range can therefore be used in addition to the vehicle movement (if present) when determining depth information or object coordinates, if this adjustment range is taken into account alongside the usual odometry data, specifically the articulation angle, which relates to the vehicle's driving state. This allows the determination of object information or depth information to be more precise and flexible, and applicable to different driving situations.

[0022] Preferably, the active actuator system is a camera adjustment system comprising actuators and / or pneumatic cylinders and / or hydraulic cylinders and / or electric servo cylinders, wherein the at least one trailer camera is directly attached to the camera adjustment system, so that when the camera adjustment system is activated, the at least one trailer camera is moved by the adjustment path to change its position. Thus, according to one embodiment, the trailer camera can be adjusted directly without moving the vehicle or its components, with the camera adjustment system being mounted and aligned accordingly on the vehicle.

[0023] Preferably, it is further provided that an active air suspension system with air springs (ECAS) or a chassis adjustment system is used as the active actuator system, wherein by controlling the active air suspension system or the chassis adjustment system a vehicle body is adjusted in height by the adjustment path, so that the at least one trailer camera attached to the vehicle body is indirectly adjusted by the adjustment path to change the position of the at least one trailer camera.

[0024] This allows for the advantageous use of an actuator system already present in the vehicle, enabling it to perform a dual function. For example, it can raise and lower the vehicle body for air suspension, stabilize (rolling, tipping), etc., and simultaneously adjust the trailer camera to different positions. The air suspension or chassis adjustment system simply needs to be activated in the relevant situation, which is possible in all driving situations, including when stationary. The trailer camera can then be freely mounted on the vehicle body to move with it.

[0025] Preferably, it is further provided that a component adjustment system is controlled as an active actuator system, wherein by controlling the component adjustment system a component of the vehicle, for example an aerodynamic component, is adjusted by the adjustment path, so that the at least one trailer camera attached to this component is indirectly adjusted by the adjustment path to change the position of the at least one trailer camera.

[0026] This allows the use of an actuator system that doesn't raise, lower, or adjust the entire vehicle structure, but only individual parts or components. Such adjustment systems are already present in certain vehicles, so they don't need to be retrofitted. The trailer camera then simply needs to be attached to this component.

[0027] The aforementioned active actuator systems can be provided individually or in combination with each other, for example to increase variability and to enable combined adjustments with possibly extended adjustment ranges.

[0028] Preferably, the object coordinates or object information for multiple object points are determined by triangulation from at least two images, and an object contour and / or object shape is derived from these multiple object points. The object can then preferably be subdivided into object classes based on its contour and / or shape. This allows for the simple recognition and classification of objects, especially stationary objects such as people.

[0029] According to further training, several trailer cameras are used, and each camera independently determines object information about an object from its baseline length using the described method. This allows depth information, or object information, to be obtained preferably from multiple sources, thereby increasing reliability. Furthermore, this also makes it possible to validate the object information obtained from the multiple trailer cameras.

[0030] Preferably, it is further provided that more than two images are captured from different viewpoints and that pixels are selected from each captured image that correspond to the same object point of the object in the captured environment. The object coordinates of the corresponding object point are then determined from the image coordinates of the selected pixels by triangulation, assuming a baseline length between the respective viewpoints of the trailing camera. This allows the respective object or object point to be tracked over a longer period of time in order to determine the depth information or the respective object information more accurately and robustly, possibly by means of bundle adjustment. Several pixels can also be grouped into one or more feature points, and the temporal correspondence of this feature point(s) between the respective images can be determined by triangulation.

[0031] Additionally, it can be provided that the object information obtained from adjusting the trailer camera by the adjustment path via the active actuator system, or from the adjustment path itself as odometry data, is validated with object information derived from the vehicle's odometry data. This data is selected from the following groups: wheel speed signal, vehicle speed, steering angle, articulation angle, and / or transmission data, specifically including transmission speed and gear selection. This allows object information obtained from different movements of the trailer camera to be compared for a given object.For example, if the vehicle speed is very low, the reliability of depth information determined from wheel speeds and articulation angles can no longer be guaranteed, for example with passive wheel speed sensors, so that, in addition to plausibility checks, the trailer camera can be adjusted by the adjustment range via the active actuator system and the depth information can be obtained from this.

[0032] According to the invention, a control unit and a vehicle with such a control unit for carrying out the described methods are further provided, wherein the vehicle is multi-part and comprises at least one towing vehicle and at least one trailer, wherein an articulation angle is formed between the towing vehicle and the trailer, and wherein at least one trailer camera is arranged on the trailer.

[0033] The invention is explained in more detail below using an exemplary embodiment. The figures show: Fig. 1 a driving situation of a multi-part vehicle; Fig. 1a a detailed view of the multi-part vehicle; Fig. 2a an image taken by the camera; Fig. 2b the recording of an object point with a camera from different viewpoints; and Fig. 3 a flowchart of the method according to the invention.

[0034] In Figur 1 The diagram schematically depicts a multi-part vehicle 1 consisting of a towing vehicle 2 and a trailer 3, wherein, according to the illustrated embodiment, a camera 4 with a detection range E is arranged on each of the vehicle parts 2 and 3. A towing vehicle camera 42 with a towing vehicle detection range E2 is arranged on the towing vehicle 2, and a trailer camera 43 with a trailer detection range E3 is arranged on the trailer 3. The cameras 4, 42, and 43 each output camera data KD, KD2, and KD3, respectively.

[0035] Vehicle 1 can be made up of multiple parts, as shown in Fig. 1 depicted, for example, as a truck and trailer combination with a drawbar trailer or turntable trailer, or as a semi-trailer truck with a tractor unit and semi-trailer. However, vehicle 1 can also be a single unit, as shown in Fig. 1a The orientation of camera 4 is shown. The orientation of camera 4 is chosen depending on the specific application.

[0036] The respective camera data KD, KD2, KD3 are generated depending on an environment U around the vehicle 1, towards which the respective detection area E, E2, E3 is directed. From the camera data KD, KD2, KD3, an image B consisting of pixels BPi with image coordinates xB, yB can be generated (see figure). Fig. 2a ) create, where each pixel BPi is assigned an object point PPi in the environment U (see Fig. 2b The object points PPi belong to objects O located in the environment U, to which specific absolute object coordinates xO, yO, zO can be assigned in space. Depending on the position SP of the respective camera 4, 42, 43, object points PPi of an object O are mapped onto different image points BPi or with different image coordinates xB, yB in the images B.

[0037] The camera data KD, KD2, KD3 from the respective cameras 4, 42, 43 are transmitted to a control unit 5, which is configured to determine object information Ol by means of a triangulation T, generally known to those skilled in the art, based on the camera data KD, KD2, KD3 and on selected odometry data DD of the vehicle 1, which relate to the current driving situation of the vehicle 1 or the sub-vehicles 2, 3 and thus also characterize the movement of the camera 4. The object information Ol specifies, in particular, spatial characteristics of the respective object O detected by the cameras 4, 42, 43 in the environment U.

[0038] Object information such as Ol includes, for example, the absolute object coordinates xO, yO, zO (world coordinates) of object O in space, and / or a distance A between a reference point PB, for example a rear side 1a of vehicle 1 (in the case of a single-part vehicle 1) or of trailer 3 (in the case of a multi-part vehicle 1) or a current position SP of the respective camera 4, 42, 43, and the detected object O or an object point PPi on object O or a related quantity, and / or an object shape OF or an object contour OC, which is subdivided, for example, into n different object classes OKn, and / or an object dynamic OD, i.e. a temporal movement of the detected object O in space, are in question.

[0039] The object information Ol is determined according to the Structure-From-Motion (SfM) method, in which ST1, ST2, ST3 (see below) are determined in partial steps. Fig. 3 ) an object O is recorded by a camera 4 from at least two different viewpoints SP1, SP2 (see Fig. 2b Through triangulation T, depth information regarding object O, or the respective object information Ol, can be obtained in a further step ST4. How to Fig. 2b As described, image coordinates xB, yB are determined for at least one first image point BP1i in the first image B1 and for at least one second image point BP2i in the second image B2, which are each assigned to the same object point PPi (ST4.1).

[0040] To simplify the process, a certain number of pixels BP1i, BP2i in the respective image B1, B2 can be combined into a feature point MP1, MP2 (see Fig. 2a ), wherein the image points to be summarized BP1i, BP2i are chosen such that the respective feature point MP1, MP2 is assigned to a specific uniquely localizable feature M on the object O (see Fig. 2b ). The feature M can, for example, be a corner ME or an edge MK on the object O, which can be extracted from the entire images B1, B2 and whose pixels BP1i, BP2i can be summarized in the feature points MP1, MP2.

[0041] As an approximation, the object shape OF or object contour OC can be at least estimated by triangulation T from the image coordinates xB, yB of the individual image points BP1i, BP2i or the feature points MP1, MP2, which are assigned to the same object points PPi or feature M in the at least two images B1, B2. For this purpose, the image coordinates xB, yB of several image points BP1i, BP2i or several feature points MP1, MP2 can be subjected to triangulation T to obtain object coordinates xO, yO, zO, which, however, do not necessarily lie on the object O in space.

[0042] Without knowing the exact base length L, i.e., the distance between the different viewpoints SP1 and SP2 of camera 4, triangulation T yields object coordinates xO, yO, zO in unscaled form. Therefore, only an unscaled object shape OF or object contour OC can be derived from such determined object coordinates xO, yO, zO, which is sufficient for determining the shape or contour. For triangulation T, an arbitrary base length L can initially be assumed.

[0043] To enable the determination of further object information Ol through triangulation T, the actual base length L is additionally used. If according to Fig. 2b If the relative positions, and thus the base length L, between the different viewpoints SP1 and SP2 of camera 4, from which the two images B1 and B2 were taken, are known or have been determined, then the absolute object coordinates xO, yO, zO (world coordinates) of object O, object point PPi, or feature M can also be determined by triangulation T. From this, the distance A between the reference point PB and the detected object O, or an object point PPi on object O, can be determined, whereby the coordinates of the reference point PB in the world coordinates follow directly from geometric considerations.

[0044] In this way, the control unit 5 can estimate a scaled object contour OC or scaled object shape PF compared to the case above if the exact object coordinates xO, yO, zO of several object points PPi or features M are determined. From the object contour OC, the object O can then be classified into a specific object class OKn. The object dynamics OD can also be taken into account, for example, a direction of movement R of the object point(s) PPi and / or an object velocity vO, if the object points PPi are considered with temporal resolution.

[0045] For example, objects identified as people can be classified in a first object class, OK1. Objects identified as stationary, such as signs, loading ramps, houses, etc., can be classified in a second object class, OK2. Objects identified as moving, such as other vehicles, can be classified in a third object class, OK3.

[0046] To determine the object information Ol even more accurately, it may be additionally provided that more than two images B1, B2 are taken and evaluated by triangulation T as described above, and / or that a bundle adjustment BA is additionally carried out.

[0047] As already described, for the SfM method, object O must be viewed from at least two different viewpoints SP1 and SP2 from camera 4, as schematically shown in Fig. 2b As shown, in substep ST2, camera 4 is moved in a controlled manner to the different viewpoints SP1 and SP2. In the scaled case, the resulting base length L between viewpoints SP1 and SP2 is determined using odometry data DD (ST4, ST4.2). Various methods can be used for this: If the entire vehicle 1 is in motion, this already results in a movement of camera 4. This means that the vehicle 1 as a whole is set in motion actively, for example by a drive system 7, or passively, for example by a gradient. If at least two images B1 and B2 are recorded by camera 4 within a time offset dt during this movement, the base length L can be determined using odometry data DD, from which the vehicle movement and thus also the camera movement can be derived.Odometry is used to determine the two viewpoints SP1 and SP2 assigned to images B1 and B2.

[0048] Odometry data DD can be, for example, wheel speed signals SR from active and / or passive wheel speed sensors 6a, 6p on the wheels of the vehicle 1 (see. Fig. 1 ) can be used. From these, depending on the time offset dt, it can be determined how far the vehicle 1 or the camera 4 has moved between the viewpoints SP1, SP2, from which the base length L can be derived. However, it is not necessary to rely solely on vehicle odometry, i.e., the evaluation of vehicle movement based on motion sensors on vehicle 1. Visual odometry can also be used as a supplement or alternative. With visual odometry, a camera position can be continuously determined from the camera data KD of camera 4 or from information in the captured images B, B1, B2, provided that at least initially, object coordinates xO, yO, zO of a specific object point PPi are known. The odometry data DD can therefore also contain a dependency on the camera position determined in this way, since the vehicle movement between the two viewpoints SP1, SP2, or SP2 can be derived from it.The base length L can also be derived directly.

[0049] To improve the accuracy of the odometric determination of the base length L during movement of vehicle 1, additional odometry data DD available in vehicle 1 can be used. For example, a steering angle LW and / or a yaw rate G can be used to also take into account the rotational movement of vehicle 1.

[0050] In the invention, for a two- or multi-part vehicle 1, where triangulation T is to be performed using the images B recorded by the trailer camera 43, an articulation angle KW between the towing vehicle 2 and the trailer 3 is additionally used to take into account the exact dynamics of the trailer 3, particularly during shunting operations or cornering. Thus, in order to determine the individual viewpoints SP1, SP2 of the trailer camera 43 or the base length L exactly, the movement of the trailer relative to the towing vehicle 2 within the time period dt is also considered.

[0051] Preferably, the articulation angle KW is measured using an active articulation angle sensor 16, which is arranged at a coupling point 17. The two vehicle parts 2, 3 pivot around each other about this coupling point 17. In the case of a semi-trailer truck, this coupling point 17 is located, for example, at the kingpin. In the case of a drawbar trailer, it is located at the trailer coupling on the towing vehicle 2. In addition to actively measuring the articulation angle KW, it is also possible to determine the articulation angle KW from images B of a camera 4, for example, a rear-facing towing vehicle camera 42 and / or a forward-facing trailer camera 43.

[0052] A bending angle KW determined in this way or otherwise can then be taken into account in the odometry data DD in order to obtain the base length L for the triangulation T, so that the extraction of the depth information of the object O detected by the trailer camera 43 can be carried out more accurately.

[0053] If the single-part vehicle 1 or the multi-part vehicle 1 with its vehicle parts 2, 3 is not in motion, or if the motion within the time offset dt is so small that the odometry data DD is so inaccurate that a reliable determination of the base length L is not possible, the camera 4 can also be set in motion by an active actuator system 8 in sub-step ST2. The motion of the camera 4 caused by the actuator system 8 differs from the motion of the vehicle 1 considered so far, in particular in that the actuator system 8 only sets the camera 4 or a vehicle section connected to the camera 4 in motion. The motion of the vehicle 1 as a whole, or a driving state Z of the vehicle 1, is therefore not changed, so that a stationary vehicle 1 remains stationary SS when the actuator system 8 is actively controlled.

[0054] The actuator system 8 is controlled by the control unit 5 via actuator signals SA. This can occur, for example, if the control unit 5 detects that the odometry data DD, which characterizes the movement of the entire vehicle 1 (i.e., the wheel speed signals SR and / or the steering angle LW and / or the yaw rate G and / or the camera data KD and / or transmission data DG, which allows the estimation of a vehicle speed v1 from a transmission speed and a gear engaged), is not accurate or detailed enough to determine the base length L. This can be the case if the vehicle 1 has been detected as stationary SS or if the vehicle speed v1 is lower than a speed limit vt.

[0055] When the actuator system 8 is activated, the camera 4 is moved directly or indirectly to different positions SP1 and SP2, so that the environment U can be captured in at least two different images B1 and B2. This allows the SfM method to be performed as described above. To determine the base length L, the control unit 5 uses a displacement W by which the camera 4 is moved between the two positions SP1 and SP2 by the actuator system 8. The displacement W is transmitted from the actuator system 8 to the control unit 5. The control unit 5 can therefore additionally incorporate the displacement W of the actuator system 8 into the odometry data DD to determine the base length L.

[0056] Various systems within the vehicle 1 are considered as actuator systems 8, which are located in Fig. 1aThe schematic diagrams shown here are examples for a single-unit vehicle 1, but they can also be used on sub-vehicles 2, 3 of multi-unit vehicles 1. For example, the camera 4 can be arranged on a camera adjustment system 9 with one or more actuators 9a, pneumatic cylinders 9b, hydraulic cylinders 9c, electric servo cylinders 9d, or similarly acting actuators, wherein the camera adjustment system 9 is attached to the vehicle 1 such that the detection area E is aligned as desired. In this case, the camera 4 can be moved to the different positions SP1, SP2 by adjusting the actuator(s) 9a, pneumatic cylinder 9b, hydraulic cylinder 9c, or servo cylinder(s) 9d by a specific travel W when actuated.

[0057] Another possibility for an active actuator system 8 is an active air suspension system 10 (ECAS, Electronically Controlled Air Suspension), which, in a single-unit vehicle 1 or, in the case of a multi-unit vehicle 1, in a towing vehicle 2 or also in a trailer 3, uses air springs 10a designed as spring bellows to ensure that a vehicle body 11 can be adjusted at a height H relative to the vehicle axles 1b, 2b, 3b of the vehicle 1 or the towing vehicle 2 or the trailer 3, i.e., raised or lowered. For this purpose, the pressure in the air springs 10a can be specifically adjusted.This can be used to achieve optimal suspension regardless of road conditions or load conditions, to dynamically compensate for changes in axle load distribution, to avoid swaying or pitching during cornering, or to adjust the height H of the vehicle body 11 during coupling of a towing vehicle 2 to a trailer 3 as well as during loading and unloading operations, for example at a loading ramp.

[0058] If the respective camera 4, 4a, 4b is arranged on the vehicle body 11 of the vehicle 1, the towing vehicle 2, or the trailer 3, the camera 4 can be adjusted, preferably in height H, by a travel W by means of a targeted control of the active air suspension system 10 by the control unit 5, in order to position it at two different viewpoints SP1, SP2. Since the travel W is known to the active air suspension system 10 and / or can be measured, it can also be transmitted to the control unit 5 so that the control unit can take the travel W effected by the active air suspension system 10 into account in the odometry data DD in order to determine the base length L.

[0059] In this way, the control unit 5, when the vehicle 1 is stationary (SS), can instruct the respective camera 4 to adjust via the active air suspension system 10, so that the respective object information Ol can also be determined for at least one object point PPi using an SfM method via triangulation T. The control unit 5 can also specify the adjustment path W as a target value, which the active air suspension system 10 is to set by changing the pressure in the air springs 10a. However, to determine the object shape OF or the object contour OC unscaled via triangulation T, the adjustment path W (or the base length L) can also be disregarded, for example, if the adjustment path W is not measured or cannot be measured.

[0060] In addition to an active air suspension system 10, any comparable active suspension adjustment system 12 can also be used as a further active actuator system 8. This system is capable of adjusting the height H of the vehicle body 11 and thus positioning the camera 4 mounted on it at two different positions SP1 and SP2. A component adjustment system 13 is also possible as an active actuator system 8. This system can raise or lower only a part or component of the vehicle body 11, to which the camera 4 is attached, for example, a driver's cab 14, by the adjustment range W. Other possible components include aerodynamic components 15, such as aerodynamic wings or spoilers, on which a camera 4 can be mounted and which can be actively adjusted to precisely move the camera 4 by an adjustment range W.

[0061] This provides a range of possibilities for actively and selectively positioning camera 4 at different positions SP1, SP2 in order to capture two images B1, B2 of an object O and to determine the respective object information Ol (scaled or unscaled) for one or more object points PPi. In principle, the adjustment path W, which is induced via the active actuator system 8, can also be combined with the odometry data DD resulting from the vehicle movement, for example, the articulation angle KW. The active actuator system 8 can also be controlled, for example, while driving to generate an additional movement of camera 4. Reference symbol list (part of the description)

[0062] 1 Vehicle 1a Rear of vehicle 1 1b Vehicle axle 1 2 Tractor unit 2b Vehicle axle of towing vehicle 2 3 Trailer 3b Vehicle axle of trailer 3 4 Camera 42 Tractor unit camera 43 Trailer camera 5 Control unit 6a Active wheel speed sensor 6p Passive wheel speed sensor 7 Drive system 8 Active actuator system 9 Camera adjustment system 9a Actuator 9b Pneumatic cylinder 9c Hydraulic cylinder 9d Electric servo cylinder 10 Active air suspension system (ECAS) 10a Air springs 11 Vehicle body 12 Chassis adjustment system 13 Component adjustment system 14 Cab 15 Aerodynamic component 16 Active articulation angle sensor 17 Coupling point A Distance B Image B1 First image B2 Second image BA Bundle compensation BPi Image points BP1 First image point BP2i Second image point DDO Dometry data DG Gearbox data dt Time offset E Camera detection range E2 First detection range of the tractor camera E3 Second detection range of the trailer camera GG Rate of travel HH Height of the vehicle body KD Camera dataKD2 First camera data of the towing vehicle camera KD3 Second camera data of the trailer camera L Base length L Fiber optic angle M Feature MP1, MP2 Feature point M Corner (as feature) M Edge (as feature) O Object OC Object contour OD Object dynamics OF Object shape O Object information O Notable object class P Reference point P Pi Object point R Direction of movement SA Actuator signal SP Camera position SP1 First camera position SP2 Second camera position S Wheel speed signals SS Standstill T Triangulation U Environment around the vehicle v1 Vehicle speed vO Object speed vt Speed ​​limit W Adjustment travel Z Driving state

Claims

1. Method for ascertaining an object information item (Ol) relating to an object (O) in an environment (U) of a multi-part vehicle (1) made up of at least one towing vehicle (2) and at least one trailer (3), wherein at least one trailer camera (43) is arranged at least on the trailer (3), the method comprising at least the following steps: - capturing the environment (U) from a first viewpoint (SP1) using the at least one trailer camera (43) and, on the basis thereof, creating a first image (B1) consisting of first image points (BP1i) (ST1); - changing the viewpoint (SP) of the at least one trailer camera (43) (ST2); - capturing the environment (U) from a second viewpoint (SP2) using the at least one trailer camera (43) and, on the basis thereof, creating a second image (B2) consisting of second image points (BP2i) (ST3); - ascertaining an object information item (Ol) relating to an object (O) in the captured environment (U) (ST4) by - selecting at least one first image point (BP1i) in the first image (B1) and at least one second image point (BP2i) in the second image (B2), wherein the first image point (BP1i) and the second image point (BP2i) are selected in such a way that they are assigned to the same object point (PPi) of the object (O) in the captured environment (U) (ST4.1), and - determining object coordinates (xO, yO, zO) of the assigned object point (PPi) from first image coordinates (xB, yB) of the at least one first image point (BP1i) and second image coordinates (xB, yB) of the at least one second image point (BP2i) by triangulation (T) assuming a base length (L) between the two viewpoints (SP1, SP2) of the trailer camera (43), wherein the base length (L) between the two viewpoints (SP1, SP2) is ascertained on the basis of odometry data (DD) of the vehicle (1), wherein the odometry data (DD) characterize an adjustment of the trailer camera (43) between the two viewpoints (SP1, SP2), wherein the odometry data (DD) contain an articulation angle (KW) between the at least one trailer (3) and the at least one towing vehicle (2) and / or are dependent on the articulation angle (KW), in order to increase the accuracy of determining the base length (L) between the two viewpoints of the trailer camera (43).

2. Method according to claim 1, characterized in that the articulation angle (KW) is ascertained - via an active articulation angle sensor (16) at a coupling point (17) between the at least one towing vehicle (2) and the at least one trailer (3), and / or - on the basis of images (B) captured by a camera (4) on the towing vehicle (2) and / or on the trailer (3).

3. Method according to claim 1 or 2, characterized in that the odometry data (DD) are generated furthermore on the basis of an adjustment distance (W), the at least one trailer camera (43) being adjusted by the adjustment distance (W) as a result of controlling an active actuator system (8) on the trailer (8) without changing a driving state (Z) of the vehicle (1).

4. Method according to claim 3, characterized in that as an active actuator system (8) a camera adjustment system (9) having servomotors (9a) and / or pneumatic cylinders (9b) and / or hydraulic cylinders (9c) and / or electric servo cylinders (9d) is controlled, the at least one trailer camera (43) being directly attached to the camera adjustment system (9), with the result that when the camera adjustment system (9) is controlled, the at least one trailer camera (43) is adjusted by the adjustment distance (W) in order to change the viewpoint (SP) of the at least one trailer camera (43), and / or in that as an active actuator system (8) either an active pneumatic suspension system (10) with pneumatic springs (10a) or a chassis adjustment system (12) is controlled, a vehicle body (11) of the trailer (3) being adjusted in terms of its height (H) by the adjustment distance (W) as a result of controlling the active pneumatic suspension system (10) or the chassis adjustment system (12), with the result that the at least one trailer camera (43) attached to the vehicle body (11) of the trailer (3) is indirectly adjusted by the adjustment distance (W) in order to change the viewpoint (SP) of the at least one trailer camera (43), and / or in that as an active actuator system (8) a component adjustment system (13) is controlled, a component of the trailer (3), for example an aerodynamics component (15), being adjusted by the adjustment distance (W) as a result of controlling the component adjustment system (13), with the result that the at least one trailer camera (43) attached to this component is indirectly adjusted by the adjustment distance (W) in order to change the viewpoint (SP) of the at least one trailer camera (43).

5. Method according to claim 3 or 4, characterized in that the plausibility of the ascertained object information items (Ol) that result from an adjustment of the trailer camera (43) by the adjustment distance (W) by the active actuator system (8) is checked against object information items (Ol) resulting from the odometry data (DD) of the vehicle (1) which are selected from the group consisting of: wheel speed signal (SR) and / or vehicle speed (v1) and / or steering angle (LW) and / or articulation angle (KW) and / or transmission data (DG).

6. Method according to any of the preceding claims, characterized in that the object coordinates (xO, yO, zO) for a plurality of object points (PPi) are ascertained from the at least two images (B1, B2) by triangulation (T), and an object contour (OC) and / or an object shape (OF) is ascertained from the plurality of object points (PPi).

7. Method according to claim 6, characterized in that the object (O) is classified into object classes (OKi) on the basis of the object contour (OC) and / or the object shape (OF).

8. Method according to any of the preceding claims, characterized in that a plurality of trailer cameras (43) are provided and object information items (Ol) relating to an object (O) are ascertained using each trailer camera (43) independently of one another.

9. Method according to claim 8, characterized in that the plausibility of the object information item (Ol) ascertained by the plurality of trailer cameras (43) is checked.

10. Method according to any of the preceding claims, characterized in that more than two images (B) are recorded at different viewpoints (SP), and image points (BPi) assigned to the same object point (PPi) of the object (O) in the captured environment (U) are selected from each recorded image (B), object coordinates (xO, yO, zO) of the assigned object point (PPi) being ascertained from the image coordinates (xB, yB) of the selected image points (BPi) by triangulation (T) assuming a base length (L) between the respective viewpoints (SP) of the trailer camera (43).

11. Method according to any of the preceding claims, characterized in that a bundle adjustment (BA) is performed on the basis of the at least two recorded images (B1, B2).

12. Method according to any of the preceding claims, characterized in that a plurality of first image points (BP1i) in the first image (B1) are combined into a first feature point (MP1) and a plurality of second image points (BP2i) in the second image (B2) are combined to form a second feature point (MP2), the first feature point (MP1) and the second feature point (MP2) being selected in such a way that they are assigned to the same feature (M) of the object (O) in the captured environment (U), object coordinates (xO, yO, zO) of the assigned feature (M) being ascertained from first image coordinates (xB, yB) of the first feature point (MP1) and second image coordinates (xB, yB) of the second feature point (MP2) by triangulation (T) assuming a base length (L) between the two viewpoints (SP1, SP2) of the trailer camera (43).

13. Control unit (5) for performing a method according to any of the preceding claims.

14. Vehicle (1) comprising a control unit (5) according to claim 13, wherein the vehicle (1) is multi-part and has at least one towing vehicle (2) and at least one trailer (3), wherein an articulation angle (KW) is formed between the towing vehicle (2) and the trailer (3), wherein at least one trailer camera (43) is arranged on the trailer (3).