Vehicle operating device and method

Abstract

The invention relates to a device (8) for operating a vehicle (1) comprising a tractor (2) and at least one semitrailer (3) or trailer, the device (8) comprising a left camera (4.1) on the left side of the tractor (2) and a right camera (4.2) on the right side of the tractor (2), the view cones (5) of which each face in the direction opposite the direction of travel (F) and intersect the view cones (5) of the respective other camera (4.1, 4.2), wherein the baseline width (b) of the cameras (4.1, 4.2) is greater than or can be adjusted to be greater than the width (w) of the semitrailer (3) or trailer, and wherein the device (8) is configured to measure the stereoscopic measurement area (6) captured by the two cameras (4.1, 4.2) by triangulation on the basis of the image data of the cameras (4.1, 4.2), and wherein the device (8) is configured to plan a lane change of the vehicle (1), and the lane change is carried out by driving an actuator of the vehicle (1) when traffic conditions in the captured stereoscopic measurement area (6) are permitted. The invention also relates to a method for operating a vehicle (1).

Description

Technical Field

[0001] The invention relates to a vehicle operating device according to the preamble of claim 1 and a vehicle operating method according to the preamble of claim 6. Background Art

[0002] For vehicles (such as commercial vehicles) to drive autonomously, the perception and measurement of the environment is absolutely necessary. For this purpose, sensors such as lidar, cameras, radar and ultrasound are generally used.

[0003] One of the most challenging aspects of autonomous commercial vehicles is changing lanes or merging. This requires at least one rear-view camera to determine if the target lane is available for lane change. This task is made more difficult by the fact that one or more trailers and / or semitrailers of the commercial vehicle obstruct the camera’s view.

[0004] The distance behind the camera needs to be monitored depends on the traffic situation. When changing lanes in moving traffic, the area to be monitored is relatively small. When merging onto a motorway from an off-ramp at a low initial speed, on the other hand, the maximum distance of the area to be monitored is relatively large. The specific range here depends on the maximum own acceleration (which is also related to the load) and the maximum speed of other traffic participants. Commercial vehicles usually have a low acceleration capability and therefore need a relatively long time to reach the speed of moving traffic and to be able to merge into the flow of traffic without hindering or endangering other vehicles.

[0005] EP 2 555 178 B1 describes a method for detecting an object on the side of a commercial vehicle, wherein at least the following steps are performed:

[0006] - detecting objects located in a sector on one side of the commercial vehicle by means of at least one camera;

[0007] - evaluating the detected object in an evaluation unit, wherein a position of the detected object relative to the commercial vehicle is determined and a risk of collision with the commercial vehicle is evaluated;

[0008] In the event of a risk of collision, the evaluation unit transmits information to the output unit, which outputs a warning signal based on the information.

[0009] Furthermore, a commercial vehicle is described, which has a detection system for carrying out the method, wherein the detection system comprises at least one camera which can be mounted on one side of the commercial vehicle, an evaluation unit and at least one output unit. The method relieves the driver of the commercial vehicle from the burden of identifying the relevance of objects. Summary of the invention

[0010] The object of the present invention is to specify a novel vehicle operating device and a novel vehicle operating method.

[0011] According to the invention, this object is achieved by a vehicle operating device having the features of claim 1 and a vehicle operating method having the features of claim 6 .

[0012] Advantageous embodiments of the invention are the subject matter of the dependent claims.

[0013] The device according to the invention is used for operating a vehicle, in particular a commercial vehicle, which includes a tractor and at least one semitrailer or trailer, the device including a left camera on the left side of the tractor and a right camera on the right side of the tractor, the view cones / detection cones of the cameras facing in a direction opposite to the driving direction and intersecting with the view cone of the other camera, wherein the baseline width of the cameras is greater than or can be adjusted to be greater than the width of the semitrailer or trailer, wherein the device is configured to measure the stereo measurement area captured by the two cameras based on the camera image data by means of triangulation. According to the invention, the device is configured to plan a lane change of the vehicle and to implement the lane change by driving the actuator of the vehicle when the traffic conditions in the captured stereo measurement area allow the lane change. It is advantageous to select a baseline width that is as large as possible.

[0014] By placing a camera on the left and right side of the cab, optimal monitoring of the rear traffic space can be achieved, which would otherwise be invisible due to the obstruction of the view caused by the vehicle-trailer combination itself. Single-camera-based methods cannot use physical measurement principles such as triangulation for distance measurement, because they can only estimate distances through assumptions and semantic analysis (such as deep learning), which inevitably contain errors.

[0015] Commercial vehicles, especially semitrailers or trailers, can create blind spots for both cameras due to their own occlusion. Depending on the lateral distance between each camera and the semitrailer, the blind spot becomes smaller and a stereo measurement area is formed that can be seen by both cameras. This allows the stereo measurement area to be measured using stereo triangulation.

[0016] In one embodiment, the camera is fixedly mounted on the tractor or at least can be extended as required, for example by means of a mount that can be extended electrically. If the camera can be extended, the measurement result can be further improved by increasing the baseline width. This extension of the camera is also temporary, especially when a measurement is required.

[0017] In one embodiment, the camera is designed to receive light in the visible wavelength range and / or in the infrared range. The latter option is particularly advantageous at night.

[0018] In one embodiment, the baseline width is greater than 3 meters or can be adjusted to be greater than 3 meters by extending the camera. Due to the large baseline width, for example greater than 3 meters, values ​​that are particularly relevant in terms of measurement technology can be measured.

[0019] In one embodiment, at least one further sensor is provided for observing the environment behind the vehicle and is designed as a radar sensor and / or a lidar sensor, wherein the device is configured to fuse camera image data with data from the at least one further sensor and use this as a basis for planning and implementing a lane change.

[0020] According to one aspect of the present invention, a method for operating a vehicle, in particular a commercial vehicle, in particular with the aid of the above-mentioned device is proposed, wherein the vehicle has a tractor and at least one semitrailer or trailer, wherein a left camera is provided on the left side of the tractor and a right camera is provided on the right side of the tractor, the view cones of the cameras respectively facing in a direction opposite to the driving direction and intersecting with the view cone of the respective other camera, wherein the base line width of the cameras is greater than or can be adjusted to be greater than the width of the semitrailer or trailer, wherein based on the image data of the cameras, a stereo measurement area captured by the two cameras is measured by means of triangulation, wherein a lane change of the vehicle is planned and, when the traffic conditions in the captured stereo measurement area permit, the lane change is implemented by driving the vehicle actuator.

[0021] In one embodiment, the maximum overhang width of the left camera and the maximum overhang width of the right camera are determined and adjusted based on the driving conditions and the vehicle's own speed. Based on the overhang width and the known width of the semitrailer or trailer, the baseline width used for triangulation calculation is updated.

[0022] In one embodiment, the angle of deflection of the semitrailer or trailer relative to the longitudinal axis of the tractor is determined, wherein triangulation is performed if the absolute value of the angle is less than a predetermined minimum deflection angle. Otherwise, in particular, no triangulation and / or lane change is performed.

[0023] In one embodiment, in order to estimate whether the traffic situation allows a lane change, an approaching object in the stereo measurement area, its distance from the vehicle and its movement trajectory relative to the vehicle are determined. In one embodiment, the relative speed between the vehicle and the approaching object is also estimated and taken into account.

[0024] The lane change mentioned in this application may refer to the vehicle merging from the acceleration lane into the main traffic road, or the normal lane change of the vehicle on a multi-lane road.

[0025] The solution according to the present invention can be used not only for commercial vehicle trailer combinations, but also for other vehicles, such as cars, cars with trailers, pickup trucks or buses, in particular articulated buses. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The embodiments of the present invention will be explained in detail below with reference to the accompanying drawings, wherein:

[0027] Figure 1 is a detailed diagram of a commercial vehicle, including a tractor and a semi-trailer.

[0028] Figure 2 is a schematic diagram of a commercial vehicle, wherein the semitrailer and tractor are oriented in a straight line.

[0029] Figure 3 is a schematic diagram of a commercial vehicle in which the semitrailer is oriented non-linearly with the tractor.

[0030] Figure 4 is a schematic diagram of a semitrailer showing the stereoscopic measurement area and blind spots of the camera, where the semitrailer is oriented in a straight line with the tractor.

[0031] Figure 5 is a schematic curve diagram showing the relationship between the length x and the lateral distance from the camera to the semi-trailer.

[0032] Figure 6 is a schematic diagram of a semitrailer, showing the stereoscopic measurement area and blind spot of the camera, where the semitrailer is not oriented in a straight line with the tractor.

[0033] Figure 7 yes Figure 6 Another schematic diagram of the semi-trailer in this situation.

[0034] Figure 8 is a schematic diagram of a device for evaluating the traffic situation behind a commercial vehicle.

[0035] Corresponding parts are provided with the same reference symbols in all the figures. DETAILED DESCRIPTION

[0036] Figure 1 1 is a detailed schematic diagram of a vehicle 1, in particular a commercial vehicle 1, which comprises a tractor 2 and a semitrailer 3. In other embodiments, at least one trailer may be provided instead of the semitrailer 3.

[0037] A camera 4 . 1 , 4 . 2 is provided on the left and right side of the tractor 2 , respectively, whose viewing cone 5 faces in the direction opposite to the driving direction F, and thus is a rear-view camera.

[0038] The cameras 4.1, 4.2 are mounted on the tractor 2 in a fixed or extendable manner, for example by means of a fixed or telescopic mount 17.1, 17.2, respectively.

[0039] The cameras 4.1, 4.2 can be designed to receive light in the visible wavelength range and / or in the infrared range (thermal imaging camera). The latter option is advantageous at night.

[0040] The base width b of the cameras 4.1, 4.2, ie their spacing from one another, must be greater than the width w of the semitrailer 3 or trailer. It is advantageous to select the base width b to be as large as possible.

[0041] By arranging a camera 4.1 on the left side of the driver's cab and a camera 4.2 on the right side of the driver's cab, optimal monitoring of the rear traffic space is achieved, which would otherwise be blocked by the vehicle-trailer combination itself, resulting in the corresponding left-turn and / or right-turn area not being visible. Single-camera-based methods cannot use physical measurement principles such as triangulation for distance measurement, because they can only estimate distances through assumptions and semantic analysis (such as deep learning), which inevitably contain errors.

[0042] The commercial vehicle 1 (vehicle-trailer combination), in particular the semitrailer 3 or trailer, will create a blind area 7 for the two cameras 4.1, 4.2 due to its own occlusion. Depending on the respective lateral distance a between the cameras 4.1, 4.2 and the semitrailer 3, the scope of the blind area 7 will become smaller and a stereo measurement area 6 will be formed that can be seen by both cameras 4.1, 4.2. In this way, the stereo measurement area 6 can be measured by stereo triangulation. Since the baseline width b is large, for example greater than 3 meters, values ​​that are particularly important in terms of measurement technology can be determined.

[0043] In addition, if the cameras 4.1, 4.2 can be extended, the measurement effect can be further improved. The extension of the cameras 4.1, 4.2 can also be temporary, especially when measurement is required.

[0044] Figure 2 is a schematic illustration of a commercial vehicle 1 , in which a semitrailer 3 and a tractor 2 are aligned in a straight line. Figure 3 1 is a schematic diagram of a commercial vehicle 1 in which the semitrailer 3 and the longitudinal axis LA of the tractor 2 are not aligned in a straight line, ie at an angle α to each other. The angle α is not equal to zero.

[0045] Figure 4is a schematic diagram of a semitrailer 3 showing the stereo measurement area 6 and the blind area 7 of the cameras 4.1, 4.2, wherein the semitrailer 3 is oriented in a straight line with the tractor 2. The base width b is equal to the sum of the width w of the semitrailer 3 and the respective lateral distance a of the cameras 4.1, 4.2 to the semitrailer 3. The blind area 7 is caused by the obstruction of the viewing cone 5 by the semitrailer 3 and starts from the rear end of the semitrailer with respect to the driving direction F. The blind area 7 ends behind the semitrailer 3 at a length x measured from the front end of the semitrailer 3 with respect to the driving direction F. The following relationship applies to this:

[0046]

[0047] Wherein, L is the length of the semitrailer 3.

[0048] The stereo measurement region 6 starts at a length x and extends in a direction opposite to the direction of travel F.

[0049] Figure 5 2 is a schematic diagram for illustrating the relationship between the length x and the lateral distance a from the cameras 4.1, 4.2 to the semitrailer 3. At the length x2≈2x, the entire width of the lane in which the commercial vehicle 1 is traveling can be seen.

[0050] Figure 6 is a schematic diagram of a semitrailer 3 showing the stereoscopic measurement area 6 and the blind area 7 of the cameras 4.1, 4.2, wherein the semitrailer 3 and the tractor 2 are oriented non-linearly, ie at an angle ∝≠0 to each other. Figure 7 This is another schematic diagram of the semitrailer 3 in this case. The base width b is equal to the width w of the semitrailer 3 and the distance between the cameras 4.1 and 4.2 at the point P, such as the steering pin. =The sum of the respective lateral distances a of the semitrailer 3 at the point about which the semitrailer is deflected relative to the tractor 2. The blind spot 7 is caused by the obstruction of the viewing cone 5 by the semitrailer 3 and starts from the rear end of the semitrailer with respect to the driving direction F. The blind spot 7 ends behind the semitrailer 3 on the extension of the longitudinal axis LA of the tractor 2 at a length x from the front end of the semitrailer 3 with respect to the driving direction F. The following relationship applies to this:

[0051] q=a+w / 2,

[0052] E1=(L,w / 2),

[0053] E2=(L,-w / 2),

[0054] J=(x′,0),

[0055] F1=(-sin(α),cos(α))·q,

[0056] F2=(+sin(α),cos(α))·q,

[0057] x=cos(α)·x′,

[0058] Among them, q is an auxiliary parameter, E1 and E2 represent the rear angles of the semitrailer 3 or trailer, J represents the intersection point as the starting point of the stereo measurement area, F1 is the focus of the camera 4.1, F2 is the focus of the camera 4.2, and x' is the distance of the starting point of the stereo measurement area along the axis of the semitrailer 3 or trailer. The focal length of the camera is denoted by the reference symbol f.

[0059] When the angle α is large, the stereo measurement area 6 may not intersect the road segment that is desired to be monitored.

[0060] Figure 7 is a schematic diagram of a device 8 for evaluating the traffic situation behind a commercial vehicle 1 .

[0061] The device 8 comprises a left camera 4.1, a right camera 4.2 and at least one further sensor 9 which may be provided, the further sensor being used to observe the environment behind the commercial vehicle 1, for example, at least one radar sensor and / or at least one lidar sensor. The data collected by the cameras 4.1 and 4.2 are processed in a stereo image module 10, for example, by means of triangulation, to create a stereo image of the traffic situation behind the commercial vehicle 1. A fusion module 11 is optionally provided, which processes the stereo image and the data of the further sensor 9 into a fused image of the traffic situation behind the commercial vehicle 1. The fused image is provided to a behavior and planning module 13 together with the data of a digital map 12. The behavior and planning module 13 plans the lane change of the commercial vehicle 1 and controls an actuator regulator 14 which is configured to drive the actuator of the commercial vehicle 1 to implement the lane change. In addition, the behavior and planning module 13 is connected to a rearward stereo module 15 which has a calculation unit 16 for calculating the maximum overhang distance a of the left camera 4.1 based on the driving conditions and the own speed of the commercial vehicle 1. le The maximum extension distance a of the right camera 4.2 ri The calculation unit 16 calculates the extension distance a based on the le and a ri The known width w of the semitrailer 3 provides the stereo image module 10 with the current baseline width b required for calculating the stereo image. Here, the current baseline width b can be transmitted to the stereo image module 10, for example, continuously or periodically. In addition, the computing unit 16 controls the electrically extendable supports 17.1 and 17.2 of the cameras 4.1 and 4.2 to adjust the extension distance a. le and a ri .

[0062] After the measurement request has been completed, it can be provided that the electrically extendable supports 17.1, 17.2 are retracted into their original positions.

[0063] In addition, the rear stereo module 15 has an angle determination unit 18, which determines the angle α of the semitrailer 3 deflected about the point P relative to the longitudinal axis LA of the tractor 2 when the behavior and planning module 13 sends a rear telemetry request to the rear stereo module 15. If the absolute value of the angle α is less than the maximum deflection angle α max , the stereoscopic image module 10 is enabled. Otherwise, the stereoscopic image module 10 is disabled.

[0064] With the proposed solution, it is possible to measure the traffic situation behind, for example, at a distance of up to 300 meters or more. As auxiliary sensors 9, lidar and / or radar can be used in a supporting manner.

[0065] The proposed solution provides the equivalent of looking over the shoulder, i.e. looking sufficiently backwards before merging and / or changing lanes to estimate the distance to an approaching object. At the same time, the relative speed between the vehicle and the approaching object should also be estimated if possible. Radar sensors are particularly suitable for this purpose, but lidar sensors can also be used. In addition, it should be determined which lane the approaching object is travelling in and whether this is relevant for the planned lane change.

[0066] Reference numerals list

[0067] 1 Vehicle, Commercial Vehicle

[0068] 2 Tractor

[0069] 3 Semi-trailer

[0070] 4.1 Camera, left camera

[0071] 4.2 Camera, right camera

[0072] 5 viewing frustum

[0073] 6 Stereo measurement area

[0074] 7 Blind Spot

[0075] 8. Installation

[0076] 9 Another sensor

[0077] 10 Stereo Image Module

[0078] 11 Fusion Module

[0079] 12Digital Map

[0080] 13Behavior and Planning Modules

[0081] 14Actuator Regulator

[0082] 15 Rear-facing stereo modules

[0083] 16 computing units

[0084] 17.1 Support

[0085] 17.2 Support

[0086] 18 Angle determination unit

[0087] aThe lateral distance of the camera

[0088] a le Maximum outreach

[0089] α min Minimum deflection angle

[0090] a ri Maximum outreach

[0091] b Baseline width

[0092] f Focal length

[0093] E1 rear angle

[0094] E2 back angle

[0095] F Driving direction

[0096] F1 Camera 4.1 Focus

[0097] F2 Camera 4.2 Focus

[0098] J Intersection

[0099] L Length

[0100] LA longitudinal axis

[0101] Point P

[0102] w Width

[0103] x length

[0104] x′ distance

[0105] α Angle

Claims

1. A device (8) for operating a vehicle (1), comprising a tractor (2) and at least one semitrailer (3) or trailer, wherein: The device (8) comprises a left camera (4.1) on the left side of the tractor (2) and a right camera (4.2) on the right side of the tractor (2), the visual cones (5) of the cameras respectively facing the direction opposite to the driving direction (F) and intersecting with the visual cones (5) of the corresponding other cameras (4.1, 4.2), wherein the baseline width (b) of the cameras (4.1, 4.2) is greater than or can be adjusted to be greater than the width (w) of the semitrailer (3) or trailer, wherein the device (8) is configured to measure the stereo measurement area (6) captured by the two cameras (4.1, 4.2) by triangulation based on the image data of the cameras (4.1, 4.2), The invention is characterized in that the device (8) is configured to plan a lane change of the vehicle (1) and implement the lane change by driving an actuator of the vehicle (1) when the traffic conditions in the acquired stereo measurement area (6) allow the lane change.

2. The device (8) according to claim 1, characterized in that The cameras (4.1, 4.2) are mounted on the tractor (2) in a fixed manner or at least in a manner that allows them to be extended as required.

3. The device (8) according to claim 1 or 2, characterized in that The cameras (4.1, 4.2) are designed to receive light in the visible wavelength range and / or in the infrared range.

4. The device (8) according to one of the preceding claims, characterized in that The base line width (b) is greater than 3 meters or can be adjusted to be greater than 3 meters by extending the cameras (4.1, 4.2).

5. The device (8) according to one of the preceding claims, characterized in that At least one further sensor (9) designed as a radar sensor and / or a lidar sensor for observing the environment behind the vehicle (1) is provided, wherein the device (8) is configured to merge the image data of the camera (4.1, 4.2) with the data of the at least one further sensor (9) and provide a basis for planning and implementing a lane change.

6. A method for operating a vehicle (1), in particular with the aid of a device (8) according to one of the preceding claims, the vehicle comprising a tractor (2) and at least one semitrailer (3) or trailer, wherein: A left camera (4.1) is provided on the left side of the tractor (2) and a right camera (4.2) is provided on the right side of the tractor (2), the visual cones (5) of the cameras are respectively oriented in the direction opposite to the driving direction (F) and intersect with the visual cones (5) of the corresponding other cameras (4.1, 4.2), wherein the base line width (b) of the cameras (4.1, 4.2) is greater than or can be adjusted to be greater than the width (w) of the semitrailer (3) or trailer, wherein the stereo measurement area (6) captured by the two cameras (4.1, 4.2) is measured by triangulation based on the image data of the cameras (4.1, 4.2), characterized in that the lane change of the vehicle (1) is planned, and when the traffic conditions in the captured stereo measurement area (6) allow, the lane change is implemented by driving the actuator of the vehicle (1).

7. The method according to claim 6, characterized in that: The maximum extension width (a) of the left camera (4.1) le ) and the maximum outward extension width (a ri ) is determined and adjusted based on the driving conditions and the own speed of the vehicle (1), and based on the overhang width (a le , a ri ) and the known width (w) of the semitrailer (3) or trailer to update the baseline width (b) to perform triangulation calculations.

8. The method according to claim 6 or 7, characterized in that: Determine the angle (α) of deflection of the semitrailer (3) or trailer relative to the longitudinal axis (LA) of the tractor (2), wherein the angle (α) is determined when the absolute value of the angle (α) is less than a predetermined minimum deflection angle (α min ) when performing triangulation.

9. The method according to any one of claims 6 to 8, characterized in that: In order to estimate whether the traffic situation allows a lane change, an object approaching in the stereo measurement area (6), the distance of the object from the vehicle (1) and the movement trajectory of the object relative to the vehicle (1) are determined.

10. The method according to claim 9, characterized in that: The relative speed between the vehicle (1) and the approaching object is also estimated.

Images