Method, computer program, control unit or central computing unit and vehicle

Abstract

A method for assisting a user in parking a trailer, comprising the following steps: capturing (110) a sequence of camera images using at least one camera (221, 231) of the vehicle (220) and / or the trailer (230), wherein the camera images depict at least a part of the environment (200) located behind and / or to the side of the vehicle (220) and the trailer (230); storing (140) at least one captured camera image; generating (160) an overall view (300) of the environment (200) from a virtual perspective based on at least one current and one stored camera image; and preferably displaying (180) at least one trajectory (360, 460) for parking the trailer (230) in the overall view.

Description

[0001] The present invention relates to a method for assisting a user in parking a trailer. The invention also relates to a computer program comprising instructions which, when executed by a computer, cause the computer to perform the steps of the inventive method. Furthermore, the invention relates to a control unit or a central processing unit having a processing unit configured to perform the steps of the inventive method. Furthermore, the invention relates to a vehicle comprising a control unit or a central processing unit according to the invention. State of the art

[0002] WO 2018 / 149625 A1 discloses a display method for use in a vehicle, wherein images of a region outside the vehicle are captured and stored. Furthermore, a composite image is generated from a current image and a stored image, and at least a portion of the composite image is displayed. The composite image comprises a first region generated from the current image and a second region generated from the stored image, wherein the second region is not visible within the current image.

[0003] Document EP 2 181 898 A1 discloses a driver assistance system in which a tractor unit is connected to a trailer. A camera captures images behind the trailer. A driver assistance system transforms the images and generates images from a bird's-eye view, i.e., from a virtual perspective vertically from above the roadway. Furthermore, a predicted movement trajectory of the trailer is determined. The predicted movement trajectory is superimposed onto the generated bird's-eye view image.

[0004] Due to the length of the trailer and the resulting distance of the vehicle from the parking space, it is typically difficult for a driver to determine the correct starting point for a parking maneuver while reversing while driving forward, making parking with a trailer difficult even with a rear-facing camera image. This often results in suboptimal turning angles and / or complicated, multi-step maneuvers when parking with a trailer.

[0005] The object of the present invention is to improve a method for assisting a user in parking a trailer. Disclosure of the invention

[0006] The above object is achieved according to the invention according to independent claims 1 and 12 to 14.

[0007] The invention relates to a method for assisting a user when parking a trailer. In one step of the method, a sequence of camera images is captured by a camera, wherein the captured camera images depict the surroundings of the vehicle. The camera is arranged on the vehicle and / or the trailer. Preferably, several cameras are arranged on the vehicle and at least one camera on the trailer, so that advantageously essentially the entire surroundings of the vehicle are captured by means of the vehicle cameras and the trailer camera. For this purpose, the at least one camera has, in particular, a wide-angle lens (“fish-eye” lens). The captured camera images depict at least part of the surroundings behind and / or to the side of the vehicle and the trailer, in particular the surroundings behind and / or to the side of the trailer.In a further step of the method, at least one captured camera image is saved, whereby the camera image can be saved unprocessed, in processed and / or transformed form. Subsequently, an overall view of the surroundings from a virtual perspective is generated and displayed based on at least one current and one saved camera image. To generate the overall view, the captured camera images are transformed in particular into a view from the virtual perspective. The virtual perspective of the overall view is directed in particular vertically from above onto the roadway and represents a so-called bird's-eye view. At least parts of the saved camera images are advantageously used in the overall view to depict surrounding areas or are projected into areas of the overall view that are not depicted by any currently captured camera images.

[0008] Furthermore, at least one trajectory of the trailer and optionally one trajectory of the vehicle are preferably displayed in the overall view, wherein the trajectory of the trailer particularly represents a single-movement movement of the trailer of the vehicle. In other words, the trajectory represents a movement of the trailer without a change of direction. Advantageously, the trajectory represents a single-movement movement of the trailer of the vehicle into a perpendicular or parallel parking space at the side of the road. For example, the type of parking space present in the surrounding area, i.e., a perpendicular or parallel parking space, can be detected by user input or selection or automatically recognized based on the captured camera images.The displayed or superimposed trajectory allows the user to quickly recognize when he can begin the parking process, which would be difficult without the process due to the length of the trailer and the resulting distance to the free parking space.

[0009] It can be provided that a first and second trajectory for the movement of the trailer are displayed or superimposed simultaneously, with the first trajectory representing a single-movement movement of the trailer into a perpendicular parking space and the second trajectory representing a single-movement movement of the trailer into a parallel parking space. The first and second trajectories are superimposed or displayed, for example, one above the other and / or semi-transparently.

[0010] Preferably, the turning angle of the displayed trajectory between the alignment of the trailer's longitudinal axis at a target position, in a parking position, or in a free parking space, and the vehicle's longitudinal axis before the parking maneuver is 90°. In other words, in this embodiment, a predefined trajectory of the trailer with a fixed, predefined turning angle of 90° is displayed, with the displayed trajectory advantageously moving or continuously shifting in the display according to the movement of the vehicle. As a result, the typically preferred trajectory with the usually desired turning angle is displayed or superimposed on the user. The displayed trajectory advantageously allows the user to recognize very quickly and proactively when they can begin the parking maneuver.

[0011] In a further embodiment, at least two trajectories, each with a different turning angle between the alignment of the trailer's longitudinal axis at a target position and the vehicle's longitudinal axis, are displayed before the parking maneuver, which trajectories represent, in particular, different levels of difficulty for reversing with the vehicle and trailer. In one variant of this embodiment, in particular, the trajectory with the smallest possible single-turn turning angle and / or the trajectory with a turning angle of 90° during single-turn driving are displayed. In another variant of this embodiment, in particular, the trajectory with the specified turning angle of 90° and a predicted trajectory determined based on the vehicle's current odometry data are displayed.This design provides the user with a clear and concise visualization of the solution space of sensible or possible trajectories for parking with a trailer.

[0012] In a further embodiment, it can advantageously be provided that information about the difficulty category of the turning maneuver with the trailer is displayed along with the displayed trajectory. The difficulty category is determined, in particular, depending on the turning angle between the alignment of the trailer's longitudinal axis at a target position and the vehicle's longitudinal axis before the parking maneuver and / or depending on the necessary changes in the vehicle's steering angle to execute the displayed trajectory. This configuration clearly visualizes the difficulty levels between different trajectories for parking with the trailer.

[0013] In a further development, the difficulty category of the turning maneuver can be determined depending on at least one detected object, with the width of the detected parking space, a curb, and / or adjacent parked vehicles being recognized as objects. For example, the difficulty category increases with the decreasing width of the detected parking space and / or with adjacent parked vehicles and / or with a curb; in particular, the difficulty category is also determined depending on the height of the curb. This extension advantageously allows the difficulty level to be determined and displayed to the user in stages.

[0014] In a particularly preferred embodiment, at least one location of a parking space for the trailer is determined based on the captured and / or stored camera images, in particular by a trained machine recognition method, preferably a neural network. Alternatively or additionally, a user input for selecting a parking space in the overall view is recorded, for example using a drag & drop method. Subsequently, the location of the detected and / or selected parking space for the trailer is indicated for the user by a marker in the display of the overall view, so that the user is advantageously informed of a suitable parking space for the trailer.In addition, it can advantageously be provided that a recommended starting position for a parking maneuver and / or a possible starting area for a parking maneuver is determined based on the detected or selected parking space and / or a determined trajectory into the detected parking space and / or based on camera images of dynamic objects in the vicinity of the vehicle and / or the road width. The recommended starting position or the starting area can be determined in front of or behind the vehicle and / or in the vehicle's own lane and / or using the lane of oncoming vehicles or dynamic objects. The recommended starting position for the parking maneuver and / or the possible starting area for a parking maneuver can then be indicated to the user in the overall view by a further marker.This design provides the user with visual support when finding or selecting a parking space and, advantageously, also when approaching the starting position.

[0015] Furthermore, an automatic or semi-automatic guidance of the vehicle with the trailer to the recommended starting position or into the possible starting area for a parking maneuver of the trailer is advantageously provided, wherein, in particular, the vehicle's steering and / or the vehicle's drive are controlled. In other words, after the parking space has been detected and / or the user has selected the parking space, the starting position or starting area can be approached automatically or semi-automatically, thus increasing the probability of a successful parking maneuver.

[0016] Furthermore, in another embodiment, odometry data of the vehicle are optionally recorded, which represent the vehicle's state of motion and optionally the trailer angle between the vehicle's coupling and the longitudinal axis of the trailer or the trailer's drawbar. Subsequently, in this embodiment, the trajectory of the trailer and / or the vehicle is displayed in the overall view, optionally depending on the vehicle's odometry data and the detected position of the parking space and / or the selected position of the parking space, wherein the trajectory represents a movement of the vehicle's trailer toward the detected and / or selected parking space and / or optionally additionally the vehicle's trajectory. In other words, in this embodiment, the trajectory of the trailer and / or the vehicle can be displayed and updated in the overall view.Through this continuation, the user is advantageously shown the predicted trajectory of the trailer and / or the vehicle with the trailer based on the current vehicle status or the odometry data, such as based on the current position to the parking space, the current steering angle and / or the current vehicle speed and / or the current attachment angle between the coupling of the vehicle and the drawbar or the longitudinal axis of the trailer.

[0017] Since reversing with a trailer is relatively difficult, a further embodiment of the invention can provide for the user to have this maneuver performed remotely by an expert. For this purpose, in one step of the method, a request signal for remote control of the vehicle is first sent, in particular as soon as the recommended starting position and / or the possible starting area have been reached. Subsequently, a control signal is received which comprises control parameters for the vehicle. The control parameters can advantageously be recorded, entered or specified by a remote-controlling person and / or determined by a control program on a server device. The remote-controlled guidance of the vehicle then takes place based on the received control signal.It can be provided that the overall view, which changes during the guidance of the vehicle, and / or a video signal is sent from a camera of the vehicle and / or the trailer, and a control signal is continuously received, wherein the control signal is particularly adapted as a function of the continuously changing overall image and / or the video signal. Accordingly, in this embodiment, it can be provided that the vehicle with the trailer is automatically guided to the recommended starting position or into the possible starting area based on sensors by automatically controlling the steering and / or the drive after the detected and / or selected parking space, and then a manual remote control or automatic remote control of the vehicle is carried out to park the trailer.

[0018] The invention also relates to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method according to the invention.

[0019] The invention further relates to a control unit or a central processing unit, comprising a signal input for providing a first signal representing at least one currently captured camera image. The control unit or the central processing unit also has a signal output for outputting an output signal representing at least the generated overall view of the surroundings from the virtual perspective based on at least the current and one stored camera image, wherein at least the trajectory representing a single-movement movement of the trailer of the vehicle is advantageously displayed in the overall view. Furthermore, the control unit or the central processing unit comprises a processing unit, in particular a processor, which is configured to carry out the steps of the method according to the invention.

[0020] The invention further relates to a vehicle comprising the control unit according to the invention or the central processing unit according to the invention.

[0021] Further advantages will become apparent from the following description of embodiments with reference to the figures. Fig. 1: Flowchart of the process as a block diagram Fig. 2: View of the vehicle with trailer in the surrounding area Fig. 3: Overall view with predefined trajectory displayed Fig. 4: Overall view with the determined trajectory displayed Examples of implementation

[0022] In Fig. 1 shows a flowchart of the method for assisting a user in parking a trailer schematically as a block diagram. In a method step 110, a sequence of camera images is captured by at least one camera, wherein the captured camera images depict the surroundings of the vehicle and the camera is arranged on the vehicle and / or the trailer. The camera images depict at least part of the surroundings behind and / or to the side of the vehicle. The camera, in particular, has a wide-angle lens, so that the captured camera images or the captured current camera image are distorted during capture.The current camera image is processed in an optional step 120, for example, the color values ​​of different camera images captured with different cameras are aligned, and the captured camera image is transformed into a view from a virtual perspective vertically from above onto the roadway. Furthermore, the current camera image is preferably rectified in an optional step 120. In an optional step 130 of the method, odometry data of the vehicle are captured, which represent the vehicle's state of motion. The captured odometry data advantageously include the current steering angle of the vehicle and / or the current coupling angle between the coupling or the longitudinal axis of the vehicle and the drawbar or the longitudinal axis of the trailer, and / or the current speed of the vehicle.Furthermore, in a step 140 of the method, at least one current camera image of the captured sequence of camera images is saved. In a further optional step 150 of the method, at least one position of a parking space for the trailer is recognized based on at least one captured camera image and / or the at least one stored camera image, in particular by a trained machine recognition method, preferably a neural network. Subsequently, in step 160, an overall view of the surroundings from a virtual perspective is generated and displayed based on at least one current and one stored camera image. The generated overall view of the surroundings thus comprises the current camera image transformed into the virtual perspective and the stored camera image transformed into the virtual perspective. The virtual perspective is in particular a perspective perpendicular from above onto the roadway.In the overall view, portions of the stored camera images are used to display surrounding areas that are not captured by any current camera image. In an optional preferred embodiment of the invention, the location of the detected parking space is indicated for the user in the generated overall view in step 170. Furthermore, in the method, in step 180, at least one trajectory is displayed in the overall view. Preferably, the turning angle of the displayed trajectory between the alignment of the trailer's longitudinal axis at a target position and the vehicle's longitudinal axis before the parking maneuver is 90°.When displaying the trajectory 180, it can be provided that a first and a second trajectory are displayed simultaneously or alternately, wherein the first trajectory represents a single-movement movement of the vehicle's trailer into a perpendicular parking space and the second trajectory represents a single-movement movement of the vehicle's trailer into a parallel parking space. Furthermore, in step 180, in particular at least two trajectories, each with a different turning angle between the alignment of the trailer's longitudinal axis at a target position and the vehicle's longitudinal axis before the parking maneuver, can be displayed, wherein in particular the trajectory with the smallest possible turning angle and / or the trajectory with the turning angle of 90° is displayed when driving in one move. It can be provided that in step 180, information about the difficulty category of the turning maneuver with the trailer is displayed for the displayed trajectory.The difficulty category is determined in particular as a function of the turning angle between the alignment of the trailer's longitudinal axis at a target position and the vehicle's longitudinal axis before the parking maneuver. The difficulty category of the turning maneuver can further be determined as a function of at least one detected object, wherein the width of the detected parking space, a curb, and / or adjacent parked vehicles are detected as objects. The display 180 of the trajectory in the overall view advantageously takes place as a function of the vehicle's odometry data and the detected position of the parking space, wherein the trajectory represents a predicted or expected movement of the vehicle's trailer toward the detected parking space. In other words, the trajectory is advantageously displayed in step 180 only if reversing into the parking space is possible.In an advantageous further embodiment of the invention, in a further optional step 190, a request signal for remote control of the vehicle is transmitted, in particular to a server device. Subsequently, in optional step 191, a control signal is received, wherein the control signal is transmitted in particular by the server device. The control signal advantageously comprises control parameters for the vehicle. Subsequently, in optional step 192, remote-controlled guidance of the vehicle takes place based on the received control signal, wherein in particular a control signal is continuously transmitted, comprising the odometry data of the vehicle and / or the generated overall view and / or at least one camera image from a camera of the vehicle and / or the trailer, and a control signal is continuously received.

[0023] In Fig. 2 is a schematic plan view of vehicle 220, comprising vehicle cameras 221, with trailer 230 in the surroundings 200 on a road or roadway 210. Trailer 230 is coupled to vehicle 220, and trailer 230 has a trailer camera 231. Trailer 230 is thus detachably connected to vehicle 220. Vehicle cameras 221 and trailer camera 231 capture the surroundings 200 of vehicle 220. Vehicle 220 moves along roadway 210 in a forward direction, as indicated by arrow 222. Parking spaces are arranged next to the roadway, with the middle parking space being free. In other words, there is a free parking space 240 between the parked other vehicles 250 in one of the parking spaces, into which the trailer 230 can be reversed by the vehicle 220 and parked. The free parking space is delimited by boundary lines.The free parking space has a width of 241. The distance between the other vehicles is 242. The distance 242 is typically greater than or equal to the width 241 of the free parking space 240.

[0024] In Fig. 3 schematically illustrates the generated and displayed overall view 300. The overall view 300 is based on at least one current camera image and one stored camera image, with at least portions of the stored camera images being displayed in areas of the overall view that are not captured by any current camera image. The overall view 300 has a virtual perspective vertically from above onto the roadway 210.The overall view depicts the roadway 210 or street, the free parking space 240, and the other vehicles 250. The depiction may be graphic and / or abstracted and / or adapted, for example, by blurring and / or contour enhancement of the camera images. The parking space 240 may be color-coded, for example, and / or the other vehicles 250 may be depicted, for example, by transformed current and / or stored camera images and / or by synthetic objects loaded from memory. The vehicle 220 and the trailer 230 are advantageously depicted in the overall view 300 by overlaying loaded images, objects, or textures.In the generated and displayed overall view 300, the trajectory 360 is also displayed. In particular, the trajectory 360 is displayed continuously, continuously, or permanently as soon as a speed of the vehicle 220 is below a threshold value and / or an input from the driver or user regarding the desired parking maneuver is detected, or the parking maneuver has been automatically recognized, for example, based on navigation data. In this exemplary embodiment, the displayed trajectory 360 advantageously has a fixed or constant predefined turning angle of 90°.Due to the permanently displayed trajectory with the specified turning angle of 90°, the user or driver of the vehicle 220 can very easily recognize when he has reached a suitable starting position for the parking maneuver for parking the trailer 230 in the free parking space 240 or can begin the parking process.

[0025] In Fig. 4 is the generated and displayed overall view 300 as in Fig. 3 shown schematically. In Fig. 4 is, however, in contrast to Fig.3 In the overall view, the free parking space 240 is marked by a marking 341, which represents the parking space 240 detected by a trained machine detection method. The display of the marking 341 or identification advantageously occurs as soon as the parking space 240 has been detected. In addition, a further marking can be displayed or shown in the overall view, which represents a recommended starting position 342 for the parking process. The recommended starting position 342 is preferably determined depending on the detected and / or selected parking space and in particular based on a trajectory to the detected and / or selected parking space. The recommended starting position 342 can include the recommended starting position 342a of the vehicle and / or the recommended starting position of the trailer 342b.Additionally or alternatively, a further marker can be displayed or shown in the overall view, representing a possible starting area 343 for the parking process, from which the parking space can generally be reached by the vehicle 220 and / or the trailer 230. The trajectory 460 is preferably only displayed when the vehicle 220 has reached a suitable starting position, that is, for example, the recommended starting position 342 or a starting position in the possible starting area 343, from which the detected parking space for the trailer 230 can be reached. This configuration allows the driver or user to easily recognize when the parking process can be started. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] WO 2018 / 149625 A1

[0002] EP 2 181 898 A1

[0003]

Claims

[1] A method for assisting a user in parking a trailer, comprising the following steps: • capturing (110) a sequence of camera images by means of at least one camera (221, 231) of the vehicle (220) and / or the trailer (230), wherein the camera images depict at least part of the surroundings (200) located behind and / or to the side of the vehicle (220) and the trailer (230), • storing (140) at least one captured camera image, and • Display (160) of an overall view (300) of the environment (200) from a virtual perspective based on at least one current and one stored camera image. [2] The method according to claim 1, wherein the following step is additionally carried out • Display (180) of at least one trajectory (360, 460) for parking the trailer (230) in the overall view. [3] Method according to claim 2, wherein a first and second trajectory (360, 460) are displayed simultaneously, wherein the first trajectory represents a single-movement movement of the trailer (230) of the vehicle (220) into a transverse parking space (240) and the second trajectory represents a single-movement movement of the trailer (230) of the vehicle (220) into a parallel parking space. [4] Method according to claim 2, wherein the turning angle of the displayed trajectory between the orientation of the longitudinal axis of the trailer (230) in the parking position and the longitudinal axis of the vehicle (220) before the parking operation is 90°. [5] Method according to one of claims 2 to 4, wherein at least two trajectories (360) each having a different turning angle between the alignment of the longitudinal axis of the trailer (230) at a target position and the longitudinal axis of the vehicle (220) are displayed before the parking operation. [6] Method according to one of claims 2 to 5, wherein information about the difficulty category of the turning maneuver with the trailer is displayed for the displayed trajectory (360, 460), wherein the difficulty category is determined in particular as a function of the turning angle between the alignment of the longitudinal axis of the trailer (230) at a target position and the longitudinal axis of the vehicle (220) before the parking maneuver. [7] Method according to claim 6, wherein the difficulty category of the turning operation is determined as a function of detected objects, wherein the width (221) of the detected parking space, a curb and / or a distance (242) between adjacent parked vehicles are detected as objects. [8] Method according to one of the preceding claims, comprising the following steps • Detection (150) of at least one position of a parking space (240) for the trailer (230) based on the captured and / or stored camera images, in particular by means of a trained machine recognition method, preferably a neural network, and / or • Recording a user input to select the location of a parking space (240) for the trailer (230) in the displayed overall view (300), and • Display (160) of the overall view (300), wherein the position of the detected parking space (240) is marked by a marker (341) and additionally in particular a recommended starting position (342) for a parking maneuver and / or in particular a possible starting area (343) for a parking maneuver for the user in the overall view by at least one further marker, wherein the recommended starting position (342) for a parking maneuver and / or the possible starting area (343) for a parking maneuver are determined based on the detected and / or selected parking space (240) and / or a determined trajectory into the detected and / or selected parking space (240) and / or detected dynamic objects in the surroundings of the vehicle (220) and / or the roadway width. [9] The method according to claim 8, wherein the following step is performed • automatic or semi-automatic guidance of the vehicle (220) and / or the trailer (230) to the recommended starting position (342) or into the possible starting area (343) for a parking operation of the trailer (230), wherein in particular the steering of the vehicle (220) and / or the drive of the vehicle (220) are controlled. [10] A method according to any one of claims 2 to 7 and claim 8 or 9, wherein the following step is carried out • Optionally, acquisition (130) of odometry data of the vehicle (220), whereby • the display (180) of the trajectory (460) in the overall view (300) optionally takes place depending on the odometry data of the vehicle (220) and based on the detected or selected position of the parking space (240), wherein the trajectory (460) represents a predicted or recommended movement of the trailer (230) towards the detected or selected parking space (240). [11] Method according to one of the preceding claims, wherein the following step is carried out • Sending a request signal for remote control of the vehicle (220), in particular as soon as the recommended starting position and / or the possible starting area have been reached, and • Receiving a control signal comprising control parameters for the vehicle (220), and • remote-controlled guidance of the vehicle (220) based on the received control signal for parking the trailer (230). [12] A computer program comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the method according to any one of the preceding claims 1 to 11. [13] Control unit or central processing unit comprising at least the following components • a signal input for providing a first signal which represents at least one currently captured camera image, • a signal output for outputting an output signal which represents at least one generated overall view (300) of the environment (200) from a virtual perspective based on at least the current and one stored camera image, wherein in the overall view, in particular, at least one trajectory (360, 460) is displayed, which represents a single-movement movement of the trailer (230) of the vehicle (220), and • a computing unit, in particular a processor, which is configured to carry out the steps of the method according to one of claims 1 to 11. [14] Vehicle (220) comprising a control unit or a central processing unit according to claim 12.

Images