Method for approaching a target object in the environment of a vehicle, control unit, vehicle and computer program

Abstract

The invention relates to a method for approaching a target object (7) in the environment of a vehicle (1), wherein the vehicle (1) has at least one camera (6) that captures at least a partial area of ​​the environment, comprising the steps: - Taking at least one camera image with the camera (6), wherein the target object is at least partially depicted in the camera image, - Determining a relative position of the target object (7) to the vehicle (1) and of an area (10) traversable by the vehicle (1) based on at least one camera image, - Determining a target position (12) for the vehicle (1) depending on the relative position of the target object (7) and the determined drivable area (10), - Determining a path (14) from a current position of the vehicle (1) to the target position (12), - Displaying at least one path information (15) describing the path (14) on a display device (3) and / or controlling at least one actuator of the vehicle (1) depending on the determined path (14).

Description

[0001] The invention relates to a method for approaching a target object in the environment of a vehicle, wherein the vehicle has at least one camera that captures at least a partial area of ​​the environment. The invention further relates to a control unit, a vehicle, and a computer program.

[0002] To facilitate maneuvering in tight spaces or parking maneuvers for drivers, it is known to display camera images of the vehicle's surroundings on a vehicle display system. For example, the area in front of the vehicle can be displayed on a screen on the dashboard using a front camera, or the area behind the vehicle using a rear camera. Additional support can be provided by overlaying the camera images with graphics that add further information such as driving trajectories and the like.

[0003] German patent application DE 10 2016 225 066 A1 describes a surround-view system for a vehicle. The system comprises at least one physical camera for capturing an image of the vehicle's surroundings from a non-centered position, an image processing unit, and a display unit. The display unit is configured to show a projected representation of the image captured by the physical camera and at least one geometric shape, with the geometric shape being displayed as an overlay on the image from the physical camera.

[0004] When using vehicles, such as passenger cars, there are situations where the vehicle needs to be positioned with relative precision relative to a point in its surroundings. Such a situation might occur, for example, in a parking garage when operating a ticket machine, or at a gas station when approaching a pump or charging station. The challenge here is that the vehicle's surroundings in the immediate vicinity of the point to be approached must be determined with a high degree of accuracy.

[0005] The invention is therefore based on the objective of providing a method which enables precise approach to a target object in the environment of a vehicle.

[0006] According to the invention, the method for approaching a target object in the environment of a vehicle, which has at least one camera capturing at least a partial area of ​​the environment, comprises the following steps: - Taking at least one camera image with the camera, whereby the target object is at least partially depicted in the camera image, - Determining the relative position of the target object to the vehicle and of an area traversable by the vehicle based on at least one camera image, - Determining a target position for the vehicle depending on the relative position of the target object and the determined drivable area, - Determining a path from the vehicle's current position to the target position, - Displaying at least one path information describing the path on a display device and / or controlling at least one actuator of the vehicle depending on the determined path.

[0007] The vehicle could be, for example, a passenger car, a truck, or another type of motor vehicle. Training the vehicle to function as a robot is also possible.

[0008] The vehicle includes at least one camera capable of capturing part of its surroundings. This camera can be a front camera, a side camera, a rear-view camera, and / or a combination of several such cameras. In particular, the vehicle includes multiple cameras capable of capturing its entire surroundings. These cameras can, for example, form a surround-view system capable of capturing the entire area around the vehicle.

[0009] To enable precise approach to a target object in the vehicle's vicinity, a camera on the vehicle, for example, a forward-facing front camera, captures an image in which the target object is at least partially depicted. The use of at least one additional camera, such as a reversing camera, a side camera, and / or other cameras on the vehicle, is also possible in this context, particularly if the target object is to be approached next to or behind the vehicle.

[0010] Based on the camera image, which depicts the target object completely or at least partially, the system then determines the target object's relative position to the vehicle and the area the vehicle can traverse. The relative position between the target object and the vehicle describes, for example, the vehicle's distance to the target object and / or the target object's location in world coordinates or in a vehicle-specific coordinate system.

[0011] The defined drivable area is, in particular, the area between the vehicle and the target object, as well as at least partially adjacent to or around the target object. This drivable area can be, for example, a road surface or another level surface that a vehicle can drive on. The drivable area can include low, traversable obstacles such as low curbs or the like. Higher obstacles that are not traversable, or at least should not generally be driven over, are specifically excluded from the drivable area. The drivable area thus defines the area in the vehicle's vicinity within which the vehicle can move when it is to be positioned at the target object.

[0012] The next step involves determining a target position for the vehicle based on the relative position of the target object and the identified drivable area. The target position is located entirely within this identified drivable area. It can also be situated in the immediate vicinity of the target object, although the precise location of the target position relative to the target object may vary for different types of target objects, as will be explained in more detail below.

[0013] Once the target position is defined, a path is determined from the vehicle's current position to the target position. This path can describe, at least in part, a possible route or trajectory along which the vehicle can be moved to the target position.

[0014] Depending on the determined path, at least one path-descriptive piece of information is displayed on a display device. Additionally or alternatively, at least one actuator of the vehicle can also be controlled based on the determined path.

[0015] By displaying path information on the display unit, the vehicle operator can, for example, receive guidance on how to safely steer the vehicle to its target position. This allows the driver to receive real-time feedback regarding steering and any deviations of the vehicle's current position from the determined path. Instructions for planned driving maneuvers that the vehicle can perform autonomously can also be provided in this way.

[0016] The control of the at least one actuator, depending on the determined path, can be implemented in such a way that the vehicle can follow the path autonomously or at least semi-autonomously. The at least one actuator of the vehicle can, for example, be a longitudinal guidance actuator and / or a lateral guidance actuator.

[0017] The inventive method, particularly when implemented by computer, has the advantage that by using the camera image to determine the relative position between the target object and the vehicle, as well as the drivable surface, precise approach to the target object is enabled without the need for additional vehicle sensors. Compared to the use of ultrasonic sensors, which can also be used to detect distances to surrounding objects, the use of the camera image has the advantage that the surroundings can be captured at a greater range, and a higher spatial resolution can be achieved when detecting the target object, thus also achieving greater accuracy in position determination.

[0018] Compared to the use of sensors such as radar or lidar sensors, the possibility of precisely approaching the target object using at least one camera represents a more cost-effective implementation, especially in areas next to and / or behind the vehicle, which are usually not covered by radar or lidar sensors.

[0019] Displaying path information has the advantage of making it easier for the driver to approach the target object, even in unfamiliar and / or confined spaces, especially during partially manual driving. For example, the driver can use the path information to make quicker and better decisions about how to steer the vehicle to reach the optimal position relative to the target object, without having to rely solely on their visual perception. This helps avoid errors in selecting a suitable position for interacting with, driving over, or passing through the target object. As a result, the efficiency of approaching the target position is increased, leading to time savings for the driver and potentially other road users.

[0020] Advantageously, the method according to the invention can determine optimal target positions in a multitude of situations and provide the driver with corresponding instructions for moving the vehicle to the target position. This avoids both unnecessary maneuvering and accidents or collisions while maneuvering towards the target object.

[0021] According to the invention, the target object can be a device that can be mechanically and / or electrically coupled to the vehicle, a device comprising a manually operable user interface, a device that can be driven over by the vehicle, and / or a device that can be driven through by the vehicle. A device that can be mechanically and / or electrically coupled to the vehicle could, for example, be a fuel pump or a charging station for an electric vehicle designed for wired or wireless charging. Mechanical coupling can be achieved, for example, via the hose with the fuel nozzle or via a connection to a charging cable. An electrical connection can include not only an electrical connection via a charging cable but also an inductive electrical connection in the case of an inductive charging station.

[0022] A device with a manually operated user interface could be, for example, a ticket machine from which a parking ticket or similar item can be dispensed after pressing a button, and / or a payment device, such as at a toll station or similar location. A device that a vehicle can drive over or through could be, for example, a drive-through window, a car wash, or an infrastructure object such as a passageway, an underpass, a lift in a workshop, a device on factory or company premises, or similar.

[0023] By approaching a target object more precisely, interaction with that object can be advantageously simplified, as the vehicle is positioned in such a way as to be easily refueled or charged at a fuel pump or charging station, for example. Operating ticket machines and similar devices can also be made more efficient and convenient with a suitable vehicle positioning relative to the target object.

[0024] According to the invention, the target object can be selected based on a user selection from a suggested set of possible target objects. The set of possible target objects can be determined, for example, automatically using map data and / or by evaluating at least one camera image. The set of possible target objects can be suggested to a user of the vehicle, for example, on the display device, and the driver can select a target object from the set of suggested target objects, for example, by marking the desired target object on a touchscreen, by a voice command, and / or by using a control element or the like.

[0025] In a preferred embodiment of the invention, the target object and / or the area accessible to the vehicle can be determined from the at least one camera image by means of semantic segmentation. Semantic segmentation allows specific image areas or individual pixels of the camera image to be assigned to different object classes. In this way, different types of target objects in the vehicle's surroundings, as well as accessible surfaces, inaccessible surfaces, and / or other environmental objects, such as other vehicles, infrastructure objects, etc., can be recognized in the camera image. A road surface, a parking lot surface, or the like can be identified as an accessible surface.

[0026] In a preferred embodiment of the invention, the relative position of the object can be determined from at least one camera image using a structure-from-motion algorithm, and / or the relative position of the object can be additionally determined as a function of a distance measurement obtained using a distance sensor of the vehicle. In particular, the relative position can be determined using the structure-from-motion algorithm from camera images taken at two or more successive times.

[0027] If the vehicle, in addition to at least one camera, also includes one or more distance sensors, such as ultrasonic sensors, radar sensors, lidar sensors, and / or further cameras, then at least one distance measurement from one or more of these sensors can be used to determine the relative position of the object. For example, in the case of a distance sensor designed as a camera, the distance measurement can be determined as depth information. By considering at least one distance measurement, a further improvement in the accuracy of the determined relative position can be advantageously achieved.

[0028] According to the invention, the target position can be determined depending on the type of target object and / or depending on the position of a section of the target object that can be coupled to the vehicle, a section that can be operated by a user, a section that can be driven over by the vehicle, and / or a section that can be driven through by the vehicle. By taking the type of target object into account, it can be determined, for example, which side a loading device or a ticket machine should be approached from, with the target position then being located on the side of the target object to be approached.The determined location of the target position can also take into account the position of a section of the target object that can be connected to the vehicle, such as a charging cable, a fuel nozzle, and the like, and / or a section that can be operated by a vehicle user, such as a button, a ticket input, a coin slot, and the like. The target position can be chosen, for example, so that, assuming standardized vehicle dimensions, the connectable section is easily accessible when the vehicle is parked in the target position, or so that a manually operated section can be easily reached by the vehicle user from inside the vehicle. Additionally or alternatively, the target position can be determined based on a drivable or drive-through section of the target object. In particular, the target position can be located within the drivable or drive-through section.

[0029] In a preferred embodiment of the invention, the target position can be determined based on the position of a driver's side window and / or the position of a section of the vehicle that can be mechanically and / or electrically coupled to the target object, in particular the position of a fuel filler flap, a charging socket, and / or an inductive charging device. These positions can be individually considered for each vehicle; that is, the actual positions on the specific vehicle can be used.

[0030] Advantageously, the target position can be chosen, for example, such that, with the vehicle positioned in the target position, the driver's side window is located directly opposite a manually operable section of the target object. Similarly, a mechanically and / or electrically connectable section of the vehicle can be positioned opposite a corresponding section of the target object. This allows for convenient operation of the manually operable section from inside the vehicle, as well as convenient mechanical and / or electrical coupling between the vehicle and the target object.

[0031] According to the invention, the target position and / or the position of a section of the target object can be continuously determined from a plurality of continuously recorded camera images, with the target position and / or the determined path being corrected as the target position and / or the section position is continuously determined. The continuously recorded camera images can be acquired, in particular, during movement of the vehicle along an initially determined path to an initially determined target position. By tracking the target object and / or the section of the target object in these camera images, a more precise determination of the target position can be achieved, and a corresponding correction of the initially determined target position can be made.To reach the corrected target position, the initially determined path can be further corrected, particularly based on the vehicle's current or imminent position along the initially determined path. Similarly, the target position and / or path can also be corrected by tracking a connectable or controllable section of the target object.

[0032] According to the invention, the drivable area, the target position, and / or the path can be determined as a function of one or more vehicle parameters describing the vehicle's geometry. The vehicle parameters are determined using a vehicle model and / or extrinsic camera parameters of at least one camera. By taking the vehicle parameters into account, the target position can be determined adapted to the actual width and / or length of the vehicle. The vehicle parameters can be stored as a vehicle model in a storage device of the vehicle, for example, in a storage device of a control unit designed to carry out the method according to the invention. Additionally or alternatively, the vehicle parameters can also be determined using extrinsic camera parameters, which can also be stored in the vehicle's storage device.

[0033] According to the invention, the display device can show path information including a trajectory to be followed to reach the target position, instructions to the driver regarding operating actions to be performed to follow the path, and / or at least one intermediate position to be assumed by the vehicle on the way to the target position. Additionally, the target position can also be displayed on the device. The instructions to the driver can, for example, include information for the lateral and / or longitudinal guidance of the vehicle, such as steering angles and / or directions of movement to be set, and can be output in real time depending on the vehicle's position on the determined path and / or a deviation of the vehicle's position from the determined path.

[0034] In a preferred embodiment of the invention, the determination of the target position and / or path can be based on user preference information. This user preference information is determined based on past control actions performed by the driver of the vehicle and / or based on target positions visited by the driver at the target object and / or at least one other target object, particularly one of the same type. The user preference information can, for example, be stored in the vehicle's memory and continuously updated and / or supplemented over several journeys. Advantageously, the vehicle's position at the target object and / or the determined path to the target position can thus be personalized or adapted to the preferences and / or requirements of the driver.User preference information can be obtained, for example, by evaluating various driving parameters during a movement of the vehicle by the driver using artificial intelligence methods.

[0035] According to the invention, for the control of the actuator, a user of the vehicle can first be given a notification on a display device indicating the determined control of the at least one actuator, wherein the control only takes place after an operating action has been performed, in particular after a manual operation or a voice command. The user of the vehicle can thus infer from the notification which actuator control or which vehicle movement is suggested or intended, and trigger it by performing the operating action. The control of the at least one actuator can take place within the framework of a driver assistance system for semi-autonomous or autonomous vehicle movement.

[0036] According to the invention, the display device can be a vehicle display, in particular a head-up display, a projection device, a screen and / or a touch display, and / or a mobile display, in particular a smartphone or a tablet. Using a head-up display or a projection device, the path information can, for example, be projected directly into the driver's field of vision. On a vehicle display, the path information can, for example, be superimposed on a camera image from at least one camera and / or another camera of the vehicle, so that the display is, in particular, an augmented reality display. Displaying the path information as a graphic overlay in a surround-view view and / or in a top-down view of the vehicle is also possible.

[0037] When using a mobile display device, a vehicle user can, for example, monitor and / or initiate autonomous vehicle movement from outside the vehicle. The mobile display device can also show additional information such as notifications and / or reminders related to the vehicle and / or the target position, particularly depending on the user's selections.

[0038] For a control device according to the invention, it is provided that it is configured to carry out a method according to the invention when providing at least one camera image that at least partially represents a target object.

[0039] A vehicle according to the invention is provided to comprise at least one camera capturing at least a partial area of ​​the vehicle's environment and a control device according to the invention.

[0040] A computer program according to the invention comprises instructions that cause a control device to execute a method according to the invention when at least one camera image, at least partially representing a target object, is provided. The computer program can be stored on a data carrier, in particular a non-transient data carrier, such as a CD-ROM, a hard drive, or flash memory.

[0041] All advantages and embodiments described above in relation to the method according to the invention apply accordingly to the control device, the vehicle according to the invention, and the computer program according to the invention, and vice versa. Furthermore, all advantages and embodiments described in connection with the control device, the vehicle according to the invention, or the computer program according to the invention are transferable analogously to the other respective subject matter of the invention.

[0042] Further advantages and details of the invention will become apparent from the exemplary embodiments described below and from the drawings. These are schematic representations and show: Fig. 1 an embodiment of a vehicle according to the invention, Fig. 2 a block diagram of an embodiment of a method according to the invention and Fig. 3. A top-down view of a vehicle and its surroundings.

[0043] In Fig. Figure 1 shows an embodiment of a vehicle 1. The vehicle 1 can be, for example, a motor vehicle, in particular a passenger car, a truck, or another type of commercial vehicle. It is also possible that the vehicle 1 is an unmotorized vehicle such as a trailer or that it is a combination of a towing vehicle and a trailer. The vehicle 1 can also be a rail-bound vehicle, such as a tram or the like, or a robot, for example, a robot in an automated warehouse, a delivery robot, or a support robot in the medical field.

[0044] The vehicle 1 comprises an embodiment of a driver assistance system 2, which includes one or more display devices 3 and a control unit 4. The display device 3 can, for example, be one or more screens or touchscreens arranged in the interior of the vehicle 1. The display device 3 can also be configured as a head-up display or as a projection device that can project graphic information, for example, onto the windshield. The display device 3 is connected to the control unit 4 via a communication link 5. The communication link 5 can, for example, be a point-to-point connection or a bus connection such as a CAN bus or the like.

[0045] The control unit 4 is designed to generate a representation of the environment from the camera images of one or more cameras 6 of the vehicle and can, for example, be implemented as a microcontroller, a processor, or another type of computing device, or may include such a computing device. The control unit 4 may have a storage device in which calculation instructions and / or data can be stored digitally.

[0046] The cameras 6 can capture the vehicle's surroundings in front of, behind, and / or beside the vehicle 1. For this purpose, the camera images from the cameras 6 can be transmitted to the control unit 4, for example, via the communication link 5, in particular as a video stream, and preferably at least approximately in real time. The cameras 6 can be at least partially fisheye cameras, which can have a detection range with a horizontal and / or vertical opening angle of 180° or more. The cameras 6 form, in particular, a surround-view camera system that can capture the entire area around the vehicle 1.

[0047] The environmental representation can depict, in particular, the area in front of, beside, and / or behind vehicle 1, using the image data from one of the cameras 6 or combining the image data from several cameras 6 into a single environmental representation. The environmental representation can be a perspective view from a virtual camera position, whereby the camera images can be transformed, for example, by projecting them onto a virtual projection surface into a viewpoint different from that of the actual camera 6. For example, the environmental representation can be displayed as a top view or a perspective side view of vehicle 1 and, for instance, a section of the vehicle's surroundings located in front of vehicle 1.

[0048] The control unit 4 is designed to implement an embodiment of a method for approaching a target object in the environment of a vehicle. This method can assist a user of the vehicle 1 in manually approaching a target object and / or enable autonomous and / or semi-autonomous approach to a target object.

[0049] In Fig. Figure 2 shows a block diagram of an exemplary embodiment of a method for approaching a target object. This method is described using the information in Fig. The 3 shown overhead view of vehicle 1 and target object 7 is explained.

[0050] In step S1 of the procedure, at least one camera image is captured with the front-facing camera 6, which records a portion of the vehicle's surroundings located in front of the vehicle 1. The target object 7 is at least partially depicted in the captured camera image. The target object 7 could, for example, be a ticket machine at the entrance to a parking lot. The target object 7 includes a manually operated user interface 8, which could be, for example, a button that the driver of the vehicle 1 must press to issue a parking ticket and open an associated barrier 9.

[0051] In step S2 of the procedure, the relative position of the target object 7 to the vehicle 1 and to a surface 10 accessible to the vehicle 1 is determined based on at least one camera image. The accessible surface 10 is, for example, a road surface accessible to the vehicle 1 within the area of ​​the environment captured by the camera 6; in this case, it is an accessible surface 10 located in front of the vehicle 1. The accessible surface 10 is bounded by the target object 7, the barrier 9, and several non-accessible surrounding objects 11.

[0052] The determination of the drivable area 10 from the at least one camera image can be carried out, for example, using semantic segmentation. Different object types can be assigned to different sections in the camera image. The target object 7 in the camera image can also be identified using semantic segmentation. Semantic segmentation makes it possible, for example, to recognize the barrier 9 as such in the camera image. Other surrounding objects 11, which could be infrastructure objects such as walls, hedges, other vehicles, or the like, can also be recognized in the camera image and used to determine the drivable area 10.

[0053] Furthermore, the relative position between target object 7 and vehicle 1 is determined from the camera image. For this purpose, depth information can be extracted and / or used from one or more of the camera images. This depth information can be determined, for example, using semantic segmentation, taking into account the extrinsic camera parameters. Additionally or alternatively, the relative position between target object 7 and vehicle 1 can also be determined from several continuously recorded camera images using a structure-from-motion algorithm. These continuously acquired camera images could, for example, have been recorded during a movement of vehicle 1 relative to its current position, thus providing additional information regarding the relative position between target object 7 and vehicle 1.

[0054] Additionally or alternatively, at least one distance measurement obtained by a distance sensor (not shown) on vehicle 1 can be used to determine the relative position between the target object 7 and the vehicle 1. The distance sensor could be, for example, an ultrasonic sensor, a radar sensor, a lidar sensor, or another camera on vehicle 1. The distance measurement could be, for example, a distance value determined by time-of-flight measurement or, as described above, depth information obtained by a camera. Due to the redundancy, considering a distance measurement obtained from a distance sensor allows for a more precise determination of the relative position between the target object 7 and vehicle 1.

[0055] Subsequently, in step S3 of the procedure, a target position 12 for the vehicle 1 is determined depending on the relative position of the target object 7 to the vehicle 1 and the determined drivable area 10. The target position 12 is in Fig. Figure 3 is shown with a dashed line. The determination of the target position 12 can depend on the type of target object 7. That is, when determining the target position 12, it can be taken into account that the target object 7 is a ticket counter with a manually operated user interface 8. The target position 12 can be chosen so that, assuming standard dimensions for vehicle 1, manual operation of the user interface 8 from inside vehicle 1 is simple and convenient. It is also possible to consider the position of the user interface 8 as the position of a user-operated section of the target object 7 when determining the target position 12.

[0056] When determining the drivable area 10 and / or the target position 12, one or more vehicle parameters describing the geometry of vehicle 1 can be taken into account. These vehicle parameters can be determined, for example, using a vehicle model and / or extrinsic camera parameters from camera 6. The vehicle model and / or the extrinsic camera parameters can be stored, for example, in a memory device of the control unit 4 so that they can be considered when determining the target position 12. For example, the vehicle parameters can include information on the width, length, and / or other geometric dimensions of vehicle 1.

[0057] Furthermore, the target position 12 can also be determined depending on the position of a driver-side side window 13 of the vehicle 1. The position of the side window 13 and / or further information regarding the vehicle 1 can also be stored in the memory of the control unit 4. In this specific example, the target position 12 is determined such that the user interface 8 can be conveniently operated through the open side window 13 of the vehicle 1 at the target position 12. For this purpose, the target position 12 can be selected so that the side window 13 is located directly opposite the user interface 8.

[0058] In step S4, a path 14 is determined from the current position of vehicle 1 to the target position 7. The determination of the parent path 14 can depend on one or more vehicle parameters describing the geometry of vehicle 1, as previously described for target position 12. Path 14 describes, for example, a trajectory that moves vehicle 1 from its current position to target position 12. If no possible path exists for the determined target position 12, a corresponding notification can be given to a user of vehicle 1, for example, via the display unit 3.

[0059] In step S5, at least one path information 15 describing path 14 is then displayed on the display device 3 of vehicle 1. Additionally or alternatively, at least one actuator of vehicle 1 can also be controlled depending on the determined path 14. The path information 15 can, for example, be, as described in Fig. Figure 3 shows a representation of a travel trajectory 15 corresponding to path 14. Additionally or alternatively, further path information can also be displayed, for example, instructions to the driver of vehicle 1 regarding operating actions to be performed in order to follow path 14 and / or path information describing at least one intermediate position to be occupied by vehicle 1 on the way to the target position 12.

[0060] In addition to or as an alternative to displaying the path information on a display device, an actuator of the vehicle 1 can be controlled depending on the determined path 14. The at least one actuator can be a lateral guidance actuator and / or a longitudinal guidance actuator of the vehicle 1, whereby the control of the actuator enables autonomous or semi-autonomous movement of the vehicle 1 from its current position to the target position 12. Before the actuator is controlled, a user of the vehicle 1 can first be notified of the determined, impending control of the actuator on the display device 3 or another display device. The actuator can then be controlled after an operating action, such as a manual operation or a voice command.

[0061] The path information 15 can, for example, be superimposed on the camera image from camera 6 to achieve an augmented representation of reality. It is also possible to display the path information 15 in an environment view derived from the camera image and / or other camera images from one or more of the cameras 6 of vehicle 1. In this way, the path information 15 can also be displayed in a top-down view, as shown in Fig. Figure 3 is shown as a graphic overlay that superimposes the camera images. This representation provides the driver of vehicle 1 with a simple and understandable aid for approaching the target object 7.

[0062] In addition to displaying the path information 15 on the display device 3 of vehicle 1, it is also possible for the path information 15 to be displayed on a mobile display device, such as a smartphone or tablet, to a user of vehicle 1. This allows the approach to the target object 7 to be controlled from outside of vehicle 1.

[0063] It is possible that the position of the target object 7 and / or the position of the user interface 8 can be continuously determined from a multiple of continuously recorded camera images. Depending on the continuous determination, a correction to a previously determined target position 12 and / or a previously determined position of the user interface 8 can be made while the vehicle 1 moves along a previously determined path 14. In this way, a more precise target position 12 can be established in relation to the target object 7. The correction of the path 14 can be made, in particular, based on a current position of the vehicle 1 or a position reached in the near future along the previously determined path.

[0064] To further enhance user comfort, the determination of the target position and / or path 14 can be based on user preference information. This user preference information can be determined based on past control actions performed by the driver with vehicle 1 and / or based on target positions visited by the driver at the target object 7 or at another, particularly identical, target object. The user preference information can be continuously determined during the operation of vehicle 1 over several driving cycles and, for example, stored in a memory device of the control unit 4.

[0065] In addition to the ticket counter described as an example of the design of target object 7, other designs of target object 7 are possible. For example, target object 7 could also be a device that can be mechanically or electrically coupled to the vehicle, such as a fuel pump, a charging station for inductive or wired charging, or the like. Other designs of devices comprising a manually operated user interface 8 are also possible. For example, target object 7 could be a toll station or the like. Furthermore, target object 7 could also be a device that vehicle 1 can drive onto or through, such as a car wash, a lifting platform, or the like.

[0066] Analogous to the above, the target position 12 can also be determined depending on the position of a section that can be connected to the vehicle 1, for example, a fuel nozzle, a charging cable, or the like. Furthermore, a section of the target object 7 that can be driven on by the vehicle and / or a section that can be driven through by the vehicle can also be taken into account when determining the target position 12, in particular such that the target position 12 lies within the section of the target object 7 that can be driven on or through.

[0067] Furthermore, the target position can also be determined by determining the position of a section of the vehicle 1 that can be mechanically and / or electrically coupled to the target object, for example, the position of a fuel filler flap, a charging socket and / or an inductive charging device or the like. The position of such a coupled section of the vehicle 1 can be stored in a memory device of the control unit 4 of the vehicle 1, as described above. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2016 225 066 A1

[0003]

Claims

[1] Method for approaching a target object (7) in the vicinity of a vehicle (1), wherein the vehicle (1) has at least one camera (6) capturing at least a partial area of ​​the vicinity, comprising the steps: - Taking at least one camera image with the camera (6), wherein the target object is at least partially depicted in the camera image, - Determining a relative position of the target object (7) to the vehicle (1) and of an area (10) traversable by the vehicle (1) based on at least one camera image, - Determining a target position (12) for the vehicle (1) depending on the relative position of the target object (7) and the determined drivable area (10), - Determining a path (14) from a current position of the vehicle (1) to the target position (12), - Displaying at least one path information (15) describing the path (14) on a display device (3) and / or controlling at least one actuator of the vehicle (1) depending on the determined path (14). [2] Method according to claim 1, characterized by , that the target object (7) is a device that can be mechanically and / or electrically coupled to the vehicle (1), a device comprising a manually operable user interface (8), a device that can be driven on by the vehicle (1) and / or a device that can be driven through by the vehicle (1). [3] Method according to claim 1 or 2, characterized by , that the target object (7) and / or the area (10) that can be traversed by the vehicle (1) is determined from the at least one camera image by means of semantic segmentation. [4] Method according to any of the preceding claims, characterized by, that the relative position of the target object (7) is determined from the at least one camera image using a structure-from-motion algorithm and / or that the relative position of the target object (7) is additionally determined as a function of a distance measurement value obtained using a distance sensor of the vehicle (1). [5] Method according to any of the preceding claims, characterized by , that the target position (12) is determined depending on a type of target object (7) and / or depending on the position of a section that can be coupled to the vehicle (1), a section that can be operated by a user, a section that can be driven on by the vehicle (1) and / or a section that can be driven through by the vehicle (1) of the target object (7). [6] Method according to any of the preceding claims, characterized by, that the determination of the target position (12) is carried out depending on the position of a driver's side window (13) and / or the position of a section of the vehicle (1) that can be mechanically and / or electrically coupled to the target object (7), in particular the position of a fuel filler flap, a charging socket and / or an inductive charging device. [7] Method according to any of the preceding claims, characterized by , that the target position (12) and / or the position of the or a section of the target object (7) are continuously determined from a plurality of continuously recorded camera images, whereby a correction of the determined path (14) is carried out depending on the continuous determination of the target position (12) and / or the position of the section. [8] Method according to any of the preceding claims, characterized by, that the drivable area (10), the target position (12) and / or the path (14) are determined as a function of one or more vehicle parameters describing a geometry of the vehicle (1), wherein the vehicle parameters are determined on the basis of a vehicle model and / or on the basis of extrinsic camera parameters of the at least one camera (6). [9] Method according to any of the preceding claims, characterized by , that the display device (3) shows as path information a trajectory to be travelled to reach the target position (12), instructions to a driver of the vehicle (1) regarding operating actions to be carried out in order to follow the path (14) and / or at least one intermediate position to be taken by the vehicle (1) on the way to the target position (12). [10] Method according to any of the preceding claims, characterized by, that the determination of the target position (12) and / or the path (14) is carried out depending on user preference information, wherein the user preference information is determined depending on past control actions of a driver of the vehicle (1) and / or depending on target positions (12) visited by the driver at the target object (7) and / or at least one other, in particular of the same type, target object. [11] Method according to any of the preceding claims, characterized by , that for the control of the actuator a user of the vehicle (1) is first given a hint of the determined control of the actuator on the or a display device (3), whereby the control only takes place after the performance of an operating action, in particular after a manual operating action or a voice command. [12] Method according to any of the preceding claims, characterized by, that as a display device (3) a display device of the vehicle (1), in particular a head-up display, a projection device, a screen and / or a touch display, and / or a mobile display device, in particular a smartphone or a tablet, is used. [13] Control device which is configured to perform a method according to one of the preceding claims when providing at least one camera image that at least partially represents a target object (7). [14] Vehicle comprising at least one camera (6) capturing at least a partial area of ​​the vehicle's (1) surroundings and a control device (4) according to claim 13. [15] Computer program comprising commands which cause a control device (4) to execute a method according to one of claims 1 to 13 when providing at least one camera image that at least partially represents a target object (7).

Images