Procedures to assist the driver of a motor vehicle

Abstract

Method for assisting the driver of a motor vehicle in carrying out a parking maneuver, in which, during a decoupling phase in which at least one control element of the motor vehicle, which in normal operation of the motor vehicle serves to control at least one drive system of the motor vehicle, is decoupled from the at least one drive system, an optical representation of a position of the motor vehicle relative to its environment is changed depending on operating actions which are exerted on the at least one control element, without changing the physical position of the motor vehicle in the environment, characterized by that during the decoupling phase, operating actions performed on at least one control element are detected by sensors, that, depending on the sensor-detected operating actions, control signals are generated to control at least one drive system, that the generated control signals are temporarily stored and that the temporarily stored control signals are used in an execution phase that lies after the decoupling phase for carrying out an at least partially automatic parking process of the motor vehicle.

Description

[0001] The invention relates to a method for assisting the driver of a motor vehicle in carrying out a driving maneuver.

[0002] Such a method is described in the applicant's German patent application with publication number DE 10 2006 026 092 A1.

[0003] Document DE 103 09 934 A1 discloses a driving simulator with a simulation unit comprising at least a central control unit, a vision simulator and an interface via which the simulation unit is connected to a control unit in the operating state, wherein the connection of the simulation unit to the control unit is a bidirectional communication link and the control unit is a component of a motor vehicle.

[0004] Document DE 601 05 684 T2 discloses a method for providing support during a vehicle driving process by displaying an image depicting environmental conditions around the vehicle on a display unit, the method comprising: a first step that allows a user to specify an end position on the image on the display unit, which is a position of the vehicle at one end of a predetermined driving process, and a second step of obtaining, for the end position, a starting position, which is a position of the vehicle at one beginning of the predetermined driving process, according to a prescribed movement pattern that represents a movement of the vehicle in the predetermined driving process.

[0005] The object of the present invention is to advantageously further develop or modify a generic method.

[0006] This problem is solved by a method having the features of claim 1. Advantageous embodiments and further developments of the present invention are set out in the dependent claims.

[0007] The invention is described in this document using a parking maneuver as an example. However, the inventive concept can readily be applied to other driving maneuvers with a defined target position – for example, exiting a garage and reaching a position suitable for continuing the journey, or even to a turning maneuver. For the sake of conciseness and to facilitate clear examples, the following text generally refers to a parking maneuver.

[0008] A key difference between the invention and known methods is that the parking process can first be tested virtually by the operator.

[0009] The testing takes place in a decoupling phase, in which at least one control element (e.g., steering wheel, accelerator pedal) of the motor vehicle, which in normal operation serves to control at least one drive system (e.g., steering, longitudinal drive) of the motor vehicle, is decoupled from that drive system. During this decoupling phase, no interventions are made to the drive systems. According to the invention, in this decoupling phase, depending on operating actions performed on the at least one control element, an optical representation of the motor vehicle's position relative to its environment is changed, without altering the motor vehicle's physical position in the environment.In particular, the optical representation, which is modified according to the invention, can consist of a (photorealistic or stylized) virtual image of the motor vehicle, which is moved within an image of the motor vehicle's surroundings or even directly within the surroundings. The optically represented "position of the motor vehicle with respect to its surroundings" is not the current physical position of the motor vehicle, but a potential or planned future position of the motor vehicle.

[0010] The operator thus has the option of virtually testing the parking procedure. At the end of the test, the visual display preferably marks – assuming successful testing – a parking position intended by the driver for the vehicle. The visual display of the vehicle's position relative to its surroundings can, for example, be shown on a display unit inside the vehicle. However, the visual display of the vehicle's position relative to its surroundings can also be achieved by generating at least one projection object in the vehicle's vicinity. A method for generating a projection object in the vehicle's vicinity using projection means provided on the vehicle is described below.

[0011] Preferably, the optically displayed position of the vehicle changes in the same way as the actual position of the vehicle relative to its real environment would change during normal operation of the vehicle with the same operating actions. A physical, and in particular at least a kinematic, model of the vehicle can be used to determine the optically displayed position of the vehicle as a function of the operating actions. Preferably, control signals relating to the control of the drive elements are initially generated as input variables to the model, depending on the results of sensor-based detection of the operating actions. Instead of control signals relating to the control of the drive elements, separate model control signals can also be generated that correspond to such control signals at the model level.

[0012] Preferably, the optical representation of the position of the motor vehicle in relation to its surroundings is achieved by displaying an image of the motor vehicle at a corresponding position within an image of the surroundings of the motor vehicle (e.g., when displayed on an on-board monitor) or by displaying an image of the motor vehicle in the real environment (e.g., when generating a projection object in the surroundings of the motor vehicle).

[0013] A key aspect of the invention is that decoupling allows the driver to use the vehicle's controls (e.g., steering wheel, accelerator pedal), which in normal operation control the vehicle's drive systems (e.g., steering, longitudinal drive), for virtual testing or to control the virtual testing itself. This enables particularly meaningful testing, as a successful test run in this way not only reveals the theoretical feasibility of the parking maneuver but also whether the driver is actually capable of effectively controlling the vehicle using the controls.Furthermore, if the driver is subsequently to participate in the parking maneuver, they can effectively practice the precise operating actions they will need to master in a later execution phase during the decoupling phase. These advantages would be lost if the virtual test run were primarily controlled by another operating element, such as a joystick.

[0014] The decoupling phase can preferably be initiated by the driver of the motor vehicle through a controlled action. The decoupling of control elements and drive elements required for the invention is relatively easy to implement in modern and future motor vehicles using so-called X-by-Wire technologies. In normal operation, there is then only an electronic coupling, which can be easily removed by changing the signal.

[0015] In the simplest case, normal operation resumes after the decoupling phase. This transition can occur automatically (e.g., if the success of the virtual parking maneuver is automatically detected) or upon a driver input. The driver can then park the vehicle in normal operation, thereby applying the training effect of the previous test. If the virtual test of the parking maneuver reveals that it is not feasible, or only very difficult and / or risky, the driver can simply continue driving without parking at that location.

[0016] According to a preferred embodiment of the invention, during the decoupling phase, operating actions performed on the at least one control element are detected by sensors, and control signals for controlling the at least one drive system are generated based on these sensor-detected actions. The generated control signals are temporarily stored, and these stored signals are used in an execution phase following the decoupling phase to carry out an at least partially automatic parking maneuver of the vehicle. Preferably, the control signals generated and temporarily stored in this embodiment of the invention are directly suitable for controlling the at least one drive system. They can then be used directly for controlling the at least one drive system during the execution phase.In principle, the generation and temporary storage of control signals can take place with any type of control signal and at any level of abstraction. The crucial point is that the generated and temporarily stored control signals can later be used to generate control signals that can directly control at least one drive system.

[0017] In such an embodiment of the invention, the parking maneuver, previously performed virtually without risk, can, if successful, actually be physically executed in the execution phase. The driver has thus not only performed the virtual parking maneuver in the decoupling phase for testing purposes, but their operating actions during the decoupling phase actually affect the real execution of the parking maneuver in the execution phase. The execution in the execution phase can be fully or partially automatic.

[0018] The driver can also be guided simply by receiving operating instructions to repeat their actions during the execution phase, just as they proved successful during the decoupling phase. These instructions can relate, in particular, to the timing and extent of a steering input. The mere issuance of operating instructions can be considered the lowest level of automation for a semi-automated parking maneuver.

[0019] In its simplest form, a fully automated parking process can be implemented by passing on control signals, recorded during the decoupling phase and directly suitable for controlling all relevant drive systems of the vehicle, unchanged and in sync with the timing of these signals during the execution phase. Alternatively, the recorded signals or the resulting trajectory can be processed, for example, by smoothing. With appropriate processing of the recorded control signals, it may even be possible to achieve a target position during virtual testing that previously required multiple maneuvers (multiple maneuvers) and subsequent corrections, using fewer maneuvers.

[0020] However, any post-processing or optimization of the control signals should preferably only be carried out if it is known that a potentially modified trajectory can still be traversed without collisions or hazards. Alternatively, known obstacles can be incorporated into the planning of a post-processed or optimized trajectory.

[0021] Preferably, the driver can also initiate a virtual re-trajectory, in whole or in part, by means of a corresponding control action. This is particularly relevant if the trajectory has been smoothed and / or post-processed. For this purpose, an image used for visual representation in accordance with the invention can be moved along the trajectory through the environment. Preferably, the driver can influence the speed and / or direction of the re-trajectory by means of further control actions (cf. jog shuttle on a video recorder). In particular, if the driver is to actively participate in the subsequent physical parking maneuver, they can use such a simulation to study the necessary maneuvers.

[0022] Preferably, a fully or partially automatic execution in the execution phase, as described above in the preferred embodiment of the invention, is triggered by a confirmation action from the driver.

[0023] According to another preferred embodiment of the invention, during the decoupling phase, operating actions applied to the at least one control element are also detected by sensors, and control signals for controlling the at least one drive system are generated depending on the sensor-detected operating actions. These control signals can also be of any type and at any level of abstraction. However, they must be suitable to serve as the basis for a model-based determination of the vehicle's motion data. The model-based determination of the motion data is preferably based on a physical, in particular kinematic, model of the vehicle. The motion data can, for example, describe or include a speed profile or a trajectory of the vehicle.Preferably, the motion data is also used to determine the position of the vehicle, which is to be displayed visually, as already mentioned. This visual representation of the position can then be based on the same physical model of the vehicle. In this embodiment of the invention, not only control signals are temporarily stored, but the calculated motion data is also temporarily stored, and this temporarily stored motion data is used in an execution phase following the decoupling phase for carrying out an at least partially automatic parking procedure of the vehicle.During the at least partially automated parking process, the movement data can then serve, for example, as target values ​​for a control procedure in which suitable control signals are generated depending on the target values ​​and current physical movement data in order to move the vehicle according to the temporarily stored movement data.

[0024] The cached control signals or movement data can be stored permanently for later repetition (e.g., daily complex entry into a garage from a defined starting position or exit from a garage). The cached control signals or movement data can also be stored for other logging purposes (e.g., accident reconstruction).

[0025] According to an advantageous embodiment of the invention, a trajectory of the motor vehicle is determined based on control signals generated during the decoupling phase. Furthermore, a trajectory of at least one other road user is determined from communication data received by the motor vehicle's communication means, and the trajectory of the motor vehicle is checked for collision-free operation with the trajectory of the at least one other road user before the execution phase. Preferably, fully or partially automatic execution in the execution phase is only enabled if the trajectories are collision-free. Of course, "safety margins" can also be taken into account. In this way, effective accident prevention can be achieved. A warning can also be issued to the driver in the event of an impending collision.

[0026] The determination of the motor vehicle's trajectory, required for such a further development of the invention, can be carried out based on buffered control signals and / or motion data. The phrase "based on the control signals generated in the decoupling phase" encompasses both cases, since any buffered motion data is also based on previously generated control signals. If motion data is buffered and represents or comprises a trajectory, the determination step can be performed very simply by adopting or using the motion data. Communication data from other road users can be received, for example, directly or indirectly from the other road users (e.g., via so-called car-to-car communication) or via a central data service.

[0027] According to another embodiment of the invention, a trajectory of the motor vehicle is also determined based on the control signals that may be generated during the decoupling phase. The trajectory of the motor vehicle is displayed visually to the driver before the execution phase, and the driver is given the opportunity to modify the trajectory completely or partially.

[0028] The same display means used during the decoupling phase to visually represent the vehicle's position relative to its surroundings can be used to visually display the trajectory. A separate control element can be used to change the trajectory. Alternatively, the change can be achieved by restarting the generation of control signals from the beginning or from a trajectory point selectable by the driver using at least one decoupled control element, and then converting the newly generated control signals into a new trajectory. The modified trajectory can then serve as the basis for a fully or partially automated parking maneuver.

[0029] According to yet another embodiment of the invention, during the execution phase of a potentially at least partially automatic parking maneuver, initial control signals for controlling the at least one drive system are automatically generated. Furthermore, during the execution phase, driver actions applied to the at least one control element are sensor-detected, and depending on these sensor-detected actions, second control signals for controlling the at least one drive system are generated. These second control signals are at least partially prioritized over the first control signals when controlling the at least one drive system. This prioritization allows the driver to override the automatic execution of a fully or partially automatic parking maneuver during the execution phase.This allows the driver to react to potentially changed boundary conditions or planning errors.

[0030] As already mentioned, the optical representation of the position of the motor vehicle in relation to its surroundings can be achieved in the method according to the invention by generating at least one projection object in the surroundings of the motor vehicle.

[0031] The following considers the case where the position of the motor vehicle relative to its surroundings is represented by generating a target object at the respective position – hereinafter referred to as the target position – as a projection object using projection means provided on the motor vehicle. The projection-based generation of the target object in the surroundings then serves as the optical representation according to the invention of the position of the motor vehicle relative to its surroundings.

[0032] The following explanations extend in part beyond the application case focused on by the invention, namely supporting the driver during a driving maneuver. However, the facts relevant to the present invention are readily apparent.

[0033] Displaying the target object as a projection object outside the vehicle offers an advantageous alternative in many cases to other display methods, such as projection onto the windshield in a so-called head-up display (HUD).

[0034] Firstly, a projection at a projection point outside a vehicle can be designed so that it is also visible to other road users. This allows other road users to be informed about an upcoming parking maneuver and the potential space requirements of the projecting vehicle. The intended parking space can also be clearly reserved for other road users by means of the projected image.

[0035] Another advantage of generating the target object as a projection object directly at the target position, compared to other display methods, is that it can potentially achieve a positionally accurate representation (relative to elements of the real vehicle environment) from the viewer's perspective in a simpler and / or improved manner. This is made possible by generating projection objects outside the vehicle, as the result of the projection into the vehicle environment does not depend, or at least not to the same extent as, for example, a projection onto the windshield, on the viewer's eye position and / or viewing angle. Therefore, for the realization of the target object's representation by projection into the vehicle environment, means of detecting the viewer's eye position and / or viewing angle are generally not required.For example, augmentation methods are known in connection with night vision systems and devices for the highlighted display of traffic signs, in which an object located outside the vehicle is visually highlighted for the driver of a motor vehicle by projecting optical information onto the windshield of the motor vehicle in such a way that it appears to the driver as an arrow pointing at the object or as a border around the object. However, if the driver moves their head to the side and this movement is not taken into account by the respective system, the projected information does not appear correctly positioned in the augmented overall image from the driver's perspective. If a corresponding optical information (here, for example, an arrow or a circle) is displayed, the driver's head will be misaligned.If, however, an ellipse is generated as a projection object in the vicinity of the motor vehicle with corresponding optical distortion, the overall image presented to the driver of the motor vehicle is largely independent of his eye position within the motor vehicle.

[0036] Compared to displaying the target object on an on-board monitor or a similar display unit, generating the target object as a projection in the vehicle's surroundings has the advantage that the operator does not have to take their gaze away from the surroundings and onto the on-board monitor. Instead, they see the target object directly at its target position.

[0037] The generation of projection objects by means of projection devices provided on a motor vehicle can be advantageous in many operating situations. The following explanations also apply, and in particular apply, to projection objects that serve as target objects or auxiliary objects in the preferred embodiment of the method according to the invention, which provides for the generation of a target object by projection into the vehicle's surroundings. Auxiliary objects in such a method include, in particular, steering arrows to clarify possible directions of travel of the target object or warning symbols to indicate a risk of collision.

[0038] By generating at least one projection object using projection devices provided on a motor vehicle, at least one anticipated future location of the vehicle can be marked, in particular, at least an area can be marked. An area marking, in particular, allows the observer to clearly and easily recognize the consequences of the vehicle's anticipated future location. An area marking is understood to be any marking that defines a specific surface area—especially on the roadway, but alternatively also on another projection surface or suspended in free space—or a specific area in three-dimensional space. The marking can consist of the respective surface area or area in space being visually filled with a solid image or with a pattern.Alternatively or additionally, the respective area or region in space can also be defined by a visual outline or in some other way. In particular, at least a flat marking allows an observer in a vehicle to assess whether a parking space is actually large enough for the vehicle and whether it can be reached without collision. Simply displaying, for example, a linear trajectory of the vehicle's center of gravity or an arrow indicating a direction of movement would, in many situations, provide only limited information. A flat marking can therefore be particularly effective in indicating the space required by the vehicle during an upcoming maneuver. Preferably, the target object is thus defined as a flat area.

[0039] A projection object can be any light phenomenon occurring outside the motor vehicle that is caused by the projection means provided on the motor vehicle. Conventional vehicle headlights and lights that primarily serve to illuminate the vehicle's surroundings (e.g., low beam and high beam) or that serve to improve the vehicle's visibility without specifically directing light onto a projection point in the vehicle's vicinity (e.g., taillights) are not considered projection means.

[0040] The projection can be configured so that a generated projection object—especially the target object and / or certain auxiliary objects—is visible only to the occupants of the projecting vehicle, particularly the driver, while remaining invisible or at least difficult to detect for other road users. This accommodates the driver's potential need to keep their planning—provided at least one projection object involves such planning—and / or information displayed in their vehicle private from other road users. It can also prevent other road users from being disturbed or confused by projections. A projection visible only to the driver can be achieved, for example, through the use of infrared projection techniques.Other projection techniques are also conceivable, such as the use of ultraviolet or polarized light. The driver and, if applicable, other occupants of the projecting vehicle can then be enabled, with the aid of optical devices, to see projected objects that are essentially invisible to other road users. These optical devices could include, for example, special filters, preferably mounted on the vehicle's windshield, or a camera with a downstream image display unit to reproduce the potentially processed camera image. The optical devices must be appropriately matched to the specific projection technique used—for example, an infrared camera can be used to make projected objects visible that are generated by infrared projection.

[0041] According to a particularly preferred embodiment of the invention, in addition to the target object, at least one auxiliary object serving to inform the driver is generated by means of the projection means provided on the motor vehicle, and the generation of the at least one auxiliary object is such that the at least one auxiliary object is essentially invisible to other road users without optical aids. For the driver of the projecting motor vehicle, however, the at least one auxiliary object, which is essentially invisible to other road users, is made visible by means of optical aids.

[0042] The cases of difficult detection and complete invisibility are summarized in this document by the phrase "essentially invisible." Unless explicitly stated otherwise, the term "invisible" is used throughout to mean "essentially invisible." When visibility is mentioned "exclusively" for the driver, it means that the information in question is "essentially invisible" to persons outside the vehicle. For the sake of completeness, it should be noted that invisibility in this context does not refer to the obscuration of an observer's view of the projected object. It refers, for example, to the invisibility of a light phenomenon due to the wavelength or polarity of the light contributing to the phenomenon.

[0043] The projection can be designed so that the generated projection objects are visible to other road users as well, or even exclusively. If the projection is designed to be visible to other road users, it can, for example, visually mark and thus reserve a required space or area in the vehicle's vicinity – such as a parking space.

[0044] Advantageously, the projection can also be designed to generate multiple projection objects, with only some of them visible to other road users, while others remain invisible. For the driver of the projecting vehicle, however, all generated projection objects are preferably visible. For example, essential core information (e.g., the target object) can be visible to both the driver and other road users, whereas supplementary information (e.g., maneuvering instructions for the driver) or auxiliary objects (e.g., control arrows indicating an operating option) are reserved for the driver only.

[0045] A projection object invisible to other road users can consist of only one object part. If this object part is invisible, the entire projection object is invisible. Likewise, a single projection object can, of course, comprise multiple object parts, with at least one object part being visible to other road users, while at least one other object part remains invisible to them. For example, a projection object can consist of a symbolic object part and a label as a further object part. The symbolic object part can then be displayed visibly to all road users, while the label is displayed only to the driver of the projecting vehicle.

[0046] The different visibility of different projected objects or object parts can be achieved in the simplest case by using different projection methods. Projected objects or object parts visible without aids are then generated by the first set of projection methods, while projected objects or object parts visible only with aids (and otherwise invisible) are generated by the second set of aids.

[0047] It is also possible to provide a control mechanism by which the driver can influence whether a projected object or a part thereof is visible to other road users. Preferably, therefore, it is possible to control whether a projected object or a part thereof is visible to other road users by means of a control mechanism provided in the motor vehicle.

[0048] A projection that appears three-dimensional to the viewer is particularly advantageous. This can be achieved, for example, in a well-known way, through a projection that is essentially two-dimensional but perceived three-dimensionally by the viewer due to perspective elements. A three-dimensional representation—especially for occupants, such as the driver, of a vehicle equipped with the projection equipment—can also be achieved through various 3D projection techniques. These techniques use technical aids (e.g., shutter glasses, red-green filters, polarization filters, etc.) to present slightly different images—for example, images that are slightly offset due to perspective—to each of the viewer's eyes, thus creating the effect of spatial perception. For improved usability, the installation of such stereoscopic projection aids is recommended.Stereoscopic vision aids are proposed for the windshield of the vehicle, or for their integration into the windshield. This is particularly feasible using one or more polarizing filters.

[0049] Projection objects are typically created by shining light onto the surface of a solid (i.e., not liquid or gaseous) body. The body's surface then serves as the projection surface. However, under certain circumstances and with the aid of suitable techniques, a projection object can also be created at a projection point "in free space"—that is, not on the surface of a solid, but, for example, "suspended in the air."

[0050] Such a projection object, generated "in free space," can, for example, be essentially two-dimensional or at least appear two-dimensional to the viewer. A fog screen, for instance, can be used as a projection surface. To achieve a three-dimensional appearance, the application of techniques that actually allow the creation of a projection object that is inherently three-dimensional is particularly advantageous. With a projection that is inherently three-dimensional, a three-dimensional representation can be presented at the same projection point to both a vehicle occupant and an observer outside the vehicle—such as another road user—because the techniques described below allow for the realization of representations whose three-dimensionality is almost independent of the viewer's perspective.Such a three-dimensional projection is particularly advantageous because of its realistic spatial appearance, which can result in high viewer attention and effective information transfer. Since such a three-dimensional projection typically protrudes spatially from or "floats" above the road surface, it is especially easy to see and is unlikely to be obscured. A three-dimensional representation of the target object is therefore particularly beneficial.

[0051] Such a three-dimensional projection is technically feasible, for example, through holographic projection, using coherent light sources (lasers). The coherent light sources necessary for holographic projection can be generated in a manner known from the prior art, for example, by splitting a laser beam using a semi-transparent mirror.

[0052] If necessary, a specially designed medium can be created at the projection point, onto which the image is projected. Alternatively or additionally, a boundary between two media with different optical properties can be created at the projection point. The boundary between the two media can then have suitable optical properties to generate a projection object.

[0053] According to a particularly advantageous embodiment of the invention, the target object is generated by performing a holographic projection onto one or more ultrasonic wavefronts. Preferably, the spatial arrangement of the ultrasonic wavefronts is controllable. An inhomogeneity in the wave-optical properties of a medium can be created using standing ultrasonic waves specifically generated for this purpose or a spatial arrangement of such standing ultrasonic waves. This inhomogeneity allows the coherent light sources used for the holographic projection to form a defraction pattern at predetermined locations in space. These locations in space thus behave like light sources in space (cf. the physical effect of defraction), thereby creating the appearance of a three-dimensional object floating in space from the perspective of an observer.A suitable ultrasonic wavefront can be created, for example, using coherent and synchronized ultrasonic transmitters. Such transmitters can generate standing waves in a medium (for example, water or—more relevant for this application—air), which may also be spatially displaceable. Because the air pressure is unevenly distributed at the relevant locations in space due to the waves, certain changes occur in the optical properties of these locations, in particular minima and maxima form with respect to specific optical properties. The resulting inhomogeneity can be used as a medium, figuratively speaking, for the formation of a defraction pattern, creating "crystallization centers."

[0054] According to another particularly advantageous embodiment of the invention, a holographic projection for generating the target object can also be created at a heat front that is at least temporarily generated at the projection point. The heat front can be created by influencing the local temperature of the air at the projection point using appropriate technical means, in particular by means of heating the air (for example, by means of another invisible, guided laser beam or several laser beams focused on a plane in space). Such techniques can, for example, locally generate water vapor or fog, since the water droplets in the air can be heated—for example, with a laser beam. A “fog wall” generated in this way can serve as the projection surface for the projection.

[0055] The aforementioned inhomogeneities in the wave-optical properties of the medium can also be created in both advantageous embodiments described above in a pulsating manner, i.e., for short periods of time, and at approximately the same location, since a projection composed of rapidly pulsating spatial regions is perceived by humans as temporally and spatially continuous. It is particularly advantageous if the device for creating wave-optical inhomogeneities is synchronized with the control of the projection means.

[0056] In the practical implementation of the described methods, it must be considered that a created projection medium may only be available in its unchanged form for a very short time. For example, a created heat front can blur within seconds or be altered, e.g., distorted, by air currents. A locally generated overpressure is usually equalized within a few milliseconds. Only in the case of a standing wave are there nodes and oscillation points in space that periodically reach their minima and maxima, which is why such a solution is preferred. To accommodate the short-term usability of the generated projection medium, the generation of projection objects can be carried out in pulsed operation with synchronization of the devices for creating the projection medium and the actual projecting device.For a projection perceived as continuous in time by humans, approximately 24 individual projections per second may suffice. Studies have shown that each individual projection need not exceed a duration of a few nanoseconds to ensure adequate recognizability.

[0057] Preferably, the motor vehicle itself comprises suitable means for creating a specially designed projection medium (for example, for generating an ultrasonic wavefront or for otherwise creating wave-optical inhomogeneities) and, optionally, for controlling its spatial arrangement. For example, suitable ultrasonic signal transmitters can be provided on the motor vehicle for this purpose. By controlling the spatial arrangement, the projection medium can be created precisely where a projection object is to be generated. With suitable control, the motor vehicle is then independently capable of generating three-dimensional projection objects using appropriate projection technology.However, it is also conceivable that at typical projection locations (such as large parking lots or parking garages), a suitable projection medium is provided by a stationary facility, which can then be used by all road users passing such a projection location.

[0058] A projection object generated "in free space" can essentially be two- or three-dimensional. The visibility of a two-dimensional projection object can also be improved if the projection is not made onto a projection surface, but rather if the projection object is generated, possibly using a projection medium such as an ultrasound wavefront, as if "floating" in free space. In this way, for example, two-dimensional projection objects can be created that actually protrude from the road surface—not just due to their perspective.

[0059] In general – that is, also with three-dimensional projections, but especially with two-dimensional projections – it can be advantageous for projection objects that might appear distorted to the viewer depending on their viewing angle to take the viewer's perspective into account during the projection process in such a way that the resulting image is at least approximately geometrically correct (undistorted). This can be achieved, in particular, by intentionally distorting the projected image to compensate for the viewing angle-dependent perspective distortion when viewing the projection object. Such a procedure is known, in a completely different context, for example, for advertisements permanently affixed to the playing surface of sports stadiums, which are aligned with a specific camera perspective so that they appear undistorted in the camera image.If the viewing direction of a viewer towards the projection object is known—for example, in the case of another road user whose position is detected by the vehicle's environmental sensors or reported via vehicle-to-vehicle communication—it is advantageous to take this known viewing direction into account during the projection in a known manner. Otherwise, assumptions regarding a viewing angle can be made (e.g., based on map data from a navigation system). Alternatively—especially if the direction in which a viewer might be located is unknown or if several possible directions need to be considered—multiple projections can simply be displayed simultaneously, each tailored to a specific viewing angle or range of viewing angles.Therefore, there is a high probability that at least one of the representations will be clearly recognizable and informative for a viewer observing from an unknown perspective. Taking the viewer's perspective into account through appropriate distortion can apply to all generated projection objects or only to individual ones. This form of representation is particularly advantageous for elements that are intended to stand out from the projection plane in the viewer's perception.

[0060] Preferably, the target object is designed so that, from the viewer's perspective, it protrudes at least partially and clearly above the road surface. It is then particularly easy to see.

[0061] It is particularly advantageous if the target object is graphically designed in such a way that it embodies a virtual motor vehicle in its environment. The target object can be designed particularly effectively if it is created as a three-dimensional projection object – possibly using a projection medium specifically created at the projection point.

[0062] Alternatively or additionally, it can also be advantageous to make the target object partially or completely transparent. This reduces the effect of occlusion from the vehicle's surroundings. Furthermore, it emphasizes the virtual nature of the target object and requires less power for projection.

[0063] In the case of optically representing the vehicle's position by generating a projection object in the vehicle's vicinity, the following special feature must be considered. Preferably, the parking maneuver "virtually tested" according to the invention begins at the vehicle's current physical position. For projection-related reasons, it can be difficult to generate the target object at a short distance from the vehicle. Therefore, it can be advantageous to generate the target object only when it is already at a sufficient distance from the vehicle to be projected. Any recording of the control signals or movement data, however, preferably begins at the starting position in order to be able to reproduce the entire parking maneuver.

[0064] By using several projection devices distributed spatially around the vehicle, it may be possible to begin projection to an improved degree at a very short distance from the vehicle.

[0065] Based on the single accompanying drawing and an exemplary embodiment, further advantages, variants, preferred embodiments and further developments of the invention are described below. Fig. 1 a scene view in which a motor vehicle generates a projection object designed as a virtual motor vehicle through projection.

[0066] Fig. Figure 1 shows the view of the driver of a motor vehicle (not depicted) through the windshield of the motor vehicle into the surroundings of the motor vehicle.

[0067] Several (also not specifically illustrated in) Fig. The vehicle's control elements (as shown in Figure 1) are coupled to the associated drive systems via so-called X-by-Wire technology. For example, the brake and accelerator pedals are only electronically coupled to the longitudinal drive, the steering wheel only electronically to the actual steering system, and the gearshift lever only electronically to the vehicle's transmission.

[0068] The vehicle is now to be parked. Several other vehicles (numbers 11, 13, 14, 15, 16, 17, 18, and 19) are parked nearby.

[0069] To test and prepare for the parking maneuver, all vehicle controls relevant to the maneuver (here: steering wheel, accelerator and brake pedals, gearshift lever) are decoupled from their corresponding drive components (e.g., steering, engine / longitudinal drive, transmission). To achieve this decoupling, the electronic interfaces between the controls and drive systems simply stop transmitting any useful signal components or instead transmit "neutral" signals. This allows the driver to operate the decoupled controls without actually moving the vehicle or influencing its movement. Instead, the driver's operation of the controls moves a projection object 12, generated by a laser projector mounted on the vehicle, in the vehicle's vicinity. This allows the driver to safely test the parking maneuver beforehand.

[0070] The driver can initiate this decoupling state by operating a designated control element inside the vehicle. This element may, for example, be labeled "Virtual Parking". Preferably, the decoupling state can only be initiated when the vehicle is physically stationary.

[0071] During the virtual parking maneuver, projection object 12 is displayed as a highly realistic representation of the vehicle in its environment. Depending on the driver's actions, it is "driven" through the environment almost exactly as the vehicle itself would move if the controls were not decoupled. The position at which the projection object must be generated is calculated based on a kinematic and kinetic motion model of the vehicle. Fig. Figure 1 shows the projection object 12 already at the end of the virtual parking maneuver in the parking position virtually approached by the driver.

[0072] The projection object 12 is designed as a three-dimensional virtual image of the motor vehicle and is visible to all road users (without optical aids). The projection object 12 is semi-transparent, which makes its virtual nature recognizable to other road users as well. The "projection rays" 8 indicate—particularly to avoid confusing other road users—which projection source (i.e., which motor vehicle) is projecting the projection object 12 (e.g., from which vehicle). Thus, the intended movement of the projecting motor vehicle, which results in the generation of the projection object 12, can also be attributed to the projecting motor vehicle by other road users.

[0073] This description assumes that the vehicle has suitable means for generating three-dimensional projection objects in its environment. However, the described processes can easily be transferred to the simpler case of a two-dimensional projection – for example, of the vehicle's footprint – onto the road surface.

[0074] During virtual testing, control signals generated by the driver through the operation of the controls are recorded. If the test is successful, these control signals can then be recalled to execute the corresponding real-world driving maneuver. The control signals generated by operating the controls in a decoupled state can therefore be used later for actual vehicle movement. The driver can thus first "virtually test" a difficult maneuver and only execute it "at the push of a button" if successful.

[0075] The recorded control signals are processed by a computing unit integrated into the vehicle, such as a control unit. The control signals are smoothed in the process. Based on these smoothed signals, a resulting trajectory is calculated using a kinematic model. This trajectory leads to the position in the vehicle's vicinity where the projected object is currently displayed. For simplicity, the unsmoothed trajectory, which results from the position profile calculated for the optical display of the projected object 12, could alternatively be recorded. The smoothed trajectory can (in Fig. (1 not shown) can also be displayed by the laser projector in the vehicle's surroundings. The projection object 12 can be temporarily hidden for this purpose.

[0076] The trajectory can be modified by the driver after the virtual parking maneuver has finished and before the subsequent physical maneuver is initiated. Advantageously, not only can a complete change be made, but also partial or segmental modifications. This eliminates the need for a complete overhaul of the recorded control signals. Furthermore, maneuver planning is simplified for the driver, as they can focus their attention on details without significantly altering the overall maneuver. It is also possible to modify only the final parking position by moving the projected object located in that position, either using a separate control element (e.g., joystick or rotary / push knob) or by using the aforementioned controls within the environment.After the final parking position has been corrected, a modified trajectory is calculated – if necessary, after obtaining confirmation from the driver – which leads from the current physical position of the vehicle to the modified final parking position without additional “corrective maneuvering”.

[0077] The driver can also initiate a virtual re-tracing of the trajectory – particularly if it has been smoothed and / or post-processed – by projection object 12, either in whole or in part. This simulation allows the driver to verify the accuracy and safety of the planned parking maneuver.

[0078] If "post-processing" a generated but not desired trajectory does not seem sensible, the driver can also discard the recorded control signals and, if necessary, start a completely new recording as part of a new virtual parking process.

[0079] Once the driver is satisfied with the final parking position of the projection object 12 and the trajectory leading there, as displayed at the end of a virtual parking maneuver (with any subsequent correction or post-processing), they activate a confirmation element inside the vehicle. This confirmation element can also be the control element – ​​possibly activated in a special way, for example by a long press – that was already used to start the virtual parking maneuver. Upon confirmation, the vehicle is driven fully automatically along the recorded trajectory into the parking position, in which Fig. Figure 1 still shows the projection object 12. To execute the fully automatic physical parking maneuver, the vehicle's drive systems are essentially supplied with the control signals recorded during the virtual parking maneuver, possibly in a processed form – e.g., smoothed. To compensate for model errors, deviations of the actual position from the trajectory are monitored and corrected by appropriate automatic interventions in the drive systems.

Claims

[1] Method for assisting the driver of a motor vehicle in carrying out a parking maneuver, in which, during a decoupling phase in which at least one control element of the motor vehicle, which in normal operation of the motor vehicle serves to control at least one drive system of the motor vehicle, is decoupled from the at least one drive system, an optical representation of a position of the motor vehicle relative to its environment is changed depending on operating actions which are applied to the at least one control element, without changing the physical position of the motor vehicle in the environment, characterized by , that during the decoupling phase, operating actions performed on at least one control element are detected by sensors, that, depending on the sensor-detected operating actions, control signals are generated to control at least one drive system, that the generated control signals are temporarily stored and that the temporarily stored control signals are used in an execution phase that takes place after the decoupling phase for carrying out an at least partially automatic parking process of the motor vehicle. [2] Method according to claim 1, characterized by , that during the decoupling phase, operating actions performed on at least one control element are detected by sensors, that, depending on the sensor-detected operating actions, control signals are generated to control at least one drive system, that model-based motion data of the motor vehicle is calculated from the generated control signals, that the calculated movement data is temporarily stored and that the temporarily stored movement data is used in an execution phase that lies after the decoupling phase for carrying out an at least partially automatic parking process of the motor vehicle. [3] Method according to one of claims 1 or 2, characterized by , that a trajectory of the motor vehicle is determined based on the control signals generated in the decoupling phase, that a trajectory of at least one other road user is determined from communication data received by the motor vehicle's communication means and that the trajectory of the motor vehicle is checked for collision-free operation with the trajectory of at least one other road user before the execution phase. [4] Method according to any one of claims 1 to 3, characterized by , that a trajectory of the motor vehicle is determined based on the control signals generated in the decoupling phase, that the trajectory of the motor vehicle is visually displayed to the driver before the execution phase and can be changed depending on the driver's operating actions. [5] Method according to any one of claims 1 to 4, characterized by , that during the execution phase of the at least partially automatic parking process, initial control signals for controlling at least one drive system are automatically generated, that during the execution phase, operating actions performed on at least one control element are detected by sensors, that, depending on the sensor-detected operating actions, second control signals are generated to control at least one drive system, that in the control of at least one drive system, the second control signals are at least partially prioritized over the first control signals. [6] Method according to any one of claims 1 to 5, characterized by , that at least one control element is designed as the steering wheel of the motor vehicle and that at least one drive system is designed as a steering system for the motor vehicle. [7] Method according to any one of claims 1 to 6, characterized by , that at least one control element is designed as the accelerator pedal of the motor vehicle and that at least one drive system is designed as a longitudinal drive of the motor vehicle. [8] Method according to any one of claims 1 to 7, characterized by, that for the optical representation of the position of the motor vehicle in relation to its surroundings, a projection object is generated at this position in the surroundings by means of projection means provided on the motor vehicle.

Images