Method and system for performing a remotely controlled parking maneuver for a vehicle using a mobile terminal - Patents.com

Abstract

To enable a remotely controlled parking maneuver to be performed comfortably while maintaining functional safety. [Solution] A method for performing a remotely controlled parking maneuver for a vehicle 1, comprising the steps of: a user with a mobile terminal 8 being outside the vehicle 1 and planning a trajectory for the parking maneuver using a driver assist system of the vehicle 1; using the driver assist system to maneuver the vehicle 1 along the planned trajectory; using a camera of the mobile terminal 8 to detect image data representing the vehicle 1 and its surroundings 5; and providing a display on a display device of the mobile terminal 8 based on the image data; in which maneuvering along the planned trajectory is performed as long as an operational input made by the user on the mobile terminal 8 representing movement of the vehicle 1 during maneuvering along the planned trajectory is detected using the mobile terminal 8; and the maneuver is aborted if this is not the case.

Description

[Technical field]

[0001] The present invention relates to a method for performing a remote-controlled parking maneuver for a vehicle, where a user with a mobile terminal is outside the vehicle. The method includes planning a trajectory for the parking maneuver using a driver assistance system of the vehicle, and steering the vehicle along the planned trajectory. The method further includes detecting image data representative of the vehicle and its surroundings using a camera of the mobile terminal. Additionally, the method includes providing a display on a display device of the mobile terminal based on the image data. Furthermore, the present invention relates to a system for performing a remote-controlled parking maneuver for a vehicle. [Background technology]

[0002] Different driver assistance systems are known from the prior art, which assist a user or driver during the operation of a vehicle. Attention is paid here to driver assistance systems or systems that can perform so-called remotely controlled parking maneuvers. In such parking maneuvers, the driver assistance system can be used to steer a vehicle into a parking space, for example. During the parking maneuver, the user or driver of the vehicle is outside the vehicle. When performing the automated parking maneuver, the appropriate operations are performed by the user on the mobile terminal.

[0003] For this purpose, it is known from the prior art, for example, that a suitable key (button) is continuously pressed on the mobile terminal. Furthermore, it is known from the prior art that a smartphone is used as the mobile terminal, and that a suitable operating input is made on the touch-sensitive display of the smartphone. For example, a circular swipe motion can be made with one finger. As long as the operating input is made, an automated parking maneuver is performed using the driver assistance system. If the operating input stops, the driving or parking maneuver is interrupted and the vehicle is braked. By the mobile terminal and the suitable operating on the mobile terminal, a dead man's switch is realized, so to speak, so that functional safety during the parking maneuver can be ensured. A user who performs such an operation during the entire parking maneuver may find the operation unpleasant. This is especially the case if the parking maneuver to be performed is longer than a predefined duration, for example 20 seconds.

[0004] To address this problem, systems are known from the prior art in which a smartphone is used as the mobile terminal. By means of the smartphone's camera it is possible to detect a time series of images representing the vehicle or the surroundings of the vehicle. It is also possible in this case to check whether the vehicle is sufficiently detected in the detected images. For example, the forward movement of the autonomously controlled vehicle can be interrupted or stopped if the proportion of the image of the vehicle in the overall image content is above or below a predefinable threshold. Such a method is disclosed, for example, in patent document 1.

[0005] Furthermore, methods are known from the prior art, in which images recorded by a mobile terminal or smartphone are used to survey a parking area or a corresponding space. In this respect, a method for determining whether a vehicle fits into a space is described in patent application WO 2007 / 023363. The method comprises detecting an image of the space in which the vehicle should move and simulating a three-dimensional representation of the space on the basis of the image. Furthermore, the method comprises determining the dimensions of the space using the three-dimensional representation and comparing the dimensions of the space with the dimensions of the vehicle. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] DE 102012016868 A1 [Patent Document 2] European Patent Application Publication No. 3101593 Summary of the Invention [Problem to be solved by the invention]

[0007] The object of the invention is to show how a remote-controlled parking maneuver of the type mentioned at the outset can be carried out more comfortably by a user while maintaining functional safety. [Means for solving the problem]

[0008] This problem is solved according to the invention by a method and a system having the features of the independent claims. Advantageous developments of the invention are the subject of the dependent claims.

[0009] The method according to the invention is used for performing a remote controlled parking maneuver for a vehicle, where a user with a mobile terminal is outside the vehicle. The method includes planning a trajectory for the parking maneuver using a driver assistance system of the vehicle. The method further includes steering the vehicle along the planned trajectory using the driver assistance system. The method also includes detecting image data representative of the vehicle and its surroundings using a camera of the mobile terminal, and providing a display on a display device of the mobile terminal based on the image data. The steering along the planned trajectory is performed as long as an operating input performed by the user at the mobile terminal, representative of the movement of the vehicle during the steering along the planned trajectory, is detected using the mobile terminal, and the steering is interrupted otherwise.

[0010] With the method, a remotely controlled parking maneuver is performed. The parking maneuver may be, for example, a parking (entry) maneuver, in which the vehicle is steered into a parking space. The parking maneuver may also be, for example, an entry maneuver, in which the vehicle is steered out of a parking space. During the parking maneuver, the vehicle is steered automatically or autonomously using a driver assistance system. This means that the driver assistance system can intervene in the longitudinal and lateral steering of the vehicle. For the parking maneuver, a trajectory is planned with the driver assistance system. For this purpose, it is possible to provide sensor data representative of the surroundings of the vehicle, for example by means of one or more surrounding sensors. In particular, it is possible with the driver assistance system to detect objects or obstacles and their position in the surroundings of the vehicle based on the sensor data. Then, based on the detected objects and the known dimensions of the vehicle, a trajectory for the parking maneuver can be planned.

[0011] During the parking maneuver, the user or the driver of the vehicle is outside the vehicle. The user can hold the mobile terminal with one or both hands during the parking maneuver. The mobile terminal is in particular a smartphone, a tablet computer or the like. The mobile terminal comprises a camera, by means of which image data or a series of images representing the surroundings and the vehicle can be captured. Furthermore, the mobile terminal comprises a display device or a display on which a suitable display can be provided. For example, the display device can continuously display the image data captured by the camera of the mobile terminal. It can then be checked in a known manner whether the vehicle is within the detection range of the camera of the mobile terminal during the parking maneuver. It is thus checked whether a predefined range of the vehicle is detected by the camera of the mobile terminal and / or displayed on the display device. If this is not the case, the parking maneuver can be aborted. For this purpose, known methods of image processing or object recognition can be used, in which the vehicle is detected in the image data.

[0012] Furthermore, during the automated or autonomous parking maneuver, a specific actuation input is made by the user at or by the mobile terminal. When the actuation input stops, the driving maneuver is interrupted and the vehicle is braked. According to the invention, the actuation input made by the user then represents the movement of the vehicle during the maneuver along a planned trajectory. Thus, during the parking maneuver, a specific actuation input has to be made by the user. The actuation input can be set individually for the planned parking maneuver. The actuation input made can represent the movement of the vehicle during the autonomously performed parking maneuver. During the actuation input, a movement is performed by the user, in particular along a planned trajectory or a copy of the trajectory actually performed on the mobile terminal. During the actuation input, the movement of the vehicle during the performance of the parking maneuver can be emulated or imitated by the user, so to speak (virtually).

[0013] If the user's operating input deviates from this predetermined operating input, the maneuvering of the vehicle or the execution of the remote-controlled parking maneuver can be interrupted. During the execution of the maneuver, the user is given the sensation that, so to speak, he can actively control the vehicle with the mobile terminal during the parking maneuver. In this way, it is possible to achieve that the user monitors the movement of the vehicle and the surroundings of the vehicle during the parking maneuver. It is thus possible to ensure that the user directs his attention to the parking maneuver. In addition, the advantage is obtained that the user does not perceive the operating input as unpleasant, compared to known methods, in which, for example, a key (button) is pressed or a movement unrelated to the parking maneuver is performed on the mobile terminal. It is therefore possible to ensure functional safety during the remote-controlled parking maneuver and provide the function of a deadman switch.

[0014] During the actuation input, data representative of the actuation input can be continuously transmitted by the mobile terminal to the driver assistance system. Alternatively or additionally, data representative of the position of the vehicle on the planned trajectory can be transmitted by the driver assistance system to the mobile terminal. It is thus possible to check whether the actuation input represents a movement of the vehicle or whether the actuation input emulates a movement of the vehicle.

[0015] Preferably, a trajectory symbol representing the planned trajectory is shown on the display of the mobile terminal. As mentioned above, it is possible to provide a display based on image data captured by the camera of the mobile terminal. The display can thus represent the surroundings and the vehicle, as is known in the case of taking pictures with a smartphone. The display can furthermore also represent a trajectory symbol. The trajectory symbol can thus be superimposed on the displayed image data. The trajectory symbol can have a curvilinear shape and represent the trajectory traveled by the vehicle during the autonomous parking maneuver. The trajectory symbol can also comprise a suitable arrow, which represents the travel direction of the vehicle during the parking maneuver. The trajectory symbol can be determined by means of the mobile terminal or a computing device of the mobile terminal. It can then be arranged to transfer suitable data representing the planned trajectory from the driver assistance system to the mobile terminal. The trajectory symbol can then be shown on the display by means of the computing device of the mobile terminal, taking into account a two-dimensional representation of the surroundings and the vehicle. The trajectory symbol so to speak shows the user how he has performed the operating inputs. It is thus possible to ensure that the user performs the necessary operating inputs during a remote-controlled parking maneuver.

[0016] According to an embodiment, a maneuver is performed as long as a swipe motion of at least one finger of the user is detected as an operating input using the detection device of the mobile terminal, the swipe motion representing the movement of the vehicle along the planned trajectory. It is therefore possible to detect a swipe motion as an operating input using the detection device of the mobile terminal, in which the user can stroke at least one finger along the detection device. The detection device can in particular be part of a touch-sensitive display of the mobile terminal or smartphone, which touch-sensitive display at the same time includes the display device of the mobile terminal. Such a swipe motion has the advantage over swipe motions known from the prior art that the swipe motion represents a parking maneuver and gives the user the impression of actively controlling the vehicle.

[0017] As mentioned above, the display device of the mobile terminal can show a track symbol or can overlay image data on which the track symbol is shown. In particular, a swipe motion along the track symbol is then performed. Thus, during the actuation input or swipe motion, the displayed or shown track symbol can be traced, as it were, with a finger. It can also be configured that a location symbol is then shown in addition to the track symbol. The location symbol can represent the current position of the vehicle on the track. During an autonomous parking maneuver, the position of the location symbol can be changed along the track. Data representing the current position of the vehicle relative to the track can be transmitted from the driver assistance system to the mobile terminal. The location symbol can then be determined and shown by the mobile terminal on the basis of the data. The location symbol indicates, as it were, to the user where the user needs to position his / her finger for the actuation input. Alternatively, it can be configured that the location symbol can be displaced by the actuation input of the user or by a swipe motion along the track symbol. Thus, the actuation input is shown to the user in a simple and intuitive manner. In addition, it is possible to easily check whether the operating inputs correspond to the settings: if the swiping movement deviates from the displayed trajectory symbol or the displayed position symbol, the parking maneuver can be aborted.

[0018] If the swiping movement clearly deviates from the indicated trajectory symbol, the autonomous parking maneuver can be interrupted. If the user performs any swiping movement, for example with at least one finger, on the detection device, the parking maneuver can be interrupted. Similarly, if the user removes at least one finger from the detection device or the touch-sensitive display, the parking maneuver can be interrupted. If the parking maneuver is interrupted, the user can be informed of this by appropriate information. If the user then performs or continues the actuation input, the parking or driving maneuver can be continued again.

[0019] According to an alternative embodiment, the maneuver is performed as long as a movement of the mobile terminal is detected as an actuation input, the movement of the mobile terminal representing the movement of the vehicle along the planned trajectory. Another aspect of performing the actuation input is to move the entire mobile terminal or the entire smartphone. Thus, during the execution of an autonomous parking maneuver, the entire mobile terminal, pointed by the user towards the vehicle, can be moved, whereby the actuation input imitates the movement of the vehicle during the parking maneuver. Here too, it can be arranged to continuously check whether a predefined percentage of the vehicle is detected by the camera of the mobile terminal during the actuation input. If this is not met, the parking maneuver can be aborted. Thus, it is provided to the user, so to speak (virtually), that he can control the vehicle by means of the mobile terminal. In this way, functional safety during the parking maneuver can be ensured. The movement of the mobile terminal or smartphone can be continuously determined by at least one motion sensor of the mobile terminal. By means of the at least one motion sensor, the movement, acceleration and / or tilt can be detected. Known smartphones or tablet computers usually include such motion sensors. Therefore, the mobile terminal does not have to be modified for the execution of the method.

[0020] Here, in particular, a vehicle symbol representing the external contour of the vehicle is shown on the display of the mobile terminal, and the maneuver is performed as long as the vehicle symbol essentially overlaps with the image of the vehicle on the display. The vehicle symbol can represent the external contour of the vehicle with respect to the provided display. In other words, the vehicle symbol can represent the external contour of the vehicle in the image data. The position of the vehicle symbol on the display can be determined depending on the current position of the vehicle during the parking maneuver. During the execution of the parking maneuver, it can be checked whether the mobile terminal is held or moved by the user in such a way that the vehicle symbol essentially overlaps with the image of the vehicle on the display or the image of the vehicle or a predefined proportion of it is located within the vehicle symbol. If a predefined deviation is recognized between the vehicle symbol and the image of the vehicle on the display, the parking maneuver can be aborted.

[0021] When the vehicle symbol overlaps with the vehicle replica in the display, the parking maneuver can be continued again. In this case, the user has to move the mobile terminal so that the vehicle symbol coincides with the vehicle replica again. The vehicle symbol can thus be "snapped" so to speak into the vehicle replica. When a match between the vehicle symbol and the vehicle replica is recognized, a corresponding feedback can be output by means of the mobile terminal. The feedback can be visual, audible (acoustic) and / or tactile. It is also possible to set up such an approach for starting a parking maneuver for the first time. To start an automated parking maneuver, the user can adjust the orientation of the mobile terminal so that the vehicle symbol coincides with the vehicle replica in the display.

[0022] Furthermore, it is advantageous if feedback evaluating the execution of the actuation input is output to the user by means of the mobile terminal. As mentioned above, the user can be appropriately informed if the parking maneuver is interrupted. For this purpose, suitable feedback can be output to the user by means of the mobile terminal. The feedback can be output visually, for example by a suitable indication on the display, audibly, for example by a suitable sound output, and / or tactilely, for example by a suitable vibration alarm, or the like. It is also possible to configure the device to output suitable suggestions to the user during the execution of the actuation input. The suggestions can include, for example, information on how the actuation input should be performed. The suggestions can include, in particular, suggestions on the direction or speed at which the actuation input is executed. It is also possible to configure the device to provide feedback to the user on how well the executed actuation input matches the predetermined actuation input. Here, for example, a color indication can be used. In this way, it is possible to assist the user during the execution of the actuation input.

[0023] It is furthermore advantageous if the parking maneuver is performed as long as the actuation input is performed at a predefined actuation speed and is otherwise interrupted. The actuation speed at which the actuation input is performed depends in particular on the current speed of the vehicle during the execution of the parking maneuver. The actuation input should therefore be performed in particular such that it represents the speed of the vehicle during the parking maneuver. The user is thus given the impression that he controls the movement of the vehicle during the parking maneuver when the actuation input is performed correctly. Here too, it is also possible to set the actuation speed or to output appropriate suggestions or feedback to the user evaluating the actuation speed of the actuation input. It is thus possible to assist the user when executing the actuation input.

[0024] In another embodiment, before the trajectory is planned, at least one parking range is detected as a goal for the parking maneuver using the mobile terminal based on the image data, and the at least one parking range is presented to the user for selection. If the user is outside the vehicle, he can point the mobile terminal or smartphone in the direction of the desired parking range. The parking range can in particular be a parking space. Image data representing the parking range can then be provided using the camera of the mobile terminal. Based on the image data, the parking range or parking space can then be surveyed using the mobile terminal. For this purpose, known methods based on image processing or object recognition can be used, for example "structure from motion (SFM)" or the like. After the surveying of the parking range or parking space, it can then be checked using the mobile terminal based on the known dimensions of the vehicle whether the parking range is suitable for the vehicle. If the parking range has sufficient dimensions for parking the vehicle, this can be appropriately displayed to the user on the display of the mobile terminal. For example, an appropriate overlay can be performed in which the parking surface is shown in green. If the dimensions of the parking surface are not sufficient for the vehicle, this can be displayed as red in the display or by means of an appropriate overlay.

[0025] It is also possible to provide the user with the possibility to select a parking surface that is recognized as suitable for parking the vehicle. Appropriate information describing the selected parking surface and / or the dimensions of the selected parking surface can then be transmitted from the mobile terminal to the vehicle or to the driver assistance system. For this purpose, in particular, it is arranged for the relative position between the vehicle and the mobile terminal to be determined. Appropriate data can for example be exchanged to determine the relative position between the mobile terminal and the vehicle. Based on the runtime of the data exchanged at different times, the relative position between the vehicle and the mobile terminal can then be calculated by triangulation. Alternatively or additionally, a satellite-aided positioning system, for example a GPS, can be used to determine the position of the vehicle and / or the position of the mobile terminal. Thus, for example, the desired parking surface or parking space does not have to be recognized and / or surveyed beforehand by the vehicle's surroundings sensors as the goal of the parking maneuver. Likewise, the vehicle does not have to be positioned in relation to the parking surface. The functionality of the remotely controlled parking maneuver can thus be expanded.

[0026] According to another embodiment, the position of at least one parking area, which is detected as a goal for the parking maneuver based on measurements by at least one surrounding sensor of the driver assistance system, is displayed on the display of the mobile terminal. Suitable parking surfaces or parking spaces can be surveyed by the surrounding sensors of the driver assistance system and recognized as possible goals for the parking maneuver. Suitable data or information representing the recognized parking surfaces can then be transmitted from the driver assistance system to the mobile terminal. Based on this data, a suitable display or overlay can then be provided on the display, which displays the recognized parking area to the user. If the recognized parking surface is outside the display range of the display of the mobile terminal, suitable suggestions can be output to the user as to where the recognized parking surface is located in the surroundings or in which direction the user needs to move the mobile terminal. This also allows a further expansion of the functionality of the remotely controlled parking maneuver.

[0027] The system according to the invention is used for performing a remote controlled parking maneuver for a vehicle, a user with a mobile terminal being outside the vehicle. The system includes a driver assistance system of the vehicle for planning a trajectory for the parking maneuver and for steering the vehicle along the planned trajectory. Furthermore, the system includes a mobile terminal having a camera for detecting image data representative of the vehicle and its surroundings and a display device for providing a display based on the image data, whereby the mobile terminal is configured to recognize control inputs made by the user at the mobile terminal, which are representative of the movement of the vehicle during the maneuver along the planned trajectory. Furthermore, the driver assistance system is configured such that the maneuver along the planned trajectory is performed as long as the mobile terminal recognizes the control inputs, so as to abort the maneuver if the mobile terminal does not recognize the control inputs.

[0028] The driver assist system comprises at least one surrounding sensor, by means of which the surroundings of the vehicle can be detected. In particular, sensor data representative of the surroundings and objects in the surroundings can be provided by the at least one surrounding sensor. The at least one surrounding sensor can be formed as a radar sensor, a Lidar sensor, an ultrasonic sensor and / or a camera. Furthermore, the driver assist system can comprise a control device, by means of which the sensor data can be evaluated. In addition, control signals can be output by the control device for automatically or autonomously steering the vehicle. Thus, longitudinal and lateral steering of the vehicle can be taken over. In addition, a trajectory for a parking maneuver can be calculated by the control device based on objects detected in the surroundings. In addition, the driver assist system can comprise a communication unit for wirelessly transmitting and / or receiving data.

[0029] The mobile terminal may in particular be formed as a smartphone or tablet. The mobile terminal may comprise a display device for providing a display and a detection device for detecting an operation. Furthermore, the mobile terminal comprises a camera capable of providing image data of the vehicle and the vehicle's surroundings. Furthermore, the mobile terminal comprises at least one motion sensor, by means of which a movement (motion) of the mobile terminal may be detected. The mobile terminal may also comprise a communication device, so as to be able to wirelessly exchange data with a communication device of the driver assistance system. Furthermore, the mobile terminal comprises a computing device connected to the camera, the display device, the detection device and the communication device for data transmission. The computing device may be formed by a processor of the mobile terminal or the smartphone.

[0030] For carrying out the method or for operating the system, it is possible to determine the relative position between the mobile terminal and the vehicle. For this purpose, data can be exchanged wirelessly via a communication device at different times and / or GPS data can be used. The relative position of the mobile terminal with respect to the vehicle can be determined by means of triangulation. Furthermore, data of at least one motion sensor of the mobile terminal can be taken into account. It is also possible that a method of image processing based on neural networks is used for determining the relative position of the vehicle with respect to the mobile terminal. It is also possible that the mobile terminal is equipped with a stereo camera and that the relative position is determined based on image data of the stereo camera.

[0031] Optical flow methods can be used to determine the position of the vehicle in different images or image data. Thus, the vehicle can be tracked or traced in the video. Once the camera position or the mobile terminal's relative position to the vehicle is determined, the above-mentioned trajectory symbol and / or vehicle symbol can be determined and overlaid. Here, only the field of view of the camera is needed, which can be determined, for example, based on a pinhole camera model. The field of view can be calculated based on the camera's resolution, pixel size and focal length. With the known field of view, only algebraic transformations, for example rotations and translations, are necessary to display the symbol or overlay at the correct size and at the correct position. Motion sensor data and optical flow methods can be used to check the operating input in the form of a movement of the mobile terminal. A typical smartphone processor has adequate computing capabilities to perform the above-mentioned steps of the method.

[0032] Another aspect of the invention relates to a computer program comprising commands which, when executed by a computing device of a mobile terminal and a control device of a driver assistance system, cause the program to carry out the method according to the invention and advantageous embodiments thereof.Furthermore, the invention relates to a computer readable (memory) medium comprising commands which, when executed by a computing device of a mobile terminal and a control device of a driver assistance system, cause the program to carry out the method according to the invention and advantageous embodiments thereof.

[0033] The preferred embodiments and advantages presented for the method according to the invention apply correspondingly to the system according to the invention, to the computer program according to the invention and to the computer-readable (memory) medium according to the invention.

[0034] Further features of the invention are evident from the claims, the drawings and the description thereof. The features and combinations of features described hereinabove and those merely shown in the description and / or drawings below can be used not only in the respective combinations described, but also in other combinations or alone, without departing from the scope of the invention.

[0035] The present invention will now be described in detail based on preferred embodiments with reference to the accompanying drawings. [Brief description of the drawings]

[0036] [Figure 1] FIG. 1 shows a schematic diagram of a vehicle with a driver assistance system and a user with a mobile terminal in the vicinity of the vehicle. [Diagram 2] FIG. 1 is a diagram illustrating a mobile terminal and its components; [Diagram 3] FIG. 2 is a diagram showing a parking area around a vehicle captured by a camera of a mobile terminal. [Figure 4] FIG. 2 illustrates a mobile terminal on which a trajectory symbol is displayed during the execution of an automated parking maneuver for a vehicle. [Diagram 5] FIG. 1 illustrates a portion of a display of a mobile terminal on which a vehicle symbol is displayed while an autonomous parking maneuver is being performed for the vehicle. [Figure 6] FIG. 6 shows a display (part) on a mobile terminal according to FIG. 5 according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] In the various figures, identical or functionally identical elements are provided with the same reference numbers.

[0038] In Fig. 1, a vehicle 1, here configured as a passenger car, is shown in a schematic representation in a plan view. The vehicle 1 comprises a driver assistance system 2, which likewise comprises an electronic control device 3. Furthermore, the driver assistance system 2 comprises at least one surroundings sensor 4, by means of which the surroundings 5 ​​of the vehicle 1 can be detected. In the present embodiment, the driver assistance system 2 comprises two surroundings sensors 4, which can be configured for example as radar sensors. Furthermore, the driver assistance system 2 comprises a communication device 6, by means of which data can be transmitted and received wirelessly.

[0039] Sensor data representative of the surroundings 5 ​​of the vehicle 1 can be provided by the surroundings sensors 4. The sensor data can be transmitted from the surroundings sensors 4 to the control device 3. Furthermore, the driver assistance system 2 can be used to plan a trajectory for a parking maneuver, during which the vehicle 1 can be parked (entered) into a parking space or removed (exited) from a parking space. In addition, the driver assistance device 2 can comprise further surroundings sensors 4, for example ultrasonic sensors, by means of which objects or obstacles in the surroundings 5 ​​of the vehicle 1 can be detected during the execution of the parking maneuver. The control device 3 can be used to transmit appropriate control signals to the steering, the drive motor (drive engine) and / or the brake system of the vehicle 1 in order to operate the vehicle 1 automatically or at least partially autonomously during the parking maneuver.

[0040] In the surroundings 5 ​​of the vehicle 1, there is a user 7 with a mobile terminal 8. The mobile terminal 8, which can in particular be formed as a smartphone or a tablet computer, is diagrammatically illustrated in FIG. 2. The mobile terminal 8 and the driver assistance system 2 together form a system 21. The mobile terminal 8 comprises a camera 9, by means of which images or image data of the surroundings 5 ​​and the vehicle 1 can be provided. Furthermore, the mobile terminal 8 comprises a display device 10, by means of which a display can be provided. The display can be provided on the basis of image data recorded by the camera 9. In addition, the mobile terminal 8 comprises a detection device 11, by means of which an operating input on the display device 10 can be detected. In this example, the display device 10 and the detection device 11 are formed by touch-sensitive displays.

[0041] In addition, the mobile terminal 8 includes a motion sensor 12, which can be used to detect the motion of the mobile terminal 8. The motion sensor 12 can detect the rotation, tilt, acceleration, or the like of the mobile terminal 8. The motion sensor 12 can detect an operation input in the form of the motion of the entire mobile terminal 8. The mobile terminal 8 also includes a communication device 13, which is used to wirelessly exchange data with the communication device 6 of the driver assistance system 2. Finally, the mobile terminal 8 includes a computing device 14, which is connected to the camera 9, the display device 10, the detection device 11, the motion sensor 12, and the communication device 13 for data transmission.

[0042] Corresponding data can be exchanged at different times between the communication device 13 of the mobile terminal 8 and the communication device 6 of the driver assistance system 2 so that the relative position between the mobile terminal 8 and the vehicle 1 can be determined. It can also be arranged to use suitable GPS receivers of the vehicle 1 and the mobile terminal 8 to determine their respective positions or their relative positions with respect to each other. The position of the vehicle 1 with respect to the mobile terminal 8 can also be determined on the basis of image data and suitable image processing.

[0043] 3 shows the surroundings 5 ​​of a vehicle 1 and a mobile terminal 8, which displays part of the surroundings 5 ​​on a display device 10. Here, the mobile terminal 8 or a smartphone is used to search for a suitable parking surface 15 for the vehicle 1. Here, a user 7 is around the vehicle 1 and positions the mobile terminal 8 in such a way that a possible parking surface 15 is located in the detection range or field of view of the camera 9. By suitable methods of image processing, the parking space or parking surface 15 can be surveyed and by a computing device 14 of the mobile terminal it is possible to check whether the dimensions of the parking surface 15 are sufficient for parking the vehicle 1 on the parking surface 15.

[0044] If the dimensions of the parking surface 15 are sufficient for parking the vehicle 1, this can be appropriately indicated to the user. In this example, a symbol 16 is shown (overlaid) to the user, which indicates the suitability of the selected parking surface 15 for the vehicle 1. It can then be arranged that the symbol 16 is displayed in an appropriate colour, for example green. If the selected parking surface 15 is not sufficient in terms of dimensions for parking the vehicle 1 on the parking surface 15, the appropriate symbol 16 can be displayed in red. Furthermore, on the display device 10 of the mobile terminal 8, a further symbol 17 is shown to the user, thus offering the user the possibility to select the parking spot or parking surface 15 in question. If the user 7 selects the parking surface 15, corresponding data can be transmitted from the mobile terminal 8 to the driver assistance system 2. The data can represent the position and dimensions of the parking surface 15. Then, using the driver assistance system 2, a trajectory can be calculated for a driving or parking manoeuvre to the parking surface 15.

[0045] FIG. 4 shows the surroundings 5 ​​of the vehicle 1 and a mobile terminal 8 on which the vehicle 1 and a part of the surroundings 5 ​​are further displayed. Here, a parking maneuver is currently performed using the driver assistance system 2. The vehicle 1 is then at least partially autonomously steered backwards into a parking space. The parking mechanism can now be performed until at least a predetermined part of the vehicle 1 is located in the detection range of the camera 9 or is displayed on the display device 10. In other cases, the parking maneuver can be interrupted. In addition, it is arranged that the parking maneuver is performed as long as a predetermined operating input is made by the user 7 at the mobile terminal 8. Here, the operating input is arranged to represent or emulate a movement of the vehicle 1 during the parking maneuver.

[0046] In this example, a track symbol 18 is furthermore shown on the display device 10 during the parking maneuver. The track symbol 18 represents the progression or movement of the vehicle during the parking maneuver. During the parking maneuver, the user 7 is then arranged to perform a swiping movement with at least one finger on the detection device 11 or on the touch-sensitive display as an actuation input, the swiping movement being performed along the track symbol 18. In other words, the user 7 traces the track symbol 18 on the display device 10 with his finger during the actuation input. In this example, a position symbol 19 is furthermore shown, which can represent the current position of the vehicle 1 on the track.

[0047] FIG. 5 shows the surroundings 5 ​​of the vehicle 1 and the mobile terminal 8 displaying the vehicle 1 and a part of the surroundings 5 ​​in a parking maneuver according to another embodiment. Here too, the vehicle 1 is steered backwards into a parking space at least partially autonomously. In this example too, a trajectory symbol 18 and a position symbol 19 are shown. In this case, a vehicle symbol 20 is further shown, which represents the outer contour of the vehicle 1 or a representation of the vehicle 1 on the display device 10. In this case, if the vehicle symbol 20 coincides with the representation of the vehicle 1 on the display device 10, a parking maneuver can be started using the driver assistance system 2. During the maneuvering of the vehicle 1, here the entire mobile terminal 8 is moved as an actuation input. The movement during the actuation input is then performed along a planned trajectory. The movement of the mobile terminal 8 is shown here by an arrow symbol 21. A corresponding arrow symbol 21 can also be shown to the user on the display device 10. The movement of the mobile terminal 8 can be determined continuously by means of the position sensor 12.

[0048] In this example, the user has to move the mobile terminal 8 so that the vehicle symbol 20 coincides with the representation of the vehicle 1. If there is a certain deviation between the vehicle symbol 20 and the representation of the vehicle 1, the parking maneuver is interrupted and the vehicle 1 is braked. This is shown here explicitly in FIG. 6. The vehicle symbol 20 then has a clear deviation from the representation of the vehicle 1 on the display device 10. In this case, if the vehicle symbol 20 is matched with the representation of the vehicle 1, the parking maneuver can be continued again. The present invention may also include the following aspects: 1. A method for performing a remote controlled parking maneuver for a vehicle (1), comprising: a user (7) having a mobile terminal (8) outside the vehicle (1), the method comprising the steps of: - planning a trajectory for a parking maneuver using a driver assistance system (2) of the vehicle (1), - steering the vehicle (1) along a planned trajectory using a driver assistance system (2), - detecting, by means of a camera (9) of a mobile terminal (8), image data representative of the vehicle (1) and its surroundings (5); - providing a display on a display device of the mobile terminal (8) based on the image data; In the method, - A method characterized in that steering along the planned trajectory is performed as long as an operating input made by a user (7) at a mobile terminal (8), representing a movement of the vehicle (1) during steering along the planned trajectory, is detected using the mobile terminal (8), and steering is aborted otherwise. 2. The method according to claim 1, characterized in that a trajectory symbol (18) representing the planned trajectory is shown on the display of the mobile terminal (8). 3. A method according to 1 or 2 above, characterized in that steering is performed as long as a swipe motion of at least one finger of the user (7) is detected using a detection device (11) of the mobile terminal (8) as an operation input, and the swipe motion represents movement of the vehicle (1) along the planned trajectory. 4. A method according to any one of 1. to 3. above, characterized in that the operation is performed as long as the movement of the mobile terminal (8) is detected as the operational input, and the movement of the mobile terminal (8) represents the movement of the vehicle (1) along the planned trajectory. 5. The method according to claim 4, characterized in that a vehicle symbol (20) representing the external contour of the vehicle (1) is shown on the display of the mobile terminal (8) and steering is performed as long as the vehicle symbol (20) overlaps with the representation of the vehicle (1) on the display. 6. The method according to any one of 1. to 5. above, characterized in that feedback evaluating the execution of the operation input is output to the user (7) using a mobile terminal (8). 7. A method according to any one of 1 to 6 above, characterized in that the parking maneuver is performed as long as an operation input is performed at a predetermined operation speed, otherwise the maneuver is interrupted. 8. A method according to any one of 1. to 7. above, characterized in that prior to planning the trajectory, at least one parking area (15) as a goal for the parking maneuver is detected using a mobile terminal (8) based on image data, and the at least one parking area (15) is presented to the user for selection. 9. A method according to any one of 1. to 8. above, characterized in that the position of at least one parking area (15) detected as a goal for the parking maneuver based on measurements by at least one surrounding sensor (4) of the driver assistance system (2) is displayed on a display device (10) of the mobile terminal (8). 10. A system (21) for performing a remotely controlled parking maneuver for a vehicle (1), comprising: a user (7) having a mobile terminal (8) outside the vehicle (1); a driver assistance system (2) for planning a trajectory for a parking maneuver and for steering the vehicle (1) along the planned trajectory; a mobile terminal (8) having a camera (9) for detecting image data representative of the vehicle (1) and the surroundings (5) of the vehicle (1) and a display device (10) for providing a display based on the image data; In a system comprising: the mobile terminal (8) is configured to detect operational inputs made by a user on the mobile terminal (8) representative of a movement of the maneuvered vehicle (1) along a planned trajectory, The driver assistance system (2) is configured such that the maneuver along the planned trajectory is performed as long as the mobile terminal (8) detects a control input, and is interrupted otherwise. A system characterized by:

Claims

1. A method for performing a remote controlled parking maneuver for a vehicle (1), comprising: a user (7) having a mobile terminal (8) being outside the vehicle (1) and comprising the following steps: - planning a trajectory for a parking maneuver using a driver assistance system (2) of the vehicle (1), - steering the vehicle (1) along a planned trajectory using a driver assistance system (2), - detecting, by means of a camera (9) of a mobile terminal (8), image data representative of the vehicle (1) and its surroundings (5); - providing a display on a display device (10) of the mobile terminal (8) based on the image data; In the method, - a method characterized in that a maneuver along the planned trajectory is performed as long as an actuation input made by a user (7) at a mobile terminal (8), representing a movement of the vehicle (1) during the maneuver along the planned trajectory, is detected using the mobile terminal (8), otherwise the maneuver is interrupted, and that prior to planning of the trajectory, based on image data, at least one parking area (15) as a goal for the parking maneuver is detected using the mobile terminal (8), and the at least one parking area (15) is presented to the user for selection.

2. 2. The method according to claim 1, characterized in that a trajectory symbol (18) representative of the planned trajectory is shown on the display of the mobile terminal (8).

3. 3. The method according to claim 1 or 2, characterized in that the steering is performed as long as a swipe motion of at least one finger of the user (7) is detected by means of a detection device (11) of the mobile terminal (8) as an operational input, the swipe motion representing a movement of the vehicle (1) along the planned trajectory.

4. 3. The method according to claim 1 or 2, characterized in that the steering is performed insofar as a movement of the mobile terminal (8) is detected as an operational input, the movement of the mobile terminal (8) representing the movement of the vehicle (1) along the planned trajectory.

5. 5. The method according to claim 4, characterized in that a vehicle symbol (20) representing the external contour of the vehicle (1) is shown on the display of the mobile terminal (8) and the maneuver is performed as long as the vehicle symbol (20) overlaps with the representation of the vehicle (1) on the display.

6. The method according to any one of claims 1 to 5, characterized in that a feedback evaluating the execution of the operational input is output to the user (7) by means of a mobile terminal (8).

7. 7. The method according to claim 1, wherein the parking maneuver is performed as long as the actuation input is performed at a predefined actuation speed, otherwise the maneuver is interrupted.

8. 8. The method according to claim 1, further comprising displaying on a display device (10) of a mobile terminal (8) a position of at least one parking area (15) detected as a goal for the parking maneuver based on measurements by at least one surrounding sensor (4) of the driver assistance system (2).

9. A system (21) for performing a remote controlled parking maneuver for a vehicle (1), comprising: a user (7) having a mobile terminal (8) outside the vehicle (1); a driver assistance system (2) for planning a trajectory for a parking maneuver and for steering the vehicle (1) along the planned trajectory; a mobile terminal (8) having a camera (9) for detecting image data representative of the vehicle (1) and the surroundings (5) of the vehicle (1) and a display device (10) for providing a display based on the image data; In a system comprising: the mobile terminal (8) is configured to detect operational inputs made by a user on the mobile terminal (8) representative of a movement of the maneuvered vehicle (1) along a planned trajectory, the driver assistance system (2) is configured such that the maneuver along the planned trajectory is performed as long as the mobile terminal (8) detects an actuation input and is otherwise interrupted, and that, prior to the planning of the trajectory, at least one parking area (15) as a goal for the parking maneuver is detected using the mobile terminal (8) on the basis of image data, and that the at least one parking area (15) is presented to the user for selection; A system characterized by:

Images