Computer-implemented method for assisting a user of an autonomous driving function of a vehicle by means of a mobile electronic device, computer program, computing apparatus, electronic device and vehicle

Abstract

The invention relates to a computer-implemented method for assisting a user of an autonomous driving function of a vehicle by means of a mobile electronic device, comprising the following steps: providing vehicle data; detecting a first input of the user by means of an input device of the mobile electronic device for starting the autonomous driving function; determining an upcoming travel trajectory of the vehicle for the autonomous driving function based on the first input and the provided vehicle data; determining information for the user on the basis of the determined upcoming travel trajectory; displaying the determined information relating to the autonomous driving function on a display apparatus of the electronic device; detecting a second input of the user for confirming the determined information or for adapting the autonomous driving function, the second input being detected by means of the input apparatus of the mobile electronic device; and emitting a control signal for the vehicle on the basis of the detected second input to a receiving apparatus of the vehicle, the control signal being configured to activate execution of the autonomous driving function of the vehicle.

Description

[0001] Description

[0002] title a user of an autonomous driving function of a by means of a mobile electronic device electronic device and

[0003] The present invention relates to a computer-implemented method for supporting a user of an autonomous driving function of a vehicle by means of a mobile electronic device. The invention also relates to a computer program for carrying out this method. Furthermore, the invention relates to a computing device and an electronic device comprising the computing device. The invention also relates to a vehicle.

[0004] State of the art

[0005] EP 1 886 094 B1 discloses a method for providing driving assistance to a vehicle, wherein sensors detect the vehicle's surroundings and at least partially display the detected environment. It is further disclosed that driving assistance is provided to move the vehicle from its current position to a target position, with the target position being displayed. The target position is moved and / or rotated within the display of the vehicle's surroundings by means of an input unit, and is surrounded by a frame that defines the area within which the target position can be moved and / or rotated to its maximum extent.

[0006] German patent application DE 10 2012 222 972 A1 discloses a method for determining the trajectory of a driving maneuver, wherein a user enters a character on a touch-sensitive display device, and a target pose is recognized based on the entered character, and a trajectory from the current position of the vehicle to the target pose is determined. German patent application DE 10 2014 209 477 A1 discloses a method for operating a parking assistance system in a vehicle, wherein the vehicle's surroundings are detected by means of an environment detection system and displayed on a display unit. After confirmation by an operator, the vehicle performs a parking maneuver into a parking space, at least semi-automatically, with the operator specifying the position of the parking space on the display unit.

[0007] Document EP 2 626 279 B1 discloses a method for providing an exit option from a vehicle. The vehicle is stopped in front of a parking space and the exit option is provided. The vehicle then parks itself autonomously into the parking space.

[0008] Patent application 10 2017 204 916 A1 discloses a method for controlling an automated driving process of a vehicle. The method begins with the sending of an authentication request, in particular by a remote control. Subsequently, initial authentication information from the vehicle is received, in particular by the remote control. Then, a second authentication piece of information is generated based on the first authentication piece of information. Following this, control information for the automated driving process is received by the vehicle, and the automated driving process is controlled based on the second authentication piece of information and the control piece of information.

[0009] Document DE 10 2017 217 784 A1 discloses a method for supporting a maneuvering operation of a motor vehicle, wherein the maneuvering operation is carried out autonomously under the supervision of a user.

[0010] Document EP 3 310 079 B1 discloses a method for verifying the reliability of V2X data. Document EP 3 314 917 B1 discloses improved protocols for short-range services for vehicle-to-anything (V2X) communication. Document EP 3 868 167 A4 discloses a method for vehicle-to-everything (V2X) communication in a wireless communication system.

[0011] The object of the present invention is to improve the autonomous driving function of a vehicle by means of a mobile electronic device. Disclosure of the invention

[0012] The above problem is solved according to the invention in accordance with independent claims 1 and 8 to 11.

[0013] The present invention relates to a computer-implemented method for supporting a user of an autonomous driving function of a vehicle by means of a mobile electronic device. The method includes the provision of vehicle data. The provided vehicle data preferably includes the vehicle's dimensions. Alternatively or additionally, the vehicle data includes at least one seating position and / or a characteristic of a user of the vehicle, for example, the seating position of a small child and / or the seating position of a person with a physical disability. Alternatively or additionally, the vehicle data includes service requirements, for example, the required amount of fuel and / or the current fuel level, the required amount of electricity and / or the current state of charge of a vehicle battery, tire pressure, and / or external and / or internal cleaning requirements for the vehicle.Alternatively or additionally, the vehicle data can include a preferred saved parking position and / or a desired parking duration. In a further step of the process, an initial user input is captured via an input device of the mobile electronic device to initiate the start of the autonomous driving function. The input device of the mobile electronic device includes, in particular, a touchscreen and / or a button on a smartphone or remote control. Alternatively, the input device can be configured to capture the initial input as an audio command via an optional microphone of the user's input device and / or as a user gesture via an optional camera of the input device.The first input represents, in particular, a user's wish for the vehicle to park autonomously in a parking space, wherein the parking space is preferably detected by means of a sensor system of the vehicle or by means of at least one sensor of the mobile electronic device, or is recognized based on the data of the sensor system or the data of the sensor, wherein the detected parking space is displayed to the user, for example, on the display device of the mobile electronic device.Alternatively, the first input as an autonomous driving function represents a desired autonomous journey of the vehicle to search for or detect a parking space in the vehicle's vicinity, wherein the vicinity preferably also includes neighboring streets or the current area, for example, a district in which the vehicle is located. In this case, the first input also specifically represents a desired autonomous parking of the vehicle into a detected parking space after the space has been located, as part of the autonomous driving function. Alternatively, the first input as an autonomous driving function represents a desired autonomous journey of the vehicle to a more distant service location, for example, a gas station or a charging station.Alternatively, the first input, as an autonomous driving function, represents a desired autonomous journey of the vehicle through a nearby bottleneck in the vehicle's vicinity, wherein the bottleneck is detected, in particular, by means of the vehicle's sensor system or by means of at least one sensor of the mobile electronic device, or recognized based on the data of the sensor system or the sensor, and wherein the detected bottleneck is preferably displayed to the user on the display device of the electronic device. Subsequently, in a step of the method according to the invention, an upcoming driving trajectory of the vehicle for the autonomous driving function is determined based on the first input and the provided vehicle data, wherein the upcoming driving trajectory is determined or adapted at least based on currently acquired sensor data from a sensor system of the vehicle.The sensor data is acquired, in particular, by means of at least one camera, a radar sensor, and / or an ultrasonic sensor of the electronic device and / or the vehicle. The determined driving trajectory is displayed to the user, at least partially, on the display of the electronic device. The driving trajectory is determined by a processing unit of the mobile electronic device, by means of a server, and / or by means of a processing unit of the vehicle, whereby, where necessary, data exchange between the mobile electronic device and the server or the vehicle's processing unit is provided via radio link.The upcoming driving trajectory is optionally determined based on map data and / or server data and / or data from the optional sensor of the mobile electronic device, wherein the optional sensor of the mobile electronic device is in particular configured to detect the environment of the mobile electronic device, and wherein a position or orientation of the detection direction is preferably assigned to these detected sensor data of the sensor of the mobile electronic device.For example, the system determines the upcoming driving trajectory to reach a sensor-detected parking space in the vicinity. Based on vehicle data such as the seating position of a small child and / or a person with a physical disability, a target parking position within the detected space is automatically adjusted by determining the driving trajectory. This results, for example, in the advantage of providing these individuals with more space to exit the vehicle. In a further step, information is generated for the user based on the determined driving trajectory. This information is then displayed on a screen of the electronic device.The determined information preferably represents the target parking position in the detected parking space, adapted based on the provided vehicle data. This position is advantageously displayed in a sensor-acquired environmental model, for example, in a known top-down or surround view, or in a synthetic view comprising virtual 3D objects detected based on the acquired data from the vehicle's sensor system. Alternatively, the determined information represents the calculated driving trajectory for the vehicle's autonomous journey to find the parking space, with the driving trajectory being displayed, for example, on a map.The information obtained may also, alternatively or additionally, include a cost estimate for performing the requested service, such as a quote and / or the projected costs for refueling the vehicle and / or charging the battery and / or the parking fees due. Furthermore, the displayed information may include an indication of feasibility and / or a projected timeframe for implementing the autonomous driving function. In a subsequent step of the process, a second user input is recorded to confirm the information obtained or to adjust the autonomous driving function. This second input is captured using the input device of the mobile electronic device.In a subsequent step of the process, a control signal for the vehicle is sent to a receiver in the vehicle based on the detected second input. This control signal is configured to activate the vehicle's autonomous driving function. If the detected second input represents an adjustment of the autonomous driving function or a rejection of the displayed information by the user, an alternative driving trajectory is automatically determined, for which different information can be obtained, or the process is terminated. The inventive method displays the final result of the desired autonomous driving function to the user, allowing them to review, adjust, or activate the autonomous driving function.This increases the user's control and / or comfort when remotely activating an autonomous driving function.

[0014] In one embodiment of the invention, the control signal is additionally configured to activate the transmission of a V2X radio signal by a transmitter in the vehicle. Alternatively or additionally, the V2X radio signal is transmitted by means of the mobile electronic device. The V2X radio signal can represent the target parking position for the vehicle and optionally the provided seating position and / or optionally the provided attribute of the vehicle's user. The V2X radio signal thus informs, for example, other traffic about the planned route or maneuver, so that they can advantageously take the planned route or maneuver into account, for example, by waiting, braking, or moving to a different parking space.Alternatively or additionally, the V2X radio signal represents the vehicle's parking request when searching for a parking space and / or a request to provide electronic billing data for the automatic calculation of a parking fee. The V2X radio signal can thus be received and responded to by other vehicles or infrastructure facilities to support the vehicle or its user, for example, if another vehicle wants to leave a parking space soon or if a parking meter issues or sends a parking ticket virtually via radio connection. Alternatively or additionally, the V2X radio signal represents a request to perform the service and / or a request to provide electronic billing data for the automatic billing of the requested services.This further simplifies or even enables the activated autonomous driving function. Alternatively or additionally, the V2X radio signal represents a warning to other vehicles and / or people regarding the vehicle's autonomous driving through a bottleneck in the vicinity or a current low vehicle speed, which is preferably less than or equal to 30 km / h, and ideally less than or equal to 20 km / h. This makes the activated autonomous driving function safer, as other road users are warned early of a potential hazard or traffic jam. The additional transmission of the V2X radio signal thus allows the functionality of the autonomous driving function to be further automated, expanded, and / or made safer. In other words, this increases user comfort and / or safety when using the autonomous driving function.

[0015] In a preferred embodiment of the invention, the second input confirms or modifies a position or target parking position in a parking space, wherein the parking space was recognized as an autonomous driving function after the first input for a desired parking maneuver. The user is shown, as determined information, at least a representation of the surroundings with the determined target parking position in the recognized parking space in the vicinity and / or parking conditions, such as parking fees and / or no-parking information, on or by means of the display device of the electronic device. The target parking position and / or the parking conditions are determined based on vehicle data and optionally based on sensor data of the vehicle's surroundings and / or optionally depending on server data about parking spaces, by means of the electronic device, a server, and / or a computing unit of the vehicle.The sensor data is acquired, in particular, by means of at least one camera, a radar sensor, and / or an ultrasonic sensor of the electronic device and / or the vehicle, preferably taking into account the seating position provided in the vehicle data and / or the user's characteristics. This design enables particularly convenient autonomous parking of the vehicle.

[0016] In a further embodiment, in an additional step, at least one live camera image from at least one camera of the vehicle is received by the vehicle's transmitting device on the electronic device that maps the vehicle's surroundings. The received camera image is displayed on the electronic device for the user to monitor the automated driving function. The displayed camera image allows the user to continuously monitor the autonomous driving function without having to be present in the vehicle.

[0017] The invention also relates to a computer program comprising instructions which, when the program is executed by a computer, cause it to perform the steps of the method according to one of the preceding claims.

[0018] The invention further relates to a computing device in an electronic device, comprising a first signal input for providing a first signal representing provided vehicle data, wherein the provided vehicle data is stored in a memory of the electronic device and can be received by the vehicle and / or by a server. The computing device further comprises a second signal input for acquiring a first and second user input, wherein the first input represents a start of the autonomous driving function and the second input represents confirmation of the determined information or an adjustment of the autonomous driving function. The computing device further comprises a first signal output for outputting a display signal, wherein the display signal is configured to control a display device of the electronic device to show the determined information on the display device.Furthermore, the computing device includes a second signal output for transmitting a control signal to a receiving device of the vehicle based on the detected second input, wherein the control command is configured to activate the execution of the autonomous driving function of the vehicle. The computing device also includes a computing unit, in particular a processor, configured to perform the steps of the method according to the invention.

[0019] The invention further relates to the electronic device, comprising at least one computing device according to the invention. The invention also relates to a vehicle, comprising a receiving unit configured to receive a control signal from an electronic device according to the invention. The vehicle also includes a control unit, the control unit being configured to control at least the steering and the drive unit based on the received control signal from the electronic device to perform an autonomous driving function. Optionally, the vehicle includes a transmitting unit configured to transmit a V2X radio signal depending on the received control signal.Furthermore, the vehicle includes a sensor system designed to detect the vehicle's surroundings, wherein the sensor data are in particular at least partially sent to a server facility and / or the electronic device, wherein the sensor system in particular includes a vehicle camera.

[0020] Further advantages will become apparent from the following description of exemplary embodiments with reference to the figures.

[0021] Figure 1: Flowchart of the process as a block diagram

[0022] Examples of implementation

[0023] Figure 1 schematically depicts a flowchart, in block diagram form, of the process for supporting a user of an autonomous driving function of a vehicle using a mobile electronic device. In a first step, vehicle data is provided. This vehicle data is stored, for example, in an electronic memory of the vehicle or the mobile electronic device and / or on a server. Generally, the vehicle and / or the mobile electronic device is / are configured to exchange data with each other and / or with the server, or to retrieve data from it and optionally store it on a respective electronic memory, in particular the vehicle data.The vehicle can be configured to acquire vehicle data using at least one sensor and store it in the vehicle's electronic memory and / or transmit it to the mobile electronic device and / or the server. The process is carried out, in particular, by means of the mobile electronic device's computing unit, so that the vehicle data is made available to the computing unit within the mobile electronic device. The provided vehicle data preferably includes the vehicle's dimensions, at least one seating position, and / or a characteristic of a vehicle user, for example, the seating position of a small child and / or a seating position of a person with a physical disability, and / or a service requirement, for example, the vehicle's fuel level or battery charge level.In a second step 200, an initial user input is captured via an input device of the mobile electronic device to initiate the start of the autonomous driving function. Subsequently, in a third step 300, an upcoming driving trajectory of the vehicle for the autonomous driving function is determined based on the initial input and the provided vehicle data, in particular by means of the computing device of the mobile electronic device. Alternatively or additionally, the upcoming driving trajectory can be determined by means of the server device or a computing device of the vehicle. Preferably, in the third step 300, the mobile electronic device sends a request radio signal to the server device, wherein the request radio signal represents the vehicle data, the captured initial input, and the request or command to determine the driving trajectory.The determination of the driving trajectory is then carried out by the server device based on the request radio signal received from the mobile electronic device. Subsequently, in the third step, the server device sends a result radio signal to the mobile electronic device, representing the determined driving trajectory. The upcoming driving trajectory is additionally determined at least based on sensor data acquired from a sensor system of the vehicle and optionally also based on map data. The sensor data is acquired in particular by means of at least one camera, a radar sensor, and / or an ultrasonic sensor of the electronic device and / or the vehicle.For this purpose, the acquired sensor data is transmitted via radio signal from a transmitter in the vehicle to the mobile electronic device and / or the server device. This transmission of the sensor data is preferably initiated by a further radio signal, which is transmitted based on the acquired initial input via a transmitter in the mobile electronic device. In a fourth step 400 of the method, the determined driving trajectory is displayed to the user, at least partially, on the display of the mobile electronic device. In a fifth step 500, which can also be performed before the fourth step 400, information is determined for the user based on the determined upcoming driving trajectory. In a sixth step 600, the determined information for the autonomous driving function is displayed to the user on the display of the mobile electronic device.In the seventh step (700), a second user input is displayed to confirm the determined information or to adjust the autonomous driving function. This second input is captured using the input device of the mobile electronic device. In the eighth step (800), a control signal for the vehicle is transmitted to a receiver based on the captured second input. This control signal is configured to activate the vehicle's autonomous driving function. In an optional ninth step (900), a V2X radio signal is transmitted by a transmitter in the vehicle. The transmitted control signal is configured to activate the transmission of the V2X radio signal by the vehicle's transmitter. Alternatively or additionally, the optional V2X radio signal is transmitted using the mobile electronic device.In an additional optional step 950, at least one current camera image is received and displayed by the electronic device, wherein the camera image is captured by at least one camera of the vehicle and transmitted by the vehicle's transmitter. The camera image depicts the vehicle's surroundings and is displayed to the user via the electronic device's display for monitoring the automatic driving function.

[0024] The second input, captured in step 700, allows the user to optionally confirm or change the location or target parking position within a parking space; for example, the target parking position can be freely selected. Preferably, in the sixth step 600, the user is shown, on the display of the electronic device, at least a representation of the surroundings with the determined target parking position in the detected parking space in the vicinity and / or parking conditions, such as parking fees and / or no-parking information. The target parking position and / or parking conditions are determined based on vehicle data and optionally based on sensor data from the vehicle's surroundings and / or optionally depending on server data about parking spaces, using the electronic device, a server, and / or the vehicle's computer.

Claims

Claims 1. Computer-implemented method for assisting a user of an autonomous driving function of a vehicle by means of a mobile electronic device, comprising the steps • Provision (100) of vehicle data, • Recording (200) an initial user input via an input device of the mobile electronic device to initiate the start of the autonomous driving function, • Determination (300) of an upcoming driving trajectory of the vehicle for the autonomous driving function based on the initial input and the vehicle data provided, wherein the upcoming driving trajectory is determined at least on the basis of sensor data acquired from a sensor system of the vehicle, wherein the determined driving trajectory is in particular at least partially displayed to the user on the display device of the mobile electronic device, • Determination (500) of information for the user based on the determined upcoming journey trajectory, • Display (600) of the determined information on the autonomous driving function on a display device of the mobile electronic device, • Recording (700) a second user input to confirm the information obtained or to adjust the autonomous driving function, wherein the second input is recorded by means of the input device of the mobile electronic device, and • Transmission (800) of a control signal for the vehicle based on the detected second input to a receiving device of the vehicle, wherein the control signal is configured to activate the execution of the autonomous driving function of the vehicle.

2. The method of claim 1, wherein the control signal is additionally configured to activate a transmission (900) of a V2X radio signal by a transmitting device of the vehicle and / or, wherein the transmission (900) of the V2X radio signal is transmitted using the mobile electronic device.

3. Method according to any of the preceding claims, wherein the V2X radio signal represents the following • the target parking position for the vehicle and optionally the provided seating position and / or optionally the provided characteristic of the vehicle user, • the vehicle's parking preference when searching for a parking space and / or billing data for automatic billing of a parking fee, • the order for the performance of the service and / or billing data for automatic billing of the requested services, and / or • a warning to other vehicles and / or persons regarding the autonomous driving of the vehicle through a narrow passage in the vicinity.

4. Method according to any of the preceding claims, wherein the vehicle data provided comprises the following data: • Dimensions of the vehicle, • at least one seating position and / or characteristic of a user of the vehicle, for example a seating position of a small child and / or a seating position of a person with a physical disability, and / or • a service requirement, for example a required amount of fuel and / or a current fuel level, a required amount of electricity and / or a current charge level of a vehicle battery, tire pressure and / or an external and / or internal cleaning requirement for the vehicle, • a preferred saved parking position, and / or • a desired parking duration.

5. Method according to one of the preceding claims, wherein the detected second input confirms or changes a location or target parking position in a parking space, wherein the user is shown on the display device of the electronic device at least a representation of the surroundings with the determined target parking position in the detected parking space in the vicinity and / or parking conditions, such as parking fees and / or no-parking information, as determined information, wherein The target parking position and / or the parking conditions are determined based on the vehicle data and optionally based on sensor data acquired from the vehicle's surroundings and / or optionally depending on server data relating to parking spaces by means of the electronic device, a server facility and / or a computing device of the vehicle, wherein the sensor data are acquired in particular by means of at least one camera, a radar sensor and / or an ultrasonic sensor of the electronic device and / or the vehicle, preferably taking into account the seat position provided in the vehicle data and / or the provided characteristic of the vehicle user.

6. A method according to any of the preceding claims, wherein the displayed determined information represents the following: • a target parking position adapted based on vehicle data in a detected parking space, and / or • the determined driving trajectory for the autonomous driving of the vehicle, and / or • a cost estimate for carrying out a requested service, for example, a quote and / or projected costs for refueling the vehicle and / or charging the battery and / or the parking fees due for a parking duration assumed based on the vehicle data, and / or • an indication of feasibility and / or a predicted timeframe for the implementation of the autonomous driving function.

7. Method according to one of the preceding claims, wherein in an additional step (950) at least one current camera image from at least one camera of the vehicle is received by the electronic device which maps the environment of the vehicle, and wherein the received camera image is displayed on the electronic device for the user to monitor the automatic driving function.

8. Computer program comprising instructions which, when the program is executed by a computer, cause it to perform the steps of the method according to any of the preceding claims.

9. Computing device in an electronic device, comprising at least the following components • a first signal input to provide a first signal representing provided vehicle data, wherein the provided vehicle data is stored on a memory of the electronic device, which can be received from the vehicle and / or from a server device, • a second signal input for capturing a first and second user input, where the first input represents an initiation of the start of the autonomous driving function and the second input represents a confirmation of information obtained about the autonomous driving function or an adjustment of the autonomous driving function, • a first signal output for outputting a display signal, wherein the display signal is configured to control a display device of the electronic device in order to display the determined information on the display device, • a second signal output for sending a control signal to a receiving device of the vehicle based on the detected second input, wherein the control command is configured to activate the execution of the autonomous driving function of the vehicle, and • a computing unit, in particular a processor, configured to perform the steps of the method according to any one of claims 1 to 7, wherein the computing unit 10. Electronic device comprising at least one computing device according to claim 9.

11. Vehicle, in particular, comprising • a receiving unit configured to receive a control signal from an electronic device according to claim 10, and • a vehicle control unit, wherein the control unit is configured to control at least the steering and the drive unit based on the control signal received from the electronic device to perform an autonomous driving function, and optionally • a transmitter unit configured to transmit a V2X radio signal depending on the received control signal, and optionally • a sensor system designed to detect the vehicle's surroundings, wherein the sensor data is in particular at least partially sent to a server facility and / or the electronic device, wherein the sensor system in particular includes a vehicle camera.

Images