Method and device for providing a virtual assistant function with a motor vehicle and motor vehicle

Abstract

The invention relates to a method and a device for providing a virtual assistant function (10) with a motor vehicle (1), wherein the method comprises providing a virtual assistant function (10) on a first device in a motor vehicle (1), detecting a stop action of the motor vehicle (1) and providing the virtual assistant function on a second device (100) outside the motor vehicle (1) in response to the stop action.

Description

[0001] The present invention relates to a method and a device for providing a virtual assistant function with a motor vehicle and to a motor vehicle with such a device.

[0002] Motor vehicle infotainment systems can provide a user with a wide range of information. However, once the user leaves the vehicle, they may have a need for additional information, particularly information they have not yet fully grasped.

[0003] It is therefore an object of the present invention to provide an improved method and an improved device for providing a virtual assistant function with a motor vehicle.

[0004] This problem is solved by the subject matter of the independent claims. Advantageous further developments are specified in the dependent claims.

[0005] According to a first aspect, a method for providing a virtual assistant function with a motor vehicle is provided.

[0006] The method comprises the steps of providing a virtual assistant function on a first device in a motor vehicle, detecting a stop operation of the motor vehicle and providing the virtual assistant function on a second device outside the motor vehicle in response to the stop operation.

[0007] The method is intended, in particular, to enable a user to use a virtual assistant from a motor vehicle, particularly seamlessly, even after leaving the vehicle. In particular, it is intended to enable the virtual assistant to be "taken along" from the vehicle. Various types of display and / or playback devices can be used for this purpose, as will be described below.

[0008] In a first step, a virtual assistant function is provided on a first device in a motor vehicle.

[0009] A virtual assistant function is a user interface with which a user, in particular a user of the motor vehicle, for example a driver and / or passenger, can interact and which can in particular provide the user with assistant functions.

[0010] In particular, the virtual assistant function is provided in the form of a so-called avatar. An avatar is a realistic or near-realistic virtual, two- or three-dimensional representation, for example in the form of a person or a human-like figure.

[0011] In particular, the human-shaped avatar can display gestures and / or hand signals to the driver. The avatar can be represented photorealistically or nearly so, or alternatively, as a cartoon, animal, and / or fantasy figure. Additionally or alternatively, the virtual assistant function can also include information, such as text, an image, a hint, or the like.

[0012] The form and, in particular, the appearance of the virtual assistant function or avatar can be personalized by a user. For example, the avatar can adopt or be set to have a phenotypic appearance similar to the user. Alternatively or additionally, the avatar can adopt or be set to have an appearance similar to a user's pet.

[0013] In particular, two or more avatars can be provided, for example one for each user or occupant of the motor vehicle.

[0014] Interacting with or using the virtual assistant function may include receiving user requests and providing information or functions, particularly vehicle functions, as will be explained in more detail below.

[0015] Likewise, the interaction may include addressing the user, for example, to encourage and / or entertain them. In particular, the interaction may include utilizing artificial intelligence, such as an artificial neural network, to support and / or enhance the functions of the virtual assistant feature.

[0016] The first device is, in particular, permanently installed in the motor vehicle. For example, the first device is a central infotainment unit, a head-up display, and / or a dedicated display or presentation device. In particular, the first device can also be a device designed to display a three-dimensional image, in particular a hologram, such as a projector.

[0017] Providing the information on the first device includes, in particular, displaying the virtual assistant function, for example, as an image, video, and / or animation. In particular, this can be a three-dimensional animation that spatially reproduces the type of interaction in the vehicle. Alternatively or additionally, the information can also be provided acoustically on the first device, for example, by playing a sound and / or a voice output via a playback device.

[0018] The virtual assistant function is provided on the first device, in particular, during operation or while driving and / or with the ignition switched on, and can also provide interactions related to the driving or operation of the motor vehicle. For example, the virtual assistant function can provide information about route guidance, traffic, the energy consumption of the motor vehicle, and / or functions of the motor vehicle.

[0019] In a further step, a stop operation of the motor vehicle is recorded.

[0020] A stop operation is an operation of the motor vehicle, in particular by a driver, which indicates or indicates a stop, parking, ending a journey and / or leaving the motor vehicle.

[0021] A stop operation can, for example, be an actuation or switching off of the ignition, an opening of a door, in particular a driver's door, a standing up from a seat, in particular a driver's seat, a closing of a door, in particular the driver's door, further in particular after an ignition has been switched off, a seat has been left and / or a door has been opened and / or a locking of the motor vehicle.

[0022] In a further step, in response to the stop operation, the virtual assistant function is provided on a second device outside the motor vehicle.

[0023] The second device is different from the first device and can be configured, in particular, as a device assigned to a user, in particular the driver. In particular, it is a mobile device, such as a smartphone, a tablet, a phablet, a wearable, such as a smartwatch and / or smartglasses, in particular those configured to display or generate augmented and / or virtual reality.

[0024] In particular, the second device is not physically connected to the motor vehicle, in particular not fixedly connected to it, and is designed differently. However, the second device can be coupled to the motor vehicle and / or the first device or have a communication connection therewith.

[0025] This can be done, for example, via a wired communication connection, such as a USB connection, or a wireless communication connection, such as near-field communication, Bluetooth communication, mobile radio communication, in particular 3G, 4G, 5G, and / or a wireless network connection, such as WIFI.

[0026] In particular, the stop operation can also be a termination of a previously described communication connection, for example, because a wired communication connection is disconnected, for example, by removing the second device from a cable, or because a wireless communication connection is terminated, for example, because the second device moves or is removed from the radio range. This can also detect, for example, the user leaving or leaving the motor vehicle, and thus transfer the virtual assistant function to the mobile device.

[0027] Alternatively or additionally, the second device may also be a device different from a mobile terminal, such as a display device arranged in a building, in particular a residential or office building to which the user of the motor vehicle has access, such as a monitor or a television.

[0028] The provision of the virtual assistant function can also be carried out on a display device of the second device and / or by output on a playback device, as previously described.

[0029] In particular, the virtual assistant function is provided in the same or at least similar manner as in the motor vehicle or on the first device. For example, the avatars on the first device and the second device look similar or are identical.

[0030] In particular, the provision of the virtual assistant function can take place without user interaction, without action and / or without actuation of the first and / or second device by a user.

[0031] The second device is particularly designed to carry out or at least effect embodiments of the method according to the invention, for example together with the first device and / or further devices and units, as will be described below.

[0032] The method according to the invention makes it possible to transfer the virtual assistant function from the motor vehicle to a mobile terminal and thus to take it out of the motor vehicle in order to provide a user with a particularly seamless assistant function.

[0033] For example, the virtual assistant function provides navigation instructions, a section of which, particularly the final section, must be covered on foot or by a means of transport other than a car. The user can thus seamlessly continue the assistant function after stopping and parking the car, thus navigating or guiding the section of the route.

[0034] According to a further development, the method further comprises the step of determining a position of the first device, wherein the provision of the virtual assistant function on the second device takes place depending on the determined position.

[0035] A position of the first device is determined. A position can, in particular, be a sufficiently definable location, such as one or more coordinates, an address such as a street, house number, town and / or postal code, an area, a region, a state, or the like.

[0036] The position can be detected using suitable detection units, such as one or more GPS and / or GNSS sensors, in particular using map data. For example, the position of the first device, and thus in particular of the motor vehicle, is determined via a navigation system built into it.

[0037] The virtual assistant function is provided on the second device depending on the determined position of the first device.

[0038] This can be achieved, for example, via a wireless communication connection of the motor vehicle or the first device and / or the second device, such as near-field communication, Bluetooth communication, mobile radio communication, in particular 3G, 4G, 5G, and / or a wireless network connection, such as WIFI and / or Car2X, in particular Car2Infrastructure. In particular, a cloud service to which the first device and / or the second device are connected can also be used for this purpose.

[0039] The position can be detected, in particular, before the virtual assistant function is provided on the second device and / or simultaneously with or after the detection of the stop operation of the motor vehicle. In particular, the virtual assistant function on the second device can also be provided in response to the determined position.

[0040] This advanced feature makes it possible to determine where the vehicle is located, and in particular, where it has been stopped and parked. This allows the user to be guided back to their vehicle using the virtual assistant function.

[0041] This is especially helpful in places the user has never been before, such as a new city.

[0042] Additionally or alternatively, this training can be used to detect whether the user who has used the assistance function in the vehicle has arrived at a location, such as his place of residence or work, and has stopped and parked the vehicle there.

[0043] In particular, in this case, although a mobile device may be available and / or provided, the virtual assistant function may not be displayed there, but a device at the position may be selected to provide the virtual assistant function.

[0044] This enables a seamless transition of the virtual assistant function from the vehicle to a device, in particular one or more display units, at that location. In particular, the user can then be greeted by the virtual assistant at their home or place of work as soon as they enter the location.

[0045] According to a further development, the method further comprises the steps of determining a position of the first and / or second device and providing the virtual assistant function on a third device depending on the determined position.

[0046] A position of the first and / or second device is determined. This can be done, as previously described, using suitable sensors on both the first device and the second device.

[0047] Depending on the determined position, the virtual assistant function is provided on a third device.

[0048] The third device is a device different from the first and second devices. For example, the second device is a mobile device, and the third device is a display device at the user's home or workplace. In particular, the third device can be a device assigned to the user, which the user owns and / or can control.

[0049] For this purpose, the first device or the motor vehicle, the second device and the third device can be connected to one another by one or more wireless communication connections, as described above, and in particular also use one or more cloud services, as also described above.

[0050] For example, the virtual assistant function can initially be provided temporarily on a mobile device upon reaching the home or work location until the user reaches a device at the location, whereupon the virtual assistant function can then be provided additionally or exclusively on a display device at the location, such as a monitor or television.

[0051] In particular, by taking into account not only the position of the first device but also the position of the second device, a particularly seamless transition from device to device can be made possible.

[0052] For example, it is possible to record which room the user is in at a location, such as a bedroom instead of a living room, because they are carrying a mobile device, and then to provide the virtual assistant function on a display device located there, depending on the room.

[0053] The virtual assistant function can be provided on the third device both in addition to and as an alternative to the second device.

[0054] According to a further development, the virtual assistant function is provided on the third device by the second device.

[0055] In particular, the third device is a device that does not have a mobile radio communication or a wireless network connection, and thus in particular does not have access to a network, the motor vehicle and / or a cloud, but only via a near-field communication, such as Bluetooth.

[0056] Thus, the virtual assistant function on the third device can be enabled via or through the second device. This can be done, in particular, in such a way that the second device, such as a mobile device, does not have to be removed from the pocket and / or unlocked. Instead, the virtual assistant can be provided on the third device without any action or interaction from the user, solely through proximity and existing near-field communication with the third device.

[0057] This further development is particularly convenient and allows the inclusion of less connected devices, such as older monitors or televisions that are not or less “smart”.

[0058] According to a further development, the third device is a display unit in public space.

[0059] Such a display unit in public space is a unit that is not assigned to the user and, in particular, over which the user cannot dispose. For example, a display unit in public space is a digital billboard, illuminated sign, or neon sign, for example, in a pedestrian zone, a shop window, or a public square.

[0060] In this case, the display unit in the public space can be designed to be either smart, i.e. have mobile radio communication and / or a wireless network connection, or less smart or connective, and only have near-field communication and thus provide the digital assistant function either via a mobile terminal, for example in the form of the second device, or via a network and in particular via a cloud.

[0061] Alternatively or additionally, the second device may also be such a display unit in a public space.

[0062] This development makes it possible to use display units that the user doesn't have access to for displaying the virtual assistant function. For example, billboards along a user's driving or walking route can be used to interact with the user and, in particular, to provide information such as navigation instructions.

[0063] According to a further development, the third device is a display unit of a vehicle different from the motor vehicle.

[0064] In particular, the display unit may also be a central infotainment unit, a head-up display and / or a dedicated display or presentation device, as described above.

[0065] This allows the user who leaves their vehicle and enters or boards another vehicle to take their virtual assistance function with them.

[0066] Alternatively or additionally, the display unit may be an external display unit of the second vehicle, which is also visible, in particular exclusively, from the outside. This may, for example, be arranged in one or more windows and / or a body part and, in particular, be facing outwards so that an outsider who does not have access to the vehicle can view it.

[0067] This makes it possible to provide a user with the digital assistance function for parked or moving cars that they are not familiar with, or that are passing by.

[0068] The display unit of the further vehicle can be designed to be either smart, i.e. have mobile radio communication and / or a wireless network connection, or less smart or connective, and only have near-field communication and thus provide the digital assistant function either via a mobile terminal, for example in the form of the second device, or via a network and in particular via a cloud.

[0069] Alternatively or additionally, the second device may also be such a display unit of a vehicle different from the motor vehicle.

[0070] According to a further development, the method further comprises the steps of receiving an information request from a user, retrieving the requested information and providing the retrieved information.

[0071] First, an information request is received from a user. This can be done, in particular, via voice input. Alternatively or additionally, input can also be made via a keyboard, particularly a soft keyboard.

[0072] This information request can be received and processed on the first, second, and / or third device. In particular, the information request is received on the device currently providing the virtual assistance function. Alternatively, the information request can be received on a mobile device, for example, in the form of the second device.

[0073] The request content can be analyzed, after which the requested information is retrieved. The retrieval can occur either from a memory of one of the first, second, and / or third devices and / or by retrieval from a remote computing device, such as a cloud.

[0074] In particular, further information can also be taken into account, such as a current position of the first, second and / or third device.

[0075] The retrieved information is provided to the user, which can also be done on the first, second and / or third device, in particular on the device that is currently providing the virtual assistance function and / or on a mobile terminal, for example in the form of the second device, as described above.

[0076] The provision is carried out in particular by the virtual assistance function or the avatar or is effected by this.

[0077] The information can, for example, be information that can be retrieved from a reference book or a search engine, such as information about a tourist attraction or the weather.

[0078] The information may in particular also include information about the motor vehicle, in particular a position, a fuel or energy level, a charge level, a temperature, in particular in the interior, and / or an operating state, such as whether a function of the motor vehicle is active and / or the doors are locked.

[0079] This further training makes it possible to provide information, particularly relating to the vehicle, to the user in a particularly simple, intuitive and seamless manner even after leaving the vehicle using the virtual assistance function.

[0080] According to a further development, the method further comprises the steps of receiving an actuation request from a user and carrying out the requested actuation on the motor vehicle.

[0081] The activation request, like the information request, can be made, in particular, by voice input. Alternatively or additionally, input can also be made via a keyboard, in particular a soft keyboard.

[0082] This actuation request can be received and processed at the first, second, and / or third device. In particular, the actuation request is received at the device currently providing the virtual assistance function. Alternatively, the actuation request can be received at a mobile device, for example, in the form of the second device.

[0083] The request content can be analyzed, whereupon the requested operation is provided to the motor vehicle or the first device and carried out there.

[0084] The actuation can, in particular, relate to the actuation of a function of the motor vehicle. For example, the actuation request includes air conditioning, in particular to a target temperature, door opening, and the start and / or end of a charging process, which is then carried out on the motor vehicle.

[0085] This advanced training makes it possible to operate the vehicle particularly easily, intuitively and seamlessly, even after leaving the vehicle, using the virtual assistance function.

[0086] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0087] According to a further aspect, a device is provided. The device may comprise a data processing device or a processor device configured to carry out an embodiment of the method according to the invention.

[0088] In particular, the device is designed as a mobile terminal, as described above, and interacts or communicates with the first device and / or the third device in order to carry out or at least effect the embodiments described above.

[0089] For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit) or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can have program code which, when executed by the processor device, is configured to carry out the embodiment of the method according to the invention. The program code can be stored in a data memory of the processor device. The processor device can, for example, be based on at least one circuit board and / or on at least one SoC (System on Chip).

[0090] According to a further aspect, a motor vehicle is provided which comprises such a device.

[0091] The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.

[0092] As a further solution, the invention also encompasses a computer-readable storage medium comprising program code which, when executed by a computer or computer network, causes the computer to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated on the Internet, for example, as a so-called app store server and / or cloud server. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code may be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).

[0093] The invention also encompasses combinations of the features of the described embodiments. The invention therefore also encompasses implementations that each comprise a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.

[0094] With regard to the embodiments of the device, the motor vehicle and the storage medium as well as the associated advantages, reference is made to the previously described embodiments of the method and the associated advantages.

[0095] Exemplary embodiments of the invention are described below. Shown are: Fig. 1 is a schematic side view of an embodiment of a method and apparatus for providing a virtual assistant function; Fig. 2 is a schematic side view of another embodiment of a method and apparatus for providing a virtual assistant function; and Fig. 3 a schematic side view of another embodiment of a method and apparatus for providing a virtual assistant function.

[0096] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0097] In the figures, the same reference symbols designate elements with the same function.

[0098] Fig. 1 shows a schematic view of an embodiment of a method and a device 100 for providing a virtual assistant function 10 with a motor vehicle 1.

[0099] The device 100, which is shown here by way of example as a mobile terminal in the form of a smartphone, is designed to carry out or at least effect the method steps described below.

[0100] According to the method according to the invention, a virtual assistant function 10 in the form of a human-like avatar is first provided on a first device, not shown in detail, in the motor vehicle 1.

[0101] A stop operation of the motor vehicle 1 is then detected and the virtual assistant function 10 is provided on the second device 100 outside the motor vehicle 1 in response to the stop operation.

[0102] In this case, a position of the first and / or second device 100 can be determined, and in response to the determined position, in this case the impending arrival of a user of the second device 100 at his house 3, the virtual assistant function 10 is also provided on a third device 200, here in the form of a television in the house 3.

[0103] The virtual assistant function 10 is provided via a remote computing unit 4, shown here as a cloud by way of example, which is connected to the motor vehicle 1 or the first device, the second device 100 and the third device 200 arranged therein via a wireless data communication line.

[0104] An information request can be received from the user 2, the requested information can be retrieved from the remote computing unit 4 and provided to the user 2 by the virtual assistant function.

[0105] Likewise, an actuation request can be received from the user 2 and this actuation can be carried out on the motor vehicle 1, also with the help of the remote computing unit 4.

[0106] Fig. 2 shows a schematic view of another embodiment of a method and a device 100 for providing a virtual assistant function 10 with a motor vehicle 1.

[0107] In contrast to the Fig. In the embodiment shown in Figure 1, the third device is shown in the form of a digital neon sign in public space, which is arranged on a high-rise building 5 and there provides the virtual assistant function 10 for the user 2.

[0108] Furthermore, the communication between the first device, the second device 100 and the third device 200 does not take place via a cloud, as in Fig. 1, but via direct connections between the motor vehicle 1 and the second device 1000 and the third device 200.

[0109] In particular, Fig. 2 shows that the provision of the virtual assistant function 10 on the third device 200 is carried out by the second device 100, for example by a Bluetooth coupling.

[0110] Fig. 3 shows a schematic view of another embodiment of a method and a device 100 for providing a virtual assistant function 10 with a motor vehicle 1.

[0111] In contrast to the Fig.2, the third device is designed in the form of a display unit of a vehicle 6 different from the motor vehicle 1, which is parked in the vicinity of the motor vehicle 1.

[0112] It is shown here by way of example that the display unit is an external display unit of the second vehicle, which is arranged on a front side of the vehicle 6, in particular on a radiator grille, and is thus only visible from the outside by a passerby, such as the user 2, so that an outsider who does not have access to the vehicle 6 can still see it.

[0113] Overall, the examples show how a virtual assistance function in the form of an avatar can be taken anywhere by a user.

Claims

[1] Method for providing a virtual assistant function (10) with a motor vehicle (1), comprising: - Providing a virtual assistant function (10) on a first device in a motor vehicle (1); - detecting a stop operation of the motor vehicle (1); and - Providing the virtual assistant function on a second device (100) outside the motor vehicle (1) in response to the stop operation. [2] The method of claim 1, further comprising: - Determining a position of the first device; wherein the provision of the virtual assistant function (10) on the second device (100) takes place depending on the determined position. [3] The method of claim 1, further comprising: - determining a position of the first and / or second device (100); and - Providing the virtual assistant function (10) on a third device (200) depending on the determined position. [4] The method of claim 3, wherein the provision of the virtual assistant function (10) on the third device (200) is performed by the second device (100). [5] Method according to claim 3 or 4, wherein the third device (200) is a display unit in a public space. [6] Method according to claim 3 or 4, wherein the third device (200) is a display unit of a vehicle different from the motor vehicle. [7] Method according to one of the preceding claims, further comprising: - Receiving an information request from a user (2); - Retrieve the requested information; and - Providing the retrieved information. [8] Method according to one of the preceding claims, further comprising: - receiving an actuation request from a user (2); and - Carrying out the requested operation on the motor vehicle (1). [9] Device (100) designed to carry out a method according to one of the preceding claims. [10] Motor vehicle (1) comprising a device (100) according to claim 9.

Images