Method, computer program and apparatus for a key of an augmented reality device and a vehicle, key of an augmented reality device and a vehicle

Abstract

The embodiments relate to methods, computer programs, and apparatus for keys to augmented reality devices and vehicles, and keys to augmented reality devices and vehicles. A method (10) for an augmented reality device (300) includes locating (12) a key (310) of a vehicle (330) in the environment of the augmented reality device (300). The method (10) further includes displaying (14) information about the key (310) to a user of the augmented reality device (300).

Description

Technical Field

[0001] This invention relates to the fields of augmented reality and vehicle communication. Embodiments provide methods, computer programs, and apparatus for keys to augmented reality devices and vehicles, and keys for augmented reality devices and vehicles. More particularly, but not exclusively, embodiments relate to the concept of using a vehicle key as a trigger to display information about the key and the associated vehicle on an augmented reality device. Background Technology

[0002] Vehicle keys have evolved from purely mechanical devices to wireless remote controls that offer users additional features or controls. Examples of such additional features include opening the trunk or windows, honking the horn, flashing lights, and activating the parking heater. Most vehicle keys have simple casings with only a few buttons and offer little information beyond unlocking the car and a few additional functions. Car keys are becoming increasingly digital and integrated into smartphones and mobile apps, allowing users to perform more complex tasks, such as locating the vehicle. Furthermore, some providers have developed car keys based on NFC (Near Field Communication) or UWB (Ultra Wideband) chips, which can be used directly from smartphones and shared with other users if desired.

[0003] Document KR20210001438A discloses a smart key and a method for using GPS to notify a user of a vehicle's location. The smart key uses a camera to acquire a realistic image superimposed on a virtual object indicating the vehicle's location, and applies augmented reality technology. The smart key for notifying the vehicle's location using GPS includes a GPS module, a main control unit, and a display unit. The GPS module collects location information. The main control unit uses the GPS module to calculate the displacement between the vehicle's pre-recorded location and its current location to obtain the vehicle's direction and distance. The display unit displays the vehicle's direction and distance information obtained by the main control unit.

[0004] The current solution does not link the car key to relevant information from the vehicle. There is a need for an improved concept for verifying information from the vehicle to the user. This need is met by the appended independent claims. Summary of the Invention

[0005] The embodiments are based on the finding that an augmented reality user interface can be linked to a vehicle key, displaying relevant information to the user, such as the current location of the vehicle associated with the key, the vehicle's current mileage (gasoline / charging), and functions such as locking / unlocking the car. A further finding is that this can be achieved by detecting the user's key from the augmented reality device. For example, the user may have any type of augmented reality device, such as augmented reality glasses. The augmented reality device may potentially communicate with the key via Bluetooth or other communication technologies. The device can identify the vehicle linked to the key and also send / receive commands to / from the key (if needed), the vehicle, and other devices (such as other users if the key is being shared).

[0006] An embodiment provides a method for an augmented reality device. The method includes locating a vehicle key in the environment of the augmented reality device. The method further includes displaying information about the key to a user of the augmented reality device. The embodiment can thus provide information based on a vehicle key detected in the user's environment.

[0007] For example, displaying information includes showing graphic and / or textual information next to a key in the environment. Thus, embodiments can provide an efficient way to present such information to a user.

[0008] In some embodiments, the method may further include: identifying the vehicle associated with the key. Information about the associated vehicle can then be enabled.

[0009] For example, the method further includes communicating with a key to obtain information about the vehicle. Communicating with the key allows for the identification of the key and the associated vehicle, which can then be used to obtain further information.

[0010] In some embodiments, the method may further include communicating with the vehicle to obtain information about the vehicle. Such communication may enable the acquisition of recent information from the vehicle, such as sensor data like temperature, image / video data, location, traffic information from the vehicle's environment, etc. Other embodiments may retrieve information about the vehicle or its status stored in a key or any other networked device.

[0011] The method may further include displaying information about the vehicle to a user of the augmented reality device. Therefore, the embodiment can efficiently provide information to the user of the vehicle even when the user is away from the vehicle. Such information provides a key linked to the vehicle.

[0012] For example, the method may further include displaying vehicle information next to the key. This information, relating to the vehicle associated with the key, is then intuitively clear to the user. In embodiments, the displayed information may be informative; for example, vehicle information may include one or more elements from a group including the vehicle's location, mileage, vehicle status, and environmental information from the vehicle. Furthermore, the vehicle information may include interactive information or elements, such as features for feature activation and / or menu navigation. Examples of features that can be activated through the displayed interactive information are vehicle locking / unlocking, parking heating or cooling, window opening and closing, driving commands (autonomous mode), etc.

[0013] The embodiment also provides a method for a vehicle key. The method includes: communicating with a user's augmented reality device and providing the user with information about the vehicle associated with the key. The embodiment enables interaction between the vehicle key and the user's augmented reality device.

[0014] In a further embodiment, the method may include authenticating the user and / or the augmented reality device before providing information about the vehicle. Embodiments can provide enhanced security by authenticating the user before displaying or providing information about the key or vehicle.

[0015] In some embodiments, the method may further include communicating with the vehicle to obtain information about the vehicle. Thus, in at least some embodiments, recent or current information can be provided to the user.

[0016] Another embodiment is a computer program having program code that, when executed on a computer, processor, or programmable hardware component, performs the methods as described herein.

[0017] A further embodiment is an apparatus including one or more interfaces configured to communicate with one or more components of a vehicle. The apparatus further includes processing circuitry configured to control the one or more interfaces and perform the methods described herein. A further embodiment is an augmented reality device and / or a key for a vehicle, including the apparatus described herein.

[0018] Another embodiment is a system having an augmented reality device and a key for a vehicle. Attached Figure Description

[0019] The following description will use only examples of apparatus and / or methods and will refer to the accompanying drawings, in which:

[0020] Figure 1A block diagram illustrating an embodiment of a method for an augmented reality device is shown;

[0021] Figure 2 A block diagram illustrating an embodiment of a method for using a key for a vehicle is shown;

[0022] Figure 3 A block diagram illustrating an embodiment of the device and an embodiment of a key for an augmented reality device or vehicle is shown;

[0023] Figure 4 A block diagram of an embodiment of the system is shown;

[0024] Figure 5 Examples of images generated by embodiments of augmented reality devices are shown; and

[0025] Figure 6 Another example of an image generated by an embodiment of an augmented reality device is shown. Detailed Implementation

[0026] Some embodiments will now be described in more detail with reference to the accompanying drawings. However, other possible embodiments are not limited to the features of these embodiments described in detail. Other embodiments may include modifications to features, as well as equivalents and substitutions of features. Furthermore, the terminology used herein to describe certain embodiments should not limit further possible embodiments.

[0027] Throughout the description of the accompanying drawings, the same or similar reference numerals refer to the same or similar elements and / or features, which may be implemented in the same or modified form while providing the same or similar functions. For clarity, the thickness of lines, layers, and / or areas in the drawings may also be enlarged.

[0028] When two elements A and B are combined using "or", this should be understood to disclose all possible combinations, such as only A, only B, and A and B, unless otherwise explicitly defined in individual cases. As alternative wording for the same combination, "at least one of A and B" or "A and / or B" can be used. This applies equally to combinations of more than two elements.

[0029] If the form is singular, such as "a," "one," or "the," and there is no explicit or implicit requirement that the use of only a single element is mandatory, further examples may also use several elements to achieve the same functionality. If the functionality is described below as being implemented using multiple elements, further examples may use a single element or a single processing entity to achieve the same functionality. It is further understood that the terms "include," "including," "comprise," and / or "comprising," when used, describe the presence of a specified feature, integer, step, operation, process, element, component, and / or group thereof, but do not preclude the presence or addition of one or more other features, integers, steps, operations, processes, elements, components, and / or groups thereof.

[0030] Figure 1 A block diagram illustrating an embodiment of method 10 for an augmented reality device is shown. An augmented reality device can be any device that allows virtual objects to be introduced into the user's perception of their environment. For example, glasses with an interactive screen can be used as an augmented reality device. Such glasses allow a user to optically perceive an image of their environment or surroundings and introduce one or more digital or virtual objects into the environment. Other augmented reality devices can use a screen or display that shows an image of the surrounding environment overlaid with virtual objects. Examples are smartphones, tablets, laptops, personal computers, etc. Another category of augmented reality devices can use a projector to project virtual objects into the user's environment or surroundings, allowing the user to perceive a realistic representation of the environment as well as additional image information of the overlay or projection of virtual objects.

[0031] In one embodiment, method 10 includes locating the key of vehicle 12 within the environment of the augmented reality device. Such location can be achieved through image processing and detection of unique optical features or properties of the key. In other embodiments, location can be accomplished using triangulation, for example, using radio or optical signals. Some embodiments may broadcast a signal in response to the key, and this response signal can be used as the basis for location. In some embodiments, the key may provide information about its location to the augmented reality device or any other device. For example, the key may communicate with a local access point or other device. Based on such communication, it may be able to determine and store its own location, either absolutely or with respect to the local access point or other device. The key can then provide its location to the augmented reality device.

[0032] like Figure 1As further illustrated, method 10 includes displaying 14 information about the key to a user of the augmented reality device. Display 14 can be implemented to overlay virtual information, such as displaying graphic and / or textual information next to the key in the environment. The user can then perceive the corresponding information overlaid on the real image (the real environment or an image of the surrounding environment).

[0033] Figure 2 A block diagram illustrating an embodiment of a method 20 for using a vehicle key is shown. The method 20 for using a vehicle key includes: communicating 22 with a user's augmented reality device. Such communication can be wireless, for example, via optical communication (such as via infrared), or radio communication (such as Bluetooth, WiFi, or cellular radio communication). The method 20 further includes: providing the user 24 with information about the vehicle associated with the key. Such information allows the user or the augmented reality device to identify the vehicle.

[0034] Figure 3 A block diagram illustrating an embodiment of device 30 and an embodiment of an augmented reality device 300 or a vehicle key 310 including device 30 is shown. Device 30 includes one or more interfaces 32 configured to communicate with one or more components of the vehicle. Device 30 further includes processing circuitry 34 coupled to the one or more interfaces 32. The processing circuitry 34 is further configured to control the one or more interfaces 32 and perform methods 10 and / or 20 as described herein. Figure 3 As indicated by the dashed line (optional from the perspective of device 30), a further embodiment includes an augmented reality device 300 including an embodiment of device 30 and a key 310 for a vehicle including an embodiment of device 30.

[0035] One or more interfaces 32 can be implemented as any component for transmitting information (e.g., transmitting, receiving, or transmitting and receiving). For example, one or more interfaces 32 can correspond to one or more contacts, inputs / outputs, and / or modules for transmitting physical signals. One or more interfaces 32 may include software interfaces / modules and / or interfaces to a transmission medium. One or more interfaces 32 can be configured to take any data processed by the processing circuitry 34 as input and / or as output. One or more interfaces 32 can enable communication with components of the vehicle, such as the vehicle key, the vehicle's modem, a server storing information about the vehicle, etc.

[0036] In this embodiment, the vehicle may be configured to operate automatically or at least partially automatically and / or be remotely operated by a control center. Specifically, the vehicle may be a car, van, truck, bus, bicycle, aerial vehicle, boat, or the like.

[0037] like Figure 3 As further shown, one or more interfaces 32 are coupled to processing circuitry 34, which may include one or more processors and control one or more interfaces 32. In this example, processing circuitry 34 may be implemented using one or more processing units, one or more processing devices, or any component for processing (such as a processor, processing circuitry, computer, or programmable hardware component operable with appropriately adapted software). In other words, the described functions of processing circuitry 34 may also be implemented in software, which then executes on one or more programmable hardware components. Such hardware components may include general-purpose processors, digital signal processors (DSPs), microcontrollers, central processing units (CPUs), etc.

[0038] Figure 4 A block diagram of an embodiment of system 400 is shown. System 400 includes embodiments of an augmented reality device 300 and a vehicle key 310. System 400 further includes a vehicle 330 and a return server 320. Figure 4 In the diagram, the arrow between the augmented reality device 300 and the vehicle key 310 indicates the communication between them. The dashed arrows indicate the different options for data communication between the vehicle 330, the return server 320, the augmented reality device 300, and the vehicle key 310.

[0039] For example, method 10, as implemented at augmented reality device 300, may further include identifying a vehicle 330 associated with a key. Key 310 may provide an identifier that allows augmented reality device 300 to directly address vehicle 330, or that enables the retrieval of information about vehicle 300 from server 320 using that identifier. For example, the identifier may be a vehicle identification number. In some embodiments, method 10 may include communicating with key 310 to obtain information about vehicle 330. Therefore, in embodiments, several options exist for the communication sequence. Key 310 may have stored information about vehicle 330, key 310 and / or augmented reality device 300 may communicate directly with vehicle 330 to retrieve information about vehicle 330, and / or key 310 and / or augmented reality device 300 may communicate with server 320, server 320 having stored information about vehicle 330 or retrieving information about vehicle 330 from vehicle 330 upon request. Therefore, different implementations are conceivable. Method 10, implemented at augmented reality device 300, can communicate directly or indirectly (via key 310 or server 320) with vehicle 330 to obtain information about vehicle 330.

[0040] Figure 5 An example of an image generated by an embodiment of an augmented reality device is shown. Figure 5The key 310 is shown placed in a bowl-shaped object. Figure 5 The image shown is perceived by a user using augmented reality glasses as an augmented reality device 300. However, Figure 5 It can also display the screen of any other augmented reality device. For example... Figure 5 As indicated, key 310 is identified by augmented reality device 300, and a label is displayed next to it. The user can then interact with the label and activate further content. For example, the user can provide input information to augmented reality device 300. Such input information can be gestures or any other input used to activate the label. Examples of such input information include gazing / looking at the label, the user's predefined hand gestures, predefined eye movements, pressing a button on the input device, etc.

[0041] Figure 6 Another example of an image generated by an embodiment of an augmented reality device is shown. Figure 6 In, such as Figure 5 The label depicted has been activated, and information about the vehicle is displayed next to the key to the user of the augmented reality device. Figure 6 The top displays an image of the vehicle along with its charging status (100%) and further status information (e.g., a lock indicating the vehicle is locked). Below, there is a bird's-eye view showing the vehicle's parking location. At the bottom, the vehicle's parking address is displayed, along with a further interactive menu element, "Show Navigation," which enables further interaction (e.g., displaying routes to the vehicle). Further interactive elements or information can be activated by any feature of the vehicle, such as locking / unlocking, opening the trunk for package delivery or retrieval, or even a request for the vehicle to retrieve something, such as in autonomous or remote operation mode. Information about the vehicle may include one or more further elements, such as the vehicle's status, environmental information from the vehicle, sensor data from the vehicle, etc.

[0042] In an embodiment, method 20, as implemented at the vehicle key 310, may further include authenticating the user and / or augmented reality device 300 before providing information about the vehicle 330. This can be accomplished, for example, through a suitable pairing or linking process, password authentication, etc. These mechanisms can prevent the provision of such information or access to unauthorized users. As previously outlined, method 20 may include communicating directly or indirectly with the vehicle 330 to obtain information about the vehicle.

[0043] The embodiment can utilize augmented reality to display vehicle-related information next to the car key. Communication between the key, the augmented reality device, and the car can use wireless or wired communication, for example, involving... Figure 4 The return server 320 is indicated in the document.

[0044] The embodiments can make it easier to provide and access vehicle information next to the vehicle key. Augmented reality can provide more precise information. The embodiments can make it possible to integrate the vehicle key into possible scenarios where the augmented reality device can be a part of our daily lives.

[0045] The embodiments may further be or relate to a (computer) program including program code that, when executed on a computer, processor, or other programmable hardware component, performs one or more of the methods described above. Therefore, the steps, operations, or processes of the different methods described above can also be performed by a programmed computer, processor, or other programmable hardware component. Examples may also cover program storage devices, such as digital data storage media, which are machine-readable, processor-readable, or computer-readable and encode and / or contain machine-executable, processor-executable, or computer-executable programs and instructions. For example, a program storage device may include or be a digital storage device, a magnetic storage medium such as a disk and tape, a hard disk drive, or an optically readable digital data storage medium. Other examples may include computers, processors, control units, (field-programmable arrays) ((F)PLAs), (field-programmable gate arrays) ((F)PGAs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), integrated circuits (ICs), or system-on-a-chip (SoC) systems programmed to perform the steps of the methods described above.

[0046] Furthermore, the disclosure of several steps, processes, operations, or functions in the specification or claims should not be construed as implying that these operations necessarily depend on the described order, unless expressly stated in individual cases or necessary for technical reasons. Therefore, the preceding description does not limit the execution of several steps or functions to a particular order. Moreover, in further examples, a single step, function, process, or operation may include and / or be decomposed into several sub-steps, sub-functions, sub-processes, or sub-operations.

[0047] If aspects of a device or system have already been described, these aspects should also be understood as descriptions of the corresponding methods. For example, a frame, device, or functional aspect of the device or system may correspond to a characteristic of the corresponding method, such as method steps. Therefore, aspects describing a method should also be understood as descriptions of the properties or functional characteristics of the corresponding frame, element, device, or system.

[0048] If aspects of a device or system have already been described, these aspects should also be understood as descriptions of the corresponding methods, and vice versa. For example, a frame, device, or functional aspect of a device or system may correspond to a feature of the corresponding method, such as method steps. Therefore, aspects describing a method should also be understood as descriptions of the properties or functional characteristics of the corresponding frame, element, device, or system.

[0049] The following claims are thus incorporated into the detailed description, wherein each claim may stand alone as a separate example. It should also be noted that although in the claims, a dependent claim refers to a specific combination with one or more other claims, other examples may also include combinations of dependent claims with the subject matter of any other dependent or independent claim. Such combinations are thus expressly presented unless, in individual cases, it is stated that a particular combination is not intended. Furthermore, the features of the claims should also be included with respect to any other independent claim, even if the claim is not directly defined as dependent on that other independent claim.

[0050] A particular aspect and feature described in a previous example may also be combined with one or more further examples to replace the same or similar features of the further examples, or additionally introduce those features into the further examples.

[0051] List of reference numerals

[0052] 10 Methods for Augmented Reality Devices

[0053] 12. Locating the vehicle key in an augmented reality environment.

[0054] 14. Display information about the key to the user of the augmented reality device.

[0055] 20 Methods for using vehicle keys

[0056] 22. Communicate with the user's augmented reality device.

[0057] 24. Provide users with information about the vehicle associated with the key.

[0058] 30 devices

[0059] 32 or more interfaces

[0060] 34 processing circuits

[0061] 300 augmented reality devices

[0062] 310 key

[0063] 320 server

[0064] Vehicle 330

[0065] 400 system

Claims

1. A method (10) for an augmented reality device (300), the method (10) comprising: Locate (12) the key (310) of the vehicle (330) in the environment of the augmented reality device (300). Display (14) information about the key (310) to the user of the augmented reality device (300); The method (10) further includes: Communicate with the key (310) and with the vehicle (330) to obtain information about the vehicle (330); Next to the key (310), display information about the vehicle (330) to the user of the augmented reality device (300).

2. The method (10) of claim 1, wherein displaying (14) information comprises: Display graphic and / or text information next to the key (310) in the environment.

3. The method (10) according to one of claims 1 or 2, further comprising: The vehicle (330) is associated with the key (310).

4. The method (10) according to one of claims 1 to 3, wherein the information about the vehicle (330) comprises: The location of the vehicle (330), the mileage of the vehicle (330), the status of the vehicle (330), and one or more elements from the group of environmental information from the vehicle (330).

5. A computer program having program code, which, when executed on a computer, processor or programmable hardware component of an augmented reality device (300), is used to perform the method (10) according to any one of claims 1 to 4.

6. An apparatus (30) comprising: One or more interfaces (32) are configured to communicate with the key to the vehicle (330) and the vehicle (330); as well as The processing circuit (34) is configured to control the one or more interfaces (32) and perform the method (10) according to any one of claims 1 to 4.

7. An augmented reality device (300) comprising the apparatus (30) according to claim 6.

Images