SYSTEMS AND METHODS FOR ANNOTING POINTS OF INTEREST ON A HEAD-UP DISPLAY

Abstract

A vehicle is disclosed that includes a head-up display (HUD), a transceiver, and a detection unit. The transceiver can be configured to receive vehicle information and information associated with a geographic area in which the vehicle may be located. The detection unit can be configured to detect a user's gaze direction. The vehicle can further include a processor configured to obtain the vehicle information, the geographic area information, and the user's gaze direction in response to the receipt of the trigger signal.The processor can determine an object in the geographic area that a user may be viewing, based on vehicle information, information associated with the geographic area, and the user's viewing direction. The processor can further render an object identifier associated with the object on the HUD display until the vehicle passes the object, in response to the object's determination.

Description

FIELD OF TECHNOLOGY

[0001] The present disclosure relates to systems and methods for annotating points of interest (POI) on a head-up display (HUD) in vehicles, and more particularly to systems and methods for annotating POI on the vehicle HUD based on a user's gaze direction. GENERAL STATE OF THE ART

[0002] Many modern vehicles include head-up displays that display important vehicle information that can assist the user when the vehicle is in motion. For example, a head-up display (HUD) can display information related to navigation directions, vehicle speed, battery state of charge (SoC) / fuel level, etc., and can assist a user while driving a vehicle.

[0003] While the vehicle information displayed on the HUD can be beneficial to the user, there may be instances where the user may desire additional information. SUMMARY

[0004] The present disclosure describes a vehicle including an augmented reality head-up display (HUD) configured to display information that can assist a driver in driving the vehicle. For example, the HUD can display vehicle speed, state of charge (SoC) / fuel level, navigation directions, etc., which can assist the driver in driving the vehicle. The HUD can further be configured to display information about objects that may be of interest to the driver, such as points of interest (POIs). In particular, the HUD can be configured to display names of buildings / structures (or other details) that the driver may see through the windshield while driving the vehicle on a road.The HUD can display a building name for a preset period of time and / or until the vehicle passes the building, which can allow the driver to see the building name on the HUD when the driver looks away from the building and back to the HUD.

[0005] The vehicle may include a gaze direction detection unit that can detect a driver's gaze direction when the driver views the building. The vehicle may be further configured to obtain a geolocation of the vehicle, a real-time vehicle orientation / posture, and a digital map of a geographic area in which the vehicle may be located. The vehicle may correlate the driver's gaze direction with the real-time geolocation of the vehicle, the vehicle orientation / posture, and the digital map of the geographic area to determine the building the driver may be viewing. The processor may further determine the building name based on the digital map of the geographic area in response to determining the building the driver may be viewing.

[0006] In some aspects, the driver may provide the vehicle with preferences for the type of buildings / structures for which the driver wishes to see corresponding names. For example, the driver may provide preferences indicating that the driver wishes to see names of landmarks, restaurants, etc., while driving on the road. In this case, in response to determining the building the driver is currently viewing, the vehicle may additionally determine the building type based on the digital map of the geographic area. The vehicle may display the building name on the HUD if the building type matches the type of building / structure whose name the driver wishes to see.

[0007] The present disclosure discloses a vehicle that displays information associated with POIs on a HUD. Because the vehicle displays information associated with buildings the driver may be viewing, the displayed information is relevant to the driver. Furthermore, the information displayed on the HUD can assist the driver in learning details about roadside buildings during low-visibility weather conditions (e.g., fog, snowfall, rain, etc.). Furthermore, the information remains on the HUD for a predefined short period of time or until the vehicle passes the building, ensuring that the displayed information does not distract the driver while driving the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The detailed description is presented with reference to the accompanying drawings. The use of the same reference numerals may indicate similar or identical elements. Different elements and / or components than those illustrated in the drawings may be used for different embodiments, and some elements and / or components may not be present in different embodiments. The elements and / or components in the figures are not necessarily drawn to scale. Throughout the disclosure, the singular and plural terms may be used interchangeably, depending on the context. Fig. 1 illustrates an exemplary environment in which techniques and structures for providing the systems and methods disclosed in this document may be implemented. Fig. 2 illustrates a block diagram of an example system for annotating an object on a head-up display (HUD) in a vehicle according to the present disclosure. Fig. 3 illustrates an exemplary view of the driver's line of sight according to the present disclosure. Fig. 4 shows an example snapshot illustrating a vehicle vector, a driver gaze direction vector, and a point of interest (POI) vector according to the present disclosure. Fig. 5 illustrates a flowchart of an exemplary method for annotating an object on a vehicle HUD according to the present disclosure. DETAILED DESCRIPTION

[0009] The disclosure is described in more detail below in this specification with reference to the accompanying drawings, in which exemplary embodiments of the disclosure are shown and are not intended to be limiting.

[0010] Fig. 1 illustrates an example environment 100 in which techniques and structures for providing the systems and methods disclosed herein may be implemented. The environment 100 may include a vehicle 102, which may take the form of any passenger or commercial vehicle, such as a car, a work vehicle, a crossover vehicle, a truck, a van, a minivan, a taxi, a bus, etc. The vehicle 102 may be a manually operated vehicle and / or configured to operate in a partially autonomous mode, and may include any powertrain, such as a gasoline engine, one or more electrically actuated motors, a hybrid system, etc. A vehicle user 104 (or a driver 104) may drive the vehicle 102 on a road 106.The road 106 may be located in a geographic area that includes one or more buildings 108, points of interest, restaurants, parking lots, gas stations / electric vehicle (EV) charging stations, and / or the like.

[0011] The vehicle 102 may include an augmented reality head-up display 110 (or HUD 110), which may be disposed on (or near) a windshield of the vehicle. The HUD 110 may be configured to display information that may assist the driver 104 when the driver 104 is driving the vehicle 102. For example, the HUD 110 may display important vehicle information, such as vehicle speed, battery state of charge (SOC) / fuel level, etc., navigation directions, information related to weather conditions, date and time, distance from a preceding vehicle, and / or the like. In some aspects, the HUD 110 may be communicatively coupled to a user device (not shown) and / or one or more vehicle components, which may enable the HUD 110 to receive and display the information described above.

[0012] In some aspects, the driver 104 may activate a POI annotation mode of the HUD 110, which may enable the HUD 110 to display additional information. For example, when the driver 104 turns on the POI annotation mode, the HUD 110 may display information associated with one or more POIs that may be located along the road 106. In some aspects, the driver 104 may turn on the POI annotation mode by actuating a dedicated actuator / button that may be located within the vehicle 102, e.g., on a vehicle steering wheel. In other aspects, the driver 104 may turn on the POI annotation mode through a voice command, a predefined hand gesture, via a user device or a vehicle infotainment system, and / or the like.

[0013] In one example aspect, the HUD 110 may display information associated with one or more objects (e.g., structures or buildings) that may be present in the geographic area in which the vehicle 102 may be traveling when the driver 104 enables the POI annotation mode. In particular, the HUD 110 may display information associated with buildings / structures ("POI") that the driver 104 may view through the vehicle's windshield. For example, the HUD 110 may display information associated with "Building 1" (which may be one of the buildings 108 located along the road 106) when the driver 104 views the building 1 through the vehicle's windshield. The information associated with the building 1 that may be displayed on the HUD 110 may include the building name (as in Fig. 1) and / or any other additional information that may be beneficial to the driver 104. As an example, the HUD 110 may display "Parking Lot of Building 1" when the driver 104 may be viewing the parking lot of Building 1 through the vehicle's windshield. Hereinafter, the terms "object" and "POT" are used interchangeably.

[0014] By displaying information associated with objects that the driver 104 may be viewing on the HUD 110, the present disclosure provides additional functionality. For example, in a conventional HUD system, when the driver 104 reaches a destination location (e.g., building 1), the HUD 110 or the user device may only display and notify that the driver 104 has reached the destination location. However, if the destination location may be a large building, a university, an office complex, and / or the like, the driver 104 may desire additional information that may assist the driver 104 in driving (e.g., knowing whether an object / structure the driver 104 may be viewing is an entrance to the building, an exit from the building, a parking lot, etc.).The present disclosure enables the driver 104 to view such additional information based on the objects the driver 104 may be viewing.

[0015] The present disclosure may also assist the driver 104 when the driver 104 may be driving through a geographical area that may be unfamiliar to the driver 104. For example, if the driver 104 is driving through a geographical area as a tourist (e.g., for the first time), the driver 104 may want to see names and additional information associated with objects / structures along the road 106. The present disclosure enables the driver 104 to view such additional information. In this case, the driver 104 may provide inputs to the vehicle 102 (e.g., via the user device or the vehicle infotainment system) indicating the types of objects about which the driver 104 may want to see additional information, and the HUD 110 may display information about the objects that may be of interest to the driver 104.For example, the driver 104 may provide input to the vehicle 102 before starting a trip or during the trip indicating that the driver 104 is interested in viewing information about points of interest as the vehicle 102 travels along the road 106. In this case, the HUD 110 may only display information about the objects / structures on the road 106 that may be points of interest when the driver 104 views them through the vehicle's windshield (and may not display information about other objects / structures).

[0016] In some aspects, the HUD 110 may display the information associated with the object (e.g., the POI) for a predefined period of time or until the vehicle 102 passes the object. To enable the HUD 110 to display information associated with the POI, the vehicle 102 may further include a detection unit (e.g., an in-vehicle camera) that can detect the driver's gaze direction when the driver 104 sees the POI along the road 106. In this case, the vehicle 102 may be connected to an external server (in Fig. 2 as server 206) that can provide the vehicle 102 with the vehicle's geolocation and a digital map of the geographic area in which the vehicle 102 may be located in real time. The vehicle 102 may additionally include an inertial measurement unit (IMU) that can detect a vehicle heading / heading (or attitude) as the vehicle 102 travels on the road 106. The vehicle 102 may correlate the detected driver gaze direction with the real-time vehicle location, the vehicle heading / heading, and the digital map of the geographic area to determine and display information (e.g., the name of the POI) associated with the POI the driver 104 is viewing on the HUD 110.

[0017] Further details on the vehicle 102 are given below in connection with Fig. 2 described.

[0018] The vehicle 102 and / or the driver 104 implements and / or performs operations as described herein in the present disclosure in accordance with the user manual and safety guidelines.

[0019] Fig. 2 illustrates a block diagram of an exemplary system 200 for annotating an object on a head-up display (e.g., the HUD 110) in a vehicle according to the present disclosure. During the description of Fig. 2 can be Fig. 3 and Fig. 4 reference may be made.

[0020] The system 200 may include a vehicle 202, a user device 204, and one or more servers 206 communicatively coupled to one another via one or more networks 208. The vehicle 202 may be the same as the vehicle 102 described above in connection with Fig. 1. The user device 204 may be associated with the driver 104 and may include, but is not limited to, a mobile phone, a laptop, a computer, a tablet, a wearable device, or any other similar device with communication capabilities. The server(s) 206 may be part of a cloud-based computing infrastructure and associated with a telematics service delivery network (SDN) associated with the vehicle 202 and other vehicles (in Fig. 2 not shown) that may be part of a commercial vehicle fleet, providing digital data services. In further aspects, the server(s) 206 may store information associated with a geographic area in which the vehicle 202 may be located. The information associated with the geographic area may include, among other things, names of objects (buildings, structures, etc.) located in the geographic area, a digital map of the geographic area, a building / structure type (e.g., an office building, a landmark, a gas station or charging station, a restaurant, etc.), and / or the like.In additional aspects, the server(s) 206 may be configured to send navigation instructions to the vehicle 202, the user device 204, and a plurality of other vehicles and user devices (not shown) based on origin and destination locations provided by respective vehicles / user devices.

[0021] The network(s) 208 illustrate(s) an example of a communications infrastructure in which the connected devices discussed in various embodiments of this disclosure may communicate. The network(s) 208 may be and / or include the Internet, a private network, a public network, or another configuration operating using any one or more of any known communications protocols, such as Transmission Control Protocol / Internet Protocol (TCP / IP), Bluetooth ® , BLE ®, Wi-Fi based on the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard, ultra-wideband (UWB) and cellular technologies such as Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), High-Speed ​​Packet Access (HSPDA), Long-Term Evolution (LTE), Global System for Mobile Communications (GSM) and Fifth Generation (5G), to name just a few.

[0022] The vehicle 202 may include a variety of units, including, but not limited to, an automotive computer 210, a vehicle control unit (VCU) 212, and a POI annotation system 214. The VCU 212 may include a plurality of electronic control units (ECUs) 216 arranged in communication with the automotive computer 210.

[0023] The user device 204 may connect to the automotive computer 210 and / or the POI annotation system 214 via the network 208, which may communicate via one or more wireless connections, and / or may connect using near field communication (NFC) protocols, Bluetooth ® protocols, Wi-Fi, ultra-wide band (UWB), and other possible data connection and sharing techniques directly to the vehicle 202.

[0024] In some aspects, the automotive computer 210 and / or the POI annotation system 214 may be installed in an engine compartment of the vehicle (or elsewhere in the vehicle 202) according to the disclosure. Further, the automotive computer 210 may operate as a functional component of the POI annotation system 214. The automotive computer 210 may be or include an electronic vehicle controller having one or more processors 218 and a memory 220. Furthermore, the POI annotation system 214 may be separate from the automotive computer 210 (as in Fig. 2) or integrated as part of the automotive computer 210.

[0025] The processor(s) 218 ​​may be arranged in communication with one or more storage devices arranged in communication with the respective computing systems (e.g., the memory 220 and / or one or more external databases located in Fig. 2 are not shown). The processor(s) 218 ​​may utilize the memory 220 to store programs in code and / or store data for performing aspects according to the disclosure. The memory 220 may be a non-transitory computer-readable memory for storing POI annotation program code. The memory 220 may include any one or a combination of volatile memory elements (e.g., dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), etc.) and any one or more non-volatile memory elements (e.g., erasable programmable read-only memory (EPROM), flash memory, electronically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), etc.).

[0026] In some aspects, the VCU 212 may share a power bus with the automotive computer 210 and may be configured and / or programmed to transfer data between vehicle systems, connected servers (e.g., the server(s) 206), and other vehicles (in Fig. 2 not shown) operating as part of a vehicle fleet. The VCU 212 may include or communicate with any combination of the ECU 216, such as a body control module (BCM) 222, an engine control module (ECM) 224, a transmission control module (TCM) 226, a telematics control unit (TCU) 228, a driver assistance technologies (DAT) controller 230, etc. The VCU 212 may further include and / or communicate with a vehicle perception system (VPS) 232 that has connectivity to and / or controls one or more vehicle sensor systems 234.The vehicle sensor system 234 may include one or more vehicle sensors, including, but not limited to, a radio detection and ranging (RADAR) sensor configured to detect and locate objects inside and outside the vehicle 202 using radio waves, seat belt buckle sensors, seat area sensors, a light detecting and ranging (LIDAR) sensor, door sensors, proximity sensors, temperature sensors, wheel sensors, ambient weather sensors, in-vehicle and external cameras, steering wheel sensors, an inertial measurement unit (IMU), etc. In an example aspect, the interior camera(s) and / or radar sensors may be configured to detect the eye movements of the driver 104 as the driver 104 drives the vehicle 202.Furthermore, the IMU may be configured to detect vehicle orientation, heading, or attitude as the vehicle 202 may be traveling on the road 106.

[0027] In some aspects, the VCU 212 may control operational aspects of the vehicle and implement one or more sets of instructions received from the user device 204 from one or more sets of instructions stored on the memory 220, including instructions that serve as part of the POI annotation system 214.

[0028] The TCU 228 may be configured and / or programmed to provide vehicle connectivity with wireless computing systems inside and outside the vehicle 202, and may include a navigation receiver (NAV receiver) 236 for receiving and processing a GPS signal, a BLE ® -Module (BLEM) 238, a Wi-Fi transceiver, a UWB transceiver and / or other wireless transceivers (in Fig. 2 not shown) used for wireless communication (including cellular communication) between the vehicle 202 and other systems (e.g., a vehicle remote key, in Fig. 2 not shown), computers, and modules. The TCU 228 may be arranged in communication with the ECUs 216 via a bus.

[0029] The ECUs 216 may control aspects of vehicle operation and communication using inputs from human drivers, inputs from an autonomous vehicle controller, the POI annotation system 214, and / or via wireless signal inputs received over the wireless connection(s) from other connected devices, such as the user device 204, the server(s) 206, among others.

[0030] The BCM 222 generally includes an integration of sensors, vehicle performance indicators, and variable throttles associated with the vehicle systems, and may include processor-based power distribution circuits that can control functions associated with the vehicle body, such as lights, windows, security, camera(s), audio system(s), speakers, windshield wipers, door locks and access control, and various convenience controls. The BCM 222 may also operate as a gateway for bus and network interfaces to communicate with remote ECUs (in Fig. 2 not shown). In some aspects, the BCM 222 may additionally control the operation of the HUD 110.

[0031] The DAT controller 230 may provide Level 1 to Level 3 automated driving and driver assistance functionality, which may include, for example, active parking assistance, trailer backup assistance, adaptive cruise control, and / or lane keeping, among other features. The DAT controller 230 may also provide aspects of user and environmental input usable for user authentication.

[0032] In some aspects, the automotive computer 210 may connect to an infotainment system 240. The infotainment system 240 may include a touchscreen interface portion and may include voice recognition features, biometric identification capabilities that may identify users based on facial recognition, voice recognition, fingerprint identification, or other biological identification means. In other aspects, the infotainment system 240 may be further configured to receive user instructions via the touchscreen interface portion and / or display notifications (including visual alert notifications), navigation maps, etc., on the touchscreen interface portion.

[0033] The vehicle 202 may further include the HUD 110 as described above in connection with Fig. 1. The automotive computer 210 and / or the POI annotation system 214 may control HUD operation via the BCM 222.

[0034] Certain computing modules may be omitted from the computing system architecture of the automotive computer 210, the VCU 212, and / or the POI annotation system 214. It is readily understood that the Fig. 2 is an example of a possible implementation according to the present disclosure and thus should not be considered limiting or exclusive.

[0035] In some aspects, the POI annotation system 214 may be integrated with and / or executed as part of the ECUs 216. Whether integrated with the automotive computer 210 or the ECUs 216 or operating as a standalone computing system within the vehicle 202, the POI annotation system 214 may include a transceiver 242, a processor 244, and a computer-readable memory 246.

[0036] The transceiver 242 may be configured to receive information / inputs from one or more external devices or systems, e.g., the user device 204, the server(s) 206, and / or the like, via the network 208. Furthermore, the transceiver 242 may transmit notifications (e.g., warning / alarm signals) to the external devices or systems. Furthermore, the transceiver 242 may be configured to receive information / inputs from vehicle components, such as the infotainment system 240, the NAV receiver 236, the vehicle sensor system 234 (including vehicle interior cameras and the IMU), and / or the like. Furthermore, the transceiver 242 may transmit signals (e.g., command signals) to vehicle components, such as the infotainment system 240, the BCM 222, the HUD 110, etc.

[0037] The processor 244 and the memory 246 may be the same as or similar to the processor 218 and the memory 220, respectively. In some aspects, the processor 244 may utilize the memory 246 to store programs in code and / or store data for performing aspects of the disclosure. The memory 246 may be a non-transitory computer-readable memory for storing the POI annotation program code. In some aspects, the memory 246 may additionally store information associated with the vehicle 202, inputs or information obtained from the user device 204 and the server(s) 206, and one or more sensory inputs received from the vehicle's sensor system 234.

[0038] In operation, the driver 104 may activate a POI annotation mode of the HUD 110 by turning on a dedicated actuator / button that may be located inside the vehicle 202, e.g., on a steering wheel of the vehicle, as described above in connection with Fig. 1. In response to the driver 104 turning on the actuator, the actuator may generate and transmit a trigger signal to the transceiver 242. In other aspects, the driver 104 may turn on the POI annotation mode through a voice command, a predefined hand gesture, via the user device 204, the infotainment system 240, and / or the like. In this case, the user device 204 or the infotainment system 240 may generate the trigger signal and transmit it to the transceiver 242.

[0039] The transceiver 242 may receive the trigger signal from the actuator, the user device 204, or the infotainment system 240 and may send the trigger signal to the processor 244. In addition, the transceiver 242 may receive inputs (e.g., real-time images) associated with the driver's eye movement and vehicle orientation / attitude from the vehicle sensing system 234, real-time vehicle geolocation from the NAV receiver 236, and the received information associated with the geographic area in which the vehicle 202 may be located from the server(s) 206. In response to receiving the vehicle information (e.g.,the real-time geolocation of the vehicle and the vehicle orientation / attitude), the inputs associated with the driver's eye movement, and the information associated with the geographic area in which the vehicle 202 may be located, the transceiver 242 may send the received information to the memory 246 for storage purposes.

[0040] The processor 244 may obtain the trigger signal from the transceiver 242. In response to obtaining the trigger signal, the processor 244 may retrieve or obtain the vehicle information, the inputs associated with the driver's eye movement, and the information associated with the geographic area from the memory 246 or directly from the transceiver 242. In some aspects, the processor 244 may determine a gaze direction of the driver 104 relative to a vehicle direction of travel based on the inputs associated with the driver's eye movement. An example view of the driver's gaze direction is shown in Fig. 3,

[0041] As in Fig. 3, the vehicle 202 may be traveling along a direction of travel 302, and the driver 104 may turn their eyes left by an angle "α" relative to the direction of travel 302 to view a building 304 while driving the vehicle 202. Similarly, the driver 104 may turn their eyes right by an angle "β" relative to the direction of travel 302 to view a building 306 while driving the vehicle 202. In some aspects, the processor 244 may be configured to determine the angles "α" and "β" based on inputs associated with the driver's eye movement obtained from the memory 246 or the transceiver 242.

[0042] Although the foregoing description describes an aspect in which the processor 244 determines the driver's gaze direction, in some aspects the gaze direction may be provided by the vehicle sensor system 234 itself or by any other vehicle component.

[0043] In response to determining or obtaining the driver's gaze direction, the processor 244 may determine an object (e.g., the building 304 or the building 306) that the driver 104 may be viewing while driving the vehicle 202 based on the vehicle information, the information associated with the geographic area, and the driver's gaze direction. Specifically, the processor 244 may correlate the vehicle information, the information associated with the geographic area, and the driver's gaze direction to determine the building (e.g., the building 304) that the driver 104 may be viewing. In one example aspect, the processor 244 may perform vector identification and addition to determine the building 304 based on the information described above. An example snapshot illustrating vector addition is shown in Fig. 4 shown.

[0044] In particular, the processor 244 may generate a vehicle vector “V_veh” (as in Fig. 4) in a coordinate space (e.g., on the digital map of the geographic area) based on the real-time geolocation of the vehicle, the vehicle orientation, and information associated with the geographic area in which the vehicle 202 may be located. The processor 244 may then generate a gaze direction vector "V_gaze" based on the driver's gaze direction and the vehicle orientation. The processor 244 may then determine an object (e.g., the building 304) that the driver 104 is viewing based on an intersection of the vector "V_gaze" with the building 304. The processor 244 may determine the vector for the building 304 "V_poi" (or the coordinates associated with the building 304) using an example expression V_poi = V_veh + V_gaze.

[0045] In response to determining the coordinates associated with the building 304, the processor 244 may determine the object (e.g., the building 304) that the driver 104 is currently viewing and an object identifier or a POI identifier for the building 304 based on the information associated with the geographic area. In some aspects, the POI identifier may include a building name (e.g., "Building 1," as described above in connection with Fig. 1) and / or other details associated with Building 304.

[0046] In response to determining the POI identifier, the processor 244 may render or display the POI identifier on the HUD 110 (e.g., via the BCM 222). In this way, when the driver 104 looks away from the building 304 and back to the HUD 110, the driver 104 may see the name (and / or other details) of the building 304 on the HUD 110. This may be a convenience for the driver 104, particularly if the driver is unfamiliar with the details of the building 304 (e.g., the name, entrances and exits, parking location, etc.) or if the driver 104 is driving the vehicle 202 in inclement weather conditions.

[0047] In some aspects, processor 244 may render the POI identifier on HUD 110 for a predefined period of time (e.g., 2 to 5 seconds). In one example aspect, the predefined period of time may be preset. In other aspects, driver 104 may set the predefined period of time via user device 204 or infotainment system 240.

[0048] In further aspects, the processor 244 may render the POI identifier on the HUD 110 until the vehicle 202 passes the building 304. In this case, the processor 244 may determine that the vehicle 202 may have passed the building 304 based on the real-time geolocation of the vehicle obtained from the NAV receiver 236. In response to determining that the vehicle 202 may have passed the building 304, the processor 244 may cause the HUD 110 to stop displaying the POI identifier. In additional aspects, the HUD 110 may display the POI identifier for a preset short period of time (which may be set by the driver 104) even after the vehicle 202 passes the building 304.

[0049] In additional aspects, the driver 104 may set preferences for the type of objects for which the driver 104 may wish to view POI identifiers on the HUD 110. For example, before starting or during a trip, the driver 104 may provide inputs or user settings to the transceiver 242 (e.g., via the user device 204 or the infotainment system 240) indicating that the driver 104 wishes to display POI identifiers for points of interest, restaurants, gas stations, or electric vehicle (EV) charging stations and / or the like on the road 106. In response to receiving the inputs from the driver 104, the transceiver 242 may generate the trigger signal (as described above) and send the trigger signal, along with the inputs received from the driver 104, to the processor 244.

[0050] The processor 244 may obtain the trigger signal and the inputs / user preferences from the transceiver 242. In response to obtaining the trigger signal, the processor 244 may track the driver's eye movement to determine the object (e.g., the building 304) that the driver 104 may be viewing, as described above. In response to determining that the driver 104 may be viewing the building 304, the processor 244 may determine, based on the information associated with the geographic area, whether the building 304 can be associated with the type of object (e.g., restaurants) for which the driver 104 wants to see the POI identifiers. The processor 244 may not render / display the POI identifier for the building 304 on the HUD 110 if the processor 244 determines that the building 304 may not be a restaurant.On the other hand, if the processor 244 determines that the building 304 may be a restaurant, the processor 244 may render / display the POI identifier for the building 304 on the HUD 110. As described above, the processor 244 may display the POI identifier for a predefined period of time or until the vehicle 202 passes the building 304.

[0051] Fig. 5 illustrates a flowchart of an example method 500 for annotating an object on a head-up display of a vehicle according to the present disclosure. Fig.5 may be described with continued reference to previous figures. The following process is exemplary and not limited to the steps described below. Moreover, alternative embodiments may include more or fewer steps than shown or described herein, and may include these steps in an order that differs from the order described in the following exemplary embodiments.

[0052] The method 500 begins at step 502. At step 504, the method 500 may include obtaining the trigger signal by the processor 244. As described above, the processor 244 may obtain the trigger signal when the driver 104 enables the POI annotation mode of the HUD 110 or when the driver 104 enters (and transmits to the transceiver 242) preferences for the types of objects for which the driver 104 wants to see POI identifiers on the HUD 110.

[0053] At step 506, the method 500 may include obtaining, by the processor 244, the vehicle information, the information associated with the geographic area in which the vehicle 202 may be located, and the user's gaze direction, as described above. At step 508, the method 500 may include determining, by the processor 244, an object (e.g., the building 304) in the geographic area that the driver 104 may be viewing based on the vehicle information, the information associated with the geographic area, and the user's gaze direction.

[0054] At step 510, method 500 may include rendering, by processor 244, an object identifier associated with the particular object on HUD 110 for a predefined period of time or until vehicle 202 passes the object, as described above. The object identifier may be, for example, an object / building name.

[0055] The method 500 may end at step 512.

[0056] In the foregoing disclosure, reference has been made to the accompanying drawings, which form a part hereof, and illustrate specific implementations in which the present disclosure may be practiced. It is understood that other implementations may be utilized and structural changes may be made without departing from the scope of the present disclosure. References in the specification to "one embodiment," "an embodiment," etc., indicate that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment.Furthermore, if a feature, structure, or characteristic is described in connection with one embodiment, those skilled in the art will recognize such a feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0057] Furthermore, the functions described in this document may be performed in one or more of hardware, software, firmware, digital components, or analog components. For example, one or more application-specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and procedures described in this document. Certain terms used throughout the specification and claims refer to specific system components. Those skilled in the art will understand that the components may be referred to by other names. This document does not intend to distinguish between components that differ in name but not in function.

[0058] It is also understood that the word "example," as used in this document, is intended to be non-exclusive and non-limiting. More specifically, the word "example," as used in this document, indicates one of several examples, and it is understood that no undue emphasis or preference is placed on the specific example described.

[0059] A computer-readable medium (also referred to as a processor-readable medium) includes any non-transitory (e.g., physical) medium that participates in providing data (e.g., instructions) that can be read by a computer (e.g., by a computer's processor). Such a medium can take many forms, including, but not limited to, non-transitory media and volatile media.

[0060] Computing devices may include computer-executable instructions, where the instructions may be executable by one or more computing devices, such as those listed above, and may be stored on a computer-readable medium.

[0061] With respect to the processes, systems, methods, heuristics, etc. described in this specification, it is understood that although the steps of such processes, etc., have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order different from the order described in this specification. Furthermore, it is understood that certain steps could be performed concurrently, other steps could be added, or certain steps described in this specification could be omitted. In other words, the descriptions of processes in this specification are for the purpose of illustrating various embodiments and should in no way be construed to limit the claims.

[0062] Accordingly, it is to be understood that the foregoing description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples listed will become apparent from a reading of the foregoing description. The scope should be determined not by reference to the foregoing description, but instead by reference to the appended claims, along with the full range of equivalents to which such claims are entitled. It is anticipated and intended that there will be future developments in the art discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. Overall, it is understood that the application is susceptible to modification and variation.

[0063] All terms used in the claims are intended to have their general meaning as known to one of ordinary skill in the art to which the technologies described in this specification apply, unless expressly stated otherwise herein. In particular, the use of the singular articles, such as "a," "an," "the," "the," "the," etc., is intended to refer to one or more of the specified elements, unless a claim expressly limits otherwise.Conditional language, such as "may," "could," "may," or "may possibly," is generally intended to convey that certain embodiments may include certain features, elements, and / or steps, while other embodiments may not include them, unless specifically stated otherwise or clear from the particular context used. Thus, such conditional language is generally not intended to imply that features, elements, and / or steps are in any way required for one or more embodiments.

[0064] In one aspect of the invention, the method includes: obtaining a user preference for a type of object of interest to the user; determining that the object is associated with the type of object of interest to the user based on the information associated with the geographic area; and rendering the object identifier in response to determining that the object is associated with the type of object of interest to the user.

[0065] In one aspect of the invention, the method includes: receiving the user preference; and obtaining the trigger signal in response to receiving the user preference.

[0066] In one aspect of the invention, the method includes rendering the object identifier on the head-up display for a preset period of time.

[0067] In one aspect of the invention, the object identifier comprises an object name.

[0068] According to the present invention, a non-transitory computer-readable storage medium is provided having stored thereon instructions that, when executed by a processor, cause the processor to: acquire a trigger signal; acquire vehicle information, information associated with a geographic area in which a vehicle is located, and a gaze direction of the user in response to acquiring the trigger signal; determine an object in the geographic area that a user is viewing based on the vehicle information, the information associated with the geographic area, and the gaze direction of the user; and render an object identifier associated with the object on a heads-up display until the vehicle passes the object in response to determining the object.

Claims

[1] Vehicle comprising: a head-up display; a transceiver configured to receive vehicle information and information associated with a geographic area in which the vehicle is located; a detection unit configured to detect a viewing direction of a user; and a processor communicatively coupled to the head-up display, the transceiver, and the detection unit, the processor configured to: Obtaining a trigger signal; Obtaining the vehicle information, the information associated with the geographical area, and the user's gaze direction in response to obtaining the trigger signal; Determining an object in the geographic area that a user is viewing based on the vehicle information, the information associated with the geographic area, and the user's viewing direction; and In response to determining the object, rendering an object identifier associated with the object on the head-up display until the vehicle passes the object. [2] The vehicle of claim 1, wherein the vehicle information includes a geolocation of the vehicle and a vehicle orientation. [3] The vehicle of claim 1, further comprising a global positioning system (GPS) receiver and an inertial measurement unit (IMU), wherein the transceiver receives the vehicle information from the GPS receiver and the IMU. [4] The vehicle of claim 1, wherein the information associated with the geographical area comprises a digital map of the geographical area. [5] The vehicle of claim 1, wherein the processor is configured to obtain the trigger signal from a user device, a vehicle infotainment system, or an actuator. [6] The vehicle of claim 1, wherein the processor is further configured to: Obtaining a user preference for a type of object that is of interest to the user; Determine that the object is associated with the type of object of interest to the user based on the information associated with the geographic area; and Render the object identifier in response to determining that the object is associated with the type of object of interest to the user. [7] The vehicle of claim 6, wherein the transceiver is further configured to receive the user preference, and wherein the processor is configured to obtain the trigger signal from the transceiver. [8] The vehicle of claim 1, wherein the processor is further configured to render the object identifier on the head-up display for a preset period of time. [9] The vehicle of claim 8, wherein the transceiver is further configured to receive inputs associated with the preset time period from a user device or a vehicle infotainment system. [10] The vehicle of claim 1, wherein the object identifier comprises an object name. [11] The vehicle of claim 10, wherein the processor is further configured to determine the object name based on the information associated with the geographic area. [12] A method for annotating an object on a head-up display of a vehicle, the method comprising: Obtaining a trigger signal by a processor; Obtaining, by the processor, vehicle information, information associated with the geographic area in which the vehicle is located, and a gaze direction of a user in response to obtaining the trigger signal; Determining, by the processor, an object in the geographic area that a user is viewing based on the vehicle information, the information associated with the geographic area, and the user's viewing direction; and In response to determining the object, rendering, by the processor, an object identifier associated with the object on the head-up display until the vehicle passes the object. [13] The method of claim 12, wherein the vehicle information includes a geolocation of the vehicle and a vehicle orientation. [14] The method of claim 12, wherein the information associated with the geographical area comprises a digital map of the geographical area. [15] The method of claim 12, wherein obtaining the trigger signal comprises obtaining the trigger signal from a user device, a vehicle infotainment system, or an actuator.

Images