System and method for controlling a head-up display in a vehicle

Abstract

To provide systems and methods for controlling a head-up display (HUD) in a vehicle.SOLUTION: One embodiment deactivates the HUD in response to a command from a driver of the vehicle, assigns a level of urgency to an item of information associated with a current vehicular context, and activates the HUD to display the item of information to the driver, when the level of urgency exceeds a predetermined threshold.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The subject matter described herein relates generally to vehicles, and more particularly to systems and methods for controlling a head-up display within a vehicle.

Background Art

[0002] It is becoming increasingly common to equip new vehicles with a head-up display (sometimes referred to as a "head-up display") (HUD). The HUD enables a vehicle's driver to view information such as the vehicle's current speed, external or internal temperature, and navigation information (e.g., an instruction to turn left at the next intersection) without taking their eyes off the road ahead. Despite the usefulness of a vehicle's HUD, there are drivers who consider the information displayed on the HUD to be distracting or obstructive, and such drivers will deactivate the operation of the HUD. If the operation of the HUD is deactivated, the driver may not be able to obtain emergency and important information matters, including matters related to safety.

Summary of the Invention

[0003] An example of a system for controlling a head-up display (HUD) in a vehicle is presented here. The system comprises one or more processors and memory communicatively coupled to one or more of those processors. The memory, when executed by one or more processors, stores a HUD deactivation module that contains instructions to stop the HUD in response to a command from the vehicle's driver. The memory, when executed by one or more processors, also stores an urgency assessment module that contains instructions to assign an urgency level to information items associated with the current status of the vehicle. The memory, when executed by one or more processors, also stores a HUD activation module that contains instructions to activate the HUD to display information items to the driver when the urgency level exceeds a predetermined threshold.

[0004] Another embodiment is a non-transient computer-readable medium for controlling a head-up display (HUD) in a vehicle, which is executed by one or more processors and stores in one or more processors commands to stop the HUD in response to a command from the driver of the vehicle. The commands also cause one or more processors to assign a level of urgency to informational items associated with the current status of the vehicle. The commands also cause one or more processors to activate the HUD to display the informational items to the driver when the level of urgency exceeds a predetermined threshold. In other embodiments, a method for controlling a head-up display (HUD) in a vehicle is disclosed. The method comprises stopping the operation of the HUD in response to a command from the driver of the vehicle. The method also includes assigning an urgency level to informational items associated with the current status of the vehicle. The method also includes activating the HUD to display the informational items to the driver when the urgency level exceeds a predetermined threshold. [Brief explanation of the drawing]

[0005] To allow for a more detailed understanding of the features enumerated above, more specific descriptions of the disclosure, some of which are illustrated in the accompanying drawings, may be provided. However, it should be noted that the accompanying drawings only illustrate possible implementations of this disclosure and should therefore not be considered a limitation on its scope. The disclosure may allow for other implementations.

[0006] [Figure 1] This figure illustrates one embodiment of a vehicle capable of implementing the system and method disclosed herein. [Figure 2] This is a functional block diagram of an embodiment of a head-up display control system. [Figure 3] This figure illustrates the interior of a vehicle equipped with a head-up display, according to an embodiment of the invention. [Figure 4] This figure illustrates information displayed on a vehicle's head-up display, according to an embodiment of the invention. [Figure 5] This figure illustrates information displayed on a vehicle's head-up display, according to another embodiment of the invention. [Figure 6] As an example of the invention, this is a flowchart of a method for controlling a head-up display in a vehicle, according to an embodiment.

[0007] For ease of understanding, the same reference numerals are used whenever possible to designate common and identical elements in the figures. Additionally, elements of one or more embodiments can be advantageously adapted for use in other embodiments described herein. [Modes for carrying out the invention]

[0008] In various embodiments of a head-up display (HUD) control system, the system evaluates the urgency or importance of various information items associated with the current vehicle status that can potentially be displayed on the HUD, which has been previously deactivated by the driver or other user. In various embodiments, if the level of urgency associated with those items exceeds a predetermined threshold, the HUD is activated to display those information items to the driver. This solves the problem that the driver may not receive urgent safety warnings or other important information if the driver has previously deactivated the HUD. In some embodiments, the system automatically deactivates the HUD again after it has displayed urgent information items to the driver.

[0009] In some embodiments, the HUD control system detects and tracks the driver's gaze direction and displays urgent information in the portion of the HUD's field of view (FOV) that coincides with the detected direction of the driver's gaze. In other words, the system displays urgent information in a specific portion of the HUD that the driver's gaze is already directed towards in order to increase the probability that the driver will see the displayed information.

[0010] In other embodiments, if the HUD control system detects that the driver's gaze is directed anywhere outside the HUD's FOV (for example, the driver is looking out the driver's side window), the system may emit some kind of audible signal (e.g., an electronic sound, a beep, a computer-generated voice warning or message) instead of activating the HUD and displaying urgent information on the HUD, or the system may emit an audible signal in addition to activating the HUD and displaying urgent information.

[0011] In some embodiments, a driver or other user involved with the vehicle can configure preferences for the HUD control system. Specifically, the system can receive one or more preferences from the driver / user via a user interface to configure different types of information items, predetermined urgency thresholds, or both.

[0012] Referring to Figure 1, an example of a vehicle 100 capable of implementing the system and method disclosed herein is shown. The vehicle 100 may include the HUD control system 170, or its components and / or modules. As used herein, “vehicle” is any form of powered transport device. In one or more implementations, the vehicle 100 may be an automobile. In some implementations, the vehicle 100 may be any other form of powered transport device. The vehicle 100 may include the HUD control system 170, or functions that support or interact with the HUD control system 170, and thus benefit from the functions considered herein. It will be understood that while the configuration is described herein in relation to an automobile, the implementations are not limited to automobiles. Rather, the implementations of the basic ideas considered herein can be applied to any type of vehicle. An example of vehicle 100 as used herein is similarly applicable to any device that can incorporate the system or method described herein.

[0013] Vehicle 100 also includes various elements. It will be understood that in various implementations, vehicle 100 does not need to have all of the elements shown in Figure 1. Vehicle 100 can have any combination of the various elements shown in Figure 1. Furthermore, vehicle 100 can have additional elements in addition to the elements shown in Figure 1. In some embodiments, vehicle 100 can be implemented without one or more of the elements shown in Figure 1, including the HUD control system 170. Although the various elements are shown in Figure 1 as being located inside vehicle 100, it will be understood that one or more of these elements can be located outside vehicle 100. Furthermore, the elements shown can be physically separated by a considerable distance. As shown in Figure 1, vehicle 100 can communicate with one or more other network nodes 185 via network 190. Other such network nodes 185 may include, for example, cloud servers, edge servers, roadside units (RSUs), or other infrastructure (e.g., traffic signals at intersections), users' mobile devices, and / or other connected vehicles (vehicles equipped with always-on internet connectivity). For example, vehicle 100 can receive traffic or weather information from one or more cloud or edge servers. When communicating with servers, vehicle 100 can use technologies such as cellular data. When communicating with other vehicles, vehicle 100 can use technologies such as dedicated narrow-area communication (DSRC) or Bluetooth® Low Energy (Bluetooth® LE) (low-power communication mode).

[0014] Some possible elements of vehicle 100 are shown in Figure 1 and described in relation to the subsequent drawings. However, descriptions of many of the elements in Figure 1 are provided after the discussion of Figures 2-6 for the purpose of brevity of this description. Additionally, it will be noted that, for the sake of brevity and clarity of the examples, reference numbers are repeated between different figures where appropriate to indicate corresponding or similar elements. Furthermore, the discussion outlines the gist of numerous specific details in order to provide a complete understanding of the embodiments described herein. However, those skilled in the art will understand that the embodiments described herein can be put into practice using various combinations of these elements.

[0015] Vehicle 100 includes a sensor system 120. The sensor system 120 may include one or more vehicle sensors 121. The vehicle sensors 121 may include one or more positioning systems, such as dead reckoning systems, or Global Navigation Satellite Systems (GNSS), such as the Global Positioning System (GPS). The vehicle sensors 121 may also include, for example, a vehicle-bus sensor that outputs speed and steering angle data related to vehicle 100. The sensor system 120 may also include one or more environmental sensors 122. The environmental sensors 122 may include a radar sensor 123, a LiDAR (radio-aided lighting) sensor 124, a sonar sensor 125, and a camera 126. The environmental sensors 122 can be used, for example, to detect and recognize objects in the environment outside vehicle 100 (e.g., other vehicles, pedestrians, cyclists, animals, traffic signs, obstacles, construction machinery, etc.). The environmental sensor 122 is particularly useful for supporting the functions of the advanced driver-assistance system (ADAS) 180.

[0016] Vehicle 100 also includes a communication system 130. The communication system 130 includes an input system 131, an output system 132, and an audio device 134. The output system 132 may include one or more display devices. For example, the output system 132 may include a center console touchscreen display. In the various embodiments considered herein, the output system 132 includes a HUD 133. In addition to the HUD 133, the output system 132 may include other display devices (not shown in Figure 1).

[0017] As mentioned above, vehicle 100 includes ADAS 180. ADAS 180 can use information obtained via sensor system 120, map data 116, and other sources (e.g., traffic information server or other connected vehicles) to predict or detect potentially dangerous traffic situations and help the driver of vehicle 100 avoid an accident. In some embodiments, ADAS 180 issues warnings or alerts so that the driver can take corrective action. In other embodiments, ADAS 180 intervenes temporarily in the steering and / or acceleration of vehicle 100 to avoid an accident when necessary. For example, in some embodiments, ADAS 180 can detect that vehicle 100 is too close to a preceding (leading) vehicle in the same lane and issue a forward collision warning and / or actively intervene by automatically applying the brakes to prevent vehicle 100 from rear-ending the preceding vehicle. Similar warnings and / or proactive interventions can be applied, for example, to lane departures and potentially dangerous cross-traffic at intersections.

[0018] Referring to Figure 2, one embodiment of the HUD control system 170 of Figure 1 is further illustrated. In this embodiment, the HUD control system 170 is shown to include one or more processors 110 of the vehicle 100 of Figure 1. Generally, one or more processors 110 can be part of the HUD control system 170, the HUD control system 170 can include one or more processors separate from the one or more processors 110 of the vehicle 100, or the HUD control system 170 can access one or more processors 110 through a data bus or other communication path, depending on the embodiment.

[0019] In one embodiment, memory 210 stores a HUD deactivation module 220, an urgency assessment module 230, a HUD activation module 240, a gaze detection module 250, and a configuration module 260. Memory 210 is random access memory (RAM), read-only memory (ROM), a hard disk drive, flash memory, or other suitable memory for storing modules 220, 230, 240, 250, and 260. Modules 220, 230, 240, 250, and 260 are, for example, computer-readable instructions that, when executed by one or more processors 110, cause that one or more processors 110 to perform the various functions disclosed herein.

[0020] As shown in Figure 2, the HUD control system 170 interfaces with the sensor system 120, the communication system 130 (in particular including the HUD 133), and the ADAS 180 (see Figure 1). In some embodiments, the HUD control system 170 can also communicate with other network nodes 185 (e.g., cloud servers, edge servers, RSUs or other infrastructure systems, user mobile devices, and / or other connected vehicles) via the network 190, as discussed above in relation to Figure 1.

[0021] The HUD control system 170 can store data such as user selection settings 280 in the database 270. The user selection settings 280 will be further considered below.

[0022] The HUD operation stop module 220 generally includes an instruction to stop the operation of the HUD 133 in response to a command from a human driver of the vehicle 100, when executed by one or more processors 110. In some embodiments, the human driver can issue an operation stop command through a menu-driven user interface based on, for example, buttons, switches, knobs, or touchscreens. In other embodiments, the driver can stop the operation of the HUD 133 through a voice command processed by a voice recognizer interfacing with the HUD operation stop module 220, or through a gesture (e.g., hand movement) processed by a gesture-based user interface interfacing with the HUD operation stop module 220.

[0023] What "the operation is stopped" of the HUD 133 can mean can be different according to special embodiments. In some embodiments, when the driver or other user stops the operation of the HUD 133, the power of the HUD 133 is turned off. In different embodiments, when the driver or other user stops the operation of the HUD 133, it puts the HUD 133 into a low-power standby mode. In such a low-power standby mode, the display is not operating and nothing is displayed. In still other embodiments, when the driver or other user stops the operation of the HUD 133, it makes the HUD 133 in a state where the power is fully supplied but nothing is displayed on the display.

[0024] The urgency assessment module 230 generally includes instructions that cause the one or more processors 110, when executed by the one or more processors 110, to assign a level of urgency to information items associated with the current situation of the vehicle (i.e., the situation regarding vehicle 100). Generally, the situation of the vehicle includes (1) the current state or status of vehicle 100 (such as position, speed, route, operating status of vehicle components or systems), (2) one or more environmental states outside of vehicle 100 (e.g., vehicles, pedestrians, obstacles, weather conditions, traffic density, road conditions, etc.), or (3) a combination of the foregoing. For example, vehicle 100 having tires with low pressure in one or more tires detected via vehicle sensor 121 (the state or status of the vehicle) is one example of a situation of the vehicle. Another example is vehicle 100 approaching an intersection where a pedestrian (an object detected by environmental sensor 122) is crossing the road in front of vehicle 100 at a crosswalk. Another example is vehicle 100 traveling at 58 miles per hour on a road wet due to a storm. Yet another example is vehicle 100 located at a special location (e.g., represented by GPS coordinates) and traveling 100 feet behind a leading vehicle detected in the same traffic lane. The situation of the vehicle can be a non-dangerous (low risk of danger) or a dangerous (high risk of danger to vehicle 100 and its occupants or other humans using the road) state.

[0025] The urgency assessment module 230 assigns a level of urgency to information items associated with a given vehicle condition. In some embodiments, the urgency assessment module 230 assigns such information items a numerical urgency value or score (e.g., 0 to 10, or 0 to 100). In some embodiments, the urgency assessment module 230 assigns information items associated with a vehicle condition a broader urgency level, such as "low," "medium," or "high." In some embodiments, these techniques are combined. For example, one information item may have an urgency score of "25," which falls into the "low" urgency category. Another information item may have an urgency score of "52," which falls into the "medium" urgency category. Yet another information item may have an urgency score of "83," which falls into the "high" urgency category.

[0026] A situation in which ADAS 180 outputs a forward collision warning to the HUD control system 170 indicating that vehicle 100 will rear-end a lead vehicle in 5 seconds if no corrective action is taken is an example of a potentially dangerous vehicle situation. In this situation, the instruction "Apply the brakes!" that can be output to HUD 133 (after HUD 133 is activated) is an example of information with a high level of urgency. Conversely, information indicating that vehicle 100 is currently traveling at 44 miles per hour in a 45 miles per hour speed zone has a relatively lower level of urgency. For whatever reason, a driver who has stopped the operation of HUD 133 is likely to want to see the former ("Apply the brakes!") appear on the restarted HUD 133, but is unlikely to want to see the latter ("44 miles per hour").

[0027] Other examples of urgent informational matters include, but are not limited to, text or graphic messages associated with lane departure warnings received from ADAS180, text or graphic messages associated with intersection traffic warnings received from ADAS180, severe weather warnings, vehicle health warnings (e.g., engine oil pressure, fuel level, charge status, engine coolant temperature, low tire pressure), open door warnings, seat belt warnings, speed limit warnings, and navigation diagrams (e.g., for urgent navigation notifications such as not to go the wrong way on a one-way street). In some embodiments, incoming text or email messages that are marked as urgent by the sender, or identified as such from the analysis of their content, are also likely to receive an assigned level of urgency from the Urgency Assessment Module 230. In the case of safety warnings from ADAS180, the shorter the time period until an undesirable event (e.g., collision or lane departure) is predicted to occur, the higher the level of urgency the Urgency Assessment Module 230 will assign to the associated informational matter. Those skilled in the art will recognize that there is a close relationship between the level of urgency that the urgency assessment module 230 assigns to an information item and the level of risk (or danger) associated with the condition of the vehicle to which the information item relates.

[0028] Other examples of information items with a low level of urgency associated with them, without limitation, include GPS navigation information, reports of favorable weather conditions, speedometer readings, odometer readings, and external temperature readings.

[0029] The HUD activation module 240 is typically executed by one or more processors 110 and includes a command to one or more processors 110 to activate the HUD 133 (which was previously deactivated) to display information to the driver when the level of urgency associated with an information item exceeds a predetermined threshold. The predetermined threshold can vary depending on the embodiment. For example, in one embodiment, the predetermined threshold is a numerical value to which the urgency score, a number assigned to the information item by the urgency assessment module 230, is compared. In other embodiments, the predetermined threshold can correspond to an urgency category such as "medium". When the urgency assessment module 230 assigns a "high" urgency level to an information item, the urgency level exceeds the "medium" threshold level, and the HUD activation module 240 restarts the HUD 133 to display the information item.

[0030] As is known to those skilled in the art, there are different types of HUDs used in vehicles. Various embodiments of the HUD control system 170 described herein can work in conjunction with any of the diverse types of HUDs. For example, in one embodiment, the HUD 133 is a projection-based display system that projects text and graphics onto at least a portion of the windshield of the vehicle 100. This is illustrated in Figure 3.

[0031] Figure 3 illustrates the interior 300 of a vehicle 100 equipped with a HUD 133 according to an embodiment as an example of the invention. In this embodiment, the projection portion of the HUD 133 (not shown in Figure 3) can be positioned behind the instrument cluster panel (meter panel: a group of instruments in the driver's seat) of the vehicle 100. As is also known to those skilled in the art, by employing optical techniques and devices, the displayed information or image can be made to appear as if it were farther away from the driver than it actually is (i.e., farther away from the inner surface of the windshield). In some embodiments, the field of view (FOV) of the HUD 133 extends about 20 degrees to the left and right from the center of the display area, i.e., from the viewpoint of the driver seated in the driver's seat. In other embodiments, the FOV of the HUD 133 includes the entire windshield of the vehicle 100. In the example shown in Figure 3, the FOV of the HUD 133 is limited to a specific portion of the windshield 310.

[0032] Figure 4 illustrates an information item 410 displayed on the HUD 133 according to an embodiment as an example of the invention. In the example in Figure 4, the urgency assessment module 230 assigns a high level of urgency to the information item 410 ("Apply the brakes!") associated with a forward collision warning received from the ADAS 180. In this case, the HUD activation module 240 determines that the level of urgency associated with the information item 410 exceeds a predetermined threshold. Therefore, the HUD activation module 240 activates the HUD 133 to display the urgent information item 410.

[0033] In some embodiments, the HUD deactivation module 220 described above is executed by one or more processors 110 and includes a further command to that one or more processors 110 to automatically deactivate the HUD 133 after it has displayed the information item 410 to the driver. How long the urgent information item 410 remains visible on the HUD 133 before the HUD 133 is deactivated again depends on the situation and the particular embodiment. Generally, the information item 410 remains displayed as long as the vehicle situation that caused it is relevant to the driver of the vehicle 100. This may be two or three seconds in some situations and slightly longer in others.

[0034] The gaze detection module 250 is typically executed by one or more processors 110 and includes instructions that cause one or more processors 110 to detect the direction of the driver's gaze in the vehicle 100. To detect the direction of the driver's gaze, the gaze detection module 250 can use cameras inside the vehicle 100 in conjunction with gaze detection / tracking algorithms well known in this technology. Some of these algorithms detect the direction of the driver's line of sight and use geometric theorems to determine what the driver is looking at inside or outside the vehicle 100 (e.g., external objects through the windshield 310, external objects through the side windows, the instrument panel, etc.). Determining the direction of the driver's gaze (i.e., the direction the driver is looking and what the driver is looking at) can support several additional functions included in some embodiments of the HUD control system 170.

[0035] For example, in one embodiment, the HUD activation module 240, when executed by one or more processors 110, includes further instructions to cause one or more processors 110 to display information items 410 on the portion of the HUD 133's FOV (i.e., the portion of the windshield through which the driver is currently looking out) that matches the detected gaze direction. This is illustrated in Figure 5.

[0036] Figure 5 illustrates information item 410 ("Apply the Brakes!") displayed on the HUD 133 of vehicle 100 according to an embodiment as an example of the invention. In the example of Figure 5, the gaze detection module 250 determines that the driver is currently looking out through the portion of the windshield 310 where "Apply the Brakes!" (the information item 410 whose level of urgency exceeds the predetermined threshold discussed above) is displayed. The projection portion of the HUD 133 adjusts the position where the information item 410 is displayed to match its detected position. This function helps ensure that the driver sees the information item 410 associated with the forward collision warning generated by the ADAS 180.

[0037] In other examples, when executed by one or more processors, the HUD activation module 240 includes further instructions to cause one or more processors to emit an audible signal via the audio device 134 when the driver's detected gaze direction (detected by the gaze detection module 250) is not within the FOV of the HUD 133. In other words, the HUD activation module 240 causes an audible signal to be emitted when the gaze detection module 250 detects that the driver is not looking at a location within the FOV of the HUD 133. For example, as considered above, if it is detected that the driver is looking out through the driver's side window, the HUD activation module 240 may cause the HUD 133 to emit some kind of audible signal (e.g., an electronic sound, a beep, or a computer-simulated voice warning or message) instead of activating the HUD 133 to display emergency information 410 on the HUD 410. In a different embodiment, the HUD activation module 240 may activate the HUD 133 to attract the driver's attention to the displayed information item 410, and in addition to displaying the urgent information item 410 on the HUD 133, it may also emit an audible signal.

[0038] The configuration module 260 is generally executed by one or more processors 110 and includes instructions that cause one or more processors 110 to receive one or more selection settings from the driver (or other user) to configure various types of information items 410, predetermined urgency thresholds, or levels of urgency associated with both, depending on the embodiment. In some embodiments, the configuration module 260 gives the driver some control over the level of urgency that the urgency assessment module 230 assigns to certain types of information items 410 associated with specific vehicle conditions. For example, in one embodiment, the driver may choose to increase the urgency level of certain types of severe weather warnings, particularly if those types of severe weather warnings are of particular concern in the area where the driver lives and works. For example, in a certain area of ​​the United States, tornadoes occur more frequently than in other areas, and the driver may want to give those types of warnings a higher priority.

[0039] In some embodiments, the driver can configure an urgency threshold for at least some types of information items 410 to control which information items 410 cause the HUD activation module 240 to activate the HUD 133 to display the information items 410 after the driver has stopped operating the HUD 133. For example, in one embodiment, the driver can specify that the HUD 133 should be activated to display information items 410 in a special configurable category (e.g., "weather warning") that has an urgency score greater than "75" (on a scale of 0 to 100). In other embodiments, the driver can specify that the HUD 133 should be activated to display information items 410 in a special category that has a "high" level of urgency (in an urgency scheme including "low," "medium," and "high"). In some embodiments, the urgency threshold is the same for all types or categories of information items 410. In other embodiments, there may be multiple different urgency thresholds for different types or categories of information items 410.

[0040] Figure 6 is a flowchart of Method 600 for controlling a HUD 133 in a vehicle, according to an embodiment as an example of the invention. Method 600 is considered in terms of the HUD control system 170 in Figure 2. Although Method 600 is considered in combination with the HUD control system 170, it should be recognized that Method 600 is not limited to being implemented within the HUD control system 170, and the HUD control system 170 is just one example of a system that can implement Method 600.

[0041] In block 610, the HUD deactivation module 220 deactivates the HUD 133 in response to a command from the driver of the vehicle 100. As discussed above, in some embodiments, the driver can issue a deactivation command via a menu-driven user interface, such as a button, switch, knob, or touchscreen. In other embodiments, the driver can deactivate the HUD 133 via a voice command processed by a voice recognition device that interfaces with the HUD deactivation module 220, or via a gesture (e.g., a hand movement) processed by a gesture-based user interface that interfaces with the HUD deactivation module 220.

[0042] In block 620, the urgency assessment module 230 assigns a level of urgency to the information item 410 associated with the current vehicle status. As discussed above, in some embodiments, the urgency assessment module 230 assigns a numerical urgency value or score (e.g., 0 to 10, or 0 to 100) to the information item 410. In some embodiments, the urgency assessment module 230 assigns a broader urgency level, such as "low," "medium," or "high," to the information item associated with the vehicle status. In some embodiments, these techniques are combined as described above.

[0043] In block 630, the HUD activation module 240 activates the HUD 133 to display the information item 410 to the driver when the urgency level associated with the information item 410 exceeds a predetermined threshold. As discussed above, in one embodiment, the predetermined threshold is a numerical value to which the urgency score, assigned to the information item 410 by the urgency assessment module 230, is compared. In other embodiments, the predetermined threshold can correspond to an urgency category such as "medium". If the urgency assessment module 230 assigns a "high" urgency level to the information item, the urgency level exceeds the "medium" threshold level, and the HUD activation module 240 restarts the HUD 133 to display the information item.

[0044] In some embodiments, method 600 also includes the HUD deactivation module 220 automatically deactivating the HUD 133 after the HUD 133 has displayed information item 410 to the driver, as discussed above. In some embodiments, method 600 utilizes a gaze detection module 250 that detects the driver's gaze direction to support the HUD activation module 240 in displaying the urgent information item 410 in the portion of the HUD 133's FOV that coincides with the driver's detected gaze direction. In some embodiments, method 600 includes the HUD activation module 240 emitting an audible signal when the driver's gaze direction detected by the gaze detection module 250 is not within the FOV of the HUD 133. In some specific embodiments, the audible signal may be emitted in combination with displaying an urgent information item 410 (for example, an audible signal used to attract the driver's attention to the HUD 133), or the audible signal may be emitted instead of activating the HUD 133 to display such information item 410. In some embodiments, the method 600 includes the configuration module 260 receiving one or more selection settings from the driver to configure the level of urgency, a predetermined threshold as discussed above, or both, which are assigned to various types of information items 410 by the urgency assessment module 230.

[0045] Herein, Figure 1 is examined in sufficient detail as an example vehicle environment in which the systems and methods disclosed herein can be implemented. In some examples, vehicle 100 can be configured to selectively switch between autonomous mode, one or more semi-autonomous operating modes, and / or manual mode. Such switching, also referred to as a handover when transitioning to manual mode, can be implemented in appropriate methods known or to be developed in the future. “Manual mode” means that all or most of the vehicle’s navigation and / or steering are performed according to input received from a user (e.g., a human driver / operator).

[0046] Vehicle 100 may include one or more processors 110. In one or more configurations, one or more processors 100 may be the main processors of vehicle 100. For example, one or more processors 110 may be electronic control units (ECUs). Vehicle 100 may include one or more data stores 115 for storing one or more types of data. Data stores 115 may include volatile and / or non-volatile memory. Examples of suitable data stores 115 include RAM, flash memory, ROM, PROM (programmable read-only memory), EPROM, EEPROM (electrically erasable programmable read-only memory), registers, magnetic disks, optical disks, hard drives, or any other suitable storage media, or any combination thereof. Data stores 115 may be components of one or more processors 110, or data stores 115 may be functionally connected to one or more processors 110 for use by those processors 110. The term “functionally connected” as used throughout this description may include direct or indirect connections, including connections without direct physical contact.

[0047] In one or more configurations, one or more data stores 115 may include map data 116. Map data 116 may include maps of one or more geographical areas. In some examples, map data 116 may include information or data about roads, traffic control devices, road signs, structures, features, and / or large objects that can serve as landmarks in one or more geographical areas. In one or more configurations, map data 116 may include one or more terrain maps 117. Terrain maps 117 may include information about the ground, terrain, roads, surface, and / or other features of one or more geographical areas. In one or more configurations, map data 116 may include one or more static obstacle maps 118. Static obstacle maps 118 may include information about one or more static obstacles located within one or more geographical areas.

[0048] One or more data stores 115 may contain sensor data 119. In this context, “sensor data” means any information about the sensors equipped on the vehicle, including the function and other information about such sensors. As described below, the vehicle 100 may include a sensor system 120. The sensor data 119 may be associated with one or more sensors of the sensor system 120. For example, in one or more configurations, the sensor data 119 may contain information about one or more lidar sensors 124 of the sensor system 120.

[0049] As described above, the vehicle 100 may include a sensor system 120. The sensor system 120 may include one or more sensors. "Sensor" means any device, component, and / or system that can detect and / or sense something. One or more sensors may be configured to detect and / or sense in real time. As used herein, the term "real time" means a level of processing responsiveness that a user or system perceives as sufficiently immediate for a particular process or decision to be made, or that allows a processor to keep pace with some external process.

[0050] In an arrangement where the sensor system 120 includes multiple sensors, the sensors can function independently of each other. Alternatively, two or more sensors can operate in combination with each other. In such a case, the two or more sensors can form a sensor network. The sensor system 120 and / or one or more sensors can be functionally connected to one or more processors 110, data stores 115, and / or other elements of the vehicle 100 (including any of the elements shown in Figure 1).

[0051] The sensor system 120 may include any suitable type of sensor. Various examples of different types of sensors are described here. However, it will be understood that the implementation is not limited to the specific sensors described. The sensor system 120 may include one or more vehicle sensors 121. The vehicle sensors 121 can detect, determine, and / or sense information about the vehicle 100 itself. In one or more configurations, the vehicle sensors 121 may be configured to detect and / or sense changes in the position and / or orientation of the vehicle 100, for example, based on inertial acceleration. In one or more configurations, the vehicle sensors 121 may include one or more accelerometers, one or more gyroscopes, inertial measurement units (IMUs), dead reckoning systems, global navigation satellite systems (GNSS), global positioning systems (GPS), navigation systems 147, and / or other suitable sensors. The vehicle sensors 121 may be configured to detect and / or sense one or more characteristics of the vehicle 100. In one or more configurations, the vehicle sensor 121 may include a speedometer for determining the current speed of the vehicle 100.

[0052] Alternatively, or additionally, the sensor system 120 may include one or more environmental sensors 122 configured to acquire and / or sense driving environment data. "Driving environment data" includes any data or information about the external environment in which the vehicle is located, or one or more parts thereof. For example, one or more environmental sensors 122 may be configured to detect, measure, and / or sense obstacles in at least part of the external environment of the vehicle 100, and / or information / data about such obstacles. One or more environmental sensors 122 may be configured to detect, measure, measure, and / or sense other things in at least part of the external environment of the vehicle 100, such as nearby vehicles, lane markers, signs, traffic signals, traffic signs, lanes, crosswalks, curbs close to the vehicle 100, or objects outside the road.

[0053] Various examples of sensors in the sensor system 120 are described here. The example sensors may be part of one or more environmental sensors 122 and / or one or more vehicle sensors 121. Furthermore, the sensor system 120 may include operator sensors that function to track or monitor aspects related to the driver / operator of the vehicle 100. However, it will be understood that the implementations are not limited to the specific sensors described. For example, in one or more configurations, the sensor system 120 may include one or more radar sensors 123, one or more lidar sensors 124, one or more sonar sensors 125, and / or one or more cameras 126.

[0054] Vehicle 100 may further include a communication system 130. The communication system 130 may include one or more components configured to facilitate communication between Vehicle 100 and one or more communication sources. As used herein, a communication source is a person or device with which Vehicle 100 can communicate, such as an external network, a computing device, an operator or passenger of Vehicle 100, or other. As part of the communication system 130, Vehicle 100 may include an input system 131. An "input system" includes any device, component, system, element or arrangement, or group thereof, that enables information / data to be input into the machine. In one or more examples, the input system 130 may receive input from a vehicle occupant (e.g., driver or passenger). Vehicle 100 may also include an output system 132. An "output system" includes any device, component or arrangement, or group thereof, that enables information / data to be presented to one or more communication sources (e.g., a person, a vehicle occupant, etc.). The communication system 130 may further include certain elements that are part of the input system 131 or the output system 132, such as the HUD 133 and one or more audio devices 134 (e.g., a speaker and a microphone), or that can interact with them.

[0055] Vehicle 100 may include one or more vehicle systems 140. Various examples of one or more vehicle systems 140 are shown in Figure 1. However, vehicle 100 may include more, fewer, or different vehicle systems. Although special vehicle systems are defined separately, it should be recognized that each or any of those systems or parts may be combined or separated via hardware and / or software within vehicle 100. Vehicle 100 may include a propulsion system 141, a braking system 142, a steering system 143, a throttle system 144, a transmission system 145, a signaling system 146, and / or a navigation system 147. Each of these systems may include one or more devices, components, and / or combinations thereof that are currently known or will be developed in the future.

[0056] One or more processors 110 can be functionally connected to communicate with various vehicle systems 140 and / or their individual components. For example, returning to Figure 1, one or more processors 110 can be in a communication state to send information to and receive information from various vehicle systems 140 in order to control the movement, speed, steering, course, direction, etc., of the vehicle 100.

[0057] Vehicle 100 may include one or more modules, at least some of which are described herein. A module can be implemented as computer-readable program code that, when executed by a processor 110, implements one or more of the various processes described herein. The processor 110 may be a device such as a CPU capable of receiving and executing one or more threads of instructions for the purpose of performing tasks. One or more modules may be components of one or more processors 110, and one or more modules may run on and / or be distributed among other processing systems to which one or more processors 110 are functionally connected. A module may contain instructions (e.g., program logic) that are executable by one or more processors 110. Alternatively, or additionally, one or more data stores 115 may contain such instructions.

[0058] In one or more configurations, one or more of the modules described herein may include artificial or computational intelligence elements such as neural networks, fuzzy logic, or other machine learning algorithms. Furthermore, in one or more configurations, one or more modules may be distributed among multiple modules described herein. In one or more configurations, two or more of the modules described herein may be combined into a single module.

[0059] Detailed embodiments are disclosed herein. However, it should be understood that the disclosed embodiments are intended to be illustrative only. Accordingly, the details of the specific structures and functions disclosed herein should not be interpreted as restrictive, but merely as grounds for the claims and as representative grounds to instruct those skilled in the art to adopt various embodiments herein in substantially any appropriately detailed structure. Furthermore, the terms and phrases used herein are not intended to be restrictive, but are intended to provide an understandable description of possible embodiments. Various embodiments are shown in Figures 1-6, but embodiments are not limited to the exemplified structures or applications.

[0060] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products across various implementations. In this regard, each block in a flowchart or block diagram can represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions described in a block may occur in a different order than shown in the diagram. For example, depending on the functions involved, two consecutively shown blocks may execute substantially simultaneously, or blocks may execute in reverse order.

[0061] The systems, components, and / or methods described herein can be implemented in hardware or a combination of hardware and software, and can be implemented centrally in a single processing system or distributed across several interconnected processing systems. Any type of processing system or other device adapted to perform the methods described herein is suitable. A typical combination of hardware and software may be a processing system having computer-readable program code that, when deployed and executed, controls the processing system to perform the methods described herein. The systems, components, and / or methods can also be embedded in computer-readable storage devices such as computer program products or other data program storage devices that are machine-readable and substantially contain a program of machine-executable instructions for performing the methods described herein. These elements can also be embedded in application products that, when deployed in a processing system, are capable of performing these methods, possessing all the features that enable the implementations of the methods described herein.

[0062] Furthermore, the arrangements described herein can take the form of a computer program product contained in one or more computer-readable media having computer-readable program code stored, contained, or embedded. Any combination of one or more computer-readable media can be used. The computer-readable media can be a computer-readable signal medium or a computer-readable storage medium. The phrase “computer-readable storage medium” means a non-temporary storage medium. The computer-readable storage medium can be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof, but is not limited to the following. More specific examples (not an exhaustive list) of computer-readable storage media include: In other words, this includes electrical connections having one or more wires, portable computer diskettes, hard disk drives (HDDs), solid-state drives (SSDs), RAM, ROM, EPROM or flash memory, optical fibers, portable compact disk read-only memory (CD-ROMs), digital multipurpose discs (DVDs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium can be any tangible medium that contains or can store programs for use by or in conjunction with instruction execution systems, apparatus, or devices.

[0063] The program code contained in the computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical fiber, cable, RF, or any suitable combination thereof. The computer program code for performing the operations for this configuration is Java®. TMThe program code can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and traditional procedural programming languages ​​such as the C programming language or similar languages. The program code can run entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can connect to the user's computer through any type of network, including a LAN or WAN, or can connect to an external computer (for example, through the Internet using an Internet Service Provider).

[0064] In the above description, certain details are highlighted to provide a complete understanding of various realizations. However, those skilled in the art will understand that the invention can be practiced without these details. In other examples, well-known structures are not shown, nor are they described in detail, to avoid unnecessarily obscuring the description of the realization. Unless the circumstances require otherwise, the word “equipped” and its variations such as “equipped” and “containing” should be interpreted in an open, comprehensive sense as “including,” but not restrictive, throughout the specification and the claims that follow. Furthermore, the headings provided herein are for convenience only and do not describe the scope or meaning of the claimed invention.

[0065] Throughout this specification, references to “one or more embodiments” or “one embodiment” mean that any special features, structures, or characteristics described in relation to an embodiment are included in at least one embodiment. Therefore, phrases “in one or more embodiments” or “in one embodiment” found in various places throughout this specification do not necessarily all refer to the same embodiment. Furthermore, special features, structures, or characteristics can be combined in any suitable way in one or more embodiments. Also, as used in this specification and the accompanying claims, the singular forms “one” and “it” also include plurals unless the context explicitly indicates otherwise. It should also be noted that the term “or” is generally used to include “and / or” unless the context explicitly indicates otherwise.

[0066] The headings and subheadings used herein (such as “Background Art” and “Summary of the Invention”) are intended solely for the purpose of providing an overall structure of the subject matter within this disclosure and are not intended to limit the disclosure of the technology or any aspect thereof. The enumeration of multiple embodiments describing features is not intended to exclude other embodiments having additional features or incorporating different combinations of the features described. The terms “equipment” and “include,” and their variations, as used herein, are intended to be non-restrictive so as not to exclude other similar items that may also be useful in the apparatus or method of the technology, whether in a sequence or in a list. Similarly, the terms “possible to” and “can be,” and their variations, are intended to be non-restrictive so as not to exclude other embodiments of the technology that do not include certain elements or features, when an enumeration of embodiments that can or may be equipped with a certain element or feature.

[0067] The broad teachings of this disclosure can be implemented in a variety of forms. Therefore, although this disclosure includes specific examples, the true scope of the disclosure should not be limited in this way, as other modifications will be obvious to those skilled in the art by studying the specification and the claims below. References herein to one or more embodiments mean that any particular feature, structure, or characteristic described in relation to an embodiment or special system is included in at least one or more embodiments or embodiments. The phrase "in one embodiment" (or a variation thereof) does not necessarily refer to the same embodiment or embodiment. It should also be understood that the steps of the various methods considered herein do not need to be performed in the same order as shown, and that not every step of each method is required in every embodiment or embodiment.

[0068] Generally, the term "module" as used herein includes routines, programs, objects, components, data structures, etc., that perform a special task or realize a special data type. In a further embodiment, memory generally stores the described module. The memory associated with the module may be a buffer or cache embedded in the processor, RAM, ROM, flash memory, or other suitable electronic storage medium. In a further embodiment, the module considered in this disclosure is realized as an application-specific integrated circuit (ASIC), a hardware component of a system-on-a-chip (SoC), a programmable logic array (PLA), or other suitable hardware component embedded with a defined set of configurations (e.g., instructions) for performing the disclosed function.

[0069] The term “one” as used herein is defined as one or more. The term “multiple” as used herein is defined as two or more. The term “other” as used herein is defined as at least the second or subsequent. The terms “contains” and / or “have” as used herein are defined as containing (i.e., open language). The phrase “at least one of ~ and ~” as used herein means any and all possible combinations of one or more of the associated and enumerated items, and that include them. For example, the phrase “A, B, and at least one of B” includes A only, B only, C only, or any combination thereof (e.g., AB, AC, BC, or ABC).

[0070] The foregoing descriptions of the realizations are provided for illustrative and descriptive purposes only. They are not intended to exhaust or limit the disclosure. Individual elements or features of a particular realization are not generally limited to that particular realization and are interchangeable and can be used in a selected realization, even if not specifically shown or described, as long as they are applicable. They can also be modified in many ways. Such modifications should not be considered deviations from the disclosure, and all such modifications are intended to fall within the scope of the disclosure.

[0071] While the foregoing is directed toward embodiments of the disclosed apparatus, system, and method, other further embodiments of the disclosed apparatus, system, and method can be devised without departing from the basic scope of the disclosure. That scope is determined by the subsequent claims. The inventions disclosed herein include the following embodiments: [Aspect 1] A system for controlling the head-up display (HUD) inside a vehicle, One or more processors, A memory that is communicably coupled to one or more processors, A HUD operation stop module, which, when executed by one or more processors, includes a command to stop the operation of the HUD in response to a command from the driver of the vehicle, An emergency assessment module, which, when executed by one or more processors, includes instructions that cause one or more processors to assign an emergency level to information items associated with the current status of the vehicle, A HUD activation module, which is executed by one or more processors, includes a command to activate the HUD to display the information to the driver when the level of urgency exceeds a predetermined threshold, The memory that stores it, A system equipped with these features. [Aspect 2] The system according to embodiment 1, wherein the HUD operation stop module, when executed by the one or more processors, includes a further command to the one or more processors to automatically stop the operation of the HUD after the HUD has displayed the information to the driver. [Aspect 3] The gaze detection module further includes, which, when executed by the one or more processors, contains instructions that cause the one or more processors to detect the direction in which the driver is looking, The system according to embodiment 1, wherein the HUD activation module, when executed by the one or more processors, includes further instructions to the one or more processors to display the information items in the portion of the HUD's field of view that coincides with the gaze direction. [Aspect 4] The system according to Embodiment 1, wherein the information items are one of the following: a forward collision warning received from the vehicle's advanced driver-assistance system (ADAS), a lane departure warning received from the ADAS, and a cross-traffic warning received from the ADAS. [Aspect 5] The system according to Embodiment 1, wherein the information items are one of the following: severe weather warnings, vehicle condition warnings, open door warnings, seatbelt warnings, speed limit warnings, navigation diagrams, text messages, and email messages. [Aspect 6] The gaze detection module further includes, which, when executed by the one or more processors, contains instructions that cause the one or more processors to detect the direction in which the driver is looking, The HUD activation module, when executed by one or more processors, includes further instructions to cause one or more processors to emit an audible signal when the gaze direction is not within the field of view of the HUD. The system described in Embodiment 1. [Aspect 7] The system of Embodiment 1, wherein when the HUD stops operating, it is in one of the following states: powered off, low-power standby mode, or powered but no display. [Aspect 8] The system according to embodiment 1, further comprising a configuration module that, when executed by the one or more processors, causes the one or more processors to receive one or more selection settings from the driver to configure at least one of the urgency level and the predetermined threshold. [Aspect 9] A non-temporary computer-readable medium for controlling a head-up display (HUD) in a vehicle, which, when executed by one or more processors, provides to the one or more processors: In response to a command from the driver of the vehicle, the operation of the HUD is stopped. Assign an urgency level to the information items associated with the current status of the vehicle. When the level of urgency exceeds a predetermined threshold, a command is stored to activate the HUD in order to display the information to the driver. Non-temporary computer-readable media. [Aspect 10] The non-temporary computer-readable medium according to embodiment 9, wherein the instruction includes a further instruction causing one or more processors to detect the driver's gaze direction and to display the information in the portion of the HUD's field of view that coincides with the gaze direction. [Aspect 11] The non-temporary computer-readable medium according to embodiment 9, wherein the information is one of the following: a forward collision warning received from the vehicle's advanced driver-assistance system (ADAS), a lane departure warning received from the ADAS, and a cross-traffic warning received from the ADAS. [Aspect 12] The information contained herein is one of the following: severe weather warnings, vehicle condition warnings, open door warnings, seatbelt warnings, speed limit warnings, navigation diagrams, text messages, and email messages, in the non-temporary computer-readable medium according to Embodiment 9. [Aspect 13] A method for controlling a head-up display (HUD) in a vehicle, In response to a command from the driver of the vehicle, the operation of the HUD is stopped, Assigning a level of urgency to information items associated with the current status of the vehicle, When the level of urgency exceeds a predetermined threshold, the HUD is activated to display the information to the driver. A method that has [Aspect 14] The method according to embodiment 13, further comprising the function of automatically stopping the operation of the HUD after the HUD has displayed the information to the driver. [Aspect 15] To detect the direction in which the driver is looking, The information items are displayed in the portion of the HUD's field of view that coincides with the gaze direction. The method according to embodiment 13, further comprising the above. [Aspect 16] The method according to embodiment 13, wherein the information is one of the following: a forward collision warning received from the vehicle's advanced driver-assistance system (ADAS), a lane departure warning received from the ADAS, and a cross-traffic warning received from the ADAS. [Aspect 17] The method according to embodiment 13, wherein the information items are one of the following: severe weather warnings, vehicle condition warnings, open door warnings, seatbelt warnings, speed limit warnings, navigation diagrams, text messages, and email messages. [Aspect 18] To detect the direction in which the driver is looking, When the gaze direction is not within the field of view of the HUD, an audible signal is emitted. The method according to embodiment 13, further comprising the above. [Aspect 19] The method according to aspect 13, wherein when the HUD is stopped from operating, it is in one of the following states: powered off, low-power standby mode, or powered but no display. [Aspect 20] The method according to embodiment 13, further comprising receiving one or more selection settings from the driver to constitute the level of urgency and at least one of the predetermined thresholds.

Claims

1. A system for controlling the head-up display (HUD) inside a vehicle, One or more processors, A memory that is communicably coupled to one or more processors, A HUD shutdown module, which, when executed by one or more processors, includes a command to stop the operation of the HUD in response to a command from the driver of the vehicle, An emergency assessment module, which, when executed by one or more processors, includes instructions that cause one or more processors to assign an emergency level to information items associated with the current status of the vehicle, A HUD activation module, which is executed by one or more processors, includes an instruction to one or more processors to activate the HUD to display the information to the driver when the level of urgency exceeds a predetermined threshold, The memory that stores it, Equipped with, The gaze detection module further includes, which, when executed by the one or more processors, contains a command to cause the one or more processors to detect the direction in which the driver is looking, The HUD boot module, when executed by one or more processors, includes further instructions to cause one or more processors to display the information in the portion of the HUD's field of view that coincides with the gaze direction. system.

2. The system according to claim 1, wherein the HUD shutdown module, when executed by the one or more processors, includes a further command to the one or more processors to automatically shut down the operation of the HUD after the HUD has displayed the information to the driver.

3. The system according to claim 1, wherein the information items are one of the following: a forward collision warning received from the vehicle's advanced driver assistance system (ADAS), a lane departure warning received from the ADAS, and a cross-traffic warning received from the ADAS.

4. The system according to claim 1, wherein the information items are one of the following: severe weather warnings, vehicle condition warnings, open door warnings, seatbelt warnings, speed limit warnings, navigation diagrams, text messages, and email messages.

5. The HUD startup module includes, when executed by one or more processors, a further instruction causing one or more processors to emit an audible signal when the gaze direction is not within the field of view of the HUD. The system according to claim 1.

6. The system according to claim 1, wherein when the HUD is stopped from operating, it is in one of the following states: power off, low-power standby mode, or power supplied but no display.

7. The system according to claim 1, further comprising a configuration module that, when executed by the one or more processors, includes instructions causing the one or more processors to receive one or more selection settings from the driver to constitute at least one of the urgency level and the predetermined threshold.

8. A non-temporary computer-readable medium for controlling a head-up display (HUD) in a vehicle, which, when executed by one or more processors, provides to the one or more processors: In response to a command from the driver of the vehicle, the operation of the HUD is stopped. Assign an urgency level to the information items associated with the current status of the vehicle. When the level of urgency exceeds a predetermined threshold, the HUD is activated to display the information to the driver. The driver's gaze direction is detected, and the information is displayed in the portion of the HUD's field of view that coincides with the gaze direction. It stores the instructions. Non-temporary computer-readable media.

9. The non-temporary computer-readable medium according to claim 8, wherein the information is one of a forward collision warning received from the vehicle's advanced driver-assistance system (ADAS), a lane departure warning received from the ADAS, and a cross-traffic warning received from the ADAS.

10. The non-temporary computer-readable medium according to claim 8, wherein the information is one of the following: severe weather warnings, vehicle condition warnings, open door warnings, seatbelt warnings, speed limit warnings, navigation diagrams, text messages, and email messages.

Images