Display method, display device, storage medium and vehicle

Abstract

The invention relates to the technical field of projection display, in particular to a display method, display equipment, a storage medium and a vehicle. The projection area formed by the HUD display device is divided into the block display area and the extended display area, the block display area supports on-demand display of different functions in different blocks, and the extended display area can synchronously display different auxiliary contents according to the state of the block display area viewed by a user. And linkage between the block display area and the extended display area is realized. The information distribution of the projection area is flexibly configured, the auxiliary content of conventional display is expanded, and the watching experience of a user is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of projection display technology, and in particular to a display method, a display device, a storage medium and a vehicle. Background Art

[0002] HUD (Head Up Display) is a new way to realize in-vehicle display by using reflection on the vehicle windshield. Specifically, the display light is emitted by the optical machine of the HUD display device, and is projected on the windshield through the corresponding optical lens to produce a corresponding virtual image, which forms an enhanced display effect with the real world outside the windshield. However, the interface configuration in the projection area of ​​the HUD display device is pre-designed, and the size and position of each display element are relatively fixed. This can neither meet the user's personalized configuration needs nor make full use of the information display space in the projection area, making it difficult for users to have a good viewing experience. Summary of the invention

[0003] The purpose of the present application is to provide a display method, a display device, a storage medium and a vehicle, which solve the technical problems in the prior art that the information distribution projected by the HUD display device is fixed, the arrangement of display elements lacks flexibility, and the space utilization rate of the projection area is low.

[0004] In order to solve the above technical problems, this application adopts the following technical solutions.

[0005] In a first aspect, the present application provides a display method, comprising: The HUD display device projects display light onto a transparent surface to form a projection area for information display; The projection area includes a divided display area and an extended display area, the divided display area includes at least a first block and a second block, the first block is configured with a first display element having a first function, and the second block is configured with a second display element having a second function; In response to the user's viewing direction corresponding to the first block, the extended display area is combined with displaying the first auxiliary content of the first function; in response to the user's viewing direction corresponding to the second block, the extended display area is combined with displaying the second auxiliary content of the second function.

[0006] According to the above description, an optional implementation method can utilize the block display area to project basic information, facilitate arrangement in the corresponding divided blocks, display additional auxiliary content related to the basic information in the extended display area, improve the richness of the content, and at the same time efficiently utilize the carrying potential of the extended display area through time-sharing display.

[0007] In an optional implementation of the first aspect, the block display area is arranged below the extended display area.

[0008] In an optional implementation of the first aspect, the enhanced display element that triggers display is configured to be displayed in the extended display area.

[0009] According to the above description, in an optional implementation manner, the upper half of the projection area is more facing the road at a certain distance in front of the vehicle, and the enhanced display elements are configured in the extended display area above, which can facilitate the realization of virtual and real fitting in actual projection.

[0010] In an optional implementation of the first aspect, a size of the block display area is larger than a size of the extended display area.

[0011] According to the above description, the optional implementation method supports free arrangement of display elements with more functions, and reuses the extended display area to display auxiliary content of corresponding functions under specific conditions, thereby improving the utilization efficiency of information projection.

[0012] In an optional implementation manner of the first aspect, the first block is configured with a first display element having a first function, and the second block is configured with a second display element having a second function, including: The first display element is a static icon, a dynamic icon, a preview interface or a detail interface corresponding to the first function; The second display element is a static icon, a dynamic icon, a preview interface or a detail interface corresponding to the second function.

[0013] According to the above description, the optional implementation method supports the use of a variety of display forms in the block display area, and uses display forms with different information output intensities to transmit information as needed, thereby reducing unnecessary information load on users.

[0014] In an optional implementation of the first aspect, in response to the viewing direction of the user corresponding to the first block, the extended display area cooperates with displaying the first auxiliary content of the first function, including: The first auxiliary content is determined according to a driving state of the vehicle.

[0015] In an optional implementation of the first aspect, in response to the viewing direction of the user corresponding to the second block, the extended display area cooperates with displaying the second auxiliary content having the second function, including: The second auxiliary content is determined according to the driving state of the vehicle.

[0016] In an optional implementation manner of the first aspect, determining the first auxiliary content according to the driving state of the vehicle includes: In response to a risk level of the user driving the vehicle being lower than a first preset value, the first auxiliary content includes entertainment information.

[0017] In an optional implementation of the first aspect, the entertainment information includes radio channels or advertising information.

[0018] According to the above description, an optional implementation method determines the content in the extended display area based on the actual driving status of the vehicle, and further determines the auxiliary content based on the degree of user involvement in driving, thereby improving the diversity of information display and the user's driving pleasure while ensuring safety.

[0019] In an optional implementation manner of the first aspect, the first function is music playing; The determining the first auxiliary content according to the driving state of the vehicle includes: In response to the vehicle being in an assisted driving state, the first auxiliary content includes lyrics of played music.

[0020] In an optional implementation manner of the first aspect, the first function is music playing; The determining the first auxiliary content according to the driving state of the vehicle includes: In response to a driving speed of the vehicle being lower than a second preset value, the first auxiliary content includes lyrics of played music.

[0021] In an optional implementation of the first aspect, the first display element is a music playback control interface.

[0022] In an optional implementation of the first aspect, the first display element is the name or album cover of the music being played.

[0023] According to the above description, an optional implementation supports displaying corresponding lyrics in an extended display area in response to a user's intention to watch the music playback function in a safe driving state, thereby improving the vehicle entertainment experience.

[0024] In an optional implementation manner of the first aspect, the first function is navigation display; The determining the first auxiliary content according to the driving state of the vehicle includes: In response to the vehicle approaching a complex road section, the first auxiliary content includes a route indication mark attached to the road ahead.

[0025] In an optional implementation manner of the first aspect, the first function is navigation display; The determining the first auxiliary content according to the driving state of the vehicle includes: In response to the vehicle approaching a complex road section, the first auxiliary content includes detailed navigation guidance.

[0026] In an optional implementation of the first aspect, the first display element is a brief navigation guide.

[0027] According to the above description, an optional implementation method can display brief navigation instructions such as navigation direction and distance only in the first block when the vehicle is on a normal road section, and only when the vehicle approaches complex sections such as ramps will the expanded display area be triggered to display more specific navigation information, thereby improving the clarity of navigation and avoiding long-term occupation of too much projection area.

[0028] In an optional implementation manner of the first aspect, the first function is navigation display or vehicle speed display; The determining the first auxiliary content according to the driving state of the vehicle includes: In response to the vehicle approaching a traffic light intersection, the first auxiliary content includes a real-time status of the traffic light.

[0029] In an optional implementation of the first aspect, the first display element is a brief navigation guide.

[0030] In an optional implementation of the first aspect, the first display element is a vehicle speed value updated in real time.

[0031] According to the above description, an optional implementation method can display the real-time status of the traffic light, such as the current traffic light color and countdown, in the extended display area, so as to remind the driver to pay attention to the indication of the traffic light and avoid running a red light by mistake.

[0032] In an optional implementation manner of the first aspect, the first function is navigation display or vehicle speed display; The determining the first auxiliary content according to the driving state of the vehicle includes: In response to the vehicle being in an overtaking state, the first assistance content includes overtaking progress.

[0033] In an optional implementation manner of the first aspect, the first function is navigation display or vehicle speed display; The determining the first auxiliary content according to the driving state of the vehicle includes: In response to the vehicle achieving overtaking in the assisted driving state, the first assistance content includes overtaking progress.

[0034] In an optional implementation of the first aspect, the response to the vehicle being in an overtaking state is determined based on external camera analysis and / or driving intention judgment.

[0035] In an optional implementation of the first aspect, the first display element is a brief navigation guide.

[0036] In an optional implementation of the first aspect, the first display element is a vehicle speed value updated in real time.

[0037] According to the above description, in an optional implementation mode, when the vehicle is speeding, the driver is prompted with the time required for overtaking and the current completion progress, thereby assisting the driver in making correct overtaking decisions.

[0038] In an optional implementation of the first aspect, the first block and the second block are symmetrically arranged.

[0039] In an optional implementation of the first aspect, the first block and the second block are arranged asymmetrically.

[0040] In an optional implementation of the first aspect, the display method further includes: The first display element and the second display element are determined according to settings input by a user.

[0041] According to the above description, the optional implementation method supports a more free configuration mode, which improves the flexibility of information layout in the block display area.

[0042] In an optional implementation of the first aspect, the display method further includes: The first display element and the second display element are determined according to the usage frequency of the user.

[0043] According to the above description, optional implementation methods can arrange the functions that users need to view frequently in the center of the block display area according to the user's usage frequency, so as to facilitate the user's retrieval, or delete the functions that the user does not use for a long time from the block display area to facilitate the reasonable distribution of other functions in the block display area.

[0044] In an optional implementation of the first aspect, in response to the viewing direction of the user corresponding to the first block, the extended display area cooperates with displaying the first auxiliary content of the first function, including: triggering display of the first auxiliary content according to the detection that the user's sight stays on the first block for more than a first time threshold; In response to the viewing direction of the user corresponding to the second block, the extended display area cooperates with displaying the second auxiliary content having the second function, including: The display of the second auxiliary content is triggered according to the detection that the user's sight stays on the second block for more than a second time threshold.

[0045] In an optional implementation of the first aspect, the first time threshold is the same as the second time threshold.

[0046] In an optional implementation of the first aspect, in response to the viewing direction of the user corresponding to the first block, the extended display area cooperates with displaying the first auxiliary content of the first function, including: triggering display of the first auxiliary content according to detecting that the user's sight stays on the first block and receiving a key input from the user; In response to the viewing direction of the user corresponding to the second block, the extended display area cooperates with displaying the second auxiliary content having the second function, including: Based on detecting that the user's sight stays on the second block and receiving the user's key input, the display of the second auxiliary content is triggered.

[0047] According to the above description, the optional implementation manner can optimize the triggering method for displaying auxiliary content in the extended display area to reduce possible false triggering.

[0048] In an optional implementation of the first aspect, the response to the user's viewing direction corresponds to the first block or the response to the user's viewing direction corresponds to the second block, which is determined according to a driver monitoring system in the vehicle.

[0049] According to the above description, an optional implementation manner may monitor the user's viewpoint position and line of sight direction in real time through a camera inside the vehicle, thereby determining the viewing focus position of the driver.

[0050] In an optional implementation of the first aspect, the display method further includes: At most one of the first auxiliary content and the second auxiliary content is displayed at the same time.

[0051] In an optional implementation manner of the first aspect, the first auxiliary content and the second auxiliary content are displayed at most one at the same time, including: The extended display area displays the first auxiliary content; In response to the second auxiliary content needing to be displayed in the extended display area, display of the first auxiliary content is hidden.

[0052] According to the above description, in the optional implementation manner, the content in the extended display area is always synchronized with the user's viewing behavior, providing the driver with the ability to choose while driving, thereby optimizing the entire information viewing experience.

[0053] In an optional implementation manner of the first aspect, the first auxiliary content and the second auxiliary content are displayed at most one at the same time, including: The extended display area displays the first auxiliary content; The display of the first auxiliary content is automatically hidden according to the driving status of the vehicle.

[0054] According to the above description, an optional implementation method can make the first auxiliary content disappear automatically after the vehicle passes through the auxiliary section, completes overtaking, etc., so as to reduce interference to the driver.

[0055] In an optional implementation of the first aspect, the display method further includes: When no triggering condition is met, the extended display area is displayed in a default mode.

[0056] According to the above description, an optional implementation manner may display the extended display area in a transparent state without displaying content or in a default mode such as displaying specific display elements, thereby achieving free switching in the extended display area.

[0057] In a second aspect, the present application provides a display device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the display method described in the first aspect when executing the computer program.

[0058] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the display method described in the first aspect are implemented.

[0059] In a fourth aspect, the present application provides a vehicle comprising the display device described in the second aspect or the computer-readable storage medium described in the third aspect.

[0060] Compared with the prior art, the present application divides the projection area formed by the HUD display device into two parts: a block display area and an extended display area. The block display area supports on-demand display of different functions in different blocks, while the extended display area can synchronously display different auxiliary content according to the state of the user viewing the block display area, thereby realizing the linkage between the block display area and the extended display area. The present application supports flexible configuration of information distribution in the projection area, and also expands the auxiliary content of the conventional display, greatly improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the description of the technical solution. Obviously, the drawings described below are only some examples recorded in the present application, and a person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

[0062] Figure 1 Schematic diagram of HUD projection display in some examples of this application.

[0063] Figure 2 This is a schematic diagram of the projection area division in some examples of this application.

[0064] Figure 3 This is a schematic diagram of the projection area division in some examples of this application.

[0065] Figure 4 This is a schematic diagram of the projection area division in some examples of this application.

[0066] Figure 5 This is a schematic diagram of the projection area division in some examples of this application.

[0067] Figure 6 Schematic diagram of the electronic and electrical architecture based on which projection displays are provided in some examples of this application.

[0068] Figure 7 This is a schematic diagram of the distribution of projection area information in some examples of this application.

[0069] Figure 8 In some examples of this application, the display area is expanded to display schematic diagrams on demand.

[0070] Fig. 9 In some examples of this application, the display area is expanded to display schematic diagrams on demand.

[0071] Fig.10 In some examples of this application, the display area is expanded to display schematic diagrams on demand.

[0072] Fig.11 In some examples of this application, the display area is expanded to display schematic diagrams on demand.

[0073] Fig.12 In some examples of this application, the display area is expanded to display schematic diagrams on demand.

[0074] Fig.13 Schematic diagram of HUD display device module in some examples of this application.

[0075] Fig.14 This is a schematic diagram of the composition of the HUD display device in some examples of this application.

[0076] Fig.15 This is a schematic diagram of projection display in a vehicle in some examples of the present application. DETAILED DESCRIPTION

[0077] The present application will be described in detail below in conjunction with the accompanying drawings, but the content of the description is merely some examples recorded in the present application and does not limit the present application. Changes in structure, method, function, etc. made by ordinary technicians in this field based on these examples are included in the protection scope of the present application.

[0078] It should be noted that the same reference numerals or marks may be used in different examples, but these do not represent an absolute connection in structure or function. In addition, the "first", "second", etc. mentioned in each example are only for the convenience of description, and do not represent an absolute distinction in structure or function, nor can they be understood as indicating or implying the relative importance or the number of corresponding objects. Unless otherwise specified, the "at least one" that may be involved in the description refers to one or more, and "more than one" refers to two or more.

[0079] In addition, when expressing a feature, the character " / " can indicate that there is an or relationship between the front and back associated objects. For example, head-up display / head-up display can be expressed as head-up display or head-up display. When expressing an operation, the character " / " can indicate that there is a division relationship between the front and back associated objects. For example, the magnification M=L / P can be expressed as L (virtual image size) divided by P (image source size). Moreover, the "and / or" in different examples is only to describe the association relationship between the front and back associated objects. This association relationship can include three situations. For example, a concave mirror and / or a convex mirror can be expressed as the existence of a concave mirror alone, the existence of a convex mirror alone, and the existence of both a concave mirror and a convex mirror.

[0080] HUD projection display mainly uses the optical reflection principle to reflect the imaging light to be displayed through a transparent surface into the viewer's eyes. The human eye can see the virtual image information in the opposite direction of the light. Correspondingly, the transparent surface can be the windshield of the vehicle, and the windshield can be used as a display screen to display the navigation instructions of the vehicle, the vehicle's driving speed, etc. Figure 1As shown, the HUD display device may include at least an optical machine 1, a first reflector 2, a second reflector 3, etc., wherein the optical machine 1 includes a backlight source and an image source (not shown), the backlight source is used to provide illumination light and adjust the brightness of the illumination light according to control, for example, the backlight source may be an LED (Light Emitting Diode), a laser, etc. Under the illumination light provided by the backlight source, the image source adjusts the corresponding display content according to control and projects the display light from the surface of the image source, for example, the image source may be an LCD (Liquid Crystal Display), a DMD (Digital Micromirror Devices), a MEMS (Micro-Electro-Mechanical System) micromirror, a LCOS (Liquid Crystal on Silicon), etc. The first reflector 2 and the second reflector 3 can project the display light projected by the optical machine 1 onto the windshield 4, so as to customize the optical path in a smaller space and meet different projection display requirements. The first reflector 2 and the second reflector 3 can be set as a concave mirror, a convex mirror, a concave lens, a convex lens, etc. according to the requirements of optical planning, and the surface shape of the lens can adopt a free-form surface. Optionally, at least one of the first reflector 2 and the second reflector 3 can also be adjusted to a certain degree of angle, so as to change the projection position of the display light on the windshield 4 to meet the needs of viewers of different heights. The display light of the optical machine 1 is finally reflected on the windshield 4 of the vehicle to form a virtual image 5. When the human eye 6 observes the virtual image 5 facing the windshield 4, it can feel a certain sense of depth, just like watching a real object at a specific distance outside the windshield. The virtual image 5 can be the navigation instruction content, vehicle speed, etc. as described above. It should be added that, according to the characteristics of different optical machines, the HUD display device can also be provided with a diffuser. In some examples, the HUD display device can also include Fresnel lenses, waveguide optical devices, diffractive optical devices, holographic optical devices, tapered optical fibers, etc.

[0081] As mentioned above, the virtual image formed by the HUD display device projecting on a transparent surface such as a vehicle windshield is generally viewed from the outside of the vehicle windshield. The specific form of the virtual image is the projection area that can carry various display elements (see Fig.15The projection area 50 is facing the road ahead and can also form a fitting relationship with the road to display corresponding enhanced display elements. The content in the projection area can be determined by the display light projected by the optical machine, and can be specifically realized by configuring the image source display content in the optical machine. Accordingly, the range of the projection area is related to the display surface range of the image source. Therefore, the HUD display device can form the information required for driving in front of the windshield, thereby reducing the frequency of the driver looking down at the instrument panel or central control screen, mobile phone, etc. However, in actual applications, the UI (User Interface) content in the projection area is currently mainly pre-designed by the HUD supplier or the automobile OEM. The position and size of the display elements arranged in the projection area are fixed after the design is completed, and the flexibility of use is greatly limited. At the same time, most of the projection area displays some permanent display information at the bottom of the projection area, such as vehicle speed display, which can basically not be modified after leaving the factory. In different scenarios, drivers pay different attention to the projection information of different functions. For example, if there is a missed call during driving, they may pay attention to the caller information, pay more attention to navigation or ADAS information in complex road conditions, and pay more attention to vehicle speed and other information in high-speed conditions. The fixed information distribution will inevitably affect the user experience. In addition, in order to realize the function of augmented reality, a large part of the space at the top of the projection area will be reserved to provide space for the display of navigation-related enhanced display elements. However, the demand for navigation does not always exist, and the actual frequency of occurrence is often low, which leads to the waste of most of the display space in the projection area. In the case that the field of view of the current projection area itself is limited, the capacity of information display is further compressed, and many positions in the projection area are blank for a long time.

[0082] In some examples, in order to improve the utilization efficiency of each position in the projection area, the adaptability of information distribution can be enhanced through flexible configuration, so as to meet the user's various information viewing needs, attention habits, and aesthetic preferences. Optionally, the HUD display device can collect the user's usage habits, so that the display interaction in the projection area can give the driver positive feedback in real time during the actual driving process. Figure 2As shown, the projection area 50 formed by the projection of the HUD display device is divided into a block display area 501 and an extended display area 502 according to the overall arrangement. The block display area 501 and the extended display area 502 can be arranged adjacent to each other, and there is no overlapping part between the two. There can be a clear dividing line between the block display area 501 and the extended display area 502, or they can be processed in a blank manner. In this example, the extended display area 502 is located above the block display area 501. Since the real scene facing the front of the windshield from bottom to top corresponds to the road from near to far, when arranging the navigation-related enhanced display elements to be displayed in the extended display area 502, the enhanced display elements can be well matched with the road ahead to achieve virtual and real fit, meeting the specific position and distance requirements. Optionally, the block display area 501 and the extended display area 502 may also adopt a left-right distribution structure, for example, the block display area 501 may be set on both sides of the extended display area 502, and the corresponding permanent display information may be arranged from top to bottom on both sides of the projection area 50, while the extended display area 502 is located in the middle, and some important information may be displayed as needed to improve the user's attention. Similarly, the extended display area 502 may be set on both sides of the block display area 501. In more examples, the extended display area 502 may be set in the middle of the projection area 50, and the block display area 501 may be set around the extended display area 502. Conversely, the extended display area 502 may also be set around the block display area 501 to provide extended interaction for the functions in the block display area 501.

[0083] In some examples, for the block display area, display elements of various functions can be freely arranged, and most of these display elements need to present the basic information required for driving through projection display, such as the speed display that realizes similar functions to the instrument panel, etc. This information is key information provided to the driver and needs to be always displayed in the projection area. Therefore, a space is opened up through the block display area to accommodate these always displayed functions, and the display elements corresponding to these functions can support simultaneous existence. And some auxiliary content is configured in the extended display area. These auxiliary contents have a link relationship with the specific function display elements in the block display area, which can be related content in the corresponding function. These auxiliary contents do not need to be displayed in the projection area all the time, but there is a need to display them when specific trigger conditions are met. Therefore, the extended display area can use a time-sharing method to reuse the same space to carry this information. The auxiliary content between different functions can be displayed in turn, or combined according to a special arrangement method. Display, save the space occupied by all auxiliary content when displayed uniformly, and reduce unnecessary information interference to the driver. In a specific example, the size of the block display area is larger than the size of the extended display area, so that the block display area reserves more space to carry multiple display elements displayed simultaneously, providing more options for users.

[0084] like Figure 3 As shown, the block display area 501 is divided into several blocks, which may specifically include block 5011 and block 5012. Block 5011 and block 5012 are distributed left and right in the block display area 501, and optionally, they may also be distributed up and down. There may be a clear dividing line between block 5011 and block 5012, or the dividing line may be hidden, and the concept of separation between each other is only provided when configuring the distribution of display elements. A display element may be independently configured in each block, and the display element is associated with a specific function. In more examples, the display element may be expressed in the form of a static icon, a dynamic icon, a preview interface, or a details interface corresponding to a specific function. Among them, the static icon may be a graphic or function name that distinguishes different functions, and the user may decide whether to select and view it specifically based on the static icon. The dynamic icon may be displayed in the form of animation on the basis of satisfying the function of distinguishing the static icon, so that it is more dynamic. The preview interface can directly display part of the information of a specific function in the display element, and the user can roughly understand the current state of the specific function or the brief information it has through the preview interface, such as the song name in the music playback function, and the brief navigation guide in the navigation display function. The details interface can provide specific information of a specific function based on the preview interface, such as the current vehicle speed value in the vehicle speed display function, and the speed value will be updated in real time as the actual vehicle speed changes.

[0085] In some examples, such as Figure 4 As shown, more blocks can be included in the block display area 501. In this example, blocks 5011, 5012, and 5013 are symmetrically arranged in a left-right distribution. Display elements of three functions can be set in blocks 5011, 5012, and 5013, or different forms of display elements corresponding to 1-2 functions can be set. For example, the preview interface and the detail interface of the first function are set in blocks 5011 and 5012 respectively. Figure 5 As shown, the block division in the block display area 501 can also be set in an asymmetrical manner. For example, in addition to the block 5011 and the block 5012 from left to right, the block 5014, the block 5015 and the block 5013 can be set from top to bottom on the right side. Different blocks have different sizes, which can adapt to display elements of different sizes. Figure 3-5 The block division method in the block display area 501 can be selected by the user in the form of a template. Furthermore, the user can also select the functions and display elements carried by each block by setting, thereby improving the flexibility of information layout. Optionally, different blocks in the block display area 501 can also be divided into positions that are convenient for viewing and positions that are not convenient for viewing, so as to Figure 4 For example, the driver's line of sight is more likely to notice the content in block 5012, so the display elements of more important functions can be arranged in the middle position first. Further, based on the frequency of historical user use of different functions, the display elements corresponding to the frequently used functions are set in the middle block. In some examples, when the number of display elements that need to be arranged in the block display area 501 is greater than the number of divided blocks, the display elements with higher priority are allocated to the divided blocks, and their priority can be determined by referring to parameters such as frequency of use and importance of information.

[0086] In order to strengthen the coordination and linkage between the segmented display area 501 and the extended display area 502, the user's line of sight will be managed first. For example, the driver of the vehicle will respond to the line of sight staying in each segmented display area 501 during driving, triggering the display of corresponding auxiliary content in the extended display area 502. The determination of these auxiliary contents is affected by the driving state of the vehicle, which will be described in detail below. In a specific example, the HUD display device realizes flexible interaction in the extended display area 502 during the projection process, and will obtain environmental information outside the vehicle and driver status information inside the vehicle, etc. This information can be realized using its own integrated sensor equipment. In some examples, such as Figure 6As shown, some of the sensing functions of the HUD display device can reuse the capabilities of the electronic and electrical architecture of the car itself. The electronic and electrical architecture of the car is currently mainly formed by a centralized structure through several domain controllers (referred to as "domain controllers"), which separate sensing from processing, and each domain controller assumes certain functions. The computing power is highly centralized and scalable, and the domain controllers can communicate with each other. In this example, the HUD display device 11 is connected to the cockpit domain controller 13, and the connection capability of the cockpit domain controller 13 can be used to obtain the corresponding sensing data. The cockpit domain controller 13 is mainly responsible for functions such as vehicle-mounted interconnection and infotainment, and is an important component for improving user experience. It can communicate with the ADAS (Advanced Driving Assistance System) domain controller 12 and the DMS (Driver Monitoring System) system 14. Among them, the ADAS domain controller 12 is responsible for integrating the on-board radar, camera and other sensors to realize environmental detection, object recognition and other functions, so it can obtain environmental information outside the vehicle, such as the condition of the vehicle's driving road. The DMS system 14 is responsible for monitoring the driver's status. In addition to identifying whether the driver is fatigued, it can also determine the direction of the driver's line of sight based on the driver's eye position and rotation status. Therefore, it can obtain the driver's status information inside the vehicle, such as whether the driver is viewing the display elements in a specific block.

[0087] In some examples, such as Figure 7 As shown, during the driving process, the vehicle will activate the HUD display device to reflect on the vehicle windshield to form a projection area 50, wherein, in the block display area 501, according to the configuration arrangement, block 5011 displays the name of the song being played, block 5012 displays the current speed value of the vehicle, and block 5013 displays a brief navigation guide. The user can view the content in each block as needed. Correspondingly, when no triggering conditions are met, for example, the user does not keep his eyes on any block, or the external environmental conditions for displaying specific auxiliary content are not met, the default mode will be used for display in the extended display area 502. In this example, the default mode is to display nothing in the extended display area 502, and keep the corresponding area of ​​the windshield completely transparent. Optionally, lane lines that fit the road or specific animation effects can also be displayed in the default mode. As Figure 8 As shown, when viewing the content in the projection area 50, the user can use Figure 6The DMS system in the example obtains the viewing direction of the user in the car, and accordingly, the length and frequency of the user's stay in different blocks can be counted, so that the user's habits can be continuously inferred. More importantly, when the user's viewing direction corresponds to a specific information viewing intention in a specific block, the corresponding expanded content, that is, auxiliary content, will be displayed in the extended display area 502. In this example, the DMS system detects that the user's human eye 6 is facing the block 5011, and the display element of the block 5011 represents the music playback function, which can be the name of the currently playing song, or the music playback control interface or the cover of the album where the currently playing song is located, etc., so it will trigger the display of auxiliary content such as the specific lyrics of the currently playing song in the extended display area 502, so as to improve the complete experience of music playback. Preferably, in order to reduce the false triggering caused by the movement of the user's line of sight, it is not only directly triggered according to the position where the human eye 6 stays, but also combined with the stay time. For example, the display of lyrics will be triggered only when a specific time threshold is exceeded. Optionally, it can also cooperate with the buttons set on the steering wheel to prevent false triggering, that is, when the line of sight of human eye 6 stays in block 5011, the user presses a specific button, and it is determined that the trigger condition is met. In this way, human-computer interaction can be achieved through the user's hand-eye linkage, so as to achieve the effect of conveniently controlling the content in the extended display area 502. Further, in order to improve the safety of function implementation, the extended display area 502 will not display the lyrics of the music playback when the vehicle is driving unsafe, reducing the interference of entertainment information to the driver. In some examples, it can be used Figure 6 The ADAS domain control in the example obtains the driving speed of the vehicle. When the driving speed of the vehicle is lower than the preset value, it is determined that the risk level of the vehicle driving is low, and the lyrics content can be displayed. Optionally, other entertainment information such as radio channels and advertising information can also be displayed on demand. Otherwise, the lyrics cannot be triggered to be displayed in the extended display area 502 to improve driving safety. In some examples, the risk level of the vehicle is determined not by the speed of the vehicle, but by whether the vehicle is in the assisted driving state. It should be noted that the assisted driving state may include adaptive cruise, pilot assistance and complete unmanned driving, that is, the driver's involvement in the vehicle driving is low, which can appropriately meet some entertainment needs. Accordingly, when the vehicle is in the assisted driving state, if the viewing direction of the human eye 6 corresponds to block 5011, it will trigger the display of entertainment information such as the lyrics of the song being played in the extended display area 502.

[0088] In some examples, such as Fig. 9As shown, according to the information needs of vehicle driving, the line of sight of human eye 6 may also stay in block 5013. For example, if there is a complex road section such as a ramp in front of the vehicle, and the driver does not know the specific route, since block 5013 contains corresponding brief navigation instructions, the driver can refer to this information to make driving decisions. In this example, the brief navigation instructions may include the direction of each lane ahead and the distance from the ramp, which is the basic information for navigation. However, this information is not very intuitive, and the driver may not necessarily understand the corresponding driving direction on the actual road. Accordingly, it can be based on Figure 6 The ADAS domain control 12 and the DMS system 14 in the example determine whether the trigger condition is met, that is, whether the line of sight of the human eye 6 stays in the block 5013 and has the intention to view the specific content of the navigation. Figure 8 The example combines judgment by time or key interaction, and the ADAS domain control 12 determines that the vehicle is about to enter a complex road section such as a ramp. At this time, the route indicator is retrieved and displayed on the extended display area 502. The route indicator belongs to an enhanced display element and can be virtual and real with the road ahead, just like the route indicator is built-in on the road. Since the route indicator directly points to the actual ramp entrance, it is more intuitive and easier for drivers to understand. Optionally, the extended display area 502 can also trigger the display of detailed navigation instructions, and the content of its navigation is more detailed than the brief navigation instructions. For example, a small road map can be called up in the extended display area 502, and the corresponding guidance information can be displayed on the small map. In some examples, when judging whether the viewing behavior of the human eye 6 meets the triggering condition, the time threshold judgment of staying in block 5013 can be different from the time threshold judgment of staying in block 5011. This may be related to the function and intention in the actual block. For example, the intention of viewing the song name in block 5011 often requires a longer stay, so a longer time threshold can be set to prevent false triggering. Optionally, the judgment thresholds for the residence time of different blocks may also be exactly the same, which is convenient for configuration in the early stage of design.

[0089] In some examples, such as Fig.10As shown, the viewing direction of the human eye 6 also corresponds to block 5013, and the corresponding anti-false triggering method can refer to the above example, but due to the different driving states of the vehicles, the auxiliary content triggered and displayed in the extended display area 502 is also different. Specifically, according to the ADAS domain control, it is determined to enter a specific intersection, and it can be further determined in combination with map information, etc. that there is traffic light control at the intersection. At this time, the real-time status of the current traffic light will be displayed in the extended display area 502, such as the model display of the red light, and the countdown time of the red light is 20 seconds, etc. This information can be obtained through map providers or traffic vehicle networking platforms. When the red light turns to green, a highlighted information prompt can also be implemented in the extended display area 502 to remind the driver that he can start to pass through the intersection. In some examples, when a vehicle enters an intersection, the real-time status of the traffic light needs to be displayed in the extended display area 502. It can also be triggered in other specific blocks corresponding to the line of sight of the human eye 6. For example, the line of sight of the human eye 6 stays on the block 5012 to check the vehicle speed. This is also a possible behavioral habit of the driver when entering the intersection. At this time, in response to the user's viewing needs, the real-time status of the traffic light can be used in the extended display area to remind the driver to pay attention to the red light to avoid accidentally running a red light.

[0090] In some examples, such as Fig.11 As shown, when the line of sight of the human eye 6 stays in block 5012, and the above example is not a false trigger, different auxiliary contents will be displayed in the extended display area 502. Since the function of vehicle speed display is configured in block 5012, if the ADAS domain control determines that the vehicle is in an overtaking state, such as the vehicle changes lanes to initiate overtaking from the left side of the front vehicle, combined with the driver checking the vehicle speed information, the overtaking progress bar will be displayed in the extended display area 502 at this time, and the overtaking progress bar will also display the completion progress according to the actual overtaking situation, prompting the user to make corresponding driving decisions. In some examples, the auxiliary content that triggers the display of the overtaking progress bar in the extended display area 502 requires not only the occurrence of an overtaking event, but also the detection that the vehicle is in an assisted driving state. This can avoid the auxiliary content in the extended display area 502 that is not good for the driver when the user is driving the vehicle. At the same time, the user is provided with the specific situation of the vehicle assisted driving in the assisted driving state, reducing the user's concerns about the safety of the vehicle driving, and facilitating the user as a driver to take over the vehicle in time according to the on-site situation. In some examples, overtaking behavior is often accompanied by the driver checking the navigation display, so the overtaking progress bar can be displayed in the extended display area 502 in response to the user's viewing direction corresponding to block 5013 and the vehicle is in an overtaking state.

[0091] It should be noted that for Figure 8-11For example, the display scheduling in the extended display area 502 may be that when a new trigger condition is met, the auxiliary content under the previous trigger condition is replaced by a new auxiliary content, that is, the first auxiliary content and the second auxiliary content that meet different trigger conditions will not be displayed in the extended display area at the same time. When the second auxiliary content needs to be displayed, the first auxiliary content will be automatically hidden. For example, Fig. 9 , the human eye 6 stays on block 5013 and the first auxiliary content is presented as a route indication sign, while the reference Fig.11 As the sight of the human eye 6 moves from block 5013 to block 5012, the first auxiliary content is replaced by the second auxiliary content, that is, the overtaking progress bar is displayed in the extended display area 502. Optionally, when the first auxiliary content is displayed, the corresponding driving state no longer exists, for example Fig.11 In the example, the vehicle has completed overtaking. At this time, even if the sight of the human eye 6 still stays on the block 5012, the corresponding overtaking progress bar will be automatically hidden in the extended display area 502. Fig.12 As shown, the sight line of human eye 6 is in the same block, such as block 5013, but there are more than two driving states that meet the requirements of triggering different auxiliary contents, such as entering a complex road section to trigger the display of the route indication sign (first auxiliary content), and entering an intersection with a traffic light to trigger the display of the real-time status of the traffic light (second auxiliary content). According to the above example, the auxiliary content that meets the conditions later can be used to replace the auxiliary content that meets the conditions first, such as the second auxiliary content replacing the first auxiliary content. Optionally, the first auxiliary content and the second auxiliary content can also be distributed in the extended display area at the same time, but the first auxiliary content and the second auxiliary content can be limited to auxiliary contents of the corresponding functions of the same block. In this example, the distribution template in which the first auxiliary content and the second auxiliary content exist at the same time is called, that is, the route indication sign can be configured in the middle position, just in line with the road ahead, and the traffic light model and countdown time in the second auxiliary content are located on both sides of the extended display area to avoid display conflicts with the first auxiliary content.

[0092] When the HUD display device implementing the display method in the above example is applied to a vehicle, the information distribution efficiency in the projection area can be optimized, and information can be displayed on demand, thereby improving the human-computer interaction experience. Fig.13As shown, the HUD display device integrated in the vehicle can be powered and data by the vehicle computer 92, or the HUD display device itself can provide power and generate data. The HUD display device can specifically include a processor 91, an Ethernet interface 901, a CAN (Controller Area Network) interface 902, a power management module 903, a running memory 904, a storage memory 905, a temperature detection 906, a motor 907, a backlight source 908, an image source 909, a positioning module 910, a radar 911, a camera 912, etc. It should be noted that Fig.13 The modules listed in the description are merely exemplary and do not constitute any limitation. In some examples, the HUD display device may further include other modules. In addition, the above modules may be implemented in one or more hardware in different examples, or a single module may be implemented by a combination of multiple hardware.

[0093] The processor 91 serves as the control center of the HUD display device, and includes one or more processing units of any type, including but not limited to a microcontroller, a microcontroller, a DSP (Digital Signal Processor) or any combination thereof. The processor 91 is used to generate an operation control signal according to a computer program, realize the control of other modules, and cooperate with corresponding modules to process the acquired or inherent data, instructions, etc.

[0094] The Ethernet interface 901 is a network data connection port for local area network communication, which defines a series of software and hardware standards. Multiple electronic devices can be connected together through the Ethernet interface 901. In this example, the processor 91 can exchange information with the vehicle computer 92 through the Ethernet interface 901, such as sending data to the vehicle computer 92 or receiving data sent by the vehicle computer 92.

[0095] The CAN interface 902 is a network data connection port of the controller area network, which provides a standard bus for the control system and embedded industrial control inside the car, and realizes the communication interaction between the control nodes. In this example, the processor 91 can also exchange information with the vehicle computer 92 through the CAN interface 902. Optionally, the processor 91 can also connect to other external devices through the CAN interface 902. In some examples, the processor 91 can also be provided with a GPIO (General-purpose input / output) interface to improve the compatibility of peripheral connections.

[0096] The power management module 903 is connected to the vehicle computer 92 and can receive the power provided by the vehicle computer 92 to provide a regulated power supply for each module of the HUD display device, thereby ensuring that the processor 91 and each module operate under normal voltage supply and avoid damage due to overvoltage.

[0097] The running memory 904 is used to store computer programs executed by the processor 91, temporarily stored calculation data, data exchanged with the storage memory, etc. The running memory 904 can be a memory such as SDRAM (Synchronous Dynamic Random-access Memory).

[0098] The storage memory 905 is used to store resources such as display content related to the HUD display device, and long-term storage of running programs and data, etc. The storage memory 905 can be a memory such as Flash (flash memory). In some examples, the processor 91 can also provide an interface to access an external memory.

[0099] Temperature detection 906 is used to detect the temperature inside the HUD display device, and may specifically include several temperature sensors. Since the resistance value of the temperature sensor changes with the change of temperature, the processor 91 can determine the resistance value of the temperature sensor at the corresponding temperature according to the voltage change between each temperature sensor and the voltage divider resistor under a fixed power supply voltage, thereby reversely deducing the temperature at the location of the temperature sensor. In some examples, the processor 91 can control several temperature sensors through the GPIO interface, and several temperature sensors can be set at different locations inside the HUD display device. The processor 91 can use time-sharing detection to obtain the temperature values ​​fed back by several temperature sensors respectively.

[0100] Motor 907 is used to drive the optical lens in the HUD display device to rotate under the control of processor 91, so as to achieve the change of the corresponding optical path. For example, when the backflow of sunlight causes the temperature rise on the surface of the image source, the motor can be used to drive the optical lens to prevent the external sunlight from reaching the surface of the image source. In some examples, the processor 91 can also drive the fan provided on the HUD display device through motor 907 to increase the speed of air exchange between the inside and outside of the HUD display device to achieve heat dissipation. Specifically, the motor 907 is connected to the processor 91 through a motor driver chip, and the motor driver chip provides high-performance power output for the motor 907, and can also communicate and control with the processor 91 through interfaces such as SPI (Serial Peripheral Interface).

[0101] The backlight source 908 is used to provide illumination light and adjust the brightness of the illumination light according to the control of the processor 91, so as to adjust the projection display brightness of the entire HUD display device. The backlight source 908 cooperates with the image source 909 to realize the main function of optical machine projection display. The backlight source 908 can be an LED (Light Emitting Diode), a laser, etc. Specifically, the backlight source 908 is connected to the processor 91 through a backlight driver chip, and the backlight driver chip provides a driving voltage for the backlight source 908 and controls the brightness of the backlight source 908 under the pulse width signal output by the processor 91.

[0102] The image source 909 is used to display an image of corresponding content and project the display light corresponding to the image according to the control of the processor 91. The image source 909 can be an LCD (Liquid Crystal Display), a DMD (Digital Micromirror Devices), a MEMS (Micro-Electro-Mechanical System) micromirror, a LCOS (Liquid Crystal on silicon), etc.

[0103] The positioning module 910 is used to monitor the position of the HUD display device and the corresponding vehicle. The positioning module 910 can be a global navigation satellite system such as GPS (Global Positioning System) and Beidou Satellite Navigation System. By measuring the distance between the satellite and the receiver on the positioning module 910 at different positions, the corresponding position and orientation data are determined. In some examples, the positioning module 910 may also include an inertial navigation system, which is based on Newton's laws of mechanics. By measuring the acceleration of the positioning module 910 in the inertial reference system, integrating it over time, and transforming it into the navigation coordinate system, the speed, yaw angle, and position data in the navigation coordinate system are obtained. Optionally, the inertial navigation system can assist the global navigation satellite system to achieve more accurate positioning and provide the processor 91 with corresponding position information.

[0104] The radar 911 is used to determine the position of a target object through electromagnetic waves, and can usually determine the distance of the target object from the vehicle where the radar 911 is located.

[0105] Camera 912 includes a vehicle body camera and an in-vehicle camera, wherein the vehicle body camera is used to determine the position of the target object through visual recognition, and the vehicle body camera can be a monocular camera or a binocular camera. The biggest difference between a monocular camera and a binocular camera is that a binocular camera can capture images from two different perspectives, thereby obtaining distance information in three-dimensional space. The in-vehicle camera is used to identify the behavior status of the driver and passengers in the vehicle, including fatigue detection, distraction detection, expression recognition, gesture recognition, sight tracking, etc. In this example, the in-vehicle camera can also implement eye tracking.

[0106] In some examples, the positioning module 910, radar 911 and camera 912 can also be directly connected to the vehicle computer 92, and are not directly connected to the processor 91 of the HUD display device. For example, the vehicle computer 92 itself is integrated with a positioning module for position tracking and a radar and camera for automatic driving. The HUD display device can obtain the collected data of the positioning module, radar and camera in real time through communication with the vehicle computer 92.

[0107] In some examples, such as Fig.14 As shown, the HUD display device implementing the above display method may specifically include a processor 931, a memory 932, an input device 933 and an output device 934, wherein the input device 933 may include an operation button integrated on the display device, etc., and the display device may receive input control instructions and data through the input device 933. The output device 934 may include an image source integrated on the display device, etc., and the display device may output corresponding instructions or data to the output device 934. Further, the memory 932 stores a computer program running on the processor 931, and the processor 931 implements the display method of the above example when executing the computer program. In some examples, a computer-readable storage medium, a computer-readable storage medium stores a computer program, and the computer program implements the display method of the above example when executed by the processor.

[0108] like Fig.15As shown, the vehicle can be provided with the above-mentioned HUD display device. Specifically, the HUD display device is integrated inside the center console 10, such as in front of the steering wheel. The corresponding display light is projected onto the opposite vehicle windshield 4 through the projection window 102 of the HUD display device. The effect of the viewer observing the windshield 4 from the cockpit is that the corresponding virtual image can be directly seen. Specifically, the projection area 50 corresponding to the windshield 4 includes a block display area and an extended display area. For details, please refer to the above example. In some examples, the vehicle can also distribute the program of the display method in the above example through the computer-readable storage medium of the above example, so that the vehicle-mounted HUD display device can be conveniently updated and upgraded. It should be noted that the above-mentioned vehicle is not limited to a car as a means of transportation, but can also include a bus, a truck, an excavator, a motorcycle, a train, a high-speed rail, a ship, a yacht, an airplane, a spacecraft, etc. The projected windshield is not limited to the front windshield of the car, but can also be a transparent surface at other positions.

[0109] In combination with the above examples, the technical solutions involved in the present application can be directly embodied as hardware, a software module executed by a control unit, or a combination of the two, that is, one or more steps and / or one or more step combinations, which can correspond to various software modules of a computer program flow, or to various hardware modules, such as ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof. For the convenience of description, the above description is divided into various modules and described separately according to their functions. Of course, when implementing the present application, the functions of each module can be implemented in the same or more software and / or hardware.

[0110] Through the description of the above examples, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution involved in the present application can be essentially or partly contributed to the prior art in the form of a software product. The software is executed by a microcontroller unit and, depending on the required configuration, may include one or more microcontroller units of any type, including but not limited to a microcontroller unit, a microcontroller, a DSP (Digital Signal Processor) or any combination thereof. The software is stored in a memory, such as a volatile memory (such as a random access memory, etc.), a non-volatile memory (such as a read-only memory, a flash memory, etc.) or any combination thereof.

[0111] In summary, the present application divides the projection area formed by the HUD display device into two parts: a block display area and an extended display area. The block display area supports on-demand display of different functions in different blocks, while the extended display area can synchronously display different auxiliary content according to the user's viewing state of the block display area, thereby realizing the linkage between the block display area and the extended display area. The present application supports flexible configuration of information distribution in the projection area, while also expanding the auxiliary content of conventional display, greatly improving the user's viewing experience.

[0112] It should be understood that although this specification includes some examples, none of these examples contains only one independent technical solution, and this description of the specification is only for the purpose of clarity. A person skilled in the art should take the specification as a whole, and the technical solutions in each example can also be appropriately combined to form other implementation methods that can be understood by a person skilled in the art.

[0113] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present application. They are not intended to limit the scope of protection of the present application. All equivalent implementation methods or modifications that do not deviate from the teachings of the present application should be included in the scope of protection of the present application.

Claims

1. A display method, characterized in that: include: The HUD display device projects display light onto a transparent surface to form a projection area for information display; The projection area includes a divided display area and an extended display area, the divided display area includes at least a first block and a second block, the first block is configured with a first display element having a first function, and the second block is configured with a second display element having a second function; In response to the user's viewing direction corresponding to the first block, the extended display area is combined with displaying the first auxiliary content of the first function; in response to the user's viewing direction corresponding to the second block, the extended display area is combined with displaying the second auxiliary content of the second function.

2. The display method according to claim 1, characterized in that: The block display area is arranged below the extended display area.

3. The display method according to claim 1, characterized in that: The first block is configured with a first display element having a first function, and the second block is configured with a second display element having a second function, including: The first display element is a static icon, a dynamic icon, a preview interface or a detail interface corresponding to the first function; The second display element is a static icon, a dynamic icon, a preview interface or a detail interface corresponding to the second function.

4. The display method according to claim 1, characterized in that: In response to the user's viewing direction corresponding to the first block, the extended display area cooperates with displaying the first auxiliary content of the first function, including: The first auxiliary content is determined according to a driving state of the vehicle.

5. The display method according to claim 1, characterized in that: The display method further includes: The first display element and the second display element are determined according to settings input by a user.

6. The display method according to claim 1, characterized in that: In response to the user's viewing direction corresponding to the first block, the extended display area cooperates with displaying the first auxiliary content of the first function, including: triggering display of the first auxiliary content according to the detection that the user's sight stays on the first block for more than a first time threshold; In response to the viewing direction of the user corresponding to the second block, the extended display area cooperates with displaying the second auxiliary content having the second function, including: The display of the second auxiliary content is triggered according to the detection that the user's sight stays on the second block for more than a second time threshold.

7. The display method according to any one of claims 1 to 6, characterized in that: The display method further includes: At most one of the first auxiliary content and the second auxiliary content is displayed at the same time.

8. A display device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the display method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the display method according to any one of claims 1 to 7 are implemented.

10. A means of transport, characterized in that: Includes the display device of claim 8 or the computer-readable storage medium of claim 9.

Images