Head-up display system, display method thereof and vehicle

Abstract

The embodiment of the invention provides a head-up display system, a display method thereof and a vehicle, the head-up display system comprises a plurality of head-up displays, the plurality of head-up displays comprise a first head-up display and a second head-up display, the first head-up display is used for displaying augmented reality information, and the second head-up display is used for displaying augmented reality information. The second head-up display is used for displaying fusion information of augmented reality and virtual reality, and the virtual image distance of a virtual image formed by the second head-up display is larger than that of a virtual image formed by the first head-up display. According to the embodiment of the invention, a virtual image with a larger visual field, more functions and richer information can be provided for a user, the imaging requirements in different scenes are met, and the HUD use experience of the user is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of display technologies, and particularly to a head-up display system, a display method thereof, and a vehicle. Background Art

[0002] A head up display (HUD) system, also known as a head-up display system, projects the light emitted by the image source of the HUD onto an imaging window (such as a retrofitted imaging plate or the windshield of a vehicle) ultimately. Users can directly see the image without lowering their heads or turning their heads, which can avoid distractions caused by the driver looking down at the instrument panel during driving, improve the driving safety factor, and also enhance the user's driving experience.

[0003] Currently, the form of the HUD is relatively single, mostly with a single HUD module assembled inside the center console, also known as the IP (Instrument Panel). Due to problems such as hardware volume and assembly space limitations, its core product parameters such as the field of view (FOV) and virtual image distance (VID) are restricted. Therefore, the windshield of the vehicle cannot be fully utilized. For example, the upper half of the window space is not fully utilized, and imaging requirements such as higher and farther real-time fusion cannot be met. Moreover, in the case of a limited imaging range and a low lower viewing angle, the longitudinal imaging cannot meet more scenario requirements, affecting the user experience. Summary of the Invention

[0004] The purpose of the embodiments of the present disclosure is to provide a head-up display system, a display method thereof, and a vehicle, so as to solve the technical problems in the prior art that due to the single form of the HUD and the fixed installation position, the formed virtual image cannot meet the requirements of multiple scenarios, affecting the user experience of using the HUD, and the technical problem that the design of forming multiple virtual image optical paths by a single head-up display is complex and the implementation difficulty is large.

[0005] To solve the above technical problems, the embodiments of the present disclosure adopt the following technical solutions:

[0006] The embodiments of the present disclosure provide a head-up display system, including a plurality of head-up displays. The plurality of head-up displays include a first head-up display and a second head-up display. The first head-up display is used to display augmented reality information, and the second head-up display is used to display the fusion information of augmented reality and virtual reality. The virtual image distance of the virtual image formed by the second head-up display is greater than the virtual image distance of the virtual image formed by the first head-up display.

[0007] In some embodiments, the first head-up display is disposed within the center console, and the second head-up display is disposed above the cockpit or within the center console, where the cockpit includes at least one of a driver's cockpit, a front passenger's cockpit, and a rear cockpit.

[0008] In some embodiments, the plurality of head-up displays further includes a third head-up display, and the virtual image distance of the virtual image formed by the third head-up display is less than the virtual image distance of the virtual image formed by the first head-up display.

[0009] In some embodiments, the head-up display system further includes an imaging window, and an anti-reflection portion is provided in the imaging area corresponding to the third head-up display on the imaging window.

[0010] In some embodiments, the virtual image formed by the first head-up display includes a near-layer virtual image and a far-layer virtual image, and the virtual image distance of the near-layer virtual image is less than the virtual image distance of the far-layer virtual image.

[0011] In some embodiments, the virtual image distance of the near-layer virtual image is 3 to 6 m, and the virtual image distance of the far-layer virtual image is 10 to 20 m; the virtual image distance of the virtual image formed by the second head-up display > 20 m.

[0012] In some embodiments, the third head-up display is disposed on the surface of the center console or within the center console, and the virtual image distance of the virtual image formed by the third head-up display is 1 to 2 m.

[0013] In some embodiments, the virtual image formed by the second head-up display is located above the virtual image formed by the first head-up display, and the virtual image formed by the first head-up display and the virtual image formed by the second head-up display do not overlap or partially overlap in the height direction.

[0014] In some embodiments, the display area of the virtual image formed by the third head-up display is the lowermost area among the display areas of the virtual images formed by the plurality of head-up displays; or

[0015] the display area of the virtual image formed by the third head-up display is located on the left side of the display area of the virtual image formed by the first head-up display; or

[0016] the display area of the virtual image formed by the third head-up display is located on the right side of the display area of the virtual image formed by the first head-up display; or

[0017] the display area of the virtual image formed by the third head-up display is located on the left and right sides of the display area of the virtual image formed by the first head-up display.

[0018] The embodiments of the present disclosure further provide a display method for a head-up display system, which is applied to the above head-up display system, and the method includes:

[0019] Determine the target head-up display corresponding to the target information according to the information category of the target information to be displayed;

[0020] Display the target information in the virtual image formed by the target head-up display, wherein the virtual images formed by the respective target head-up displays have corresponding information categories.

[0021] In some embodiments, the information category corresponding to the near-layer virtual image formed by the first head-up display includes first non-AR planar information;

[0022] The information category corresponding to the far-layer virtual image formed by the first head-up display includes augmented reality information integrating the real scene;

[0023] The information category corresponding to the virtual image formed by the second head-up display includes at least one of the augmented reality information, associated information of the augmented reality information, virtual reality information, integrated information of augmented reality and virtual reality, and entertainment life information;

[0024] The information category corresponding to the virtual image formed by the third head-up display includes second non-AR planar information.

[0025] In some embodiments, the first non-AR planar information includes at least one of instrument-related information and electronic rearview mirror information;

[0026] The augmented reality information includes at least one of AR navigation information and AR assisted driving information;

[0027] The second non-AR planar information includes at least one of associated information of the augmented reality information, electronic rearview mirror information, and an AI assistant,

[0028] The AI assistant is used to perform AI intelligent interaction with in-vehicle passengers and assist in controlling the display of the virtual image formed by the target head-up display.

[0029] In some embodiments, the determining the target head-up display corresponding to the target information according to the information category of the target information to be displayed includes:

[0030] When the information category of the target information is non-AR driving information, determine the target head-up display corresponding to the target information according to the information complexity of the target information; or,

[0031] When the information category of the target information is the target category, determine the target head-up display and / or display form corresponding to the target information according to at least one of the relative position relationship between the target object in the real environment corresponding to the target information and the vehicle, and the relationship between the target information and other information in the virtual images formed by the multiple head-up displays. The display form includes at least one of displaying in AR form, displaying in non-AR form, simplified display, and complete display.

[0032] In some embodiments, the determining the target head-up display corresponding to the target information according to the information complexity of the target information includes:

[0033] If the information complexity of the target information is higher than the complexity threshold, display the target information in the display area of the virtual image formed by the third head-up display, where the information complexity of the target information is determined according to the pixel usage rate of the target information; or,

[0034] If the information complexity of the target information is lower than or equal to the complexity threshold, display the target information in the display area of the near-layer virtual image formed by the first head-up display.

[0035] In some embodiments, the target information is POI information, the target category includes POI information, and the determining the target head-up display and display form corresponding to the target information according to the relative position relationship between the target object in the real environment corresponding to the target information and the vehicle includes:

[0036] Determine the field of view angle range where the target information is located according to the position information of the target object relative to the vehicle and the field of view angle range corresponding to each head-up display;

[0037] Determine the target head-up display and display form corresponding to the target information according to the field of view angle range where the target information is located.

[0038] In some embodiments, the method further includes:

[0039] In response to an operation on the target information currently displayed in the display area of the far-layer virtual image formed by the first head-up display or the display area of the virtual image formed by the second head-up display, obtain the associated information of the target information;

[0040] Display the associated information in the virtual image formed by the second head-up display.

[0041] In some embodiments, the method further includes:

[0042] Determine the display parameters of the target information currently displayed according to the vehicle speed, where the display parameters include at least one of the display form, display time, and display size of the target information; or

[0043] Adjust the display parameters of the target information currently displayed according to the driving scenario in which the vehicle is located, or add or cancel the display of the multimedia interaction information corresponding to the target information.

[0044] In some embodiments, the method further includes:

[0045] Obtain the target information to be displayed in the first virtual image formed by the head-up display system;

[0046] Pre-display the target information in the second virtual image formed by the head-up display system, where the first virtual image and the second virtual image are different.

[0047] In some embodiments, the method further includes:

[0048] Obtain the target information in the first virtual image whose remaining display time is less than the time threshold;

[0049] Convert and display the target information in the second virtual image.

[0050] In some embodiments, the method further includes:

[0051] Display recommended information in the third virtual image;

[0052] Based on the target recommended information selected by the user from the recommended information, after the vehicle arrives at the recommended location corresponding to the target recommended information, display the target recommended information in the fourth virtual image.

[0053] An embodiment of the present disclosure further provides a vehicle, including the above head-up display system.

[0054] The head-up display system, its display method, and the vehicle provided by the embodiments of the present disclosure can provide a virtual image with a larger field of view, more functions, and richer information for the user by setting multiple head-up displays, where the multiple head-up displays at least include a first head-up display for displaying augmented reality information and a second head-up display for displaying the fusion information of augmented reality and virtual reality, and the virtual image distance of the virtual image formed by the second head-up display is greater than the virtual image distance of the virtual image formed by the first head-up display, meeting the imaging requirements in different scenarios and improving the user's HUD usage experience. Description of the Drawings

[0055] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0056] Figure 1 The first structural schematic diagram of the head-up display system according to an embodiment of the present disclosure;

[0057] Figure 2 The second structural schematic diagram of the head-up display system according to an embodiment of the present disclosure;

[0058] Figure 3 The flowchart of the display method of the head-up display system according to an embodiment of the present disclosure;

[0059] Figure 4 The display schematic diagram of the target information according to an embodiment of the present disclosure;

[0060] Figure 5 For Figure 4 The display and pixel usage schematic diagram of the first UI module in

[0061] Figure 6 The front view of the display of another target information according to an embodiment of the present disclosure;

[0062] Figure 7 The side view of the display of another target information according to an embodiment of the present disclosure. Detailed implementation manners

[0063] Reference is made herein to the various aspects and features of the present disclosure with reference to the accompanying drawings.

[0064] It should be understood that various modifications can be made to the embodiments applied herein. Therefore, the above description should not be regarded as a limitation, but only as an example of the embodiments. Those skilled in the art will think of other modifications within the scope and spirit of the present disclosure.

[0065] The accompanying drawings included in the specification and forming a part of the specification illustrate the embodiments of the present disclosure, and together with the general description of the present disclosure given above and the detailed description of the embodiments given below, are used to explain the principles of the present disclosure.

[0066] These and other features of the present disclosure will become apparent from the following description of the preferred forms of the embodiments given by way of non-limiting examples with reference to the accompanying drawings.

[0067] It should also be understood that although the present disclosure has been described with reference to some specific examples, those skilled in the art can surely implement many other equivalent forms of the present disclosure, which have the features as described in the claims and thus are all within the protection scope defined thereby.

[0068] When combined with the accompanying drawings, the above and other aspects, features and advantages of the present disclosure will become more apparent in view of the following detailed description.

[0069] Hereinafter, specific embodiments of the present disclosure will be described with reference to the accompanying drawings; however, it should be understood that the embodiments claimed are merely examples of the present disclosure, which can be implemented in various ways. Well-known and / or repetitive functions and structures are not described in detail to avoid obscuring the present disclosure with unnecessary or redundant details. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but are merely used as a basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in substantially any suitable detailed structure in a variety of ways.

[0070] This specification may use the phrase "in one embodiment", "in another embodiment", "in yet another embodiment" or "in other embodiments", which may each refer to one or more of the same or different embodiments according to the present disclosure.

[0071] Figure 1 The structural schematic diagram of the head-up display system according to an embodiment of the present disclosure is shown. As Figure 1 shown, the first embodiment of the present disclosure provides a head-up display system, including a plurality of head-up displays, the plurality of head-up displays including a first head-up display 1 and a second head-up display 2. The first head-up display 1 is configured to display augmented reality (AR) information, and the second head-up display 2 is configured to display the fusion information of augmented reality and virtual reality (VR). The virtual image distance of the virtual image 20 formed by the second head-up display 2 is greater than the virtual image distance of the virtual image 10 formed by the first head-up display 1.

[0072] Wherein, the first head-up display 1 is an AR-HUD (Augmented Reality HUD, augmented reality head-up display), and the AR-HUD can project virtual information (i.e., AR information) onto the real environment, and fuse the real environment information and the virtual information to achieve a sensory experience beyond reality. In this embodiment, the AR information is the virtual image information displayed by fusing the real road environment, which may include the entire virtual image screen rendered by fusing the real environment, or may include the display information rendered by fusing a certain or certain target objects in the virtual image screen.

[0073] The AR information may specifically include driving information such as the current vehicle condition and navigation information (such as steering information) displayed in line with the road surface, and may also include road, traffic, and obstacle recognition information (such as the leading vehicle locking identification), as well as navigation information and assisted driving information such as warning prompt information and POI (Point of Interest) information.

[0074] The second head-up display 2 is a head-up display for displaying the fusion information of AR and VR, such as an MR-HUD (Mixed Reality HUD) or an XR-HUD (Extended Reality HUD).

[0075] The VR technology can use a computer to generate 3D content, providing users with a simulation of reality through senses such as vision and hearing, offering an immersive experience with a strong sense of "immersion" and "presence". In a VR environment, the user does not see real environment information and is completely immersed in the virtual environment. Usually, virtual reality devices such as headphones and controllers are used to interact in the environment.

[0076] The fusion information of AR and VR can not only achieve the fusion of the real environment and virtual information but also enable interaction in the real environment. The information projected by the second head-up display 2 can be the MR information projected by the MR-HUD or the XR information projected by the XR-HUD; the information projected by the second head-up display 2 can include not only the fusion information of AR and VR but also AR information and VR information. The MR technology is the fusion of the real environment and the virtual environment, where the environment simultaneously contains the physical environment and virtual information and has "real-time" characteristics. The MR information can achieve real-time interaction in the real environment. The XR technology extends to include any technology that fuses the real environment and the virtual environment.

[0077] Optionally, in this embodiment, the fusion information of AR and VR may also include driving information such as the current vehicle condition and navigation information (including steering) displayed in line with the road surface; road, traffic, and obstacle recognition information (such as the leading vehicle locking identification), as well as assisted driving information such as warning prompt information and POI information; and entertainment and life-related information (such as movies, videos, phone calls, or video conferences, etc.).

[0078] The virtual image distance of the virtual image is the distance between the virtual image formed by the head-up display and the eye box area of the head-up display. Specifically, it can be the distance between the center of the virtual image and the center of the eye box area. The eye box area of the head-up display refers to the area where the observer's eyes (also known as the human eyes) are located and where the virtual image formed by the head-up display can be observed. The eye box area has a certain size. The observer's eyes deviate from the center of the eye box by a certain distance, such as moving up and down or left and right. As long as the observer's eyes are still within the eye box area, the virtual image formed can be seen at different positions within the eye box area.

[0079] In some embodiments, the eye box areas of all the head-up displays in the above head-up display system overlap, such as complete overlap or partial overlap. When there is partial overlap, the ratio of the area of the overlapping part to the area of the eye box area of each head-up display is greater than a preset ratio. The preset ratio can be set according to requirements, and the present disclosure does not specifically limit it.

[0080] As Figure 6 shown, the image source of the head-up display (such as the first head-up display 1) emits light rays and forms a light spot, that is, the eye box area, after being reflected by the imaging window 5. Different head-up displays can form different eye box areas. All the head-up displays can be set so that the eye box areas completely overlap, so that when the observer's eyes are located within the eye box area, the virtual images formed by different head-up displays can be seen simultaneously.

[0081] Optionally, the eye box areas of the first head-up display 1 and the third head-up display 2 completely overlap. Moreover, the area of the eye box area 62 of the second head-up display 2 is larger than the area of the eye box area 61 of the first head-up display 1, and the eye box area 62 of the second head-up display 2 includes the eye box area 61 of the first head-up display 1, thereby realizing the expansion of the area where the virtual image formed by the second head-up display 2 can be viewed, facilitating the user to view the content such as entertainment and life information displayed in the virtual image in different postures.

[0082] The virtual image distance (VID2) of the virtual image 20 formed by the second head-up display 2 is greater than the virtual image distance (such as VID 11 ) of the virtual image 10 (at least including the virtual image 101) formed by the first head-up display 1. The virtual image 20 formed by the second head-up display 2 is farther than the virtual image 10 formed by the first head-up display 1, enabling the head-up display system to fuse real environment imaging in real time at a farther distance; at the same time, the virtual images formed by the first head-up display 1 and the second head-up display 2 can be located at positions with different heights and within different longitudinal field of view ranges. For example, the longitudinal field of view range of the virtual image 10 formed by the first head-up display 1 is θ1, and the longitudinal field of view range of the virtual image 20 formed by the second head-up display 2 is θ2, thereby increasing the overall display range of the head-up display system (for example, the overall longitudinal field of view range of the system is θ1 + θ2).

[0083] Virtual images with different virtual image distances can display different virtual image contents, providing users with richer virtual image information. Moreover, different head-up displays can be used to display virtual images with different functions. For example, the virtual image 20 formed by the second head-up display 2 can interact with the passengers in the vehicle to improve the user experience. For another example, the POI function can be implemented using different head-up displays.

[0084] The head-up display system provided by the embodiments of the present disclosure includes multiple head-up displays. The multiple head-up displays at least include a first head-up display 1 for displaying augmented reality information and a second head-up display 2 for displaying the fusion information of augmented reality and virtual reality. Moreover, the virtual image distance of the virtual image 20 formed by the second head-up display 2 is greater than the virtual image distance of the virtual image 10 formed by the first head-up display 1, which can provide users with virtual images with a larger field of view (including far and near distances, height ranges, etc.), more functions, and richer information, meeting the imaging requirements in different scenarios and improving the user's HUD usage experience.

[0085] Optionally, the multiple head-up displays are arranged at different positions in the vehicle to form virtual images with different virtual image distances. In a specific embodiment, some head-up displays can be arranged in the center console, and some other head-up displays can be arranged at any position in the vehicle, such as on the surface of the center console, on the seat, etc., so that imaging can be performed within a larger field of view as shown in Figure 1 . In other embodiments, multiple head-up displays can also be arranged at different positions in the center console to perform imaging within a larger field of view through the head-up displays at different positions. The positions of the head-up displays can be determined according to the situation inside the vehicle and the user's needs, etc., and the present disclosure does not specifically limit them. By forming multiple virtual images through multiple head-up displays at different positions, it is possible to avoid the defects of complex optical path design and high implementation difficulty in the head-up display caused by forming multiple virtual images through only one head-up display, and reduce the manufacturing cost of the head-up display.

[0086] In some embodiments, the first head-up display 1 is arranged in the center console 4, and the second head-up display 2 is arranged above the cockpit or in the center console 4. The cockpit includes at least one of a main driver cockpit, a co-driver cockpit, and a rear cockpit.

[0087] Exemplarily, as shown in Figure 1 , the first head-up display 1 is arranged in the center console 4, and the second head-up display 2 is arranged above the main driver cockpit. The first head-up display 1 and the second head-up display 2 can respectively emit image light rays from the upper and lower sides of the human eye, and after being reflected by the imaging window 5 (such as the windshield), they are incident on the eyebox area, so that the human eye can see the formed virtual image. The image light rays emitted from different positions by different head-up displays can not interfere with each other, thereby improving the imaging quality.

[0088] In some embodiments, the virtual image 10 formed by the first head-up display 1 includes a near-layer virtual image 101 and a far-layer virtual image 102, and the virtual image distance VID of the near-layer virtual image 101 11 is less than the virtual image distance VID of the far-layer virtual image 102 12 .

[0089] The virtual image 10 formed by the first head-up display 1 includes a double-layer virtual image of far and near. The virtual image distance VID of the far-layer virtual image 102 12 is relatively far, and the display area of the formed virtual image 20 (i.e., the area of the virtual image 20) is larger, which is used to display AR information that fuses the real scene. The AR information includes, but is not limited to, AR navigation information and AR assisted driving information. The AR navigation information includes at least one of navigation guidance information, POI information, and destination identification information. The AR assisted driving information includes at least one of lane line identification information, pre-collision warning information of the vehicle in front, and pre-vehicle locking identification. The virtual image distance VID of the near-layer virtual image 101 11 is smaller and is used to display the first non-AR planar information. The first non-AR planar information includes, but is not limited to, at least one of instrument-related information (such as vehicle speed and fuel consumption) and electronic rearview mirror (CMS, Camera Monitor System) information.

[0090] The electronic rearview mirror information can be image information of the left and right rear of the vehicle captured by a high-definition wide-angle camera to eliminate the driver's field of vision blind area and improve driving safety. It should be noted that the virtual image 10 formed by the first head-up display 1 can also be a single-layer virtual image, only displaying AR information; the virtual image 10 formed by the first head-up display 1 can also be a multi-layer virtual image, including a near-layer virtual image and multiple far-layer virtual images. AR information fused with objects at different distances in the real environment (such as the vehicle in front 10m away from the vehicle and the building 100m away from the vehicle) can be respectively displayed in the multiple far-layer virtual images.

[0091] In some embodiments, the virtual image distance VID of the near-layer virtual image 101 11 is 3 to 6 m, and the virtual image distance VID of the far-layer virtual image 102 12 is 10 to 20 m; the virtual image distance VID2 of the virtual image 20 formed by the second head-up display 2 is > 20 m, so as to provide a better visual experience for the user.

[0092] In some embodiments, as Figure 1 shown, the plurality of head-up displays further includes a third head-up display 3, and the virtual image distance of the virtual image 30 formed by the third head-up display 3 is less than the virtual image distance of the virtual image 10 formed by the first head-up display 1.

[0093] The third head-up display 3 is a P-HUD (Panoramic HUD), and the P-HUD is used to display the second non-AR planar information. The second non-AR planar information includes, but is not limited to, at least one of the associated information of the augmented reality information (associated information of the AR information, such as the secondary menu corresponding to the AR information), the electronic rearview mirror information, and the AI assistant.

[0094] The information category of the virtual image 30 formed by the third head-up display 3 can be determined in advance according to the information category of the virtual image 10 formed by the first head-up display 10. For example, when the near-layer virtual image 101 is only used to display instrument-related information, the electronic rearview mirror information can be displayed in the virtual image 30; when the near-layer virtual image 101 is used to display instrument-related information and electronic rearview mirror information, the second non-AR planar information such as the associated information of the AR information (the electronic rearview mirror information is not displayed) can be displayed in the virtual image 30, so that when the virtual images formed by the first head-up display 1 and the second head-up display 3 are displayed simultaneously, different non-AR planar information can be displayed separately.

[0095] Some non-AR planar information can usually be displayed in positions such as the in-vehicle instrument panel and the central control screen. In this embodiment, the second non-AR planar information is projected onto the imaging window 5 through the third head-up display 3, and in cooperation with the first head-up display 1, the driver does not need to look down at the instrument panel, the central control screen, etc., which can reduce the distraction of the driver's attention and improve the driving safety of the vehicle. At the same time, the third head-up display 3 can cooperate with the above-mentioned first head-up display 1 and the second head-up display 2 to achieve more levels of information display. The area of the image source in the third head-up display 3 can be set according to actual needs. When the area of the image source is large, the area of the virtual image 30 formed by the third head-up display 3 is also large, so as to achieve panoramic display.

[0096] In some embodiments, as Figure 1 shown, the head-up display system further includes an imaging window 5, and an anti-reflection portion 6 is provided in the imaging area corresponding to the third head-up display 3 on the imaging window 5.

[0097] Specifically, the anti-reflection portion 6 can be provided in the imaging area corresponding to the third head-up display 3 on the imaging window 5, so that the image light emitted by the third head-up display 3 can be reflected by the imaging window 5 as much as possible after entering this imaging area, reducing the transmission of the image light in the imaging window 5, and achieving the improvement of the brightness and contrast of the formed virtual image 30. The anti-reflection portion 6 can be an opaque coating coated on the inner surface or the outer surface of the imaging window 5, which is convenient to install.

[0098] In some embodiments, the third head-up display 3 is disposed on the surface of the center console 4 or inside the center console 4, and the virtual image distance VID3 of the virtual image 30 formed by the third head-up display 3 is 1 to 2 m.

[0099] The third head-up display 3 is arranged on the surface of the center console 4 or inside the center console 4. The virtual image distance of the virtual image 30 formed by the third head-up display 3 is less than the virtual image distance of the near-layer virtual image 101 formed by the first head-up display 1, so that the near and far distances of the near-layer virtual image 101, the far-layer virtual image 102 of the first head-up display 1 and the second head-up display 2, and the virtual image formed by the third head-up display 3 are different, which can meet the imaging requirements in different usage scenarios, facilitate the user to view, and further improve the user experience. In some embodiments, as Figure 1 and Figure 2 shown in (a) of

[0100] By setting different head-up displays, the imaging window 5 can be fully utilized, especially the upper unused area of the imaging window 5 is used for imaging to form virtual images with different heights, which is convenient for the user to observe.

[0101] Since the virtual image distance of the virtual image 20 formed by the second head-up display 2 is greater than the virtual image distance of the virtual image 10 formed by the first head-up display 1, the installation positions of the first head-up display 1 and the second head-up display 2 can be arranged such that the virtual image 20 formed by the second head-up display 2 is located above the virtual image 10 formed by the first head-up display 1; and the virtual image 10 formed by the first head-up display 1 and the virtual image 20 formed by the second head-up display 2 do not overlap or partially overlap in the height direction, so as to maximize the overall field of view of the head-up display system.

[0102] Optionally, since AR information can be displayed in both the virtual images formed by the first head-up display 10 and the second head-up display 20, and the virtual image distance of the virtual image 20 formed by the second head-up display 20 is farther, therefore, the first AR information related to driving, such as navigation guidance information, destination identification information, lane line identification information, pre-collision warning information for the vehicle ahead, and the first form of POI information (such as displaying simplified POI information), etc., is displayed in the virtual image 10 located below to facilitate assisting the user in driving; other forms of POI information (such as displaying detailed POI information), the associated information of the first form of POI information, and information related to entertainment and life, etc., are displayed in the virtual image 20 located above to display more and richer information.

[0103] Exemplarily, in a specific embodiment, the first head-up display 1 displays (i.e., displays in the virtual image formed by the first head-up display 1) AR information in the form of text and icons, such as the first POI information. The second head-up display 2 can also display the first AR information in the form of text and icons, and can also display more forms of POI information, such as pictures, videos, and three-dimensional second POI information. The POI in the real environment corresponding to the first POI information is different from the POI in the real environment corresponding to the second POI information; or, the first head-up display 1 displays (i.e., displays in the virtual image formed by the first head-up display 1) AR information in the form of text and icons, such as POI information, and the second head-up display 2 can display the associated information of the POI information (i.e., other information associated with the POI in the real environment corresponding to the POI information). Thus, the diversification display of POI information can be realized by using two head-up displays, and different forms of POI information can be displayed at different positions, improving the user's HUD usage experience. In another specific embodiment, the first head-up display 1 displays AR information such as navigation guidance integrated with the road and a front vehicle locking identifier, and the second head-up display 2 displays entertainment information such as movies and videos, thereby realizing the display of more and richer information through the head-up display system.

[0104] In some embodiments, the virtual image 10 formed by the first head-up display 1 and the virtual image 20 formed by the second head-up display 2 can be partially overlapped in the height direction.

[0105] In some embodiments, as Figure 2 shown in (b) of FIG., the display area of the virtual image 30 formed by the third head-up display 3 is the lowermost area among the display areas of the virtual images formed by the multiple head-up displays.

[0106] That is, the virtual image 30 formed by the third head-up display 3 (P-HUD) is located at the bottom of all virtual images to display the second non-AR planar information, facilitating the user to view and perform corresponding operations (such as interacting with the AI assistant) through the bottom P-HUD. Other head-up displays are used to display AR information, VR information, etc. For example, AR information is displayed in the far-layer virtual image 102 of the first head-up display 1 to provide a better real-scene fusion perception experience for the user, and VR information is displayed in the virtual image 30 of the second head-up display 2 to provide a better three-dimensional perception experience for the user.

[0107] Further, as Figure 1As shown, when the virtual image displayed by the first head-up display 10 includes a near-layer virtual image 101, the near-layer virtual image 101 displays the plane information of the first non-AR. At this time, the plane information of the non-AR can be divided, and the first non-AR plane information such as instrument-related information is displayed in the display area (near-layer instrument area) corresponding to the near-layer virtual image 101. The display height of this display area is the most comfortable, facilitating the driver to view the most concerned basic driving information. The near-layer instrument area can completely replace the instrument panel; the associated information of the POI information, electronic rearview mirror information, AI assistant, incoming call information, information pop-up window, voice feedback, quick menu bar, etc., which are the second non-AR plane information displayed in the far-layer virtual image 102, are displayed in the virtual image 30 formed by the P-HUD below the near-layer virtual image 101. This display area can display the information corresponding to convenient functions, etc., facilitating user operation.

[0108] As can be seen from the above, in the POI function environment, the third head-up display 3 can be used to display the response result after the interactive operation on the POI information. The interactive operation includes selecting the POI information, and the response result includes a secondary menu. For example, after selecting the POI information, the secondary menu of the POI information is displayed in the virtual image formed by the third head-up display 3. The secondary menu may include calling the merchant, navigating to the poi location, viewing, favoriting, POI mode selection bar, etc. For example, the POI information can be displayed in the far-layer virtual image 102 formed by the first head-up display 1 or the virtual image 20 formed by the second head-up display 2. When the user selects a certain POI information, the secondary menu of the POI information as the response result is displayed in the virtual image formed by the third head-up display 3.

[0109] In some embodiments, as Figure 2 shown in (c) below, the display area of the virtual image 30 formed by the third head-up display 3 is located on the left side of the display area of the virtual image 10 formed by the first head-up display 1; or

[0110] the display area of the virtual image 30 formed by the third head-up display 3 is located on the right side of the display area of the virtual image 10 formed by the first head-up display 1; or

[0111] the display area of the virtual image 30 formed by the third head-up display 3 is located on the left and right sides of the display area of the virtual image 10 formed by the first head-up display 1.

[0112] In this embodiment, it can be in Figure 2Based on the arrangement of the first head-up display 10 and the second head-up display 20 shown in (a), a third head-up display 30 is added, and the virtual image 30 formed by the third head-up display 30 is located on at least one side of the left and right sides of the virtual image 10 formed by the first head-up display 10, so as to display second non-AR planar information such as associated information of AR information, CMS information, AI assistant, etc., to meet the imaging requirements of multiple scenarios, and information interaction between different head-up displays can also be realized. For example, the AI assistant displayed by the third head-up display 30 can interact with the passengers in the vehicle intelligently, assist in controlling the display of the head-up display, and provide useful information and support for the user.

[0113] In a specific embodiment, when the AI assistant displayed in the virtual image 30 formed by the third head-up display 30 interacts with the driver, the driver can interact with the AI assistant by voice or other means to select the corresponding POI information. At this time, the selected POI information is highlighted (for example, highlighted or marked as the selected state) in the virtual image 10 formed by the first head-up display 10 and / or the virtual image 20 formed by the second head-up display 20, and the detailed information of the selected POI information is further displayed in the virtual image 30 formed by the third head-up display 30, providing more detailed and useful POI information for the user and realizing information interaction between different head-up displays.

[0114] Figure 3 The flowchart of the display method of the head-up display system according to the embodiment of the present disclosure is shown. As Figure 3 shown, the second embodiment of the present disclosure provides a display method of a head-up display system, which is applied to the above-mentioned head-up display system, and the method includes:

[0115] S101: Determine the target head-up display corresponding to the target information according to the information category of the target information to be displayed;

[0116] S102: Display the target information in the virtual image formed by the target head-up display, where the virtual images formed by the respective head-up displays have corresponding information categories.

[0117] Among them, the head-up display at least includes the first head-up display 10 and the second head-up display 20. When using different head-up displays for display, first determine the target head-up display according to the information category of the target information to be displayed, and display the target information in the virtual image formed by the target head-up display.

[0118] The information categories of the target information can be divided into AR information and non-AR information (i.e., non-AR planar information) according to whether it is integrated with the real environment. The non-AR information can be further divided into non-AR planar information, non-AR three-dimensional information, etc. The information categories of the target information can also be divided into static information (such as instrument-related information) and dynamic information (such as videos) according to whether the information is dynamic information. The information categories of the target information are divided according to actual needs, and the present disclosure does not specifically limit them.

[0119] The display method of the head-up display system provided by the embodiments of the present disclosure, when using multiple head-up displays for display, determines the target head-up display corresponding to the target information from the multiple head-up displays according to the information category of the target information to be displayed, and then displays the target information in the virtual image formed by the target head-up display. The target information can be hierarchically and clearly displayed in the corresponding virtual image according to the information category, which is convenient for the user to view, and the display position of the target information is the best position for the user to view, which can improve the user's viewing comfort. In addition, the present disclosure can make full use of different display areas of the imaging window 5 to display the target information of the corresponding information category, which can provide rich information content for the user and improve the HUD performance.

[0120] Optionally, the information category corresponding to the near-layer virtual image 101 formed by the first head-up display 10 includes first non-AR planar information;

[0121] The information category corresponding to the far-layer virtual image 102 formed by the first head-up display 10 includes augmented reality information integrated with the real scene;

[0122] The information category corresponding to the virtual image 20 formed by the second head-up display 20 includes at least one of the augmented reality information, the associated information of the augmented reality information, virtual reality information, the fusion information of augmented reality and virtual reality, and entertainment life information;

[0123] The information category corresponding to the virtual image 30 formed by the third head-up display 30 includes second non-AR planar information.

[0124] The associated information of the AR information is the information related to the AR information. For example, if the AR information is POI information, the associated information of the AR information can be the simplified information, detailed information, derivative information, etc. of the POI information.

[0125] As described above, in this embodiment, the target information of different information categories such as the first non-AR planar information, AR information, the associated information of the AR information, and VR information can be respectively displayed in different virtual images formed by the same head-up display and different virtual images formed by different head-up displays, improving the user's AR, VR, etc. experiences.

[0126] It can be understood that the above-mentioned planar information of AR, AR information, associated information of AR information, VR information, fusion information of AR and VR, etc. are all information related to the driving scenario, while the entertainment life information can be information related to the driving scenario (such as providing shopping information for the user according to the driving scenario) or unrelated (such as playing movies for entertainment). Since the main function of the head-up display is to assist driving, in this embodiment, the entertainment life information is described as a separate information category.

[0127] Furthermore, the first non-AR planar information includes at least one of instrument-related information and electronic rearview mirror information;

[0128] The augmented reality information includes at least one of AR navigation information and AR assisted driving information;

[0129] The second non-AR planar information includes at least one of the associated information of the augmented reality information, electronic rearview mirror information, and the AI assistant, and the AI assistant is used to perform AI intelligent interaction with the passengers in the vehicle and assist in controlling the display of the virtual image formed by the target head-up display.

[0130] By further and detailedly classifying the information categories of the first non-AR planar information, AR information, and second non-AR planar information, the virtual images of the corresponding information categories can be displayed on the imaging window 5 according to user needs, improving the user's HUD usage experience.

[0131] When instrument-related information, electronic rearview mirror information, and other first non-AR planar information are displayed in the near-layer virtual image 101 formed by the first head-up display 10, other non-AR planar information except the above first non-AR planar information is displayed in the virtual image 30 formed by the third head-up display 3, so that the display content in the near-layer virtual image 101 formed by the first head-up display 10 is different from the display content in the virtual image 30 formed by the third head-up display 30. The user can view different target information through different display areas, avoiding information confusion.

[0132] Since the display area corresponding to the near-layer virtual image 101 is a better position for the user to view the instrument-related information, when the instrument-related information is displayed in the near-layer virtual image 101, the AR information is displayed in the far-layer virtual image 102, or all the AR information is displayed in the virtual image 20 formed by the second head-up display 20, etc., the associated information of the AR information, the electronic rearview mirror information, and the second non-AR planar information such as the AI assistant can be displayed in the virtual image 30 formed by the third head-up display 30. On the one hand, it can be displayed separately from the target information displayed by other head-up displays, and on the other hand, it is convenient to realize the interaction of different information and improve the user's HUD usage experience. For example, displaying the instrument-related information in the near-layer virtual image 101 is convenient for the user to understand the vehicle driving situation, displaying the front vehicle locking mark in the far-layer virtual image 102, and displaying the associated information of the AR information such as the distance from the front vehicle and the electronic rearview mirror information in the virtual image 30 formed by the third head-up display 30, so that the user can judge whether to overtake according to the distance from the front vehicle and the environment behind the vehicle, and control the vehicle steering and speed, etc. through the instrument-related information when overtaking.

[0133] In some embodiments, in step S101, according to the information category of the target information to be displayed, determining the target head-up display corresponding to the target information includes:

[0134] S201: When the information category of the target information is non-AR driving information, determining the target head-up display corresponding to the target information according to the information complexity of the target information; or,

[0135] S202: When the information category of the target information is the target category, determining the target head-up display and / or display form corresponding to the target information according to at least one of the relative position relationship between the target object in the real environment corresponding to the target information and the vehicle, and the relationship between the target information and other information in the virtual images formed by the multiple head-up displays, where the display form includes at least one of displaying in the AR form, displaying in the non-AR form, simplified display, and complete display.

[0136] After determining that the information category of the target information in step S201 is non-AR driving information, since the non-AR driving information can be simple information such as identification information, such as lane information and traffic signal information, or complex information such as navigation guidance information for real-time navigation guidance, such as Eagle Eye Map and Turn-By-Turn (TBT) information, etc. At this time, the target head-up display corresponding to the target information can be determined according to the information complexity of the target information, etc., so that the target information is displayed in a reasonable and clear form at the corresponding display position.

[0137] Among them, the information complexity of the target information refers to the complexity of the number of constituent elements in the target information and the interrelationships, interactions, and collaborative relationships between the elements.

[0138] Further, in step S201, determining the target head-up display corresponding to the target information according to the information complexity of the target information includes:

[0139] S2011: If the information complexity of the target information is higher than the complexity threshold, display the target information in the display area of the virtual image 30 formed by the third head-up display 3, where the information complexity of the target information is determined according to the pixel usage rate of the target information; or,

[0140] S2012: If the information complexity of the target information is lower than or equal to the complexity threshold, display the target information in the display area of the near-layer virtual image 101 formed by the first head-up display 1.

[0141] Non-AR driving information can be displayed either in the near-layer virtual image 101 formed by the first head-up display 1 or in the virtual image 30 formed by the third head-up display 3. To further accurately determine the display position of the target information, in this step, the target information is displayed in the virtual image formed by the corresponding target head-up display according to the information complexity of the target information.

[0142] The target information is pre-designed through UI. In this embodiment, the information complexity of the target information is determined according to the pixel usage rate of the target information in the corresponding UI module area. The higher the pixel usage rate of the target information in the corresponding UI module area, the more complex the target information; the lower the pixel usage rate of the target information in the corresponding UI module area, the simpler the target information. When calculating the pixel usage rate, the colored pixel count is used, and the black pixels are not counted.

[0143] For example, complex target information needs to be displayed at a lower lower viewing angle to avoid blocking the road surface. As Figure 4 shown, the pixel usage rate of the first UI module 401 (vehicle speed identification module) is about 10%, and the pixel usage rate of the second UI module 402 is about 30%. Therefore, the first UI module 401 is displayed in the near-layer virtual image 101 of the first head-up display 1, and the second UI module 402 is displayed in the virtual image 30 formed by the third head-up display 3 at a lower position to avoid excessive occlusion of the real road surface.

[0144] Figure 5 (a) in shows the display schematic diagram of the first UI module 401, Figure 5 in (b) shows the pixel usage schematic diagram of the first UI module 401. As Figure 5As shown, the UI module area corresponding to the vehicle speed indicator is the vehicle speed display area (the black area in the figure). The vehicle speed display area occupies 40 * 40 = 1600 pixels. The UI of the vehicle speed indicator (the first UI module 401) uses 175 pixels in its UI module area, and its pixel utilization rate is approximately 10%. It is determined that the complexity of the target information is relatively small. There are a large number of blank areas in the vehicle speed display area, and the road surface is less blocked. Therefore, the vehicle speed indicator can be displayed in the nearer virtual image 101 at a relatively upper position (compared with the virtual image 30 formed by the third head-up display 3).

[0145] As described above, in this embodiment, according to the pixel utilization rate of the target information in the corresponding function module area (i.e., the above-mentioned UI module area), the information complexity of the target information is determined, and then the target head-up display corresponding to the target information is determined. When the pixel utilization rate of the target information in the corresponding function module area is greater than the preset utilization rate threshold, the function module including the target information is displayed in the lower area of the imaging window 5 (the display area of the virtual image 30 formed by the third head-up display 3); when the pixel utilization rate of the target information in the corresponding function module area is less than or equal to the preset utilization rate threshold, the function module including the target information is displayed in the upper area of the imaging window 5 (the display area of the nearer virtual image 101 formed by the first head-up display 1).

[0146] In the above embodiment, non-AR driving information belongs to a type of information category. When the information category of the target information is other categories, the target head-up display corresponding to the target information can also be determined according to the information complexity of the target information. For example, when the information category of the target information is video information, it can be determined that the information complexity is relatively large, and the target head-up display corresponding to the target information is determined to be the second head-up display 2, that is, the video information is displayed at a higher position in the imaging window 5, which does not affect the safe driving of the driver (if it is displayed in the area corresponding to the farther virtual image 102, it may block the driver's line of sight), and it is convenient for users to entertain.

[0147] In step S201, when it is determined that the information category of the target information is the preset target category (such as the above non-AR driving information), the target head-up display corresponding to the target information is determined.

[0148] In specific implementation, the target category can also be other categories, or can be more finely divided in other ways. For example, the target category can include AR information categories such as destination identification information, lane line identification information, POI information, etc. After determining the target category to which the information category of the target information belongs, through step S202, according to at least one of the relative position relationship between the target object in the real environment corresponding to the target information and the vehicle, and the relationship between the target information and other information in the virtual images formed by the multiple head-up displays, the target head-up display and / or display form corresponding to the target information can be determined, that is, not only can the target head-up display corresponding to the target information be determined, but also the display form of the target information on the target head-up display can be determined; or after the display form of the target information is determined first, the target head-up display corresponding to the target information can be determined according to the display form, so as to realize the diversified display of the target information.

[0149] In some embodiments, the target information is POI information, the target category includes POI information, and in step S202, according to the relative position relationship between the target object in the real environment corresponding to the target information and the vehicle, determining the target head-up display and display form corresponding to the target information includes:

[0150] S2021: According to the position information of the target object relative to the vehicle and the field of view angle ranges corresponding to the respective head-up displays, determine the field of view angle range where the target information is located;

[0151] S2022: According to the field of view angle range where the target information is located, determine the target head-up display and display form corresponding to the target information.

[0152] The target head-up display corresponding to the POI information is closely related to the position relationship between the vehicle and the POI in the real environment (such as the target building in the real environment).

[0153] Such as Figure 6 and Figure 7As shown in the figure, in this embodiment, when the target object is a POI of a target building and the target information is the POI information of the target building (such as the name and icon of the building), the position information of the vehicle, the position information of the target building, and the height information of the target building can be obtained through the positioning information and the high-precision map. The target information can be displayed to fit the target building. For example, the POI information is superimposed on a preset position of the building (such as the half-height position of the building), so that the target information is presented as AR information integrated with the real environment. At this time, according to the position information of the target object relative to the vehicle and the field of view angle ranges corresponding to each head-up display, it can be determined whether the target object appears in the field of view of the head-up display, and then the target head-up display for displaying the target information can be determined. Specifically, according to the distance and height of the target building relative to the vehicle and the field of view angle ranges corresponding to each head-up display, it can be determined whether the target building appears in the field of view of the head-up display, and then the target head-up display for displaying the POI information can be determined. When the target building appears in the field of view of the head-up display, the POI information can be displayed in the head-up display. In this embodiment, the head-up display system includes a first head-up display 1 and a second head-up display 2, that is, it is determined whether the POI information corresponding to the target building can be displayed in the field of view angle range (θ1) of the first head-up display 1 and the field of view angle range (θ2) of the second head-up display 2, and in which head-up display it is displayed.

[0154] As Figure 6 shown, both target buildings appear in the field of view angles of the first head-up display 1 and the second head-up display 2. Therefore, according to the heights of the two target buildings, the distances between the target buildings and the vehicle, and the field of view angle ranges corresponding to each head-up display, the field of view angle range where the preset position of the target building is located, that is, the field of view angle range where the POI information is located, can be determined. Then, the target head-up display corresponding to the POI information can be determined. The preset position of the first building (that is, the half-height position of the first building) is within the field of view angle of the first head-up display 1, and the preset position of the second building (that is, the half-height position of the second building) is within the field of view angle of the second head-up display 2. Therefore, it is determined that the target head-up display for the first POI information (POI1) corresponding to the first building is the first head-up display 1, and the target head-up display for the second POI information (POI2) corresponding to the second building is the second head-up display 2.

[0155] In the above embodiments, it is possible to make a separate judgment and compare the POI object (i.e., the target object corresponding to the POI information) with the field of view angle ranges of each head-up display one by one. In some other embodiments, it is also possible to superimpose the field of view angle ranges of two head-up displays to obtain the combined overall field of view angle range (θ1 + θ2). Then, when the target building appears within this overall field of view angle range, determine the specific field of view angle range of the head-up display where the POI information corresponding to the target building is located, and further determine the target head-up display.

[0156] After determining the target head-up display, it is possible to continue to determine the display form of the target information based on the distance between the target object and the vehicle (i.e., the relative position relationship between the target object in the real environment corresponding to the target information and the vehicle), or based on the relationship between the target information and other information in the virtual images formed by multiple head-up displays, or based on the distance between the target object and the vehicle and the relationship between the target information and other information in the virtual images formed by multiple head-up displays.

[0157] When the POI information is displayed in a simplified display form, for example, only the text content, icons, etc. in the POI information are displayed, so as to display the refined POI information in the virtual image. When the POI information is displayed in a complete display form, the POI information can be displayed in detail (such as text content, icons, pictures, videos, three-dimensional, etc. content).

[0158] In some embodiments, after determining the target head-up display, based on the distance between the target building and the vehicle, or combined with other information such as the position of the target building within the field of view angle of the head-up display, determine the display form of the POI information. For example Figure 6 in, the distances between two target buildings and the vehicle are relatively close, and the overall field of view angle range cannot completely cover the target buildings. The display form of the POI information can be determined according to the position, display ratio, size, etc. of the part of the target building that can be displayed within the field of view angle. Figure 6 in, the proportion of the part of the target building that can be displayed within the field of view angle is relatively small, and the POI information can be displayed in the form of a refined POI ( Figure 6 the dot in the circle) form. Similarly, when most of the target building corresponding to the POI information is located within the overall field of view angle range, the POI information can be displayed in complex forms such as pictures and videos.

[0159] When the target building is not within the overall field of view angle range (such as a target building at a long distance), an indicator (such as an arrow) can be displayed in the first head-up display 1 or the second head-up display 2, and the indicator points to the direction where the target building is located to prompt the user that there may be a POI in this direction. The arrow is not the POI information to be displayed. Therefore, it will not present a perspective effect of being larger near and smaller far away, but is displayed with a fixed size.

[0160] In step S202, when determining the target head-up display corresponding to the target information according to the relative position relationship between the target object in the real environment corresponding to the target information and the vehicle, as Figure 7 shown, the preset position of the POI corresponding to the POI information (for example, at half the height of the building) relative to the coordinates (x, y, z) of the vehicle can be mapped to the rendering space coordinates, and it is judged which coordinate space corresponding to the head-up display the rendering space coordinates are located in, so as to determine the target head-up display corresponding to the POI information. For example, the first POI information (POI1) is in the coordinate space corresponding to the first head-up display 1 (within the range of ∠BOC), and the second POI information (POI2) is in the coordinate space corresponding to the second head-up display 2 (within the range of ∠AOB). Therefore, the first POI information is displayed in the virtual image 10 of the first head-up display 1, and the second POI information is displayed in the virtual image 20 of the second head-up display 2. After determining the target head-up display of the POI information, the display form of the target information is further determined according to the relative position between the target building and the vehicle, etc., which will not be elaborated here.

[0161] In some other embodiments, after determining the target head-up display, the display form of the POI information is determined according to the relationship between the target information and other information in the virtual images formed by the multiple head-up displays.

[0162] The relationship between the target information and other information in the virtual images formed by the multiple head-up displays mainly refers to the relationship determined according to the function and use of the target information.

[0163] When the target information is POI information, the POI information can be AR information, and the display position and display form of the target information in the virtual image are restricted by the display of other function information already displayed in the virtual image. For example, after determining the target head-up display corresponding to the POI information, if it is determined that the display position of the POI in the virtual image formed by the target head-up display conflicts with other function display areas (such as the flat instrument area, the road surface information area, and the navigation area), the display form of the POI information can be changed to a simplified form so that the user can view the target information without interfering with the display of other information. When determining the target head-up display corresponding to the POI information, if the layout of the virtual image formed by the target head-up display is relatively loose, that is, the horizontal and vertical fields of view are larger, there is less other function information, or there is a clear POI position area on the virtual image, more elements such as pictures, videos, and three-dimensional scenes can be added to the display content of the target information to form a target information with richer content and more diverse forms, further improving the user experience.

[0164] In addition, if the display position of the POI information is restricted, but there is still space in the vertical field of view, the POI information can be displayed at a higher position, farther from the ground, that is, the display position of the POI information in the virtual image formed by the target head-up display is changed. For another example, when the POI information corresponding to a building is displayed in the far-layer virtual image 102 corresponding to the first head-up display 1, if there is already display content in the display position of the POI information, to ensure that the user can clearly see the POI information, the POI information can be displayed at a higher position, such as on the top of the building.

[0165] The target head-up display corresponding to the POI information, the specific display position of the POI information in the virtual image formed by the target head-up display, etc. can be adjusted in real time according to the driving of the vehicle. The above steps S2013 and S2014 are applicable not only to the AR information such as POI information, but also to the display of other AR information such as the front vehicle locking mark or the fusion information of AR and VR, which will not be elaborated here.

[0166] In the above embodiments, the target head-up display is determined first, and then the display form of the target information is determined. In some other embodiments, the display form of the target information can be determined first, and then the target head-up display can be determined. For example, after determining that the target information needs to be simplified for display, it can be displayed in the virtual image 10 formed by the first head-up display 1, rather than in the virtual image 20 formed by the second head-up display 2 with a higher imaging height, so that the user can view it without raising their head.

[0167] In some embodiments, the method further includes:

[0168] S301: In response to an operation on the target information for the display area of the far-layer virtual image 102 currently displayed by the first head-up display 1 or the display area of the virtual image 20 formed by the second head-up display 2, obtain the associated information of the target information;

[0169] S302: Display the associated information in the virtual image 20 formed by the second head-up display 2.

[0170] When the head-up display system includes the first head-up display 1 and the head-up display 2, after the user performs an interaction operation on the target information (such as POI information) displayed in the far-layer virtual image 102 or the virtual image 20, the associated information of the target information (such as the response result of the interaction operation) is displayed in the virtual image 20 formed by the second head-up display 2.

[0171] During the driving of the vehicle, when the driver or other HUD users are interested in information such as a cinema or its advertisement in the POI information displayed in the display area of the far-layer virtual image 102 or the display area of the virtual image 20, the user can, in the form of selecting the POI, make the associated information of the movie ticket purchase interface be displayed in the virtual image 20 formed by the second head-up display 2, so that it is convenient for the user to open a third-party movie sales platform to purchase tickets. Optionally, the AR information corresponding to the three-dimensional model related to the cinema (such as the three-dimensional model for displaying the internal space of the screening hall) is displayed in the virtual image 20 formed by the second head-up display 2 (above the virtual image 10 formed by the first head-up display 1) so as not to block the road surface and can provide a better AR interaction experience for the user. Since the height of the virtual image 20 formed by the second head-up display 2 is higher than that of the virtual image 10 formed by the first head-up display 1, in some embodiments, all the associated information is displayed in the virtual image 20 formed by the second head-up display 2, thus avoiding the associated information displayed from blocking the road surface.

[0172] In some other embodiments, the associated information can also be displayed in the virtual image 30 formed by the third head-up display 3. Since the height of the virtual image 30 formed by the third head-up display 3 is the lowest, the non-AR associated information such as the movie list in the ticket purchase interface can be displayed in the virtual image 30 formed by the third head-up display 3 for the convenience of the user to view during driving, while the AR associated information corresponding to the three-dimensional model related to the cinema is displayed in the virtual image 20 formed by the second head-up display 2 to achieve a reasonable display of the associated information. The target information is still displayed in the display area of the far-layer virtual image 102 formed by the first head-up display 1 or the display area of the virtual image 20 formed by the second head-up display 2.

[0173] In some embodiments, the method further includes:

[0174] S401: Determine the display parameters of the currently displayed target information according to the vehicle speed, where the display parameters include at least one of the display form, display time, and display size of the target information; or

[0175] S402: Adjust the display parameters of the currently displayed target information according to the driving scenario in which the vehicle is located, or add or cancel the display of the multimedia interaction information corresponding to the target information.

[0176] Among them, the display form of the target information includes simplified display, complete display, etc.

[0177] In step S401, the display parameters of the currently displayed target information can be determined according to the vehicle speed to meet the viewing needs of the user and provide the user with a better viewing experience. For example, taking the target information as POI information, when the vehicle speed is higher than the preset vehicle speed threshold, the vehicle speed is relatively fast. At this time, the target information can be displayed in a simplified form, such as only displaying the text content in the POI information, or the display time of the target information can be set longer, and the target information can be enlarged for display, etc., to ensure that the user can clearly see the target information and avoid visual interference caused by the information flashing quickly to the user, resulting in phenomena such as dizziness for the user; when the vehicle speed is lower than or equal to the preset vehicle speed threshold, the vehicle speed is relatively slow. At this time, the target information can be displayed in a complete form, such as simultaneously displaying the text, pictures, videos, etc. in the POI information, or the display time of the target information can be set shorter, and the target information can be scaled for display, etc., so that while the user can clearly see the target information, the slower vehicle speed can be used to view more abundant information, and visual fatigue of the user caused by some information being displayed for too long or having too large a display size can be avoided.

[0178] In step S402, the display parameters of the currently displayed target information can be adjusted according to the driving scenario in which the vehicle is located, or the multimedia interaction information corresponding to the target information can be added for display or cancelled. For example, when the target vehicle is in a traffic jam scenario (the vehicle speed is 0 in D gear and there are many surrounding vehicles), since the target vehicle is close to a stationary state, the display parameters of the target information can be adjusted as described above when the vehicle speed is low; when the target vehicle is close to a stationary state, the distance from other surrounding buildings or other vehicles is relatively close, and the position of the POI information used to mark other surrounding buildings or other vehicles remains unchanged or changes little. At this time, more forms of target information can be displayed. For example, when the POI is the target shopping mall, when the vehicle is in a traffic jam scenario (near the parking lot of the target shopping mall), the restaurants in the target shopping mall can be displayed in the display area of the virtual image 20 formed by the second head-up display 2 in the form of pictures or videos, that is, the multimedia interaction information corresponding to the target information is displayed, so that the user can have a meal later. At this time, the second head-up display 2 can also display entertainment and life information such as weather information, social media notifications, instant news, environmental protection tips, music and entertainment recommendations, environmental protection tips, etc., to facilitate the user's entertainment in the traffic jam state. Another example is that when the driving scenario of the vehicle is a rough road surface, to enable the user to concentrate on driving and ensure driving safety, the display form of the target information can be adjusted, such as displaying it in a simplified form (such as only displaying the text content in the POI information), or reducing the display time of the target information, scaling the target information for display, etc. Similarly, at this time, the multimedia interaction information corresponding to the target information can be cancelled to ensure that the user can concentrate on driving.

[0179] It can be understood that the target information and multimedia interaction information after adjusting the display parameters can be displayed in the display area of the currently displayed head-up display, or can also be displayed in the display area of other head-up displays. For example, when the target information is the POI information in text format of the target mall (for example, the target information is the mall name "XX Mall"), the POI information is displayed in the far virtual image 102 of the first head-up display 1, and the multimedia interaction information corresponding to the target information (such as the 3D scene model of the mall, the names of restaurants in the mall, restaurant introductions, etc.) can be projected and displayed in the display area of the virtual image 20 formed by the second head-up display 2.

[0180] Furthermore, for the POI information containing a 3D scene, interactive displays such as rotation or magnification can be performed to improve the user experience. At the same time, in this case, other entertainment and life information, such as weather information, social media notifications, instant news, environmental protection tips, music and entertainment recommendation information, etc., can also be displayed in the display area of the virtual image 20 formed by the second head-up display 2.

[0181] The reference factors for determining the target head-up display and / or display form corresponding to the target information described above can also be other factors, and the display form can also be other more forms (such as color gradient), which are not specifically limited in this disclosure.

[0182] In some embodiments, the method further includes:

[0183] S501: Obtain the target information to be displayed in the first virtual image formed by the head-up display system;

[0184] S502: Pre-display the target information in the second virtual image formed by the head-up display system, where the first virtual image and the second virtual image are different.

[0185] Among them, the first virtual image is the far virtual image 102 formed by the first head-up display 1 or the virtual image 20 formed by the second head-up display 2, and the second virtual image is the near virtual image 101 formed by the first head-up display 1 or the virtual image 30 formed by the third head-up display 3.

[0186] During the interaction through multiple head-up displays, when determining AR information such as POI information to be displayed in the far-layer virtual image 102 or virtual image 20, or the fusion information of AR and VR information based on the geographical location and navigation route, the target information can be pre-displayed in the near-layer virtual image 101 formed by the first head-up display 1 or the virtual image 30 formed by the third head-up display 3 in a folded state or a planar form (i.e., non-AR form), etc., that is, the target information is displayed in the form of planar information. Then, when the POI information is within the field of view angle of the first head-up display 1 or the second head-up display 2, the pre-displayed target information is fused with the real scene and displayed in the far-layer virtual image 102 formed by the first head-up display 1 or the virtual image 20 formed by the second head-up display 2 by means of state conversion or switching the display area, so that the user can see the smoothly changing target information and avoid the impact on the user's vision caused by the sudden appearance of the target information.

[0187] In some embodiments, the method further includes:

[0188] S601: Obtain the target information in the first virtual image with the remaining display time less than the time threshold;

[0189] S602: Convert and display the target information in the second virtual image.

[0190] For the POI information that is about to disappear in the far-layer virtual image 102 or virtual image 20, in this step, it is converted and displayed in the near-layer virtual image 101 formed by the first head-up display 1 or the virtual image 30 formed by the third head-up display 3, which can increase the time for the driver to obtain information and avoid some information flashing by due to the POI position or too fast vehicle speed. At the same time, the POI trajectory along the journey can be automatically collected and added (collected in the form of a preset icon), which is convenient for the user to view and record in the second virtual image in time or retrieve and view through the AI assistant, improving the information interaction effect of each head-up display.

[0191] In some embodiments, the method further includes:

[0192] S701: Display recommended information in the third virtual image;

[0193] S702: Based on the target recommended information selected by the user from the recommended information, after the vehicle arrives at the recommended location corresponding to the target recommended information, display the target recommended information in the fourth virtual image.

[0194] The third virtual image is the virtual image 20 formed by the second head-up display 2 or the near-layer virtual image 101 formed by the first head-up display 1, and the fourth virtual image is the far-layer virtual image 102 formed by the first head-up display 1.

[0195] The recommended information can be commercial information such as recommended locations or delicious foods. There is at least one piece of recommended information (for example, at least one recommended shopping mall), which can be pre-displayed on the third virtual image or displayed on the third virtual image when the vehicle travels near the recommended location. When the user interacts with the recommended information and selects one of the recommended information as the target recommended information and changes the itinerary to go to the recommended location corresponding to the target recommended information (such as the shopping mall where a certain recommended food is located), the first head-up display 1 or the second head-up display 2 performs navigation planning according to the recommended location selected by the user. When the vehicle arrives at the recommended location selected by the user, the target recommended information moves to the far-layer virtual image 102 formed by the first head-up display 1 through form conversion and is displayed by fusing with the recommended location in an AR fusion manner, realizing diversified display of the recommended information and facilitating the user to perform subsequent interactions based on the recommended information.

[0196] When the recommended information is directly the recommended location, the target recommended information is the target recommended location, and this target recommended information can be POI information and is displayed by fusing with the real target recommended location. As can be seen from the above, steps S701 - S702 specifically describe the display of the target information of the target recommended information. The display of the target recommended information (for example, a recommended restaurant can be simply displayed in the form of a fork and knife, or the restaurant sign, the interior view of the restaurant, etc.) can be set as needed, and the present disclosure does not specifically display it.

[0197] The third embodiment of the present disclosure provides a vehicle, including the above-mentioned head-up display system. The vehicle includes, but is not limited to, land vehicles such as cars, air vehicles such as aircraft, or water or underwater vehicles, etc.

[0198] It should be noted that the vehicle corresponds to the head-up display system and the display method of the head-up display system in the above embodiments. Any optional items in the embodiments of the head-up display system and the display method of the head-up display system are also applicable to the embodiments of the vehicle, and will not be elaborated here.

[0199] The fourth embodiment of the present disclosure provides an electronic device, including at least a memory and a processor. A computer program is stored on the memory, and when the processor executes the computer program on the memory, the steps of the above method are implemented.

[0200] In some embodiments, the processor executing the computer program may be a processing device including more than one general-purpose processing device, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be more than one dedicated processing device, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0201] The memory may be a read-only memory (ROM), a random access memory (RAM), a phase change random access memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), an electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), a flash drive or other forms of flash memory, a cache, a register, a static memory, a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD) or other optical memory, a cassette tape or other magnetic storage device, or any other possible non-transitory medium used to store information or instructions accessible by a computer device, etc.

[0202] The electronic device according to the embodiments of the present disclosure may include, but is not limited to, fixed terminal devices such as servers, desktop computers, digital TVs, etc., and mobile terminal devices such as in-vehicle devices (e.g., head-up display devices), handheld devices (e.g., mobile phones, tablet computers, etc.), wearable devices (e.g., smart watches, smart bracelets, etc.).

[0203] The fifth embodiment of the embodiments of the present disclosure provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0204] The computer-readable storage medium according to the embodiments of the present disclosure may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device, for example, it may be the above-mentioned memory.

[0205] The computer programs of the embodiments of the present disclosure can be organized into one or more computer-executable components or modules. Aspects of the present disclosure can be implemented with any number and combination of such components or modules. For example, aspects of the present disclosure are not limited to the specific computer-executable instructions or specific components or modules shown in the figures and described herein. Other embodiments may include different computer-executable instructions or components with more or less functionality than those shown and described herein.

[0206] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

Claims

1. A head-up display system, characterized in that, Including a plurality of head-up displays, the plurality of head-up displays include a first head-up display and a second head-up display, the first head-up display is used to display augmented reality information, the second head-up display is used to display the fused information of augmented reality and virtual reality, and the virtual image distance of the virtual image formed by the second head-up display is greater than the virtual image distance of the virtual image formed by the first head-up display.

2. The head-up display system according to claim 1, characterized in that, The first head-up display is arranged in the center console, the second head-up display is arranged above the cockpit or in the center console, and the cockpit includes at least one of a driver's cockpit, a co-driver's cockpit, and a rear cockpit.

3. The head-up display system according to claim 1, wherein, The plurality of head-up displays further include a third head-up display, and the virtual image distance of the virtual image formed by the third head-up display is less than the virtual image distance of the virtual image formed by the first head-up display.

4. The head-up display system according to claim 3, wherein, The head-up display system further includes an imaging window, and an anti-reflection part is arranged in the imaging area corresponding to the third head-up display on the imaging window.

5. The head-up display system according to claim 1, wherein The virtual image formed by the first head-up display includes a near-layer virtual image and a far-layer virtual image, and the virtual image distance of the near-layer virtual image is less than the virtual image distance of the far-layer virtual image.

6. The head-up display system according to claim 5, wherein The virtual image distance of the near-layer virtual image is 3 - 6m, and the virtual image distance of the far-layer virtual image is 10 - 20m; the virtual image distance of the virtual image formed by the second head-up display > 20m.

7. The head-up display system according to claim 3, wherein The third head-up display is arranged on the surface of the center console or in the center console, and the virtual image distance of the virtual image formed by the third head-up display is 1 - 2m.

8. The head-up display system according to any one of claims 1-7, characterized in that The virtual image formed by the second head-up display is located above the virtual image formed by the first head-up display, and the virtual image formed by the first head-up display and the virtual image formed by the second head-up display do not overlap or partially overlap in the height direction.

9. The head-up display system according to claim 3, wherein The display area of the virtual image formed by the third head-up display is the lowermost area among the display areas of the virtual images formed by the plurality of head-up displays; Or The display area of the virtual image formed by the third head-up display is located on the left side of the display area of the virtual image formed by the first head-up display; Or The display area of the virtual image formed by the third head-up display is located on the right side of the display area of the virtual image formed by the first head-up display; Or The display area of the virtual image formed by the third head-up display is located on the left side and the right side of the display area of the virtual image formed by the first head-up display.

10. A display method for a head-up display system, characterized in that, Applied to the head-up display system according to any one of claims 1 - 9, the method includes: Determine the target head-up display corresponding to the target information according to the information category of the target information to be displayed; Display the target information in the virtual image formed by the target head-up display, wherein each virtual image formed by the target head-up display has a corresponding information category.

11. The display method according to claim 10, wherein The information category corresponding to the near-layer virtual image formed by the first head-up display includes first non-AR planar information; The information category corresponding to the far-layer virtual image formed by the first head-up display includes augmented reality information integrating the real scene; The information categories corresponding to the virtual images formed by the second head-up display include at least one of the augmented reality information, the associated information of the augmented reality information, virtual reality information, the fusion information of augmented reality and virtual reality, and entertainment life information; The information categories corresponding to the virtual images formed by the third head-up display include the second non-AR planar information.

12. The display method according to claim 11, wherein The first non-AR planar information includes at least one of instrument-related information and electronic rearview mirror information; The augmented reality information includes at least one of AR navigation information and AR assisted driving information; The second non-AR planar information includes at least one of the associated information of the augmented reality information, electronic rearview mirror information, and the AI assistant, The AI assistant is used to perform AI intelligent interaction with the passengers in the vehicle and assist in controlling the display of the virtual images formed by the target head-up display.

13. The display method according to claim 10, characterized in that, The determining of the target head-up display corresponding to the target information according to the information category of the target information to be displayed includes: In the case where the information category of the target information is non-AR driving information, determining the target head-up display corresponding to the target information according to the information complexity of the target information; or, In the case where the information category of the target information is the target category, determining the target head-up display corresponding to the target information and / or the display form according to at least one of the relative position relationship between the target object in the real environment corresponding to the target information and the vehicle, and the relationship between the target information and other information in the virtual images formed by the multiple head-up displays, and the display form includes at least one of being displayed in AR form, being displayed in non-AR form, simplified display, and full display.

14. The display method according to claim 13, characterized in that The determining of the target head-up display corresponding to the target information according to the information complexity of the target information includes: If the information complexity of the target information is higher than the complexity threshold, displaying the target information in the display area of the virtual image formed by the third head-up display, where the information complexity of the target information is determined according to the pixel usage rate of the target information; or, If the information complexity of the target information is lower than or equal to the complexity threshold, displaying the target information in the display area of the near-layer virtual image formed by the first head-up display.

15. The display method according to claim 13, wherein The target information is POI information, the target category includes POI information, and the determining of the target head-up display corresponding to the target information and the display form according to the relative position relationship between the target object in the real environment corresponding to the target information and the vehicle includes: Determining the field of view angle range where the target information is located according to the position information of the target object relative to the vehicle and the field of view angle ranges corresponding to the respective head-up displays; Determining the target head-up display corresponding to the target information and the display form according to the field of view angle range where the target information is located.

16. The display method according to claim 10, wherein The method further includes: In response to an operation on target information for a display area of a far-layer virtual image currently displayed on the first head-up display or a display area of a virtual image formed by the second head-up display, obtain associated information of the target information; Display the associated information in the virtual image formed by the second head-up display.

17. The display method according to claim 10, wherein The method further includes: Determine display parameters of the currently displayed target information according to the vehicle speed, where the display parameters include at least one of a display form, a display time, and a display size of the target information; or Adjust the display parameters of the currently displayed target information according to a driving scenario in which the vehicle is located, or add or cancel the display of multimedia interaction information corresponding to the target information.

18. The display method according to claim 10, wherein The method further includes: Obtain target information to be displayed in a first virtual image formed by the head-up display system; Pre-display the target information in a second virtual image formed by the head-up display system, where the first virtual image and the second virtual image are different.

19. The display method according to claim 18, wherein, The method further includes: Obtain target information in the first virtual image with a remaining display time less than a time threshold; Convert and display the target information in the second virtual image.

20. The display method according to claim 18, wherein The method further includes: Display recommended information in a third virtual image; Based on target recommended information selected by a user from the recommended information, after the vehicle arrives at a recommended location corresponding to the target recommended information, display the target recommended information in a fourth virtual image.

21. A vehicle, characterized in that, A head-up display system according to any one of claims 1-9.

Images