Head-up display device, display control method and device, storage medium, and vehicle

Abstract

A head-up display device (170), a display control method, a display control device (1700), a storage medium, and a vehicle. The display control method may comprise: acquiring a layer of image data (50) to be displayed (S401); determining, from among projection planes (31-1, ..., 31-N) formed by a display portion (160), a display projection plane for displaying the layer (S402); and controlling the layer to display on the corresponding display projection plane (S403).

Description

Head-up display device, display control method, device and storage medium, vehicle CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 12, 2023, with application number 2023117025895, and invention name “Head-up display device, display control method, device and storage medium, vehicle”. The content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0002] The present disclosure relates to the field of assisted driving technology, and more specifically to a head-up display device, a display control method, a device and a storage medium, and a vehicle. Background Art

[0003] A head-up display (HUD) device projects light from an image source onto an imaging window (e.g., an imaging board, windshield, etc.) through, for example, a reflective optical design, to display vehicle status information such as speed and fuel level, as well as navigation, hazard warnings, and other indication information at an appropriate location in front of the driver. This allows the driver to obtain relevant information such as speed and fuel level without shifting their line of sight from the road ahead, thereby improving driving safety and the driving experience. Technical content

[0004] The present disclosure provides a head-up display device, a display control method, a device and a storage medium, and a vehicle; these can increase the vividness and immersion of a driver's visual effects and improve the user experience.

[0005] The technical solution of the present disclosure is achieved as follows:

[0006] In a first aspect, the present disclosure provides a display control method, the method comprising:

[0007] Get the layer of image data to be displayed;

[0008] Determining a display projection surface for displaying the layer from the projection surfaces formed by the display unit;

[0009] Control the layer to be displayed on the corresponding display projection surface.

[0010] In a second aspect, the present disclosure provides a display control device, the device comprising: an acquisition part, a determination part and a control part; wherein,

[0011] The acquisition part is configured to acquire a layer of image data to be displayed;

[0012] The determining section is configured to determine a display projection surface for displaying the layer from among the projection surfaces formed by the display section;

[0013] The control part is configured to control the layer to be displayed on the corresponding display projection surface.

[0014] In a third aspect, the present disclosure provides a display control device, comprising: a processor and a memory; the processor is configured to execute instructions stored in the memory to implement the display control method as described in the first aspect.

[0015] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the display control method as described in the first aspect.

[0016] In a fifth aspect, the present disclosure provides a head-up display device, comprising a display control unit and a display unit; wherein,

[0017] The display control unit is configured to obtain a layer of image data to be displayed;

[0018] Determining a display projection surface for displaying the layer from the projection surfaces formed by the display unit;

[0019] Controlling the layer to be displayed on the corresponding display projection surface;

[0020] The display unit is configured to display the layer on the corresponding display projection surface based on the control of the display control unit.

[0021] In a sixth aspect, the present disclosure provides a vehicle, comprising the head-up display device described in the fifth aspect.

[0022] The present disclosure provides a head-up display device, a display control method, a device and a storage medium, and a vehicle; after dividing image data to be displayed into different layers according to the depth of field from near to far, a corresponding display projection surface is allocated to each layer according to the distance from the driver's viewpoint from near to far, thereby increasing the vividness and immersion of the driver's visual effects when viewing the image data and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for describing the embodiments. The drawings described below are only exemplary embodiments of the present disclosure.

[0024] FIG. 1 is an exemplary top view of a vehicle provided by the present disclosure.

[0025] FIG. 2 is an exemplary perspective view from a driver's seat of a vehicle provided by the present disclosure.

[0026] FIG3 is a schematic diagram of the architecture of the head-up display device provided by the present disclosure.

[0027] FIG4 is a flow chart of a display control method provided by the present disclosure.

[0028] FIG5(A) is a schematic diagram of image data to be displayed provided by the present disclosure.

[0029] FIG5(B) is a schematic diagram of a layer division provided by the present disclosure.

[0030] FIG6 (A) is a side view schematic diagram of determining a projection surface for each layer provided by the present disclosure.

[0031] FIG6(B) is an oblique schematic diagram of determining a projection surface for each layer provided by the present disclosure.

[0032] FIG7 is a schematic diagram of a process of dividing layers provided by the present disclosure.

[0033] FIG8 is a schematic diagram of another layer division provided by the present disclosure.

[0034] FIG9(A) is a side view schematic diagram of another method for determining a projection surface for each layer provided by the present disclosure.

[0035] FIG9(B) is another oblique schematic diagram of determining the projection surface for each layer provided by the present disclosure.

[0036] FIG10(A) is a side view schematic diagram of another method of determining a projection surface for each layer provided by the present disclosure.

[0037] FIG10(B) is another oblique schematic diagram of determining the projection surface for each layer provided by the present disclosure.

[0038] FIG11(A) is a side view schematic diagram of another method of determining a projection surface for each layer provided by the present disclosure.

[0039] FIG11(B) is another oblique schematic diagram of determining the projection surface for each layer provided by the present disclosure.

[0040] FIG12(A) is a side view schematic diagram of another method for determining a projection surface for each layer provided by the present disclosure.

[0041] FIG12(B) is another oblique schematic diagram of determining the projection surface for each layer provided by the present disclosure.

[0042] FIG13 is a schematic diagram of the architecture of another head-up display device provided by the present disclosure.

[0043] FIG14 is a schematic diagram of another image data to be displayed provided by the present disclosure.

[0044] FIG15(A) is a schematic diagram of layers of another type of image data to be displayed provided by the present disclosure.

[0045] FIG15(B) is a schematic diagram of arrangement of layers of another image data to be displayed along the sight line direction provided by the present disclosure.

[0046] FIG16(A) is a schematic diagram showing the display of various layers of image data to be displayed provided by the present disclosure.

[0047] FIG16(B) is another schematic diagram of displaying layers of image data to be displayed provided by the present disclosure.

[0048] FIG17 is a schematic diagram showing the composition of a display control device provided by the present disclosure.

[0049] FIG18 is a schematic structural diagram of a display control device provided by the present disclosure. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present disclosure more apparent, the following will describe in detail exemplary embodiments of the present disclosure with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0051] FIG1 and FIG2 respectively show an exemplary top view of a vehicle and an exemplary perspective view from the driver's seat of the vehicle. In the present disclosure, the vehicle is equipped with a head-up display device 170 (see FIG3 ) that can be applied to the technical solution of the present disclosure. In some examples, the vehicle can be an internal combustion engine vehicle using an engine as a power source, a hybrid vehicle using an engine and an electric motor as power sources, an electric vehicle using an electric motor as a power source, or other types of vehicles. In the subsequent content of this specification, the vehicle equipped with the head-up display device 170 will be referred to as the present vehicle. In FIG1 , the present vehicle includes a windshield 204 located at the front of the vehicle. The driver and passengers in the passenger cabin 208 of the present vehicle can see the front of the present vehicle through the windshield 204.

[0052] 2 , the windshield 204 is visually positioned above the vehicle dashboard 206. The driver can turn the steering wheel 210 within the passenger compartment 208 to steer the vehicle, such as to change lanes, merge, and park the vehicle. In some embodiments, the steering wheel 210 can be retracted or omitted.

[0053] In FIG2 , head-up display 170 projects display information 212 (e.g., a virtual image) onto a portion of windshield 204 through one or more apertures (e.g., aperture 216) in instrument panel 206. While FIG3 illustrates an example size of display information 212, display information 212 can be presented over a larger or smaller area. Examples of display information 212 include various vehicle information and streaming information such as images and videos provided by an in-vehicle entertainment system (not shown). Head-up display 170 provides this information to the vehicle driver without the driver having to look away from objects in front of the vehicle.

[0054] Referring to the exemplary implementation architecture of a head-up display device 170 shown in FIG3 , the head-up display device 170 includes a display control unit 150 and a display unit 160. After receiving data 320 from in-vehicle systems such as the vehicle control system and in-vehicle entertainment system, the display control unit 150 processes the received data 320 to obtain display information to be displayed, and projects this display information onto the vehicle's windshield 204 via the display unit 160 for display. Specifically, after receiving the data 320, the display control unit 150 generates a signal 312. The display unit 160 includes a light source 161 and an optical path component 162. Based on the signal 312 from the display control unit 150, the light source 161 outputs light (e.g., a virtual image) for display on the windshield 204. For example, the light source 161 may include one or more lasers that output red, green, and blue light.

[0055] Optical path assembly 162 can reflect the output of light source 161 onto windshield 204 through aperture 216. A viewer (e.g., the driver) can view display information 212 in a display area projected onto windshield 204. In some examples, optical path assembly 162 can include one or more reflective mirrors (plane mirrors) and concave mirrors (magnifying mirrors). Optical path assembly 162 can reflect the output of light source 161 onto windshield 204 in the form of two or more (e.g., N) optical paths (e.g., optical paths L-1, ..., LN in FIG. 3 ), forming a virtual image 30 in front of the vehicle. Based on the control of display control unit 150, virtual image 30 can be projected onto one of N projection surfaces 31-1, ..., 31-N corresponding to the N optical paths (optical paths L-1, ..., LN) for display.

[0056] In some examples, the N projection surfaces are distributed from the driver's viewpoint in a range of 5 m to 17 m from the driver's viewpoint, in descending order of distance from the driver. In some examples, the display control unit 150 can control the display unit 160 to display the virtual image 30 on a projection surface that is farther or closer to the driver's viewpoint, so that the driver can experience an AR display visual effect when observing.

[0057] In the above-described display control scheme, although virtual image 30 is displayed on projection surfaces at different distances from the driver's viewpoint to produce an AR display visual effect, virtual image 30 is still displayed only on a single projection surface, resulting in a lack of vividness and immersion in the visual effect. Therefore, the present disclosure aims to increase the vividness and immersion of the driver's visual effect by dividing the image data to be displayed into different layers and displaying them on multiple projection surfaces formed by display unit 160, thereby enhancing the user experience.

[0058] 4 , which shows a display control method provided by the present disclosure, which can be applied to the head-up display device 170 shown in FIG3 , and in particular to the display control unit 150 in the head-up display device 170. As shown in FIG4 , the method can include steps S401 to S403.

[0059] In step S401 , a layer of image data to be displayed is obtained.

[0060] In some examples, the image data to be displayed may be image data provided by other in-vehicle systems on the vehicle, such as a navigation system or an infotainment system, or may be individual image frames of video data provided by these in-vehicle systems. Specifically, these in-vehicle systems may transmit the image data to be displayed to the display control unit 150 in the head-up display device 170 via a system bus within the vehicle.

[0061] In some examples, the image data to be displayed may also be transmitted to the display control unit 150 in the head-up display device 170 by other devices not mounted on the vehicle. For example, a mobile terminal carried by the driver or passenger of the vehicle may be connected to the head-up display device 170 of the vehicle via a wired or wireless connection. Based on this connection, the mobile terminal may transmit the image data to be displayed to the display control unit 150 in the head-up display device 170.

[0062] In the above example, the image data to be displayed is usually a two-dimensional image. Based on this, the display control unit 150 can be used to perform layered processing on the content in the two-dimensional image to divide the two-dimensional image into at least one layer. In some examples, the display control unit 150 can divide the content presented by the image data to be displayed 50 from near to far according to the depth of field shown in Figure 5 (A), and can obtain 5 layers as shown in Figure 5 (B), which include at least one of the layers of subtitle layer 51, atmosphere layer 52, foreground layer 53, main body layer 54 and background layer 55. Specifically, the elements or image content included in each layer in Figure 5 (B) are:

[0063] The subtitle layer 51 is used to display text information or video subtitles, and may also include explanatory text for the video content, and may also include information about time, speed, engine speed, etc. during the vehicle's driving process.

[0064] The atmosphere layer 52 is used to present information related to the environment, such as the visualization effect of music, the light and shadow effects of the video (such as gradient light and shadow effects), and environmental elements corresponding to the weather (such as raindrops on a rainy day, sunlight on a sunny day, etc.).

[0065] The foreground layer 53 is used to display image elements with a closer depth of field in the video or image content.

[0066] The main layer 54 is used to display the most important image elements in the video or image content. This layer occupies the main display area and is used to provide a high-quality, high-definition video experience.

[0067] Background layer 55, located behind all layers, is used to display background elements related to the video or image content, enhancing the user's panoramic perception. It can also display a virtual environment or a background that coordinates with the main content. Background layer 55 does not include any transparent elements or image content.

[0068] In addition, in some examples, the vehicle-mounted infotainment system or mobile terminal may also divide the image data 50 to be displayed according to the layer order shown in FIG. 5 , and then transmit each layer of the image data to be displayed to the display control unit 150 to avoid the display control unit 150 from performing a more complex image processing operation process, and may directly control the above five layers to be displayed through the display unit 160.

[0069] In step S402, a display projection surface for displaying a layer is determined from among the projection surfaces formed by the display unit.

[0070] In the present disclosure, in combination with the projection surfaces 31-1, ..., 31-N arranged in order of distance from the driver as shown in Figure 3, corresponding display projection surfaces can be determined for the subtitle layer 51, atmosphere layer 52, foreground layer 53, main layer 54 and background layer 55 respectively from near to far. For example, as shown in Figures 6 (A) and 6 (B), there are five projection surfaces arranged in order of distance from the driving position (e.g., driver's viewpoint 60), which are labeled 31-1, ..., 31-5, respectively, and the subtitle layer 51, atmosphere layer 52, foreground layer 53, main layer 54 and background layer 55 are also divided in order from near to far. Therefore, the display projection surface determined for the subtitle layer 51 in the present disclosure is projection surface 31-1, the display projection surface determined for the atmosphere layer 52 is projection surface 31-2, the display projection surface for the foreground layer 53 is projection surface 31-3, the display projection surface for the main layer 54 is projection surface 31-4, and the display projection surface for the background layer 55 is projection surface 31-5.

[0071] In step S403, the layer is controlled to be displayed on the corresponding display projection surface.

[0072] In this disclosure, referring to Figures 6(A) and 6(B), layers with different depths of field are sequentially assigned corresponding display projection surfaces based on their distance from the driver's viewpoint 60, from near to far. Each layer is then projected onto the corresponding display projection surface by controlling the five optical paths provided by the optical path assembly 162 in the display unit 160. When a vehicle occupant, such as the driver, views video content, this provides a more vivid and immersive visual experience compared to a display solution that projects a two-dimensional image onto a single projection surface.

[0073] The technical solution shown in Figure 4 divides the image data to be displayed into different layers according to the depth of field from near to far, and then allocates a corresponding display projection surface to each layer according to the distance from the driver's viewpoint from near to far, thereby increasing the vividness and immersion of the driver's visual effects when viewing the image data and improving the user experience.

[0074] For the five layers divided in the technical solution shown in Figure 4, in some possible implementations, taking the image data to be displayed as an image frame of video data as an example, as shown in Figure 7, the depth of field of the content presented by the image data to be displayed is divided from near to far to obtain a layer including at least one of a subtitle layer, an atmosphere layer, a foreground layer, a main body layer and a background layer, including steps S701 to S705.

[0075] In step S701, subtitle text is obtained from the subtitle file of the video data, or subtitle text in the image data to be displayed is obtained by using a text recognition technology, and the subtitle text is formed into the subtitle layer.

[0076] In step S702 , after the sound related to the natural environment is identified in the video data, the atmosphere layer is obtained by matching the image data to be displayed according to the sound related to the natural environment.

[0077] In step S703, image elements with a shorter depth of field are segmented from the image data to be displayed to obtain the foreground layer.

[0078] In step S704, the image elements of the subject are segmented from the image data to be displayed to obtain the subject layer.

[0079] In step S705, background elements are segmented from the image data to be displayed to obtain the background layer.

[0080] Regarding the subtitle layer described in the above implementation, in some examples, the video data will have a separate subtitle file. When the video data is played, the subtitle text in the subtitle file is embedded in the image frame of the video data for playback. Based on this, the subtitle text of each image frame can be directly obtained from the subtitle file, and a subtitle layer can be formed based on the subtitle text. In some examples, the video data will embed the subtitles into the image frame of the video file, so that when the video data is played, the subtitles are displayed along with the played image frame. Based on this, text recognition technology (such as OCR) can be used to identify and extract the subtitle text in the image frame, and a subtitle layer can be formed based on the subtitle text.

[0081] Regarding the atmosphere layer described in the above implementation, since this layer is easily mixed with other layers, making it more difficult to segment, the present disclosure utilizes other content in the video data as a reference for segmentation. For example, after identifying the sounds of the natural environment in the video data, the natural environment currently displayed by the image frame can be determined, and based on the determined natural environment, the image elements in the image frame corresponding to the natural environment can be segmented. For example, if there is background sound of rain in the video data, the natural environment displayed by the image frame can be determined to be a rainy environment. Based on this rainy environment, the raindrop elements in the image frame can be segmented to obtain the rainy day atmosphere layer.

[0082] For the foreground layer described in the above implementation, a deep learning model can be used for segmentation, such as semantic segmentation or instance segmentation.

[0083] For the subject layer described in the above implementation, since the image elements of the subject are easily intertwined with the background or foreground, more complex image analysis methods are required, such as segmentation using a variety of image processing and computer vision methods. For example, in the present disclosure, two or more of the following concentrated technical means can be used in combination for processing. 1. Deep learning and machine learning model technology, such as using semantic segmentation to use deep neural networks (such as CNNs) to understand the category of each pixel in the image, thereby distinguishing the main content. Or using instance segmentation to further distinguish the boundaries of each independent object, which helps to extract the subject in complex scenes, or using object detection to identify and locate specific objects in the image, which can be used to identify key elements in the main content. 2. Computer vision techniques, such as highlighting foreground subjects by subtracting the background against a static or nearly static background, or using optical flow analysis to analyze pixel motion in video sequences to help identify dynamic subjects, or using feature matching and tracking to track specific feature points across consecutive frames to help maintain the continuity of the subject content. 3. Image processing algorithms, such as edge detection to identify edges in an image to help distinguish between the subject and background, or thresholding to segment an image based on pixel intensity to help extract the distinct subject content. 4. Artificial intelligence and pattern recognition techniques, such as transfer learning to use pre-trained models and adapt them to specific subject content, or autoencoders and generative adversarial networks (GANs) for advanced image analysis and reconstruction. It should be noted that the difficulty of extracting the subject layer and the required techniques depend on many factors, including image complexity, contrast between the subject and background, dynamic changes, and lighting conditions. In practical applications, it is usually necessary to combine and optimize the above technologies to meet the requirements of specific scenarios.

[0084] For the background layer described in the above implementation, similar to the foreground layer, a deep learning model can be used to segment the background elements, such as semantic segmentation or instance segmentation.

[0085] Among the five layers obtained by the aforementioned division, since the atmosphere layer is easily mixed with other layers, the image elements of the subject in the subject layer are easily intertwined with the background or foreground. Therefore, in some possible implementations, the implementation shown in FIG7 cannot accurately divide one or more of the atmosphere layer, foreground layer, and subject layer from the image data to be displayed. Based on this, the method further includes:

[0086] When at least one of the atmosphere layer, the foreground layer, and the main body layer cannot be divided, the layer that cannot be divided is displayed through the background layer.

[0087] For example, in the above implementation, among the five layers shown in Figure 5(B), it is assumed that the main layer 54 cannot be divided using the implementation shown in Figure 7. Therefore, the main layer 54 is displayed within the background layer 55, resulting in the four layers shown in Figure 8: the subtitle layer 51, the ambiance layer 52, the foreground layer 53, and the background layer 54'. Comparing the four layers shown in Figure 8 with the five layers in Figure 5(B), it can be seen that the background layer 54' in Figure 8 includes image elements from both the main layer 54 and the background layer 55 in Figure 5(B). In other words, the background layer 54' shown in Figure 8 represents the main layer 54, which cannot be divided, displayed through the background layer 55. Of course, if one or more of the ambiance layer 52, foreground layer 53, and main layer 54 shown in Figure 5(B) cannot be divided, they can also be displayed through the background layer 55, that is, all image elements from the layers that cannot be divided are displayed within the background layer 55.

[0088] Regarding the above implementation, further speaking, when some image elements in the atmosphere layer 52, the foreground layer 53 and the main layer 54 cannot be divided into corresponding layers, these image elements that cannot be divided into corresponding layers can also be displayed through the background layer.

[0089] Regarding the technical solution shown in FIG4 , when the number of layers of image data to be displayed (set as M) is inconsistent with the number of projection surfaces formed by the display unit 160 (set as N), in some possible implementations, determining a display projection surface for displaying each layer from the projection surfaces formed by the display unit includes:

[0090] When the number of projection surfaces N is less than the number of layers M, each of the first N-1 projection surfaces arranged in order of distance from the driver's seat is determined as a display projection surface for the N-1 layers divided from near to far according to the depth of field of the presented content, and the Nth projection surface is determined as a display projection surface for the remaining M-N+1 layers;

[0091] When the number of projection surfaces N is greater than the number of layers M, each of the first M projection surfaces arranged in order of distance from the driving seat among all the projection surfaces is determined as a display projection surface of M layers divided from near to far according to the depth of field of the presented content.

[0092] Specifically, for the above implementation, using the five layers shown in Figure 5(B) as an example, the layers are arranged in descending order of depth of field: subtitle layer 51, ambiance layer 52, foreground layer 53, main body layer 54, and background layer 55. When the number of projection surfaces N is less than five, the layer with the closest depth of field is prioritized for a one-to-one correspondence with the projection surface closest to the driver's seat. If a one-to-one correspondence between projection surfaces and layers is not possible, the remaining layers are displayed on the last projection surface. For example, as shown in Figures 9(A) and 9(B), when the number of projection surfaces N is four, projection surface 31-1 is the display projection surface for subtitle layer 51, projection surface 31-2 is the display projection surface for ambiance layer 52, projection surface 31-3 is the display projection surface for foreground layer 53, and projection surface 31-4 is the display projection surface for main body layer 54 and background layer 55. As shown in Figures 10(A) and 10(B), when the number of projection surfaces N is 3, projection surface 31-1 is the display projection surface for the subtitle layer 51, projection surface 31-2 is the display projection surface for the ambiance layer 52, and projection surface 31-3 is the display projection surface for the foreground layer 53, the main body layer 54, and the background layer 55. As shown in Figures 11(A) and 11(B), when the number of projection surfaces N is 2, projection surface 31-1 is the display projection surface for the subtitle layer 51, and projection surface 31-2 is the display projection surface for the ambiance layer 52, the foreground layer 53, the main body layer 54, and the background layer 55.

[0093] For the above implementation, specifically, still taking the five layers shown in FIG5 (B) as an example, when the number of projection surfaces N is greater than the number of layers M, the projection surface closest to the driving seat is preferentially used to display the corresponding layer. For example, as shown in FIG12 (A) and FIG12 (B), when the number of projection surfaces is greater than 5, such as n, projection surface 31-1 is the display projection surface of the subtitle layer, projection surface 31-2 is the display projection surface of the atmosphere layer, projection surface 31-3 is the display projection surface of the foreground layer, projection surface 31-4 is the main layer, and projection surface 31-5 is the display projection surface of the background layer. Projection surface 31-6 and the projection surfaces thereafter do not display layers. It should be noted that as the speed of the vehicle continues to increase, the driver's line of sight will gradually extend to a farther distance. In order to adapt to the driver's line of sight, in some examples, the method further includes:

[0094] When the vehicle speed exceeds the set speed threshold, all projection surfaces, starting from the Kth projection surface after the first projection surface to each of the K+M-1th projection surfaces, are determined as display projection surfaces of M layers divided from near to far according to the depth of field of the presented content.

[0095] For the above example, for example, when the driver's line of sight does not extend to a long distance, each layer can determine the corresponding display projection layer according to the example shown in Figures 6 (A) and 6 (B). When the vehicle speed is high, such as exceeding a set speed threshold, and the driver's line of sight begins to extend to a long distance, as shown in Figures 12 (A) and 12 (B), the corresponding display projection layer can be determined for each layer starting from a projection surface after the first projection surface 31-1. For example, the Kth projection surface 31-K is the display projection surface of the subtitle layer, projection surface 31-K+1 is the display projection surface of the atmosphere layer, projection surface 31-K+2 is the display projection surface of the foreground layer, projection surface 31-K+3 is the main body layer, and projection surface 31-K+4 is the display projection surface of the background layer. It should be noted that as the vehicle speed continues to increase and exceeds a larger speed threshold, the driver's line of sight extends to a farther distance. At this time, the corresponding display projection layer can be determined for each layer starting from a projection surface after the Kth projection surface 31-K. For example, the Tth (T is greater than K) projection surface 31-T is the display projection surface of the subtitle layer, projection surface 31-T+1 is the display projection surface of the atmosphere layer, projection surface 31-T+2 is the display projection surface of the foreground layer, projection surface 31-T+3 is the main layer, and projection surface 31-T+4 is the display projection surface of the background layer. The above example can adapt to the long-distance extension of the driver's line of sight caused by the increase in vehicle speed by determining the corresponding display projection surface for each layer starting from a farther projection surface, thereby improving the vividness and immersion of the display effect.

[0096] For the five layers divided in the technical solution shown in FIG4 , the display clarity of each layer can be determined based on the focus position of the driver's viewpoint. Based on this, referring to FIG13 , it shows an exemplary implementation architecture of another head-up display device 170 provided by the present disclosure. In addition to the display control unit 150 and the display unit 160 shown in FIG3 , the head-up display device 170 also includes a line of sight tracking unit 130. In some examples, the line of sight tracking unit 130 can use a line of sight tracking sensor to track the line of sight direction of the eye box position in real time, such as the line of sight direction of the driver, or it can also capture the driver's facial image through a vehicle-mounted camera device, and determine the line of sight direction of the eye box position based on the driver's eye features in the image. Based on the line of sight tracking unit 130 shown in FIG13 , in some possible implementations, the method further includes:

[0097] Determine the focus layer of the sight focus from all layers according to the sight direction of the eye box position;

[0098] The layers other than the focused layer among all the layers are displayed according to an image quality lower than the image quality of the focused layer.

[0099] Specifically, regarding the above implementation, the image data to be displayed, as shown in FIG14 , can be divided into four layers, as shown in FIG15(A) , based on the aforementioned technical solution: a subtitle layer 1401, a foreground layer 1402, a main layer 1403, and a background layer 1404. The arrangement of these four layers along the driver's line of sight is shown in FIG15(B). After the driver's line of sight is determined by the line of sight tracking unit 130, the display control unit 150 can determine the layer on which the driver's line of sight is focused based on the driver's line of sight. For example, when the driver focuses on foreground layer 1402, the image quality of subtitle layer 1401, main layer 1403, and background layer 1404, excluding foreground layer 1402, can be reduced. For example, as shown in FIG16(A), subtitle layer 1401, main layer 1403, and background layer 1404 can be displayed at a lower resolution and / or frame rate than foreground layer 1402. When the driver focuses on the main layer 1403, the image quality of the subtitle layer 1401, the foreground layer 1402 and the background layer 1404 other than the main layer 1403 can be reduced. For example, as shown in Figure 16 (B), the subtitle layer 1401, the foreground layer 1402 and the background layer 1404 can be displayed at a lower resolution and / or frame rate than the main layer 1403.

[0100] Regarding the above implementation method, it should be noted that after determining the focus layer according to the driver's line of sight, reducing the image quality of other layers except the focus layer can reduce the computing resources, storage resources and transmission bandwidth consumed in the graphics rendering process, and can also avoid the misalignment of layers on different projection surfaces when the driver's line of sight changes, thereby avoiding the phenomenon of "image goofs".

[0101] Based on the same inventive concept as the aforementioned technical solution, FIG. 17 shows a display control device 1700 provided by the present disclosure. The device 1700 may be the display control unit 150 shown in FIG. 3 or FIG. 13 . In other words, the functional structure of the display control device 1700 shown in FIG. 17 can also be implemented by the display control unit 150. The device 1700 may include: an acquisition part 1701, a determination part 1702, and a control part 1703; wherein,

[0102] The acquisition part 1701 is configured to acquire a layer of image data to be displayed;

[0103] The determining portion 1702 is configured to determine a display projection surface for displaying the layer from the projection surface formed by the display unit;

[0104] The control part 1703 is configured to control the layer to be displayed on the corresponding display projection surface.

[0105] In some examples, the obtaining portion 1701 is configured to:

[0106] The depth of field of the content presented by the to-be-displayed image data is divided from near to far to obtain layers including at least one of a subtitle layer, an atmosphere layer, a foreground layer, a main body layer and a background layer.

[0107] In some examples, the obtaining portion 1701 is configured to:

[0108] Obtaining subtitle text from a subtitle file included in the video data, or obtaining subtitle text in the image data to be displayed using a text recognition technology, and forming the subtitle layer with the subtitle text; wherein the image data to be displayed is each image frame of the video data;

[0109] After identifying sounds related to the natural environment in the video data, obtaining the atmosphere layer by matching the sounds related to the natural environment from the image data to be displayed;

[0110] Segmenting image elements with a closer depth of field from the image data to be displayed to obtain the foreground layer;

[0111] Segmenting the image elements of the subject from the image data to be displayed to obtain the subject layer;

[0112] The background elements are segmented from the image data to be displayed to obtain the background layer.

[0113] In some examples, the obtaining portion 1701 is further configured to:

[0114] When at least one layer among the atmosphere layer, the foreground layer and the main body layer cannot be divided, the layer that cannot be divided is displayed through the background layer.

[0115] In some examples, the determining portion 1702 is configured to:

[0116] When the number N of projection surfaces formed by the display unit is consistent with the number M of layers, the corresponding display projection surfaces will be determined in the order of the subtitle layer, the atmosphere layer, the foreground layer, the main body layer and the background layer based on the projection surfaces arranged from near to far from the driving seat.

[0117] In some examples, the determining portion 1702 is configured to:

[0118] When the number of projection surfaces N is less than the number of layers M, each of the first N-1 projection surfaces arranged in order of distance from the driving seat from near to far will be determined as the display projection surface of N-1 layers divided from near to far according to the depth of field of the presented content, and the Nth projection surface will be determined as the display projection surface of the remaining M-N+1 layers.

[0119] In some examples, the determining portion 1702 is configured to:

[0120] When the number of projection surfaces N is greater than the number of layers M, each of the first M projection surfaces arranged in order of distance from the driving seat among all the projection surfaces is determined as a display projection surface of M layers divided from near to far according to the depth of field of the presented content.

[0121] In some examples, the determining portion 1702 is further configured to:

[0122] When the number of projection surfaces N is greater than the number of layers M, and the vehicle speed exceeds the set speed threshold, all projection surfaces, starting from the Kth projection surface after the first projection surface to each of the K+M-1th projection surfaces, are determined as display projection surfaces of M layers divided from near to far according to the depth of field of the presented content.

[0123] In some examples, the determining portion 1702 is further configured to:

[0124] Determine the focus layer of the sight focus from all layers according to the sight direction of the eye box position;

[0125] The control section 1703 is configured to display the layers other than the focused layer among all the layers with image quality lower than that of the focused layer.

[0126] Referring to Figure 18 , a block diagram of a display control device 1700 according to an exemplary embodiment of the present disclosure is shown. In some examples, the display control device 1700 has communication capabilities and can access a wired or wireless network. In some examples, the display control device 1700 can receive data based on the accessed wired or wireless network. It is understood that the display control device 1700 is responsible for the computation and processing of the technical solution of the present disclosure, and this disclosure is not limited thereto.

[0127] As shown in FIG. 18 , the display control device 1700 in the present disclosure may include one or more of the following components: a processor 1810 and a memory 1820 .

[0128] Optionally, processor 1810 utilizes various interfaces and circuits to connect various components within the computing device. It executes instructions, programs, code sets, or instruction sets stored in memory 1820, as well as accesses data stored in memory 1820, to perform various functions of the computing device and process data. Optionally, processor 1810 can be implemented in at least one hardware form: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). Processor 1810 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), and a baseband chip. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the touchscreen display; the NPU implements artificial intelligence (AI) functions; and the baseband chip handles wireless communications. It is understandable that the above-mentioned baseband chip may not be integrated into the processor 1810, but may be implemented by a separate chip.

[0129] Memory 1820 may include random access memory (RAM) or read-only memory (ROM). Optionally, memory 1820 includes non-transitory computer-readable storage medium. Memory 1820 may be used to store instructions, programs, code, code sets, or instruction sets. Memory 1820 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), and instructions for implementing each of the above method embodiments. The data storage area may store data generated based on the use of the computing device.

[0130] The present disclosure also provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the display control method described in the above embodiments.

[0131] The present disclosure also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium; a processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device executes to implement the display control method described in each of the above embodiments.

[0132] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in this disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0133] It should be noted that the technical solutions described in this disclosure can be combined arbitrarily without conflict.

[0134] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims. Industrial Applicability

[0135] In this embodiment, the display control device divides the image data to be displayed into different layers according to the depth of field from near to far, and allocates a corresponding display projection surface to each layer according to the distance from the driver's viewpoint from near to far, thereby increasing the vividness and immersion of the driver's visual effects when viewing the image data and improving the user experience.

Claims

1. A display control method, characterized in that: The method comprises: Get the layer of image data to be displayed; Determining a display projection surface for displaying the layer from the projection surfaces formed by the display unit; Control the layer to be displayed on the corresponding display projection surface.

2. The method according to claim 1, characterized in that The step of obtaining the layer of image data to be displayed includes: According to the depth of field of the content presented by the to-be-displayed image data, the layers are divided from near to far to obtain layers including at least one of a subtitle layer, an atmosphere layer, a foreground layer, a main body layer and a background layer.

3. The method according to claim 2, characterized in that The depth of field of the content presented by the image data to be displayed is divided from near to far to obtain a layer including at least one of a subtitle layer, an atmosphere layer, a foreground layer, a main body layer and a background layer, including: Obtaining subtitle text from a subtitle file of the video data, or obtaining subtitle text in the image data to be displayed by using a text recognition technology, and forming the subtitle layer with the subtitle text; wherein the image data to be displayed is each image frame of the video data; After identifying the sound related to the natural environment in the video data, matching and obtaining the atmosphere layer from the image data to be displayed according to the sound related to the natural environment; Segmenting the image elements with a closer depth of field from the image data to be displayed to obtain the foreground layer; Segmenting the image elements of the subject from the image data to be displayed to obtain the subject layer; The background elements are segmented from the image data to be displayed to obtain the background layer.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: When at least one of the atmosphere layer, the foreground layer and the main body layer cannot be divided, the layer that cannot be divided is displayed through the background layer.

5. The method according to claim 2, characterized in that: The step of determining a display projection surface for displaying the layer from the projection surface formed by the display unit comprises: When the number N of projection surfaces formed by the display unit is consistent with the number M of layers, the corresponding display projection surfaces will be determined in the order of the subtitle layer, the atmosphere layer, the foreground layer, the main layer and the background layer based on the projection surfaces arranged from near to far from the driving seat.

6. The method according to claim 2, characterized in that The step of determining a display projection surface for displaying the layer from the projection surface formed by the display unit comprises: When the number of projection surfaces N is less than the number of layers M, each of the first N-1 projection surfaces arranged in order of distance from the driving seat from near to far will be determined as a display projection surface of N-1 layers divided from near to far according to the depth of field of the presented content, and the Nth projection surface will be determined as the display projection surface of the remaining M-N+1 layers.

7. The method according to claim 2, characterized in that The step of determining a display projection surface for displaying the layer from the projection surface formed by the display unit comprises: When the number of projection surfaces N is greater than the number of layers M, each of the first M projection surfaces arranged in order of distance from the driving seat from near to far among all the projection surfaces is determined as a display projection surface of M layers divided from near to far according to the depth of field of the presented content.

8. The method according to claim 2 or 7, characterized in that: When the number of projection surfaces N is greater than the number of layers M, the method further includes: When the vehicle speed exceeds the set speed threshold, all projection surfaces, starting from the Kth projection surface after the first projection surface to each of the K+M-1th projection surfaces, are determined as display projection surfaces of M layers divided from near to far according to the depth of field of the presented content.

9. The method according to claim 1, characterized in that: The method further comprises: Determine a focus layer for line of sight focus from all layers according to the line of sight direction of the eye box position; The other layers, among all the layers, excluding the focused layer, are displayed according to an image quality lower than an image quality of the focused layer.

10. A display control device, characterized in that: The device comprises: an acquisition part, a determination part and a control part; wherein, The acquisition part is configured to acquire a layer of image data to be displayed; The determining section is configured to determine a display projection surface for displaying the layer from the projection surface formed by the display section; The control part is configured to control the layer to be displayed on the corresponding display projection surface.

11. A display control device, characterized in that: The device comprises: a processor and a memory; the processor is used to execute instructions stored in the memory to implement the display control method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the display control method according to any one of claims 1 to 9.

13. A head-up display device, characterized in that: The head-up display device includes a display control unit and a display unit; wherein, The display control unit is configured to obtain a layer of image data to be displayed; Determining a display projection surface for displaying the layer from the projection surfaces formed by the display unit; Controlling the layer to be displayed on the corresponding display projection surface; The display unit is configured to display the layer on the corresponding display projection surface based on the control of the display control unit.

14. The head-up display device according to claim 13, characterized in that: The apparatus further includes a gaze tracking unit configured to determine a gaze direction of the eye box position; The display control unit is further configured to determine a focus layer on which the line of sight is focused from all the layers according to the line of sight direction of the eye box position; And, displaying other layers among all layers except the focused layer according to an image quality lower than the image quality of the focused layer.

15. A vehicle, characterized in that: The vehicle includes the head-up display device according to claim 13 or 14.

Images