Display method and device and computer readable storage medium
By generating and superimposing layers of unobstructed and obscured indicator parts in the augmented reality head-up display technology, the problem that the front and rear relationship cannot be reflected when the obstacle blocks the indicators is solved, and the driver's indicators are realized Accurate perception of position and distance improves the driving experience.
Patent Information
- Application Number
- CN202411131807.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-26
AI Technical Summary
In the existing augmented reality head-up display technology, when an obstacle blocks the indicator, the front and back relationship between the obstacle and the indicator cannot be accurately reflected, resulting in the driver being unable to accurately sense the orientation and distance of the indicator.
By obtaining the indicator data and obstacle data of the vehicle while driving, creating corresponding primitives and performing in-depth test rendering, a first layer and a second layer are generated, and the first layer displays the indicator part that is not blocked by the obstacle. , the second layer displays the portion blocked by the obstacle, and superimposes the second layer on the first layer to form a target layer to display the front and back relationship between the obstacle and the indicator.
It accurately reflects the front and rear relationship between obstacles and indicators, ensuring that the driver can accurately perceive the orientation and distance of indicators, thereby providing a better visual experience.
Smart Images

Figure CN119027622A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of augmented reality head-up display technology, and in particular to a display method, device and computer-readable storage medium. Background Art
[0002] Augmented Reality Head-Up Display (AR-HUD) is a display technology that combines virtual information with real-world scenes to provide drivers with a more intuitive, safe and convenient driving experience. The most common application of AR-HUD technology in vehicles is fixed AR-HUD, which uses display screens, transparent glass, etc. as display carriers.
[0003] When displaying augmented reality (AR) indicators, there are often obstacles blocking the indicators in front, such as a navigation arrow blocked by an obstacle, a lane line blocked by an obstacle, etc. In the current technology, the indicator is directly drawn on the obstacle blocking it, that is, the indicator is displayed on the obstacle in front of it.
[0004] It can be seen from this that the display screen in the current technology cannot accurately reflect the front-to-back relationship between the obstacle and the indicator, resulting in the driver being unable to accurately perceive the direction and distance of the indicator. Summary of the invention
[0005] Based on the above problems, the present application provides a display method, device and computer-readable storage medium, which can accurately reflect the front and back relationship between obstacles and indicators, so that the driver can accurately perceive the direction and distance of the indicator with reference to the obstacles, thereby providing the driver with a better visual experience.
[0006] The embodiments of the present application disclose the following technical solutions:
[0007] In a first aspect, the present application provides a display method, comprising:
[0008] Acquire indicator data and obstacle data during vehicle driving;
[0009] Based on the indicator data and the obstacle data, creating a corresponding first indicator graphic primitive and a first obstacle graphic primitive;
[0010] Rendering all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth values corresponding to the first obstacle primitive, to obtain a first layer;
[0011] Based on the indicator data and the obstacle data, create a corresponding second indicator graphic element and a second obstacle graphic element;
[0012] Rendering pixels in the second indicator primitive whose pixel depth values are greater than the pixel depth values corresponding to the second obstacle primitive to obtain a second layer; the second layer carries a portion of the first indicator primitive that is blocked by the first obstacle primitive;
[0013] The second layer is superimposed on the first layer to obtain a target layer, and the target layer is displayed.
[0014] In a possible implementation, rendering all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth values corresponding to the first obstacle primitive to obtain a first layer includes:
[0015] generating a corresponding first depth buffer based on the pixel depth value of the first indicator primitive and the pixel depth value of the first obstacle primitive;
[0016] Based on the first depth buffer, a depth test is performed, and pixels in the first indicator primitive and the first obstacle primitive that meet a first condition are rendered to obtain a first layer; the first condition is that the pixel depth value is less than or equal to the corresponding depth value in the first depth buffer; the pixel depth values of all pixels in the first obstacle primitive meet the first condition.
[0017] In a possible implementation manner, before generating the corresponding first depth buffer based on the pixel depth value of the first indicator primitive and the pixel depth value of the first obstacle primitive, the method further includes:
[0018] A first setting operation is performed on the first indicator primitive and the first obstacle primitive; the first setting operation includes: setting a depth write state to an on state, setting a depth test state to an on state, and the type of the depth test is a first type; the first type of depth test is that when the depth value of a pixel is less than or equal to the corresponding depth value in the depth buffer, the pixel passes the depth test.
[0019] In a possible implementation, rendering pixels in the second indicator primitive whose pixel depth values are greater than pixel depth values corresponding to the second obstacle primitive to obtain a second layer includes:
[0020] generating a second depth buffer based on the pixel depth value of the second obstacle primitive;
[0021] Based on the second depth buffer, a depth test is performed to render pixels in the second indicator primitive that meet a second condition to obtain a second layer; the second condition is that the pixel depth value is greater than the corresponding depth value in the second depth buffer.
[0022] In a possible implementation manner, before generating the second depth buffer based on the pixel depth value of the second obstacle primitive, the method further includes:
[0023] Performing a first setting operation on the second obstacle primitive; the first setting operation includes: setting a depth write state to an on state, setting a depth test state to an on state, and setting the type of the depth test to a first type; the first type of depth test is that when the depth value of a pixel is less than or equal to the corresponding depth value in the depth buffer, the pixel passes the depth test;
[0024] A second setting operation is performed on the second indicator primitive; the second setting operation includes: setting the depth write state to an off state, setting the depth test state to an on state, and the type of the depth test is a second type; the second type of depth test is that when the depth value of the pixel is greater than the corresponding depth value in the depth buffer, the pixel passes the depth test.
[0025] In a possible implementation manner, after creating the corresponding first indicator graphic primitive and first obstacle graphic primitive based on the indicator data and the obstacle data, the method further includes:
[0026] Based on the obstacle data, a corresponding third obstacle primitive is created; the size of the third obstacle primitive is the same as the size of the first obstacle primitive, and the position of the third obstacle primitive is farther than the position of the first obstacle primitive by a preset length;
[0027] The rendering of all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive to obtain a first layer includes:
[0028] Render all pixels in the first obstacle primitive, all pixels in the third obstacle primitive, and pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth values corresponding to the first obstacle primitive and the third obstacle primitive to obtain a first layer.
[0029] In a possible implementation, rendering all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth values corresponding to the first obstacle primitive to obtain a first layer includes:
[0030] Obtaining a first layer by rendering all pixels in the first obstacle primitive with a first color, and rendering pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth values corresponding to the first obstacle primitive with a second color;
[0031] The rendering of the pixels in the second indicator primitive whose pixel depth value is greater than the pixel depth value corresponding to the second obstacle primitive to obtain the second layer includes:
[0032] The second layer is obtained by rendering pixels in the second indicator primitive whose pixel depth value is greater than the pixel depth value corresponding to the second obstacle primitive by a third color; the third color includes: a color different from the first color and the second color;
[0033] And / or, displaying the target layer includes:
[0034] The target layer is displayed through augmented reality head-up display technology.
[0035] In a possible implementation, the step of acquiring indicator data and obstacle data during vehicle driving includes:
[0036] Through the advanced driver assistance system and navigation system, the indicator data and obstacle data of the vehicle are obtained during driving.
[0037] In a second aspect, the present application provides a display device, including:
[0038] A data acquisition module, used to acquire indicator data and obstacle data during vehicle driving;
[0039] A first primitive creation module, used to create corresponding first indicator primitives and first obstacle primitives based on the indicator data and the obstacle data;
[0040] A first layer generation module, configured to render all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive, to obtain a first layer;
[0041] A second primitive creation module, used to create corresponding second indicator primitives and second obstacle primitives based on the indicator data and the obstacle data;
[0042] A second layer generation module is used to render pixels in the second indicator primitive whose pixel depth values are greater than the pixel depth values corresponding to the second obstacle primitive to obtain a second layer; the second layer carries a portion of the first indicator primitive that is blocked by the first obstacle primitive;
[0043] The target layer display module is used to superimpose the second layer on the first layer to obtain the target layer, and display the target layer.
[0044] In one possible implementation, the first layer generation module is specifically used to generate a corresponding first depth buffer based on the pixel depth value of the first indicator primitive and the pixel depth value of the first obstacle primitive; based on the first depth buffer, perform a depth test, render the pixels in the first indicator primitive and the first obstacle primitive that meet a first condition, and obtain a first layer; the first condition is that the pixel depth value is less than or equal to the corresponding depth value in the first depth cache.
[0045] In one possible implementation, the display device also includes: a setting operation module, used to perform a first setting operation on the first indicator primitive and the first obstacle primitive; the first setting operation includes: setting the depth write state to an on state, setting the depth test state to an on state, and the type of the depth test is a first type; the first type of depth test is that when the depth value of the pixel is less than or equal to the corresponding depth value in the depth cache, the pixel passes the depth test.
[0046] In one possible implementation, the second layer generation module is specifically used to generate a second depth cache based on the pixel depth value of the second obstacle primitive; based on the second depth cache, perform a depth test, render pixels in the second indicator primitive that meet a second condition, and obtain a second layer; the second condition is that the pixel depth value is greater than the corresponding depth value in the second depth cache.
[0047] In one possible implementation, the setting operation module is also used to perform a first setting operation on the second obstacle primitive; and perform a second setting operation on the second indicator primitive; the second setting operation includes: setting the depth write state to an off state, setting the depth test state to an on state, and the type of the depth test is a second type; the second type of depth test is that when the depth value of the pixel is greater than the corresponding depth value in the depth cache, the pixel passes the depth test.
[0048] In a possible implementation, the display device also includes: a third primitive creation module, used to create a corresponding third obstacle primitive based on the obstacle data; the size of the third obstacle primitive is the same as the size of the first obstacle primitive, and the position of the third obstacle primitive is farther than the position of the first obstacle primitive by a preset length; a first layer generation module, specifically used to render all pixels in the first obstacle primitive, all pixels in the third obstacle primitive, and pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive and the third obstacle primitive, to obtain a first layer.
[0049] In one possible implementation, the first layer generation module is specifically used to render all pixels in the first obstacle primitive with a first color, and to render pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive with a second color, to obtain a first layer; and the second layer generation module is specifically used to render pixels in the second indicator primitive whose pixel depth value is greater than the pixel depth value corresponding to the second obstacle primitive with a third color, to obtain a second layer; the target layer display module is specifically used to display the target layer through augmented reality head-up display technology.
[0050] In a possible implementation, the data acquisition module is specifically used to obtain indicator data and obstacle data during vehicle driving through an advanced driver assistance system and a navigation system.
[0051] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program or instructions, and when the computer program or instructions are executed, the method described in the first aspect is implemented.
[0052] Compared with the prior art, the present application has the following beneficial effects: through the first layer part, the front and back relationship between the obstacle and the indicator can be accurately reflected, and through the second layer part, the part of the indicator blocked by the obstacle can be correctly displayed, that is, the embodiment of the present application can correctly display the indicator, avoid the situation where the obstacle blocks the indicator, and can correctly reflect the front and back relationship between the obstacle and the indicator, so that the driver can accurately perceive the direction and distance of the indicator with reference to the obstacle, thereby providing the driver with a better visual experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0054] Figure 1 A flow chart of a display method provided in an embodiment of the present application;
[0055] Figure 2 An example diagram of a first layer provided in an embodiment of the present application;
[0056] Figure 3 An example diagram of a second layer provided in an embodiment of the present application;
[0057] Figure 4An example diagram of a target layer provided in an embodiment of the present application;
[0058] Figure 5 An example diagram of another target layer provided in an embodiment of the present application;
[0059] Figure 6 A schematic diagram of the structure of a display device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0060] As described above, when displaying an indicator, there is often a situation where it is blocked by an obstacle in front. Generally, the indicator is directly displayed on the obstacle in front of it to avoid the obstacle blocking the indicator. However, directly displaying the indicator on the obstacle in front of it that blocks it cannot reflect the sense of front and back depth of field. For example, assuming that indicator A is located behind obstacle B, and obstacle B blocks all or part of indicator A, in the current technology, indicator A is directly displayed in front of obstacle B. The final visual representation displayed to the driver is that indicator A is located in front of obstacle B, but the actual situation is that indicator A is located behind obstacle B. Therefore, the current technology cannot reflect the front and back relationship between the obstacle and the indicator, resulting in the driver being unable to accurately perceive the direction and distance of the indicator.
[0061] The present application provides a display method, including: obtaining indicator data and obstacle data during vehicle driving; creating corresponding first indicator primitives and first obstacle primitives based on the indicator data and the obstacle data; rendering all pixels in the first obstacle primitive and pixels whose pixel depth values in the first indicator primitive are less than or equal to the pixel depth values corresponding to the first obstacle primitive to obtain a first layer; creating corresponding second indicator primitives and second obstacle primitives based on the indicator data and the obstacle data; rendering pixels whose pixel depth values in the second indicator primitive are greater than the pixel depth values corresponding to the second obstacle primitive to obtain a second layer; superimposing the second layer on the first layer to obtain a target layer, and displaying the target layer. The obtained second layer carries the part of the first indicator primitive that is blocked by the first obstacle primitive. The second layer is superimposed on the first layer to obtain the target layer and display it. Through the first layer part, the front and back relationship between the obstacle and the indicator can be accurately reflected, and through the second layer part, the part of the indicator blocked by the obstacle can be correctly displayed. That is, the present application can correctly display the indicator, avoid the situation where the obstacle blocks the indicator, and can correctly reflect the front and back relationship between the obstacle and the indicator, so that the driver can accurately perceive the direction and distance of the indicator with reference to the obstacle, thereby providing the driver with a better visual experience.
[0062] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0063] Embodiment 1:
[0064] Combine the following Figure 1-Figure 4 , a display method provided by an embodiment of the present application is introduced in detail.
[0065] like Figure 1 As shown, a display method provided in an embodiment of the present application includes the following steps:
[0066] S101, obtaining indicator data and obstacle data during vehicle driving.
[0067] The indicator data refers to data related to indicators related to driving rules, route guidance, etc. For example, the indicators include lane lines, navigation arrows, etc.; the indicator data includes: real-time position data, shape data, direction data, etc.
[0068] Obstacle data refers to the data of physical objects around the vehicle that may affect driving safety. For example, obstacles include other vehicles, pedestrians, cyclists, and static obstacles on the road (such as roadblocks, stones, etc.); obstacle data includes location data, size data, shape data, dynamic attribute data (such as speed, acceleration, direction, etc.), etc.
[0069] In a possible implementation, indicator data and obstacle data during vehicle driving are obtained through an advanced driver assistance system and a navigation system.
[0070] Among them, Advanced Driver Assistance Systems (ADAS) are a series of advanced technologies and functions integrated into vehicles to improve driving safety, comfort and efficiency. Advanced Driver Assistance Systems use various sensors (such as cameras, radars, lidars, ultrasonic sensors and global positioning systems, etc.) and complex algorithms to monitor the vehicle's external environment and the vehicle's own operating status in real time, thereby providing the driver with necessary information, warnings and even direct control systems.
[0071] Among them, the navigation system is a technical system that uses satellite signals, electronic map data, sensors and other technical means to determine the location, plan the route, and guide the user from the starting point to the destination.
[0072] Exemplarily, the indicators include navigation arrows and lane lines. For lane lines, the lane line data of the vehicle is obtained through the advanced driver assistance system; for navigation arrows, the navigation arrow data of the vehicle is obtained through the navigation system. For obstacles, the obstacle data of the vehicle is obtained through the advanced driver assistance system.
[0073] S102: Create corresponding first indicator graphic primitives and first obstacle graphic primitives based on the indicator data and the obstacle data.
[0074] Among them, primitive is a basic concept in computer graphics, which refers to the basic elements or building blocks that constitute graphics.
[0075] Specifically, based on the indicator data, a corresponding first indicator primitive is created; and based on the obstacle data, a corresponding first obstacle primitive is created. Exemplarily, the indicator data includes: lane line data and pilot arrow data, and the corresponding first indicator primitive includes: lane line primitive and pilot arrow model primitive.
[0076] S103: Render all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth value corresponding to the first obstacle primitive to obtain a first layer.
[0077] In the process of 3D computer graphics and rendering, the pixel depth value can also be called the "Z value", which is used to indicate the front and back positions of each pixel in the scene. The smaller the pixel depth value, the closer the pixel is to the front; the larger the pixel depth value, the closer the pixel is to the back.
[0078] Specifically, the first layer includes all pixels in the first obstacle primitive and pixels of a portion of the first indicator primitive that is not blocked by the first obstacle primitive.
[0079] In a possible implementation, a corresponding first depth buffer is generated based on the pixel depth value of the first indicator primitive and the pixel depth value of the first obstacle primitive; a depth test is performed based on the first depth buffer, and pixels in the first indicator primitive and the first obstacle primitive that meet a first condition are rendered to obtain a first layer. The first condition is that the pixel depth value is less than or equal to the corresponding depth value in the first depth buffer, and the pixel depth values of all pixels in the first obstacle primitive meet the first condition.
[0080] The first depth buffer is a two-dimensional array, which corresponds one-to-one to the pixels of the real screen and is used to store the depth value of the primitive surface closest to the user on each pixel.
[0081] Specifically, the first depth buffer is obtained in the following manner: an initial depth buffer is set, and the initial depth buffer is a two-dimensional array for storing depth values pre-set according to pixels of the screen. Based on the pixel depth value of the first indicator primitive and the pixel depth value of the first obstacle primitive, the pixel depth value of the primitive and the corresponding depth value stored in the depth buffer are compared in sequence, and when the pixel depth value of the primitive is less than or equal to the corresponding depth value, the corresponding depth value in the depth buffer is updated to the pixel depth value of the primitive, thereby obtaining the first depth buffer.
[0082] For ease of understanding, the following takes the case where the first obstacle primitive includes obstacle primitive A and obstacle primitive B, and the first indicator primitive includes indicator primitive C as an example to describe how to obtain the first depth buffer.
[0083] An initial depth buffer is set in advance according to a real screen (for a vehicle, it is a front windshield); based on the comparison of the pixel depth value of the obstacle primitive A with the depth value of the corresponding pixel in the initial depth buffer, when the pixel depth value of the obstacle primitive A is less than or equal to the depth value of the corresponding pixel, the depth value stored in the initial depth buffer is updated to the pixel depth value of the obstacle primitive A, and a depth buffer a is obtained; based on the comparison of the pixel depth value of the obstacle primitive B with the depth value of the corresponding pixel in the depth buffer a, when the pixel depth value of the obstacle primitive B is less than or equal to the depth value of the corresponding pixel, the depth value stored in the depth buffer a is updated to the pixel depth value of the obstacle primitive B, and a depth buffer b is obtained; based on the comparison of the pixel depth value of the indicator primitive C with the depth value of the corresponding pixel in the depth buffer b, when the pixel depth value of the indicator primitive C is less than or equal to the depth value of the corresponding pixel, the depth value stored in the depth buffer b is updated to the pixel depth value of the indicator primitive C, and a corresponding first depth buffer is generated.
[0084] The first depth buffer generated in the above manner represents the depth value of the primitive surface closest to the user on each pixel. When the depth value of a pixel in the first indicator primitive is less than or equal to the depth value of the pixel in the first depth buffer, it represents that the pixel is located at the top layer, that is, the pixel is not blocked by the first obstacle primitive and is the pixel closest to the user.
[0085] In one possible implementation, before generating the corresponding first depth buffer, a first setting operation is performed on the first indicator primitive and the first obstacle primitive; the first setting operation includes: setting the depth write state to an on state, setting the depth test state to an on state, and the type of the depth test is a first type, and the first type of depth test is that when the depth value of the pixel is less than or equal to the corresponding depth value in the depth buffer, the pixel passes the depth test.
[0086] Among them, depth writing and depth testing are two important concepts in computer graphics used to deal with the occlusion relationship of objects when rendering three-dimensional scenes. They are used to ensure the correct drawing order of pixels and avoid unnecessary overlapping rendering, thereby improving rendering efficiency and realism.
[0087] Specifically, depth write determines whether to write the depth value of the pixel to the depth buffer, and its status can be configured to be on or off. When turned on, if the element passes the depth test, the depth value of the element will be written to the depth buffer, updating the depth information of the element in the depth buffer.
[0088] Further, in a possible implementation, based on the obstacle data, a corresponding third obstacle primitive is created, the size of the third obstacle primitive is the same as the size of the first obstacle primitive, and the position of the third obstacle primitive is farther than the position of the first obstacle primitive by a preset length; all elements in the first obstacle primitive, all pixels in the third obstacle primitive, and pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth values corresponding to the first obstacle primitive and the third obstacle primitive are rendered to obtain the first primitive.
[0089] The third obstacle primitive is located at a preset distance behind the first obstacle primitive to prevent depth comparison errors, and the third obstacle primitive can turn off blending, which can improve the overall visual experience of the driver.
[0090] In a possible implementation, the first layer is obtained by rendering all pixels in the first obstacle primitive with a first color, and rendering pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive with a second color. That is, the obstacle and the indicator portion not blocked by the obstacle are rendered with different colors, so that the driver can effectively distinguish between the obstacle and the indicator, thereby improving the driver's visual experience.
[0091] It should be noted that the first color may include one or more colors, the second color may include one or more colors, and the first color and the second color may include the same color, which is not specifically limited in this application.
[0092] For ease of understanding, the following Figure 2 , an example is given to introduce the actual process of obtaining the first layer in S102 and S103.
[0093] A blank first layer is created in advance, and an AR camera is created and bound to the blank first layer; the AR camera is a camera application with an augmented reality function, which can superimpose digital information and virtual elements onto images of the real world in real time.
[0094] Create a lane line primitive (the lane line primitive is a first indicator primitive), set the depth writing state to be on, set the depth test state to be on, and the type of the depth test is the first type.
[0095] Create a pilot arrow model primitive (a pilot arrow model primitive is a first indicator primitive), set the depth writing state to be on, set the depth test state to be on, and the type of the depth test is the first type.
[0096] Create the first pedestrian primitive that is closer and the first cycling primitive that is farther (the first pedestrian primitive and the first cycling primitive are the first obstacle primitives), set the depth write state to the on state, set the depth test state to the on state, and the type of the depth test to the first type. Furthermore, the color mixing function is enabled for the first pedestrian primitive and the first cycling primitive. Enabling the color mixing function of the primitive allows the pixel to be mixed with the existing pixel color in the frame buffer according to a certain formula when rendering / drawing the pixel, thereby achieving visual effects such as transparency and translucency.
[0097] Create a closer pedestrian barrier primitive and a farther cycling barrier primitive (the pedestrian barrier primitive and the cycling barrier primitive are the third obstacle primitives), the size of the pedestrian barrier primitive is the same as the size of the first pedestrian primitive, the size of the cycling barrier primitive is the same as the size of the first cycling primitive; and the pedestrian barrier primitive is located 1 cm behind the first pedestrian primitive, and the cycling barrier primitive is located 1 cm behind the first cycling primitive. Set the depth writing state to the on state, set the depth test state to the on state, and the type of the depth test is the first type. Furthermore, the pedestrian barrier primitive and the cycling barrier primitive turn off the color mixing function.
[0098] Render the above primitives to the blank first layer in sequence, that is, add the lane line primitive, the cycling barrier primitive, the cycling primitive, the navigation arrow model primitive, the pedestrian barrier primitive and the pedestrian primitive to the blank first layer in sequence to generate the corresponding first layer. That is, the rendered first layer is as follows Figure 2 shown.
[0099] S104: Create corresponding second indicator graphic primitives and second obstacle graphic primitives based on the indicator data and the obstacle data.
[0100] Specifically, based on the indicator data, a corresponding second indicator graphic element is created; based on the obstacle data, a corresponding second obstacle graphic element is created.
[0101] The pixel depth value of each pixel in the second indicator primitive is the same as the pixel depth value of the corresponding pixel in the first indicator primitive; the pixel depth value of each pixel in the second obstacle primitive is the same as the pixel depth value of the corresponding pixel in the first obstacle primitive.
[0102] S105 : Render pixels in the second indicator primitive whose pixel depth values are greater than the pixel depth values corresponding to the second obstacle primitive to obtain a second layer.
[0103] Among them, the second layer carries the part of the first indicator primitive blocked by the first obstacle primitive. That is, the pixel in the second indicator primitive whose pixel depth value is greater than the pixel depth value corresponding to the second obstacle primitive is the part of the second indicator primitive blocked by the second obstacle primitive; and the second indicator primitive and the first indicator primitive are obtained based on the same indicator data, and the second obstacle primitive and the first obstacle primitive are obtained based on the same obstacle data, so the part of the second indicator primitive blocked by the second obstacle primitive is also the part of the first indicator primitive blocked by the first obstacle primitive.
[0104] In one possible implementation, a second depth buffer is generated based on the pixel depth value of the second obstacle primitive; based on the second depth buffer, a depth test is performed, and pixels in the second indicator primitive that meet a second condition are rendered to obtain the second primitive; the second condition is that the pixel depth value is greater than the corresponding depth value in the second depth buffer.
[0105] Specifically, the second depth buffer is obtained in the following manner: setting an initial depth buffer; based on the pixel depth value of the second obstacle primitive, comparing the pixel depth value of the second obstacle primitive and the corresponding depth value stored in the depth buffer in sequence, when the pixel depth value of the second obstacle primitive is less than or equal to the corresponding depth value, updating the corresponding depth value in the depth buffer to the pixel depth value of the second obstacle primitive, thereby obtaining the second depth buffer.
[0106] The second depth buffer obtained in the above manner represents the depth value of the primitive surface closest to the user when there is only the second obstacle primitive, that is, the depth value when the second obstacle primitive is located at the top layer. When a pixel in the second indicator primitive is greater than the depth value of the pixel in the second depth buffer, it represents that the pixel is located in the lower layer of the second obstacle primitive, that is, the pixel is blocked by the second obstacle primitive, and the pixel blocked by the second obstacle primitive in the second indicator primitive is drawn. Therefore, through the second depth buffer generated based on the pixel depth value of the second obstacle primitive, all or part of the indicator blocked by the obstacle in the first layer is drawn to obtain the second layer, thereby displaying all or part of the indicator blocked by the obstacle.
[0107] In one possible implementation, before generating the corresponding second depth buffer, a first setting operation is performed on the second obstacle primitive; and a second setting operation is performed on the second indicator primitive, the second setting operation including: setting the depth write state to an off state, setting the depth test state to an on state, and the type of the depth test to a second type; the second type of depth test is that when the depth value of the pixel is greater than the corresponding depth value in the depth buffer, the pixel passes the depth test.
[0108] In a possible implementation, based on the pixel depth value of the second obstacle primitive, the second indicator primitive is rendered using a third color, wherein the third color includes a color different from the first color and the second color, so that the unobstructed portion of the indicator and the obstructed portion of the indicator can be effectively distinguished.
[0109] It should be noted that the third color may include one color different from the first color and the second color, or multiple colors different from the first color and the second color, and this application does not make specific limitations.
[0110] For ease of understanding, the following Figure 3 , an example is given to introduce the actual process of obtaining the second layer in S104 and S105. Figure 3 The indicator data and obstacle data corresponding to the second layer shown are Figure 2 The indicator data and obstacle data corresponding to the first layer shown are the same.
[0111] A blank second layer is created in advance, and a corresponding AR camera is set. In addition, the created blank second layer can also be called a blank primitive.
[0112] Create a highlighted lane line primitive (a highlighted lane line primitive is a second indicator primitive), set the depth write state to off, set the depth test state to on, and the depth test type to the second type. Further, the color of the highlighted lane line primitive is {0,128,255,255}, and is set to the edge line appearance. Among them, the color {0,128,255,255} is a typical RGBA color value, indicating a color with no red component, medium green component, full blue component and full opacity. This color combination will present a completely opaque blue-green color.
[0113] Create a highlight pilot arrow model primitive (a highlight pilot arrow model primitive is a second indicator primitive), set the depth write state to off, set the depth test state to on, and the depth test type to the second type. Further, set the highlight attribute of the highlight pilot arrow model primitive to on, and the color of the highlight pilot arrow model primitive is {0, 255, 255, 255}.
[0114] Create a closer second pedestrian primitive and a farther second cycling primitive (the second pedestrian primitive and the second cycling primitive are second obstacle primitives), set the depth write state to the on state, set the depth test state to the on state, and the depth test type to the first type. Further, the second pedestrian primitive and the second cycling primitive turn off the color mixing function and the color buffer writing function, which can be simply understood as not rendering the second pedestrian primitive and the second cycling primitive.
[0115] Render the above primitives to the blank second layer in sequence, that is, add the second cycling primitive, the second pedestrian primitive, the highlighted lane line primitive, and the highlighted navigation arrow model primitive to the blank second layer in sequence to generate the corresponding second layer. That is, the rendered second layer is as follows Figure 3 shown.
[0116] S106: Overlay the second layer on the first layer to obtain a target layer, and display the target layer.
[0117] In a possible implementation, the target layer is displayed through augmented reality head-up display technology.
[0118] For example, Figure 3 The second layer shown is superimposed on Figure 3 On the first layer shown, we get Figure 4 The target layer shown in Figure 4 The target layer is shown.
[0119] Further, if Figure 5 As shown, it is the target layer at another moment. For the navigation arrow 501, the solid display part is the part of the navigation arrow 501 that is not blocked by the obstacle, and the line outline part is the part of the navigation arrow 501 that is blocked by the obstacle. Similarly, for the lane line 502, the solid display part is the part of the lane line 502 that is not blocked by the obstacle, and the line outline part is the part of the lane line 502 that is blocked by the obstacle.
[0120] An embodiment of the present application provides a display method, including: obtaining indicator data and obstacle data during vehicle driving; creating a corresponding first indicator primitive and a first obstacle primitive based on the indicator data and the obstacle data; rendering all pixels in the first obstacle primitive and pixels whose pixel depth values in the first indicator primitive are less than or equal to the pixel depth values corresponding to the first obstacle primitive to obtain a first layer; creating a corresponding second indicator primitive and a second obstacle primitive based on the indicator data and the obstacle data; rendering pixels whose pixel depth values in the second indicator primitive are greater than the pixel depth values corresponding to the second obstacle primitive to obtain a second layer; superimposing the second layer on the first layer to obtain a target layer, and displaying the target layer. In the embodiment of the present application, the obtained second layer carries the portion of the first indicator primitive obscured by the first obstacle primitive, and the second layer is superimposed on the first layer to obtain the target layer and display it. Through the first layer portion, the front and back relationship between the obstacle and the indicator can be accurately reflected, and through the second layer portion, the portion of the indicator obscured by the obstacle can be correctly displayed, that is, the embodiment of the present application can correctly display the indicator, avoid the situation where the obstacle obscures the indicator, and can correctly reflect the front and back relationship between the obstacle and the indicator, so that the driver can accurately perceive the direction and distance of the indicator with reference to the obstacle, thereby providing the driver with a better visual experience.
[0121] Furthermore, by rendering the unobstructed part of the indicator and the obstructed part of the indicator in different colors, the user can effectively distinguish them, thereby improving the user's visual experience.
[0122] Embodiment 2:
[0123] Combine the following Figure 6 , a display device provided in an embodiment of the present application is introduced in detail.
[0124] like Figure 6 As shown, a display device provided in an embodiment of the present application includes the following modules:
[0125] The data acquisition module 601 is used to acquire indicator data and obstacle data during vehicle driving;
[0126] A first primitive creation module 602, configured to create corresponding first indicator primitives and first obstacle primitives based on the indicator data and the obstacle data;
[0127] The first layer generation module 603 is used to render all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive, to obtain a first layer;
[0128] A second primitive creation module 604, used to create corresponding second indicator primitives and second obstacle primitives based on the indicator data and the obstacle data;
[0129] The second layer generation module 605 is used to render pixels in the second indicator primitive whose pixel depth value is greater than the pixel depth value corresponding to the second obstacle primitive to obtain a second layer; the second layer carries the portion of the first indicator primitive blocked by the first obstacle primitive;
[0130] The target layer display module 606 is used to superimpose the second layer on the first layer to obtain the target layer, and display the target layer.
[0131] In one possible implementation, the first layer generation module 603 is specifically used to generate a corresponding first depth buffer based on the pixel depth value of the first indicator primitive and the pixel depth value of the first obstacle primitive; based on the first depth buffer, perform a depth test, render the pixels in the first indicator primitive and the first obstacle primitive that meet a first condition, and obtain a first layer; the first condition is that the pixel depth value is less than or equal to the corresponding depth value in the first depth cache.
[0132] In one possible implementation, the display device also includes: a setting operation module, used to perform a first setting operation on the first indicator primitive and the first obstacle primitive; the first setting operation includes: setting the depth write state to an on state, setting the depth test state to an on state, and the type of the depth test is a first type; the first type of depth test is that when the depth value of the pixel is less than or equal to the corresponding depth value in the depth cache, the pixel passes the depth test.
[0133] In one possible implementation, the second layer generation module 605 is specifically used to generate a second depth cache based on the pixel depth value of the second obstacle primitive; based on the second depth cache, perform a depth test, render pixels in the second indicator primitive that meet a second condition, and obtain a second layer; the second condition is that the pixel depth value is greater than the corresponding depth value in the second depth cache.
[0134] In one possible implementation, the setting operation module is also used to perform a first setting operation on the second obstacle primitive; and perform a second setting operation on the second indicator primitive; the second setting operation includes: setting the depth write state to an off state, setting the depth test state to an on state, and the type of the depth test is a second type; the second type of depth test is that when the depth value of the pixel is greater than the corresponding depth value in the depth cache, the pixel passes the depth test.
[0135] In one possible implementation, the display device also includes: a third primitive creation module, used to create a corresponding third obstacle primitive based on the obstacle data; the size of the third obstacle primitive is the same as the size of the first obstacle primitive, and the position of the third obstacle primitive is farther than the position of the first obstacle primitive by a preset length; a first layer generation module, specifically used to render all pixels in the first obstacle primitive, all pixels in the third obstacle primitive, and pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive and the third obstacle primitive, to obtain a first layer.
[0136] In one possible implementation, the first layer generation module 603 is specifically used to render all pixels in the first obstacle primitive with a first color, and to render pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive with a second color, to obtain a first layer; and the second layer generation module 605 is specifically used to render pixels in the second indicator primitive whose pixel depth value is greater than the pixel depth value corresponding to the second obstacle primitive with a third color, to obtain a second layer.
[0137] In a possible implementation, the target layer display module 606 is specifically configured to display the target layer through augmented reality head-up display technology.
[0138] In a possible implementation, the data acquisition module 601 is specifically configured to obtain indicator data and obstacle data during vehicle driving through an advanced driver assistance system and a navigation system.
[0139] A display device provided by an embodiment of the present application includes: a data acquisition module 601, which is used to acquire indicator data and obstacle data during vehicle driving; a first primitive creation module 602, which is used to create corresponding first indicator primitives and first obstacle primitives based on the indicator data and obstacle data; a first layer generation module 603, which is used to render all pixels in the first obstacle primitive and pixels whose pixel depth values in the first indicator primitive are less than or equal to the pixel depth values corresponding to the first obstacle primitive, to obtain a first layer; a second primitive creation module 604, which is used to create corresponding second indicator primitives and second obstacle primitives based on the indicator data and obstacle data; a second layer generation module 605, which is used to render pixels whose pixel depth values in the second indicator primitive are greater than the pixel depth values corresponding to the second obstacle primitive, to obtain a second layer; and a target layer display module 606, which is used to superimpose the second layer on the first layer, to obtain a target layer, and to display the target layer. The display device can realize correct display of the indicator, avoid the situation where the indicator is blocked by an obstacle, and can correctly reflect the front-to-back relationship between the obstacle and the indicator, so that the driver can accurately perceive the direction and distance of the indicator with reference to the obstacle, thereby providing the driver with a better visual experience.
[0140] In addition, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program or instruction is stored. When the above-mentioned computer program or instruction is executed, any one of the display methods described in Example 1 is implemented.
[0141] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the method and device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0142] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A display method, characterized in that: include: Acquire indicator data and obstacle data during vehicle driving; Based on the indicator data and the obstacle data, creating a corresponding first indicator graphic primitive and a first obstacle graphic primitive; Rendering all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth values corresponding to the first obstacle primitive, to obtain a first layer; Based on the indicator data and the obstacle data, create a corresponding second indicator graphic element and a second obstacle graphic element; Rendering pixels in the second indicator primitive whose pixel depth values are greater than the pixel depth values corresponding to the second obstacle primitive to obtain a second layer; the second layer carries a portion of the first indicator primitive that is blocked by the first obstacle primitive; The second layer is superimposed on the first layer to obtain a target layer, and the target layer is displayed.
2. The method according to claim 1, characterized in that The rendering of all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive to obtain a first layer includes: generating a corresponding first depth buffer based on the pixel depth value of the first indicator primitive and the pixel depth value of the first obstacle primitive; Based on the first depth buffer, a depth test is performed, and pixels in the first indicator primitive and the first obstacle primitive that meet a first condition are rendered to obtain a first layer; the first condition is that the pixel depth value is less than or equal to the corresponding depth value in the first depth buffer; the pixel depth values of all pixels in the first obstacle primitive meet the first condition.
3. The method according to claim 2, characterized in that Before generating a corresponding first depth buffer based on the pixel depth value of the first indicator primitive and the pixel depth value of the first obstacle primitive, the method further includes: A first setting operation is performed on the first indicator primitive and the first obstacle primitive; the first setting operation includes: setting a depth write state to an on state, setting a depth test state to an on state, and the type of the depth test is a first type; the first type of depth test is that when the depth value of a pixel is less than or equal to the corresponding depth value in the depth buffer, the pixel passes the depth test.
4. The method according to claim 1, characterized in that: The rendering of the pixels in the second indicator primitive whose pixel depth value is greater than the pixel depth value corresponding to the second obstacle primitive to obtain the second layer includes: generating a second depth buffer based on the pixel depth value of the second obstacle primitive; Based on the second depth buffer, a depth test is performed to render pixels in the second indicator primitive that meet a second condition to obtain a second layer; the second condition is that the pixel depth value is greater than the corresponding depth value in the second depth buffer.
5. The method according to claim 4, characterized in that Before generating a second depth buffer based on the pixel depth value of the second obstacle primitive, the method further includes: Performing a first setting operation on the second obstacle primitive; the first setting operation includes: setting a depth write state to an on state, setting a depth test state to an on state, and setting the type of the depth test to a first type; the first type of depth test is that when the depth value of a pixel is less than or equal to the corresponding depth value in the depth buffer, the pixel passes the depth test; A second setting operation is performed on the second indicator primitive; the second setting operation includes: setting the depth write state to an off state, setting the depth test state to an on state, and the type of the depth test is a second type; the second type of depth test is that when the depth value of the pixel is greater than the corresponding depth value in the depth buffer, the pixel passes the depth test.
6. The method according to claim 1, characterized in that After creating the corresponding first indicator graphic element and first obstacle graphic element based on the indicator data and the obstacle data, the method further includes: Based on the obstacle data, a corresponding third obstacle primitive is created; the size of the third obstacle primitive is the same as the size of the first obstacle primitive, and the position of the third obstacle primitive is farther than the position of the first obstacle primitive by a preset length; The rendering of all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive to obtain a first layer includes: Render all pixels in the first obstacle primitive, all pixels in the third obstacle primitive, and pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth values corresponding to the first obstacle primitive and the third obstacle primitive to obtain a first layer.
7. The method according to claim 1, characterized in that The rendering of all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive to obtain a first layer includes: Obtaining a first layer by rendering all pixels in the first obstacle primitive with a first color, and rendering pixels in the first indicator primitive whose pixel depth values are less than or equal to the pixel depth values corresponding to the first obstacle primitive with a second color; The rendering of the pixels in the second indicator primitive whose pixel depth value is greater than the pixel depth value corresponding to the second obstacle primitive to obtain the second layer includes: The second layer is obtained by rendering pixels in the second indicator primitive whose pixel depth value is greater than the pixel depth value corresponding to the second obstacle primitive by a third color; the third color includes: a color different from the first color and the second color; And / or, displaying the target layer includes: The target layer is displayed through augmented reality head-up display technology.
8. The method according to any one of claims 1 to 7, characterized in that: The step of obtaining indicator data and obstacle data during vehicle driving includes: Through the advanced driver assistance system and navigation system, the indicator data and obstacle data of the vehicle are obtained during driving.
9. A display device, characterized in that: include: A data acquisition module, used to acquire indicator data and obstacle data during vehicle driving; A first primitive creation module, used to create corresponding first indicator primitives and first obstacle primitives based on the indicator data and the obstacle data; A first layer generation module, configured to render all pixels in the first obstacle primitive and pixels in the first indicator primitive whose pixel depth value is less than or equal to the pixel depth value corresponding to the first obstacle primitive, to obtain a first layer; A second primitive creation module, used to create corresponding second indicator primitives and second obstacle primitives based on the indicator data and the obstacle data; A second layer generation module is used to render pixels in the second indicator primitive whose pixel depth values are greater than the pixel depth values corresponding to the second obstacle primitive to obtain a second layer; the second layer carries a portion of the first indicator primitive that is blocked by the first obstacle primitive; The target layer display module is used to superimpose the second layer on the first layer to obtain the target layer, and display the target layer.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or instruction, and when the computer program or instruction is executed, the method according to any one of claims 1 to 9 is implemented.
