Display processing method, display device, storage medium and vehicle

Abstract

The present application relates to the field of projection display technology, and in particular to a display processing method, display device, storage medium and transportation vehicle. The present application uses mapping materials to simulate the real three-dimensional effect of actual objects, and the different states of the actual objects are called by materials containing different elements to form corresponding display elements in the projection area, so as to truly reflect the real-time environmental changes of the actual objects and the three-dimensional presentation under the changes in the environment, and provide users with a real sense of feedback. The three-dimensional display effect of the present application does not rely on high-end hardware resources, and the operation is simple and efficient, which can save the hardware cost of implementation.

Description

Technical Field

[0001] The present application relates to the field of projection display technology, and in particular to a display processing method, a display device, a storage medium and a vehicle. Background Art

[0002] HUD (Head Up Display) is a new way to realize in-vehicle display by using reflection on the vehicle windshield. Specifically, the display light is emitted by the optical machine of the HUD display device, and is projected on the windshield through the corresponding optical lens to produce a corresponding virtual image, which forms an enhanced display effect with the real world outside the windshield. Among them, for the display elements projected by the HUD display device, if a 3D digital model is used to reflect the physical entity of the vehicle's surrounding environment, such as vehicles, the intuitiveness of the information display can be greatly improved. However, the rendering and display of 3D digital models currently requires a lot of hardware resources, and the existing HUD display equipment will increase costs if it meets the relevant display requirements. Summary of the invention

[0003] The purpose of this application is to provide a display processing method, a display device, a storage medium and a vehicle, which solve the technical problems in the prior art that the display device cannot display 3D digital models in a simple way, has high hardware resource requirements and high costs.

[0004] In order to solve the above technical problems, this application adopts the following technical solutions.

[0005] In a first aspect, the present application provides a display processing method, comprising: The display device is configured to display a first display element, wherein the first display element is used to represent a first object around the display device; In response to the first object being in the first state, the first material is called to perform mapping projection to realize the display of the first display element, and the first material has the stereoscopic effect of the first object in the first state.

[0006] In an optional implementation of the first aspect, the first material is a pre-designed picture stored in advance.

[0007] According to the above description, an optional implementation method uses material calling and mapping instead of real-time rendering, which reduces the requirements for hardware resources and makes the display method simpler and more direct. The pre-stored material library supports the configuration image calling of the first object in different states.

[0008] In an optional implementation of the first aspect, the three-dimensional effect is presented through at least one psychological suggestive structure of occlusion relationship, shadow simulation, linear perspective, texture gradient, and near big and far small.

[0009] According to the above description, an optional implementation method realizes the effect of direct 3D rendering by using the stereoscopic structure contained in the called 2D picture, which uses the psychological suggestive structure in the picture to make the human eye feel a certain sense of depth.

[0010] In an optional implementation manner of the first aspect, the first material having a three-dimensional effect of the first object in the first state includes: The first object is identified as a vehicle of a corresponding model, and the first material includes a three-dimensional outline and / or color of the corresponding model.

[0011] According to the above description, an optional implementation method can adaptively select corresponding image materials for display after simply collecting the model of the external vehicle based on the on-board camera, thereby improving the user's viewing experience.

[0012] In an optional implementation manner of the first aspect, the first material having a three-dimensional effect of the first object in the first state includes: According to actual lighting conditions of the first object in the first state, the first material called has a first light and shadow effect simulating the surroundings of the first object.

[0013] In an optional implementation manner of the first aspect, the first material having a three-dimensional effect of the first object in the first state includes: According to actual lighting conditions of the first object in the first state, a second material is called and superimposed on the first material to simulate a first light and shadow effect around the first object.

[0014] According to the above description, an optional implementation can also simulate the actual light and shadow around the object in the first display element according to the external sun's height and intensity, so as to create a more realistic three-dimensional effect and keep it consistent with the actual situation of the first object.

[0015] In an optional implementation of the first aspect, the display device is further configured with a second display element, which is used to indicate a carrier on which the display device is located, and a display position relationship between the first display element and the second display element is configured according to an actual position relationship between the first object and the carrier.

[0016] In an optional implementation manner of the first aspect, the first material having a three-dimensional effect of the first object in the first state includes: According to the direction of the first object relative to the carrier, the first material called has a corresponding viewing angle simulating the first object.

[0017] In an optional implementation of the first aspect, the display processing method further includes: In response to the first object turning and switching from the first state to the second state, a third material is called to perform texture projection, and the perspective of simulating the first object in the third material is different from the perspective of simulating the first object in the first material.

[0018] According to the above description, an optional implementation can display the distribution between the carrier and surrounding objects in real time. When applied to the projection display in the vehicle, the position relationship between the vehicle and surrounding vehicles can be directly seen on the windshield, assisting the driver in driving safely.

[0019] In an optional implementation of the first aspect, the display position relationship between the first display element and the second display element is configured according to the actual position relationship between the first object and the carrier, including: In response to the first object moving away from or approaching the carrier, a display size and / or a display angle of the first display element is adjusted.

[0020] In an optional implementation manner of the first aspect, in response to the first object moving away from or approaching the carrier, adjusting the display size and / or display angle of the first display element includes: When the first object moves forward and away from the carrier, the first display element is reduced in size according to the distance away.

[0021] In an optional implementation manner of the first aspect, in response to the first object moving away from or approaching the carrier, adjusting the display size and / or display angle of the first display element includes: When the first object moves backward and approaches the carrier, the first display element increases in size according to the approach distance.

[0022] In an optional implementation manner of the first aspect, in response to the first object moving away from or approaching the carrier, adjusting the display size and / or display angle of the first display element includes: When the first object is located on the left side of the carrier, the first display element is adjusted clockwise according to the distance between the first object and the carrier.

[0023] In an optional implementation manner of the first aspect, in response to the first object moving away from or approaching the carrier, adjusting the display size and / or display angle of the first display element includes: When the first object is located on the right side of the carrier, the first display element is adjusted counterclockwise according to the distance between the first object and the carrier.

[0024] According to the above description, an optional implementation manner can display the environment around the carrier as a whole, and the overall display satisfies the relationship of linear perspective to form a real three-dimensional effect.

[0025] In an optional implementation of the first aspect, the display position relationship between the first display element and the second display element is configured according to the actual position relationship between the first object and the carrier, including: When the first object is located at the left side of the carrier, the first material is called to simulate a viewing angle of the first object tilted toward the upper right corner.

[0026] In an optional implementation of the first aspect, the display position relationship between the first display element and the second display element is configured according to the actual position relationship between the first object and the carrier, including: When the first object is located at the right side of the carrier, the first material is called to simulate a viewing angle of the first object tilted toward the upper left corner.

[0027] According to the above description, an optional implementation method calls the viewing angle material that conforms to the viewing habits according to the overall linear perspective relationship to make it have a more three-dimensional effect.

[0028] In an optional implementation of the first aspect, the display position relationship between the first display element and the second display element is configured according to the actual position relationship between the first object and the carrier, including: In response to the first object approaching the carrier, the display device is further configured with a third display element, and the third display element is used to represent warning prompt content.

[0029] According to the above description, the optional implementation mode is applied in a driving scene, and a striking collision warning logo can be set near the first display element and the second display element to intuitively remind the driver of the potential danger.

[0030] In a second aspect, the present application provides a display device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the display processing method described in the first aspect when executing the computer program.

[0031] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the display processing method described in the first aspect are implemented.

[0032] In a fourth aspect, the present application provides a vehicle comprising the display device described in the second aspect or the computer-readable storage medium described in the third aspect.

[0033] Compared with the prior art, this application uses texture materials to simulate the real three-dimensional effect of actual objects, and the different states of the actual objects are called by materials containing different elements to form corresponding display elements in the projection area, so as to truly reflect the real-time environmental changes of the actual objects and the three-dimensional presentation under the environmental changes, providing users with a real sense of feedback. The three-dimensional display effect of this application does not rely on high-end hardware resources, and the operation is simple and efficient, which can save the hardware cost of implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the description of the technical solution. Obviously, the drawings described below are only some examples recorded in the present application, and a person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

[0035] Figure 1 Schematic diagram of HUD projection display in some examples of this application.

[0036] Figure 2 Schematic diagram of the vehicle and surrounding vehicles on the lane in some examples of this application.

[0037] Figure 3 This is a schematic diagram of the three-dimensional effect of objects with specific orientations in some examples of this application.

[0038] Figure 4 This is a schematic diagram of the three-dimensional effect of objects with specific orientations in some examples of this application.

[0039] Figure 5 Schematic diagram of different shadow effects of objects in some examples of this application.

[0040] Figure 6 Schematic diagram of basic type materials and overlay type materials in some examples of this application.

[0041] Figure 7 Schematic diagrams showing the first display element and the second display element in some examples of the present application.

[0042] Figure 8 Schematic diagram of HUD display device module in some examples of this application.

[0043] Fig. 9 This is a schematic diagram of the composition of the HUD display device in some examples of this application.

[0044] Fig.10 This is a schematic diagram of projection display in a vehicle in some examples of the present application. DETAILED DESCRIPTION

[0045] The present application will be described in detail below in conjunction with the accompanying drawings, but the content of the description is merely some examples recorded in the present application and does not limit the present application. Changes in structure, method, function, etc. made by ordinary technicians in this field based on these examples are included in the protection scope of the present application.

[0046] It should be noted that the same reference numerals or marks may be used in different examples, but these do not represent an absolute connection in structure or function. In addition, the "first", "second", etc. mentioned in each example are only for the convenience of description, and do not represent an absolute distinction in structure or function, nor can they be understood as indicating or implying the relative importance or the number of corresponding objects. Unless otherwise specified, the "at least one" that may be involved in the description refers to one or more, and "more than one" refers to two or more.

[0047] In addition, when expressing a feature, the character " / " can indicate that there is an or relationship between the front and back associated objects. For example, head-up display / head-up display can be expressed as head-up display or head-up display. When expressing an operation, the character " / " can indicate that there is a division relationship between the front and back associated objects. For example, the magnification M=L / P can be expressed as L (virtual image size) divided by P (image source size). Moreover, the "and / or" in different examples is only to describe the association relationship between the front and back associated objects. This association relationship can include three situations. For example, a concave mirror and / or a convex mirror can be expressed as the existence of a concave mirror alone, the existence of a convex mirror alone, and the existence of both a concave mirror and a convex mirror.

[0048] HUD projection display mainly uses the principle of optical reflection to reflect the imaging light to be displayed through a transparent surface into the viewer's eyes. The human eye can see the virtual image information in the opposite direction of the light. Correspondingly, the transparent surface can be the windshield of the vehicle, and the windshield can be used as a display screen to display the navigation instructions of the vehicle, the vehicle's driving speed, etc. Figure 1As shown, the HUD display device may include at least an optical machine 1, a first reflector 2, a second reflector 3, etc., wherein the optical machine 1 includes a backlight source and an image source (not shown), the backlight source is used to provide illumination light and adjust the brightness of the illumination light according to control, for example, the backlight source may be an LED (Light Emitting Diode), a laser, etc. Under the illumination light provided by the backlight source, the image source adjusts the corresponding display content according to control and projects the display light from the surface of the image source, for example, the image source may be an LCD (Liquid Crystal Display), a DMD (Digital Micromirror Devices), a MEMS (Micro-Electro-Mechanical System) micromirror, a LCOS (Liquid Crystal on Silicon), etc. The first reflector 2 and the second reflector 3 can project the display light projected by the optical machine 1 onto the windshield 4, so as to customize the optical path in a smaller space and meet different projection display requirements. The first reflector 2 and the second reflector 3 can be set as a concave mirror, a convex mirror, a concave lens, a convex lens, etc. according to the requirements of optical planning, and the surface shape of the lens can adopt a free-form surface. Optionally, at least one of the first reflector 2 and the second reflector 3 can also be adjusted to a certain degree of angle, so as to change the projection position of the display light on the windshield 4 to meet the needs of viewers of different heights. The display light of the optical machine 1 is finally reflected on the windshield 4 of the vehicle to form a virtual image 5. When the human eye 6 observes the virtual image 5 facing the windshield 4, it can feel a certain sense of depth, just like watching a real object at a specific distance outside the windshield. The virtual image 5 can be the navigation instruction content, vehicle speed, etc. as described above. It should be added that, according to the characteristics of different optical machines, the HUD display device can also be provided with a diffuser. In some examples, the HUD display device can also include Fresnel lenses, waveguide optical devices, diffractive optical devices, holographic optical devices, tapered optical fibers, etc.

[0049] As mentioned above, the HUD display device is more convenient for the driver to view information because the projected content is located directly in front of the driver's field of vision. In some examples, the environment around the vehicle can be acquired through acquisition technology, and the digital models corresponding to the objects in the surrounding environment can be rendered and displayed in real time in the projection area of ​​the HUD display device, such as specific vehicles in the surrounding area, which can be used to prompt the driver well, especially to prevent the driver from ignoring potential risks in the surrounding environment. In order to cooperate with the real-time display of the surrounding environment, especially the display restoration of actual objects, the scene reconstruction technology can be used to render the objects in the surrounding environment in real time, and the rendering results can be output to the image source of the HUD display device to project the display light, and finally provide an immersive digital twin experience in front of the vehicle windshield. Accordingly, scene reconstruction includes data collection, data processing and modeling, real-time tracking, etc. Among them, data collection mainly relies on efficient three-dimensional data acquisition methods such as depth cameras and laser radars to collect data information such as geometry and texture of physical space. Data processing and modeling is to convert the collected information into a three-dimensional point cloud or mesh model through preprocessing, matching, fusion and other steps, and further construct a fine three-dimensional scene model through surface reconstruction, texture mapping and other technologies. Real-time tracking is to register the reconstructed model in real time to the actual environment according to the user's perspective. However, scene reconstruction technology has higher requirements on sensor accuracy, chip performance, etc., and the development difficulty is relatively large, and the cost is also higher. In some examples, the above-mentioned real-time display of the surrounding environment can also be applied to other display devices, such as mobile phone terminals, display large screens, etc. The display device can determine the state of surrounding objects based on the collected surrounding environment data, such as the outline of a specific object, the distance of a specific object from the display device, etc., and at the same time display the layout relationship between the object and the display device in real time on the screen of the display device. Similarly, if the scene reconstruction in the above example is used to display these specific objects in real time, it will also face the problem of high hardware configuration requirements.

[0050] In some examples, taking the HUD display device used in the vehicle as an example, in order to reduce the chip processing burden and reduce the hardware cost, in the data collection stage, it is not limited to the use of expensive equipment such as laser radar, but also can use ordinary cameras to obtain simple data such as the model of surrounding vehicles, such as identifying the logo on the rear of the vehicle to determine the model of the vehicle, and then searching for vehicle model materials based on the vehicle model, instead of identifying complex data such as the outline of surrounding vehicles in real time. Optionally, it is also possible to communicate directly with adjacent vehicles, and the communication between them can adopt a unified protocol standard to determine vehicle information by obtaining simple data such as the vehicle model returned by adjacent vehicles. Furthermore, the distance information between the vehicle where the HUD display device is located and the surrounding vehicles can also be measured by low-cost millimeter-wave radar, ultrasonic radar, etc., so that the surrounding objects and positions and other information can be basically determined. More importantly, when reproducing the surrounding objects, the HUD display device does not use the scene reconstruction method of data processing and modeling, thereby avoiding the option of high-cost chip processing. Instead, it retrieves the corresponding vehicle model material for display according to the vehicle model. The vehicle outline and texture in the material are pre-stored according to the design data of the actual vehicle model. Optionally, these materials will be stereo-modeled in a specific state and output in a relatively fixed picture format. Calling pictures consumes much less hardware resources and can basically achieve the effect of real-time rendering. Furthermore, in order to reflect the three-dimensional effect of a specific object, the image material will retain at least one of the psychological suggestive structures of occlusion relationship, shadow simulation, linear perspective, texture gradient, and near big and far small during modeling. These psychological structures can help the human eye perceive depth vision when observing the content of the image. Specifically, because the human brain has undergone long-term training, it will produce hallucinations for some scenes presented in a two-dimensional plane, so that the human eye mistakenly believes that these scenes are three-dimensional when viewing. In this example, these structures are used to realize the visual experience of the 2D image directly reflecting the three-dimensional scene, and the image can support the convenience of pre-storage and call when displaying. Among them, the occlusion relationship can be based on the front and back relationship of the corresponding object under a specific viewing angle, and the rear part of the viewing angle closer to the human eye will block the front part in the same line of sight direction. Shadow simulation can reflect the shadow effect produced according to the direction of light irradiation, such as the shadow effect cast by the surface of the object itself and the plane where the object is located, and the light and dark conditions presented by different surfaces of the object. Linear perspective can be along the direction of the viewing angle, the entire line of sight will focus on a point in the distance, and the shape of the object will also converge along this radial focus point.Texture gradient can be a texture simulation based on the hierarchical changes in the projection size and projection density of objects in the field of view on the retina of the human eye. In addition, it can be accompanied by a certain air perspective effect, that is, simulating the atmosphere and air media (rain, snow, smoke, fog, dust, water vapor, etc.) to make the human eye see that nearby objects are thicker, more colorful, and clearer than distant objects. The near big and far small is based on the mechanism that the size of the image changes with the change of the object distance when the human eye forms an image. The farther objects are adjusted to be smaller in the material so that they can be reflected in the corresponding projection area when mapped. In addition, some motion perspective and different relative height presentation effects can be retained in the material content to enhance the overall three-dimensional effect.

[0051] In some examples, when the display device recognizes the surrounding objects and needs to call the corresponding materials, different materials will be called according to the current actual state of the object. The actual state specifically includes the arrangement and combination of several elements. That is, in the material library for calling, the same object has multiple materials in different states. Taking vehicles as an example, the image content corresponding to these states may include the outline of the corresponding vehicle model. For example, sedans, SUVs, light trucks, trucks, etc. all have different outlines. More specifically, vehicles of different brands and models will have different styles of outline materials. Furthermore, the state of the vehicle can also include the specific body color of the vehicle. For example, different models of vehicles of the same brand correspond to colors of different states. The body color can be searched for the color index in the material based on the collected color information. For example, the color of surrounding vehicles is identified by a camera, and different colors have separate materials for calling. In more examples, the state of the vehicle may also include the vehicle's orientation, the different distances of the location, the vehicle's conditions under different lighting conditions, etc. Different states have corresponding separate materials for calling, and these pre-stored materials can be images that are exported after setting specific states through modeling in advance. The images specifically include arrangements and combinations of occlusion relationships, shadow simulation, linear perspective, texture gradients, near big and far small, etc. under specific states. For the convenience of retrieval, index information of specific state content is also set in the material, such as index tags such as left front orientation and right rear lighting, so that it is easy to quickly locate the required material when calling. Figure 2As shown, for the vehicles around the vehicle (the vehicle carrier where the HUD display device is located), according to the actual driving situation, there may be multiple position states such as front, rear, left, right, left front, right front, left rear, and right rear, respectively. The above states can determine the position of the surrounding vehicles through the radar, camera, etc. in the above examples. When the vehicle is displayed as the center of the display and the surrounding vehicles are displayed, the corresponding position relationship will first be displayed in the projection area according to the actual relative position relationship. For example, if a specific vehicle is determined to be to the left of the vehicle through detection, then the digital model of the specific vehicle displayed by projection will also be displayed to the left of the digital model of the vehicle. The distance between them can also be displayed in equal proportion. Specifically, the display position is configured after proportional processing according to the measured distance. More importantly, due to the different positions and distances between the specific vehicles around and the vehicle, the three-dimensional effect displayed is also different. For example, in the occlusion relationship reflecting the three-dimensional effect, the rear part of the vehicle driving forward in front may be displayed more significantly, while the front part may be blocked. In addition, the different viewing angles of the specific vehicle relative to the vehicle will also affect the different presentation of the three-dimensional effect, that is, there are different occlusion relationships. Accordingly, separate call materials can be set for the same model of vehicle in multiple position states such as front, rear, left, right, left front, right front, left rear, and right rear, including elements and contents related to the psychological suggestive structure in the above position states, and different materials can be called in different positions. In more examples, in order to cooperate with the refined display effect of the vehicle during movement, more position relationship materials can be set for calling, and in the process of real-time display, different materials can be continuously called in response to different position states.

[0052] like Figure 3 , Figure 4 As shown in the figure, objects represented by vehicles have different states when their specific orientations are different, and the three-dimensional effects they display are also different. Therefore, materials with corresponding orientations will be set for use. In this example, the orientations of surrounding vehicles are based on the orientation of the vehicle itself. Figure 3 In the figure, when the side where the long side 71 is located is facing the vehicle, for example, a specific vehicle passes horizontally at the intersection in front of the vehicle, the side of the specific vehicle facing the vehicle will be fully seen, while the side away from the vehicle will be blocked, and the front and rear of the vehicle (the side where the short side 72 is located) will not be fully seen from the perspective of the orthographic projection. These will be correctly presented in the picture of the corresponding material, and these perspectives can be achieved by adjusting the direction of the digital model in the modeling software, which will be described in detail below. In a specific example, when a vehicle passing from left to right or from right to left is detected at an intersection, the material in this state can be called for display. It should be noted that the called material must also be combined with other elements such as vehicle outline and color, which are determined based on identifying the model of the vehicle passing through the intersection. Figure 4 In the example, when the direction of a specific vehicle turns 90 degrees, the side where the short side 72 is located will face the vehicle. In reality, the oncoming vehicle may face the front of the vehicle or the vehicle in front may face the rear of the vehicle. The three-dimensional effect presented by the digital model in the material is that the side where the short side 72 is located blocks the other side opposite to it, and the side where the long side 71 is located will be presented in a linear perspective. When a specific vehicle in the above state is displayed, the material that satisfies this state is displayed. In more examples, the pre-stored material library may also include materials of corresponding objects in multiple fine orientations for calling, so as to meet the flexible and changeable orientations of objects and coherent turning behaviors in actual applications. Optionally, these object materials with different orientations can be configured in advance using modeling software. For example, a digital model of the corresponding shape can be modeled first, and then the orientation of the digital model can be rotated according to the actual display needs. Specifically, it can be configured according to Figure 3 , Figure 4 The orientation is rotated once, and then the image material that meets the requirements is exported. As mentioned above, in order to facilitate retrieval, the image material also has the corresponding index information of the orientation, and the index tag can be retrieved according to actual needs when calling.

[0053] In some examples, such as Figure 5 As shown, the different states of the same object also include elements such as shadow relationships under different lighting. In the physical world, when an object is illuminated by different light sources, such as the sun in an outdoor environment, it will show different light and shadow effects on the surface of the object and the surrounding planes. The overall performance is that the surface close to the light source is brighter, while the surface away from the light source is a little darker. Correspondingly, due to the partial blocking of light, a certain shadow will be produced on the backlit surface. Therefore, in order to simulate these actual light and shadow conditions, the shadow simulation of the object to be simulated can be performed. As mentioned above, the simulated light and shadow effects can not only truly reflect the different states of the actual object under different lighting, but also provide a certain three-dimensional sense for the human eye. Figure 5In the example, although the same object has the same outline, such as the same model of vehicle, it has different materials to match the state under different lighting. In the first material, the light source illuminates the object from the right, and the surface on the right side close to the light source appears brighter, while the surface on the left side away from the light source will be darker. In the second material, the light source illuminates from the left side, so the visible surface on the left side appears brighter, and on the contrary, the surface on the right side will be darker. In the third material, the light source illuminates from the front side, so the side facing the viewing direction will be darker, and the upper surface of the front part of the object will appear brighter. The fourth material is just the opposite. Since the light source illuminates from the back side, the back half appears brighter than the front half. In more examples, the material with the arrangement and combination of lighting state elements will also reflect the shadow projected on the surrounding. Taking the right light source as an example (not shown in the figure), the shadow of the object will be seen on the left plane carrying the object (such as the road simulating the vehicle's driving), and under different light source heights, there are shadows of different lengths. This can also be refined into several materials to adapt to the call in different situations. Furthermore, more material states can be arranged and combined according to the actual direction of the light source to adapt to the effects of sunlight at different time periods. Optionally, different states of sunlight brightness can also be included to adapt to sunny days, cloudy days, early mornings, noon and other situations. The specific performance in the material will present different brightness reflection intensities on the surface of the digital model of the object. In the actual display, after the corresponding object is detected, such as the radar detecting the surrounding vehicles, not only the material search is performed with index tags such as specific vehicle models, but also the parameters such as the direction, intensity, and height of the sun are detected, such as determined by the light sensor on the vehicle, and the material is positioned in combination with the corresponding light source parameter index tag. In this way, the content in the material includes not only the elements of the appearance state of the specific object such as the outline and color, but also the elements such as the orientation and shadow effect in different three-dimensional states. Correspondingly, these materials can also be used in advance in the modeling software to adjust the parameters of the modeled digital model (which already has a specific outline and color). Specifically, different lighting directions and intensities can be set. The digital model will be adjusted to produce different three-dimensional effects by different simulated lighting in the modeling software, that is, different light and shadow effects can be simulated, and the corresponding materials can be exported for subsequent display. I will not go into details here.

[0054] In some examples, the material of a specific object will form a single material by arranging and combining elements of different dimensions, that is, in order to meet the specific state of the actual object, the single material will have a specific combination of elements such as contour, color, orientation, shadow effect, etc. In some examples, the material representing the state of a specific object can also be divided into at least two types according to the type of element content, corresponding to basic type material and overlay type material, respectively, and the number of permutations and combinations can be compressed by arranging and combining by type, such as Figure 6 As shown, the basic type material can be the contour model and other elements of the actual object. The contour model has an orientation in a specific state, but its surface is left blank, that is, it does not have the color or light and shadow effects in a specific state. For example, the surface 73 of the object in the basic type material does not originally have the corresponding color and shadow effects, but supports splicing overlay type materials to form a complete object digital model. The overlay type material includes the elements of light and shadow effects, colors and other specific states. Its area range can match the surface of the digital model to be spliced ​​in the basic type material, where different colors, light and shadow effect combinations can correspond to different states, such as Figure 6 Multiple overlay type materials correspond to different colors. Different overlay type materials are called according to actual needs to be spliced ​​with the same basic type material. In actual display, after recognizing the specific state of surrounding objects, in addition to calling specific basic type materials for display, corresponding overlay type materials are also called to the corresponding surface of the basic type materials according to the color and lighting conditions in the corresponding state, so that the displayed digital model can achieve the same state as the actual object. At the same time, it can also save the number of pre-stored materials and improve the flexibility of configuration.

[0055] In some examples, such as Figure 7 As shown, in the scenario where the vehicle-mounted HUD display device reflects the surrounding environment in real time, the HUD display device will display the road conditions of the vehicle (i.e., the vehicle equipped with the HUD display device) in a more intuitive scene reproduction manner in front of the driver's field of vision, that is, reproduce the collected surrounding environment in the projection area, which may include the actual corresponding road direction, and is specifically displayed by calling the road model 703. Specifically, the road model 703 not only has shoulders on both sides, but also includes information such as lane lines on the road. In order to present a three-dimensional viewing effect, the road model 703 reflecting the straight direction will present a linear perspective relationship that focuses on the distance in a trapezoidal shape, that is, the farther away the road is, the smaller the distance between the two sides of the road. Further, in order to reflect the distance relationship between the surrounding vehicles (the first object around the HUD display device) and the vehicle in the projection area, the HUD display device will not only display the first display element 701 corresponding to the detected specific vehicle during projection, but the first display element 701 is a first material called according to the state of the specific vehicle and configured to be displayed in the corresponding projection area (for details, please refer to Figure 2-Figure 6 ), the HUD display device will also display the second display element 702 corresponding to the vehicle. The second display element 702 can also be realized by calling the pre-stored material. Similarly, the material of the vehicle can also have different materials according to different states (similarly, refer to Figure 2-Figure 6), for example, the material may include elements such as specific driving directions on different roads and specific shadow effects under different lighting. Accordingly, the road model 703 of the road where the vehicle is located will be reproduced in the projection area, so that the driver can intuitively understand the direction of the current road, and the second display element 702 can be displayed at the relevant position between the corresponding road models 703 according to the positioning information of the vehicle itself, which corresponds to the actual position of the vehicle on the road, and the first display element 701 can also be displayed at the corresponding position between the corresponding road models 703 according to the measured relative position relationship with the vehicle, so as to reflect the corresponding positions of the surrounding vehicles and the vehicle on the actual road, so that the driver can intuitively judge the driving conditions of the surrounding vehicles. Accordingly, in the projection area, not only the actual object types corresponding to different display elements are reflected, such as the driver can judge by contour, color, etc., but also the display position between the display elements can be used to reflect the direction and distance between the actual objects. In a specific example, there is a specific proportional relationship between the distance L between the display elements in the projection area and the distance between the vehicles on the actual road, and the proportional relationship can be determined by the size of the projection area. The angle θ between the display elements is consistent with the relative direction between the actual vehicles, so that the driver can be assisted to accurately judge the specific road conditions nearby. Further, as the vehicle corresponding to the second display element 702 continues to move on the actual road, the second display element 702 will also be displayed on the road model 703 accordingly, and its moving speed corresponds to the actual vehicle speed, while the movement of the first display element 701 will be proportionally reflected according to the moving speed of the surrounding specific vehicles, ensuring that the distance between the first display element 701 and the second display element 702 in the projection area can always correctly reflect the actual relative position relationship between the surrounding vehicles and the vehicle. In some examples, the above display logic in the projection area can also be applied to other display devices such as the central control screen. It also calls the specified material and updates the display position on the display device according to the changes of the actual object. It will not be repeated here.

[0056] In some examples, if the displayed second display element reflects the digital model of the vehicle, the front view of the digital model of the vehicle can be used as the overall perspective, and when the first display element calls the material, the first object corresponding to the first display element should be considered relative to the vehicle to determine the display perspective, so as to call the material of the corresponding perspective. Optionally, in order to meet the three-dimensional effect of linear perspective, for the vehicle on the left side of the vehicle, when the first display element is displayed, it will be adjusted clockwise relative to the second display element, and the adjustment angle is determined according to the actual distance between the first object and the vehicle. The greater the distance, the greater the clockwise adjustment angle. Optionally, the tilt angle of the first display element based on linear perspective can also be directly in the called material, and a more realistic display can be achieved by directly mapping after calling. Similarly, when the identified vehicle is on the right side of the vehicle, the first display element will be adjusted counterclockwise relative to the second display element. That is, under the linear perspective relationship, the front parts of the first display elements located on both sides of the second display element facing the distance will be tilted toward the front part of the second display element. As mentioned above, this tilt angle can be achieved by calling the material and then rotating it, or by calling the material with a specific angle. For example, the material displayed on the left side of the second display element should be called to tilt toward the upper right corner, and the material displayed on the right side of the second display element should be called to tilt toward the upper left corner. In this way, a more realistic three-dimensional effect can be achieved while storing materials in multiple states and combining other elements such as specific angles and occlusion relationships.

[0057] In some examples, for the first display element and the second display element displayed dynamically, in addition to objectively reflecting the actual movement situation, in order to further optimize the stereoscopic effect, when the distance between the first display element and the second display element increases, that is, when the actual first object moves away from the vehicle, in addition to proportionally controlling the movement of the first display element relative to the second display element, the size of the first display element will be adjusted according to the linear perspective relationship, presenting a stereoscopic effect of a specific vehicle moving away. On the contrary, if the actual distance between the two is getting closer, the first display element will also increase the displayed size proportionally according to the speed of approaching, presenting an obvious approaching effect to the human eye. Optionally, when the first display element and the second display element are approaching each other, that is, when the distance between the specific vehicle and the vehicle is less than a specific threshold, in addition to adjusting the size of the first display element to prompt the driver, a third display element will be configured and displayed between the first display element and the second display element. The third display element can be a red warning sign, or a warning prompt content directly superimposed on the first display element or the second display element, intuitively prompting the driver to pay attention to the potential danger. In some examples, when the first display element moves relative to the second display element in response to changes in an actual object, if the first object represented by the first display element also changes in direction, taking the turning of a specific vehicle in the surrounding area as an example, this is equivalent to the specific vehicle switching from a first state to a second state. In order to truly reflect the actual situation, the first display element can call materials from different perspectives for display, that is, it can switch from calling the first material corresponding to the first state to calling the third material corresponding to the second state. If the switching between the first material and the third material is smooth, the effect of animation display can also be achieved.

[0058] When the HUD display device implementing the display processing method in the above example is applied to a vehicle, the driver can observe realistic road conditions while facing the windshield, especially potential risks that cannot be noticed by directly observing the window with the human eye. Figure 8 As shown, the HUD display device integrated in the vehicle can be powered and data by the vehicle computer 92, or the HUD display device itself can provide power and generate data. The HUD display device can specifically include a processor 91, an Ethernet interface 901, a CAN (Controller Area Network) interface 902, a power management module 903, a running memory 904, a storage memory 905, a temperature detection 906, a motor 907, a backlight source 908, an image source 909, a positioning module 910, a radar 911, a camera 912, etc. It should be noted that Figure 8The modules listed in the description are merely exemplary and do not constitute any limitation. In some examples, the HUD display device may further include other modules. In addition, the above modules may be implemented in one or more hardware in different examples, or a single module may be implemented by a combination of multiple hardware.

[0059] The processor 91 serves as the control center of the HUD display device, and includes one or more processing units of any type, including but not limited to a microcontroller, a microcontroller, a DSP (Digital Signal Processor) or any combination thereof. The processor 91 is used to generate operation control signals according to a computer program, realize the control of other modules, and cooperate with corresponding modules to process the acquired or inherent data, instructions, etc.

[0060] The Ethernet interface 901 is a network data connection port for local area network communication, which defines a series of software and hardware standards. Multiple electronic devices can be connected together through the Ethernet interface 901. In this example, the processor 91 can exchange information with the vehicle computer 92 through the Ethernet interface 901, such as sending data to the vehicle computer 92 or receiving data sent by the vehicle computer 92.

[0061] The CAN interface 902 is a network data connection port of the controller area network, which provides a standard bus for the control system and embedded industrial control inside the car, and realizes the communication interaction between the control nodes. In this example, the processor 91 can also exchange information with the vehicle computer 92 through the CAN interface 902. Optionally, the processor 91 can also connect to other external devices through the CAN interface 902. In some examples, the processor 91 can also be provided with a GPIO (General-purpose input / output) interface to improve the compatibility of peripheral connections.

[0062] The power management module 903 is connected to the vehicle computer 92 and can receive the power provided by the vehicle computer 92 to provide a regulated power supply for each module of the HUD display device, thereby ensuring that the processor 91 and each module operate under normal voltage supply and avoid damage due to overvoltage.

[0063] The running memory 904 is used to store computer programs executed by the processor 91, temporarily stored calculation data, data exchanged with the storage memory, etc. The running memory 904 can be a memory such as SDRAM (Synchronous Dynamic Random-access Memory).

[0064] The storage memory 905 is used to store resources such as the display content related to the HUD display device, and long-term storage of running programs and data, etc. The storage memory 905 can be a memory such as Flash (flash memory). In some examples, the processor 91 can also provide an interface to access an external memory.

[0065] Temperature detection 906 is used to detect the temperature inside the HUD display device, and may specifically include several temperature sensors. Since the resistance value of the temperature sensor changes with the change of temperature, the processor 91 can determine the resistance value of the temperature sensor at the corresponding temperature according to the voltage change between each temperature sensor and the voltage divider resistor under a fixed power supply voltage, thereby reversely deducing the temperature at the location of the temperature sensor. In some examples, the processor 91 can control several temperature sensors through the GPIO interface, and several temperature sensors can be set at different locations inside the HUD display device. The processor 91 can use time-sharing detection to obtain the temperature values ​​fed back by several temperature sensors respectively.

[0066] Motor 907 is used to drive the optical lens in the HUD display device to rotate under the control of processor 91, so as to achieve the change of the corresponding optical path. For example, when the backflow of sunlight causes the temperature rise on the surface of the image source, the motor can be used to drive the optical lens to prevent the external sunlight from reaching the surface of the image source. In some examples, the processor 91 can also drive the fan provided on the HUD display device through motor 907 to increase the speed of air exchange between the inside and outside of the HUD display device to achieve heat dissipation. Specifically, the motor 907 is connected to the processor 91 through a motor driver chip, and the motor driver chip provides high-performance power output for the motor 907, and can also communicate and control with the processor 91 through interfaces such as SPI (Serial Peripheral Interface).

[0067] The backlight source 908 is used to provide illumination light and adjust the brightness of the illumination light according to the control of the processor 91, so as to adjust the projection display brightness of the entire HUD display device. The backlight source 908 cooperates with the image source 909 to realize the main function of optical machine projection display. The backlight source 908 can be an LED (Light Emitting Diode), a laser, etc. Specifically, the backlight source 908 is connected to the processor 91 through a backlight driver chip, and the backlight driver chip provides a driving voltage for the backlight source 908 and controls the brightness of the backlight source 908 under the pulse width signal output by the processor 91.

[0068] The image source 909 is used to display an image of corresponding content and project the display light corresponding to the image according to the control of the processor 91. The image source 909 can be an LCD (Liquid Crystal Display), a DMD (Digital Micromirror Devices), a MEMS (Micro-Electro-Mechanical System) micromirror, a LCOS (Liquid Crystal on silicon), etc.

[0069] The positioning module 910 is used to monitor the position of the HUD display device and the corresponding vehicle. The positioning module 910 can be a global navigation satellite system such as GPS (Global Positioning System) and Beidou Satellite Navigation System. By measuring the distance between the satellite and the receiver on the positioning module 910 at different positions, the corresponding position and orientation data are determined. In some examples, the positioning module 910 may also include an inertial navigation system, which is based on Newton's laws of mechanics. By measuring the acceleration of the positioning module 910 in the inertial reference system, integrating it over time, and transforming it into the navigation coordinate system, the speed, yaw angle, and position data in the navigation coordinate system are obtained. Optionally, the inertial navigation system can assist the global navigation satellite system to achieve more accurate positioning and provide the processor 91 with corresponding position information.

[0070] The radar 911 is used to determine the position of a target object through electromagnetic waves, and can usually determine the distance of the target object from the vehicle where the radar 911 is located.

[0071] Camera 912 includes a vehicle body camera and an in-vehicle camera, wherein the vehicle body camera is used to determine the position of the target object through visual recognition, and the vehicle body camera can be a monocular camera or a binocular camera. The biggest difference between a monocular camera and a binocular camera is that a binocular camera can capture images from two different perspectives, thereby obtaining distance information in three-dimensional space. The in-vehicle camera is used to identify the behavior status of the driver and passengers in the vehicle, including fatigue detection, distraction detection, expression recognition, gesture recognition, sight tracking, etc. In this example, the in-vehicle camera can also implement eye tracking.

[0072] In some examples, the positioning module 910, radar 911 and camera 912 can also be directly connected to the vehicle computer 92, and are not directly connected to the processor 91 of the HUD display device. For example, the vehicle computer 92 itself is integrated with a positioning module for position tracking and a radar and camera for automatic driving. The HUD display device can obtain the collected data of the positioning module, radar and camera in real time through communication with the vehicle computer 92.

[0073] In some examples, such as Fig. 9 As shown, the display device implementing the above display processing method may specifically include a processor 931, a memory 932, an input device 933 and an output device 934, wherein the input device 933 may include an operation button integrated on the display device, etc., and the display device may receive input control instructions and data through the input device 933. The output device 934 may include an image source integrated on the display device, etc., and the display device may output corresponding instructions or data to the output device 934. Further, the memory 932 stores a computer program running on the processor 931, and the processor 931 implements the display processing method of the above example when executing the computer program. In some examples, a computer-readable storage medium, a computer-readable storage medium storing a computer program, and the display processing method of the above example is implemented when the computer program is executed by the processor.

[0074] like Fig.10 As shown, the vehicle can be provided with the above-mentioned HUD display device. Specifically, the HUD display device is integrated inside the center console 10, such as in front of the steering wheel. The corresponding display light is projected onto the opposite vehicle windshield 4 through the projection window 102 of the HUD display device. The effect of the viewer observing the front of the windshield 4 from the cockpit is that the virtual image in the projection area 50 can be directly seen. In addition to basic information such as vehicle speed, the virtual image content can also include a three-dimensional digital model with a real simulation effect, which corresponds to the actual road conditions outside. In some examples, the vehicle can also distribute the program of the display processing method in the above example through the computer-readable storage medium of the above example, so that the vehicle-mounted HUD display device can be conveniently updated and upgraded. It should be noted that the above-mentioned vehicle is not limited to a car as a means of transportation, but can also include a bus, a truck, an excavator, a motorcycle, a train, a high-speed rail, a ship, a yacht, an airplane, a spacecraft, etc. The projected windshield is not limited to the front windshield of the car, but can also be a transparent surface at other positions.

[0075] In combination with the above examples, the technical solutions involved in the present application can be directly embodied as hardware, a software module executed by a control unit, or a combination of the two, that is, one or more steps and / or one or more step combinations, which can correspond to various software modules of a computer program flow, or to various hardware modules, such as ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof. For the convenience of description, the above description is divided into various modules and described separately according to their functions. Of course, when implementing the present application, the functions of each module can be implemented in the same or more software and / or hardware.

[0076] Through the description of the above examples, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution involved in the present application can be essentially or partly contributed to the prior art in the form of a software product. The software is executed by a microcontroller unit and, depending on the required configuration, may include one or more microcontroller units of any type, including but not limited to a microcontroller unit, a microcontroller, a DSP (Digital Signal Processor) or any combination thereof. The software is stored in a memory, such as a volatile memory (such as a random access memory, etc.), a non-volatile memory (such as a read-only memory, a flash memory, etc.) or any combination thereof.

[0077] In summary, this application uses texture materials to simulate the real three-dimensional effect of actual objects, and the different states of the actual objects are called through materials containing different elements to form corresponding display elements in the projection area, so as to truly reflect the real-time environmental changes of the actual objects and the three-dimensional presentation under the environmental changes, providing users with a real sense of feedback. The three-dimensional display effect of this application does not rely on high-end hardware resources, is simple and efficient to run, and can save the hardware cost of implementation.

[0078] It should be understood that although this specification includes some examples, none of these examples contains only one independent technical solution, and this description of the specification is only for the purpose of clarity. A person skilled in the art should take the specification as a whole, and the technical solutions in each example can also be appropriately combined to form other implementation methods that can be understood by a person skilled in the art.

[0079] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present application. They are not intended to limit the scope of protection of the present application. All equivalent implementation methods or modifications that do not deviate from the teachings of the present application should be included in the scope of protection of the present application.

Claims

1. A display processing method, characterized in that: include: The display device is configured to display a first display element, wherein the first display element is used to represent a first object around the display device; In response to the first object being in the first state, the first material is called to perform mapping projection to realize the display of the first display element, and the first material has the stereoscopic effect of the first object in the first state.

2. The display processing method according to claim 1, characterized in that: The three-dimensional effect is presented through at least one psychological suggestive structure of occlusion relationship, shadow simulation, linear perspective, texture gradient, and near big and far small.

3. The display processing method according to claim 1, characterized in that: The first material has a three-dimensional effect of the first object in the first state, including: The first object is identified as a vehicle of a corresponding model, and the first material includes a three-dimensional outline and / or color of the corresponding model.

4. The display processing method according to claim 1, characterized in that: The first material has a three-dimensional effect of the first object in the first state, including: According to actual lighting conditions of the first object in the first state, the first material called has a first light and shadow effect simulating the surroundings of the first object.

5. The display processing method according to claim 1, characterized in that: The display device is further configured with a second display element, which is used to indicate a carrier on which the display device is located, and the display position relationship between the first display element and the second display element is configured according to an actual position relationship between the first object and the carrier.

6. The display processing method according to claim 5, characterized in that: The display position relationship between the first display element and the second display element is configured according to the actual position relationship between the first object and the carrier, including: In response to the first object moving away from or approaching the carrier, a display size and / or a display angle of the first display element is adjusted.

7. The display processing method according to claim 5, characterized in that: The display position relationship between the first display element and the second display element is configured according to the actual position relationship between the first object and the carrier, including: In response to the first object approaching the carrier, the display device is further configured with a third display element, and the third display element is used to represent warning prompt content.

8. A display device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the display processing method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the display processing method according to any one of claims 1 to 7 are implemented.

10. A means of transport, characterized in that: Includes the display device of claim 8 or the computer-readable storage medium of claim 9.

Images