Heads-up display processing methods, equipment, vehicles and storage media

Abstract

This application relates to the field of projection display technology, and more particularly to a head-up display (HUD) processing method, device, vehicle, and storage medium. This application divides the display area projected onto the windshield by the HUD display device into a ground-level oblique projection area and a non-ground-level orthographic projection area. Different distortion correction methods are applied to the oblique and orthographic projection areas respectively, accurately determining the impact of production and installation deviations on image distortion for each vehicle. This application can improve the distortion correction effect of multi-projection displays, and its implementation is simple, requiring no complex algorithms.

Description

Technical Field

[0001] This application relates to the field of projection display technology, and in particular to a head-up display processing method, device, vehicle, and storage medium. Background Technology

[0002] HUD (Head-Up Display) is a novel in-vehicle display method that utilizes reflection from the vehicle's windshield. Specifically, the HUD display device's optical engine emits display light, which is projected onto the windshield through corresponding optical lenses to create a virtual image. This virtual image is the projected image from the HUD display device. Besides displaying essential vehicle status information (such as vehicle speed and RPM), it can also combine navigation guidance information (such as turn arrows and trajectory lines) with the actual road through enhanced display. However, due to the complexity of light transmission, deviations in the optical lenses, windshield surface shape, and assembly can all cause distortion in the projected image. Image distortion correction needs to be performed based on the actual deviations of each vehicle. Furthermore, the diverse and complex nature of the displayed content also presents a significant challenge to image distortion correction. Summary of the Invention

[0003] The purpose of this application is to provide a head-up display processing method, device, vehicle, and storage medium, which solves the technical problem that the existing image distortion correction cannot effectively process both ground-mounted and non-ground-mounted HUD projections, and the implementation method is complicated.

[0004] To solve the above-mentioned technical problems, this application adopts the following technical solution:

[0005] In a first aspect, this application provides a head-up display processing method, including:

[0006] The HUD display device projects a display area onto the windshield of the vehicle.

[0007] The display area includes a first virtual image portion displayed on the ground and a second virtual image portion displayed on the front.

[0008] The first virtual image portion is calibrated for display based on the oblique projection standard diagram;

[0009] The second virtual image portion is calibrated for display based on the orthographic projection standard diagram.

[0010] According to the above description, in the display area of ​​the HUD display device, the first virtual image portion and the second virtual image portion are respectively calibrated using different methods to obtain the first distortion parameter and the second distortion parameter, which are used as different configuration parameters for image distortion correction.

[0011] In one alternative embodiment of the first aspect, the first virtual image portion is used to display navigation guidance information, and the second virtual image portion is used to display vehicle status information.

[0012] As described above, navigation guidance information is displayed on the ground using oblique projection, which improves the intuitiveness of navigation for users, while vehicle status information is displayed normally using orthographic projection, which helps drivers easily understand the vehicle's driving status.

[0013] In one alternative embodiment of the first aspect, the first virtual image portion and the second virtual image portion are located on the same display focal plane.

[0014] According to the above description, the first virtual image part achieves stereoscopic style correction of near-large and far-small based on the oblique projection standard map, and is located on the same display focal plane as the second virtual image part, which reduces the design requirements of optical hardware and has a relatively low cost.

[0015] In one alternative embodiment of the first aspect, the first virtual image portion is located above the second virtual image portion.

[0016] In one alternative embodiment of the first aspect, the projection position of the second virtual image portion is at the bottom position of the windshield.

[0017] In an optional embodiment of the first aspect, the step of display calibration of the first virtual image portion according to the oblique projection standard diagram includes:

[0018] The second virtual image portion is shielded, and the first virtual image portion and the oblique projection standard map are calibrated separately.

[0019] The step of displaying and calibrating the second virtual image portion according to the orthographic projection standard diagram includes:

[0020] The first virtual image portion is shielded, and the second virtual image portion is separately calibrated with the orthographic projection standard image.

[0021] According to the above description, the first distortion parameter generated by the calibration of the first virtual image portion will remove the second virtual image portion, and the second distortion parameter generated by the calibration of the second virtual image portion will remove the first virtual image portion.

[0022] In an optional embodiment of the first aspect, the step of display calibration of the first virtual image portion according to the oblique projection standard diagram includes:

[0023] The second virtual image portion is masked according to the sub-region mask, and the first virtual image portion is separately calibrated with the oblique projection standard map;

[0024] The step of displaying and calibrating the second virtual image portion according to the orthographic projection standard diagram includes:

[0025] The first virtual image portion is masked according to the sub-region mask, and the second virtual image portion is separately calibrated with the orthographic projection standard image.

[0026] Based on the above description, a sub-region mask matrix is ​​used to achieve fast masking of different regions.

[0027] In one alternative embodiment of the first aspect, the sub-region mask is defined according to a preset position of the first virtual image portion and the second virtual image portion.

[0028] In one alternative embodiment of the first aspect, the first virtual image portion and the second virtual image portion do not overlap.

[0029] In one alternative embodiment of the first aspect, the oblique projection standard map and / or orthographic projection standard map is a screen set outside the vehicle windshield, the screen having a standard dot matrix that meets the display requirements.

[0030] In one alternative embodiment of the first aspect, the oblique projection standard map and / or orthographic projection standard map are data files pre-existing in a database.

[0031] In one alternative embodiment of the first aspect, the oblique projection standard map is a standard ground-level indicator map where objects appear larger when closer and smaller when farther away.

[0032] Secondly, this application provides a head-up display processing method, including:

[0033] HUD display devices project a display area onto the windshield of a vehicle;

[0034] The display area includes a first virtual image portion displayed on the ground and a second virtual image portion displayed on the front.

[0035] The first virtual image portion is distorted according to the first distortion parameter displayed and calibrated.

[0036] The distortion of the second virtual image portion is corrected according to the second distortion parameters displayed and calibrated.

[0037] In one alternative embodiment of the second aspect, the first virtual image portion is used to display navigation guidance information, and the second virtual image portion is used to display vehicle status information.

[0038] In an alternative embodiment of the second aspect, the first virtual image portion and the second virtual image portion are located on the same display focal plane.

[0039] In one alternative embodiment of the second aspect, the first virtual image portion is located above the second virtual image portion.

[0040] In one alternative embodiment of the second aspect, the projection position of the second virtual image portion is at the bottom position of the windshield.

[0041] In an optional embodiment of the second aspect, the distortion correction of the first virtual image portion according to the displayed calibrated first distortion parameter includes:

[0042] The second virtual image portion is shielded, and the first virtual image portion is individually corrected based on the first distortion parameters;

[0043] The distortion correction of the second virtual image portion according to the second distortion parameter calibrated by the display includes:

[0044] The first virtual image portion is shielded, and the second virtual image portion is individually corrected according to the second distortion parameters.

[0045] In an optional embodiment of the second aspect, the distortion correction of the first virtual image portion according to the displayed calibrated first distortion parameter includes:

[0046] The second virtual image portion is masked according to the sub-region mask, and the first virtual image portion is individually corrected according to the first distortion parameter;

[0047] The distortion correction of the second virtual image portion according to the second distortion parameter calibrated by the display includes:

[0048] The first virtual image portion is masked according to the sub-region mask, and the second virtual image portion is individually corrected according to the second distortion parameter.

[0049] In one alternative embodiment of the second aspect, the sub-region mask is defined according to a preset position of the first virtual image portion and the second virtual image portion.

[0050] In one alternative embodiment of the second aspect, the first virtual image portion and the second virtual image portion do not overlap.

[0051] Thirdly, this application provides a head-up display processing device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the head-up display processing method described in the first or second aspect.

[0052] Fourthly, this application provides a vehicle including a HUD display device, the HUD display device having a cooperating memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the head-up display processing method described in the second aspect.

[0053] Fifthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the head-up display processing method described in the first or second aspect.

[0054] Compared with existing technologies, this application divides the display area projected onto the windshield by the HUD display device into a ground-level oblique projection area and a non-ground-level orthogonal projection area. Different distortion correction methods are applied to the oblique and orthogonal projection areas respectively, accurately determining the impact of production and installation deviations on distortion for each vehicle. This application can improve the distortion correction effect of multi-projection displays, and its implementation is simple, requiring no complex algorithms. Attached Figure Description

[0055] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description of the technical solution will be briefly introduced below. Obviously, the drawings described below are merely some examples recorded in this application, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0056] Figure 1 The following are schematic diagrams of HUD projection displays in some examples of this application.

[0057] Figure 2 The following are schematic diagrams of HUD display devices in some examples of this application.

[0058] Figure 3 The following are schematic diagrams of HUD projection display scenarios in some examples of this application.

[0059] Figure 4 The following are schematic diagrams of HUD projection display scenarios in some examples of this application.

[0060] Figure 5 The following are schematic diagrams of the HUD projection display area in some examples of this application.

[0061] Figure 6 The following are schematic diagrams illustrating non-ground-based navigation guidance in some examples of this application.

[0062] Figure 7 The following are schematic diagrams illustrating the ground-based display of navigation guidelines in some examples of this application.

[0063] Figure 8 This is a schematic diagram of ground-based display distortion calibration in some examples of this application.

[0064] Figure 9 This is a schematic diagram of ground-based display distortion calibration in some examples of this application.

[0065] Figure 10This is a schematic diagram of non-ground-based display distortion calibration in some examples of this application.

[0066] Figure 11 The following is a schematic diagram of the sub-region mask matrix in some examples of this application.

[0067] Figure 12 Here are flowcharts of header display processing methods in some examples of this application.

[0068] Figure 13 Here are flowcharts of header display processing methods in some examples of this application.

[0069] Figure 14 The following are schematic diagrams of head-up display processing devices in some examples of this application.

[0070] Figure 15 The following are schematic diagrams of vehicles in some examples of this application. Implementation

[0071] The present application will be described in detail below with reference to the accompanying drawings. However, the description is only a few examples recorded in the present application and does not limit the present application. Any changes in structure, method or function made by those skilled in the art based on these examples are included within the protection scope of the present application.

[0072] It should be noted that while the same labels or markers may be used in different examples, these do not represent an absolute structural or functional relationship. Furthermore, the use of terms such as "first," "second," etc., in the examples is merely for descriptive convenience and does not represent an absolute structural or functional distinction, nor should it be interpreted as indicating or implying relative importance or the number of corresponding objects. Unless otherwise specified, "at least one" in the description refers to one or more, and "more than one" refers to two or more.

[0073] Furthermore, when representing features, the character " / " can indicate an OR relationship between related objects. For example, "head-up display" or "head-up display" can be represented as "head-up display" or "head-up display". When representing operations, the character " / " can indicate a division relationship between related objects. For example, magnification M = L / P can be represented as L (virtual image size) divided by P (image source size). Moreover, the "AND / OR" in different examples is merely to describe the relationship between related objects. This relationship can include three cases. For example, a concave mirror and / or a convex mirror can be represented as a concave mirror alone, a convex mirror alone, or both concave and convex mirrors.

[0074] HUDs primarily utilize the principle of optical reflection, reflecting the image light to be displayed through a transparent surface into the human eye. The eye can then view the corresponding information by following the reverse direction of the light. Since a dedicated display screen is not required, this provides another convenient way to display information. In particular, placing a transparent surface (such as a windshield) within the driver's field of vision eliminates the need to shift their gaze away from the vehicle when viewing information, improving driving safety. In some examples, a HUD display device can be fixedly installed on the vehicle's center console. The HUD display device includes an optical engine and optical lenses. The backlight of the optical engine can be based on technologies such as LED (Light Emitting Diode) and laser for illumination, while the image source can be based on technologies such as LCD (Liquid Crystal Display), DMD (Digital Micromirror Devices), MEMS (Micro-Electro-Mechanical Systems) micromirrors, and LCOS (Liquid Crystal on Silicon). The optical engine, corresponding to the image source, displays the image (display content) that needs to be projected onto the imaging position and projects the display light of the image. Through the optical path planning of the optical lenses, the display light is finally reflected onto the windshield of the vehicle. The windshield, as a transparent surface that reflects the display light, can act as a display screen. The driver can directly observe the virtual image corresponding to the display content through the windshield. For example, the display content can be the vehicle's speed, navigation information, etc.

[0075] HUD displays, as a novel display method, are used in vehicles to provide in-car users with a more intuitive in-vehicle display experience. Currently, HUD displays are mainly divided into two types: aftermarket and pre-installed. Aftermarket HUDs, including C-HUDs (Combiner-HUDs), represent an early form of HUD display and can be purchased after the vehicle is purchased, based on display needs. C-HUDs are compact and can be placed directly on the center console without complex installation, but the display area is limited and the display format is relatively simple. With the development of HUD technology, pre-installed HUD displays are gradually becoming more common in vehicles, including W-HUDs (Windshield-HUDs) and AR-HUDs (Augmented Reality HUDs). Currently, W-HUDs are a representative example on the market, with the main body of the HUD display embedded inside the center console (see reference...). Figure 2This requires the HUD to be installed in a pre-reserved space before the vehicle leaves the factory. The vehicle's windshield needs to be compatible with the projection requirements of the HUD display device as a transparent surface for the projection display. However, W-HUD simply moves the projection display position to the windshield; its display format does not create a virtual-real combination with the actual scene outside the windshield. Correspondingly, AR-HUD emerged based on W-HUD. The size of the projection display area is increased, and more importantly, the projected content can be aligned with the road outside the windshield, providing a better user experience for users inside the vehicle, especially the driver.

[0076] like Figure 1 As shown, the HUD display device can include at least an optical engine 1, a first reflector 2, and a second reflector 3. In this example, the first reflector 2 and the second reflector 3 are optical lens groups that work together to achieve light path transmission. The first reflector 2 and the second reflector 3 in the optical lens group can project the display light projected by the optical engine 1 onto the windshield 4. In some examples, the first reflector 2 and the second reflector 3 can be set as concave mirrors, convex mirrors, etc., as needed. In some examples, the optical lens group can also achieve light path planning through one or more transmissive lenses. The optical engine 1 projects light to display corresponding information, and the light path is planned through the first reflector 2 and the second reflector 3. The light path can be customized in a small space and meet different projection display requirements. The display light projected by the optical engine 1 is finally projected onto the windshield 4 of the vehicle after multiple reflections by the first reflector 2 and the second reflector 3. The driver 6 inside the vehicle can see the virtual image 5 formed on the windshield 4 by the light projected by the optical engine 1, which may correspond to vehicle parameter information, etc. In some examples, the first reflector 2 and the second reflector 3 can be adjusted to a certain extent to change the projection position of the projected light on the windshield 4, adapting to different driver heights 6. It should be added that, corresponding astigmatic lenses can also be provided to adjust the imaging effect based on the characteristics of different optical engines. In some examples, the HUD display device can also include Fresnel lenses, waveguide optics, diffractive optics, holographic optics, tapered optical fibers, and other modules to achieve optical path planning and optimization.

[0077] like Figure 2 As shown, for HUD display devices integrated inside the vehicle's center console (such as W-HUD and AR-HUD as described above), in addition to the optical engine 1, the first reflector 2, and the second reflector 3, its main body is enveloped by a housing 101. The optical engine and optical lenses are housed within the internal space of the housing 101 and are stably fixed to the interior of the housing 101 by brackets or the like. (Refer to...) Figure 1The optical engine 1, the first reflector 2, and the second reflector 3 work together to achieve a certain optical path planning inside the housing 101, and finally project the display light out through the window 102 opened in the housing 101. When the HUD display device 100 is embedded in the center console of the car, the window 102 on the housing 101 faces the windshield above the center console. Accordingly, the display light projected from the window 102 will be reflected on the windshield to form a virtual image that can be seen by the human eye.

[0078] like Figure 3 As shown, the center console 10 can be equipped with... Figure 2 The HUD display device shown projects light onto the windshield 4 to form a virtual image, i.e., the display image corresponding to display area 50. For common W-HUD display formats, this mainly includes vehicle driving information and navigation information. The driver in the driver's seat can directly view the information on the windshield to understand the vehicle's current driving status and determine the driving route according to the corresponding navigation indicators. As shown in the figure, the vehicle is currently traveling at 60 kilometers per hour, and a right turn indication is displayed. Figure 3 In the example, because the navigation indicators do not align with the actual road view, in complex road environments, such as multiple intersections, drivers find it difficult to determine which intersection the indicator corresponds to. Drivers are then required to analyze the actual road conditions to determine the correct route, distracting their attention and resulting in a poor user experience. Therefore, AR-HUD utilizes superior projection display technology to precisely align the navigation guidance information reflected on the windshield with the indicated road. For example... Figure 4 As shown, in some examples, in addition to projecting vehicle status information onto the windshield 4 as normal, the navigation guidance information above the vehicle status information is displayed in a horizontal visual form in conjunction with the actual road. This gives the content in the display area 50 a realistic three-dimensional feel to the driver, and also keeps the direction of guidance completely consistent with the actual driving lane, making vehicle navigation more intuitive.

[0079] It should be noted that the projection effect of a HUD display device onto the windshield is largely related to the optical lenses and the windshield as described above. Different shapes of the optical lenses and the windshield may alter the field of view, sharpness, static distortion, and dynamic distortion of the projected display. For example... Figure 5As shown, the display area 50 contains the virtual image projected onto the windshield by the HUD display device, including a first virtual image portion 51 and a second virtual image portion 52. Both virtual images will change due to variations in the transmission of the displayed light. Taking static distortion as an example, non-conforming optical lenses or windshield surface shapes will cause distortion in the display of the aforementioned virtual image, resulting in a difference from the image displayed on the image source. Surface shape deviations can be caused by errors in the design of the optical lenses or windshield during manufacturing, or by errors in the installation of the optical lenses or windshield on the vehicle. These errors are largely unavoidable. In some examples, to reduce these errors, software can be used to pre-correct the image displayed on the image source, using the amount of pre-correction to offset the effect of surface shape deviations on display distortion. The amount of pre-correction can be determined by calibrating the projection effect of the HUD display device after installation on each vehicle to determine the pre-corrected distortion parameters. Since the HUD display device and windshield are relatively fixed after installation, their deviations will remain essentially constant. Therefore, by pre-correcting the image displayed for each projection based on the distortion parameters obtained from calibration, it can be ensured that the image displayed on the windshield conforms to the effect of normal observation. Accordingly, when the HUD display device projects an image, distortion correction processing is required for both the first virtual image portion 51 and the second virtual image portion 52.

[0080] For the human eye, objects of the same proportion appear larger when observed from a closer distance than when observed from a relatively farther distance. Therefore, in some examples, with... Figure 5 For example, in navigation applications, the first virtual image portion 51, to enhance the three-dimensionality of the display and simulate realistic visual effects, requires virtual guidance elements to exhibit visual changes from large to small as they move from near to far, indicating the driver's possible direction of travel in the next moment. It should be noted that in the context of this description, "far" and "near" refer to the side closer to the driver from their normal position in the cockpit, with the side further away from the driver considered "near" and the side further away considered "far." However, in methods of displaying images on the same focal plane, such as... Figure 6 As shown, traditional uniform-sized UI designs are square in shape, meaning the top and bottom widths are the same, failing to represent differences in distance. This only reflects the effect as seen from the front, leading to a mismatch between the viewer's perception and the actual road surface. It's only suitable for displaying information that isn't flush with the ground. Furthermore, traditional single-plane floating UIs also have a high probability of not fitting snugly to the actual road, reducing the overall experience for drivers viewing information on the windshield.

[0081] In some examples, to achieve a stereoscopic fit for the first virtual image portion, such as Figure 7As shown, relying on the experience of HMI (Human Machine Interface) engineers, they can create a series of image elements using projection to demonstrate the effect of objects appearing larger when closer and smaller when farther away. However, the HMI engineer's output is often a standard image element that is identical for each vehicle model. If the HUD in every vehicle is perfectly and consistently installed, the image element output by the HMI engineer can be used directly. However, considering the manufacturing and installation tolerances of the HUD display device itself and the windshield used for projection, the standard image output by the HMI engineer will change during virtual image projection due to the aforementioned errors. For example, the so-called ground-level image may shift to another position, or the ground-level image may become a sky-level image. In some examples, the HMI engineer can directly output a standard image, and the HUD display device can pre-correct the standard image based on calibrated distortion parameters to eliminate distortions caused by optical transmission.

[0082] In some examples, such as Figure 8As shown, when the HUD display device projects standard navigation guidance information 51 onto the windshield, on the one hand, the standard navigation indicators are all in a regular shape on a plane, which does not meet the actual ground-hugging requirements; on the other hand, the navigation indicators are distorted due to the influence of optical transmission. Therefore, it is necessary to calibrate it, on the one hand, to determine the parameter configuration required to convert the standard navigation information into a display shape where things appear larger when closer and smaller when farther away, and on the other hand, to determine the parameter configuration to compensate for the amount of image display distortion. Accordingly, when calibrating the HUD display device on a specific vehicle, a calibration screen 60 can be set outside the windshield 4. The screen 60 can be a physical signboard or road marking, or it can be a display screen. The screen 60 is provided with an oblique projection standard image 61 that matches the projection position of the HUD display device. The oblique projection standard image 61 corresponds to the standard navigation guidance information and contains a standard image that has ground-hugging capability with the actual road, and is in a normal, undistorted display shape. Optionally, it can be a standard dot matrix composed of key pixels. The corresponding head-up display processing device (such as a distortion calibration device) can use a camera to capture the positional relationship between the navigation guidance information 51 projected onto the windshield (at this time, the navigation guidance information displayed on the windshield already has a certain distortion) and the oblique projection standard image 61 on the screen. Accordingly, a set of ground-fitting matrices containing all pixels of the image corresponding to the navigation guidance information 51 can be generated, which are the distortion parameters used for ground-fitting image correction. The distortion parameters can be used to correct the image distortion of the image source before projection when the HUD display device is actually projected. That is, pre-correction is performed based on the positional offset relationship of each pixel during projection display, determined by the distortion parameters, so that the image during actual projection display (such as navigation indicators) satisfies the ground-fitting shape of near objects being larger and far objects being smaller, without distortion.

[0083] In some examples, such as Figure 9 As shown, distortion calibration can also be performed without using... Figure 8In the example, the screen method allows the controller used for calibration to pre-store a standard oblique projection image, where objects appear larger when closer and smaller when farther away. This pre-stored oblique projection image is similar to the one set on the screen, representing a standard ground-level pattern reflecting the near-large, far-small appearance of a ground-level display. When the camera used for calibration scans and acquires the navigation guidance information 51 of the HUD display device on the windshield, the virtual image corresponding to this information is essentially rectangular. Due to distortion, the central portion slightly protrudes upwards, and this distortion becomes more pronounced in many examples. During calibration, the navigation guidance information 51 is compared and analyzed with the pre-stored oblique projection image 61. Specifically, to ensure the accuracy of the camera scan, key pixels on the image corresponding to the navigation guidance information 51 are acquired. Then, based on the positions of these key pixels, interpolation or polynomial fitting is used to deduce the positions of other pixels. The deviations are then calculated between these positions and the corresponding pixel positions on the oblique projection image 61, and distortion parameters for image distortion correction are obtained accordingly. When the HUD display device in the vehicle performs image distortion correction before projection display, it can correct the image to be displayed according to the calibrated distortion parameters, and then display it on the image source of the HUD display device. Optionally, it can be corrected separately and then spliced ​​together with other images to be displayed and projected onto a designated position on the windshield.

[0084] In some examples, such as Figure 10 As shown, Figure 5 Since the second virtual image portion 52 uses a frontal information display method, it is not necessary to correct the rectangular image to a trapezoidal image (i.e., a stereoscopic shape where objects appear larger when closer and smaller when farther away). Only the distortion caused by optical transmission needs to be corrected to a normal display outline. Accordingly, a calibrated camera can be used to compare and analyze the virtual image 52 projected onto the windshield with a pre-stored orthographic projection standard image 62. The orthographic projection standard image 62 is a regular image when the image is displayed frontally, that is, the shape it should have in a planar display state when viewed directly by the human eye. Correspondingly, the orthographic projection standard image 62 can be the original image normally displayed on the image source, while the virtual image 52 obtained by the camera scanning the windshield can be the display image formed by the display light projected from the image source onto the orthographic projection standard image 62, passing through optical lenses, and finally projected onto the windshield. Optionally, the orthographic projection standard image 62 is composed of several key pixels arranged in a regular horizontal and vertical pattern, and the virtual image 52 on the windshield is also composed of several corresponding key pixels. Due to the deviation caused by optical projection, the key pixels have shifted in position, that is, the corresponding image is distorted. Therefore, it is necessary to calculate the specific distortion parameters as the basis for image distortion correction during subsequent projection display. (Refer to...) Figure 9The example describes how the distortion parameters can be obtained by analyzing the display position deviation between the virtual image 52 and the orthographically projected standard image 62. Optionally, interpolation or polynomial fitting can be used to determine the pixels other than the key pixels, and the distortion parameters corresponding to the deviation relationship can be calculated, ultimately forming a complete non-ground-attached matrix, which is the distortion parameter used for non-ground-attached image correction. (Refer to...) Figure 8 In the example, the orthographic projection standard image used for calibration can be directly set on a screen, which is fixed at the corresponding position outside the windshield during calibration. Optionally, the orthographic projection standard image can be a standard dot matrix or an image such as vehicle status information displayed on the front of the HUD display device. The calibration process involves direct comparison and adjustment via camera scanning. In some examples, once the calibration obtains the distortion parameters for non-ground-hugging image correction, image distortion correction can be performed on any image to be displayed before projection onto the HUD display device. That is, each display pixel is pre-corrected to its corresponding position based on its possible offset value. After optical transmission, the display content at the pre-corrected position can be restored to the position of the pixels that should be displayed normally.

[0085] like Figure 5As shown, the display area 50 includes a first virtual image portion 51 and a second virtual image portion 52. To meet the needs of actual projection display, the first virtual image portion 51 and the second virtual image portion 52 do not display the same type of information. Optionally, there is a certain separation between the first virtual image portion 51 and the second virtual image portion 52 to ensure that they do not overlap. In some examples, the first virtual image portion 51 is used to display navigation guidance information or other information that needs to be displayed close to the ground. The information displayed close to the ground is displayed using oblique projection to create a stereoscopic shape that fits the ground. The projected virtual image appears to the human eye as content that is already present on the ground itself; for example, the virtual image appears to be drawn directly on the actual road. The second virtual image portion 52 is used to display vehicle status information or other information that needs to be displayed outside the ground. The information displayed outside the ground is displayed in the traditional screen display manner. It does not have the stereoscopic feel of simulating actual space but is in a flat state, suitable for viewing data-related information, and is simpler and clearer. In this example, the first virtual image portion 51 is positioned above the second virtual image portion 52. Optionally, the second virtual image portion 52 is positioned at the bottom of the windshield. This allows for a clear division of functions in the information display layout. The navigation guidance information of the first virtual image portion 51 can be effectively coordinated with the road ahead of the vehicle, while the vehicle status information of the bottom second virtual image portion 52 can provide real-time feedback on the vehicle's current speed, RPM, and other information, which the driver can clearly view. In some examples, the first virtual image portion 51 and the second virtual image portion 52 can also have other positional relationships, such as the first virtual image portion 51 being in the middle of the second virtual image portion 52, with the second virtual image portion 52 surrounding the first virtual image portion 51.

[0086] To reduce the hardware cost of the HUD display device, the first virtual image portion 51 and the second virtual image portion 52 are located on the same display focal plane. The display focal plane refers to the image containing the virtual image at a relatively fixed distance from the human eye. This is related to the optical path and lens focal length in the HUD display device. Different virtual image distances on the display focal plane can provide drivers with different senses of depth. In this example, the first virtual image portion 51 and the second virtual image portion 52 need to present different visual effects on the same display focal plane. As mentioned above, the first virtual image portion 51 needs to present a stereoscopic visual effect of objects appearing larger when closer and smaller when farther away, while the second virtual image portion 52 needs to present a planar visual effect of a frontal display. (Referring to...) Figures 8-10For example, the distortion calibration and image distortion correction achieved by the two different visual effects are completely different. If a uniform method is used to calibrate or correct the first virtual image portion 51 and the second virtual image portion 52, one of the virtual image portions will be projected abnormally. In some examples, the first virtual image portion 51 and the second virtual image portion 52 are subjected to two distortion calibration processes. One process forms a ground-fit matrix for the area where the ground-fit image is located, and the other process forms a non-ground-fit matrix for the area where the non-ground-fit image is located. Optionally, the two distortion calibration processes can be processed in parallel. During image distortion correction, the source image to be displayed will undergo two image distortion correction processes. One process corrects the distortion of the ground-fit image portion of the source image based on the ground-fit matrix, and the other process corrects the distortion of the non-ground-fit image portion of the source image based on the non-ground-fit matrix. Optionally, the two image distortion correction processes can be processed in parallel, as will be detailed below.

[0087] In some examples, since the first virtual image portion 51 and the second virtual image portion 52 need to be corrected separately, it is necessary to quickly identify the region where the corresponding image is located from the display area 50. For example... Figure 11 As shown, a sub-region mask is predefined. The sub-region mask represents the first virtual image portion 51 and the second virtual image portion 52 using different encodings within a matrix. This matrix can represent each pixel in the display area 50. In this example, the sub-region mask matrix can use all zeros to represent all pixels in the first virtual image portion 51 and all ones to represent all pixels in the second virtual image portion 52. Correspondingly, the image to be displayed in the display area 50 can be ORed with the sub-region mask matrix. This preserves the pixel content of the first virtual image portion 51 (selecting the first virtual image portion 51), while the second virtual image portion 52 is set to all ones and can be removed or ignored. Similarly, the image to be displayed in the display area 50 can be ANDed with the sub-region mask matrix. This preserves the pixel content of the second virtual image portion 52 (selecting the second virtual image portion 52), while the first virtual image portion 51 is set to all zeros and can be removed or ignored. Once the first virtual image portion 51 and the second virtual image portion 52 are selected, distortion calibration and image distortion correction can be performed respectively. In some examples, the sub-region mask is configured according to a pre-planned display area distribution, and can then be reused in later correction processes. The regions corresponding to the first and second virtual image portions can also be marked with encodings different from those in the above examples, such as swapping regions with all zeros and all ones.

[0088] like Figure 12 As shown, in some examples, the head-up display processing method specifically includes:

[0089] Step S1201: Project a display area onto the vehicle's windshield using a HUD display device. The display area includes a first virtual image portion displayed on the ground and a second virtual image portion displayed from the front. In this example, to determine the distortion parameters for the HUD display device's projection display based on the deviation generated by each vehicle, a specified image needs to be projected onto the windshield using a HUD display device installed in the vehicle, forming a corresponding display area on the windshield. To meet actual display requirements, the display area includes a first virtual image portion and a second virtual image portion for actual display. The first and second virtual image portions respectively display different types of calibration images. For example, the first virtual image portion projects standard navigation guidance information from the HUD display device, while the second virtual image portion projects a standard orthographic dot matrix image from the HUD display device.

[0090] Step S1202: The first virtual image portion is calibrated according to the oblique projection standard diagram. After the HUD display device on the vehicle projects the image onto the windshield in step S1201, the projected display result needs to be scanned using a camera or similar device and then compared with the projection standard diagram. For details, please refer to... Figure 8 , Figure 9 For example, selecting the first virtual image portion and disabling the comparison processing of the second virtual image portion, the projection display result of the first virtual image portion is calibrated using a standard oblique projection image to obtain a first distortion parameter. This first distortion parameter represents the oblique projection display offset caused by optical influences on the HUD display device in a specific vehicle. Specifically, it can include the degree of pixel position offset in all oblique projection displays. The offset value can be used to pre-correct the image to compensate for the resulting offset. See [link / reference] for details. Figure 11 The example uses a sub-region mask that defines the first virtual image portion and the second virtual image portion to achieve masking.

[0091] Step S1203: Perform display calibration on the second virtual image portion according to the orthographic projection standard diagram. (Refer to...) Figure 10 For example, since the second virtual image is displayed in a non-ground-mounted orthographic projection mode, an orthographic standard image is used to project the corresponding area of ​​the second virtual image onto the windshield. The projection result is obtained by scanning with a camera, and then compared and analyzed using the orthographic standard image. Specifically, the second virtual image is selected, and the comparison processing of the first virtual image is disabled (see [reference needed] for details). Figure 11(Example) The projection display result of the second virtual image is calibrated using a standard orthographic projection image to obtain a second distortion parameter. The second distortion parameter represents the orthographic projection display offset caused by optical influences on the HUD display device in a specific vehicle. Specifically, it can include the degree of pixel position offset of all orthographic projection displays. The offset value can be used to pre-correct the image to offset the degree of offset. It should be noted that steps S1202 and S1203 are not absolutely sequential; the two steps can also be processed simultaneously after camera scanning to obtain the first distortion parameter and the second distortion parameter. The corresponding first distortion parameter and second distortion parameter can be sent to the corresponding HUD display device for storage and used for image distortion correction during daily projection display.

[0092] like Figure 13 As shown, in some examples, the head-up display processing method specifically includes:

[0093] Step S1301: The HUD display device projects a display area onto the vehicle's windshield. This display area includes a first virtual image portion displayed close to the ground and a second virtual image portion displayed from the front. In this example, while the vehicle is in motion, the driver can view the HUD display device integrated into the vehicle's center console (see details in [reference needed]). Figure 1 , Figure 2 View the projected information on the windshield. (Refer to...) Figure 4 As shown, a corresponding display area is formed on the windshield, including a first virtual image portion and a second virtual image portion. The first virtual image portion can display navigation guidance information, and the second virtual image portion can display vehicle status information. Due to the limitations of optical projection imaging, if the first and second virtual image portions are not corrected for image distortion, a distorted display image will appear on the windshield. The first virtual image portion needs to align the navigation guidance information with the ground; therefore, without proper correction, it will not only exhibit general image shape distortion but also fail to match the actual road outside the windshield due to the lack of stereoscopic shape processing. To meet actual display requirements, [the following can be done]: Figure 12 The first and second distortion parameters obtained from the distortion calibration in this example are used to correct image distortion so that the image projected onto the windshield, regardless of its position, meets the corresponding display requirements.

[0094] Step S1302: Perform distortion correction on the first virtual image portion according to the first distortion parameter specified in the display calibration. For a display area having a first virtual image portion and a second virtual image portion, image distortion correction is only performed on the first virtual image portion according to the first distortion parameter. That is, the display pixels of the first virtual image portion can be pre-corrected based on the offset value of each pixel in the first distortion parameter to adapt to the effects of optical distortion. Optionally, it can utilize... Figure 11 The sub-region mask defined in the code is used to block the second virtual image portion, and the first virtual image portion of the image to be displayed is selected for image distortion correction.

[0095] Step S1303: Correct the distortion of the second virtual image portion according to the second distortion parameter specified in the display calibration. Compared to step S1302, the first virtual image portion can be masked by a sub-region mask, the second virtual image portion can be selected, and the second distortion parameter can be used to correct the image distortion of the second virtual image portion to be displayed. It should be noted that steps S1302 and S1303 are not absolutely sequential; the two steps can simultaneously acquire the corrected image, which is then projected onto the windshield by the HUD display device to achieve a normal display effect.

[0096] In some examples, such as Figure 14 As shown, the head-up display (HUD) processing device includes a processor 1401, a memory 1402, an input device 1403, and an output device 1404. Specifically, it can be a computer device or a standalone host computer control device. In some examples, the HUD display device itself can be used to implement image distortion correction. The input device 1403 can include a mouse, keyboard, etc. The HUD processing device can receive input control commands and data through the input device 1403, such as selecting a specific standard dot matrix pattern. The output device 1404 can include a scanning camera, etc. The HUD processing device can output corresponding commands or data to the output device 1404, such as controlling the scanning camera to acquire the projection display result on the windshield during distortion calibration. The memory 1402 stores a computer program running on the processor 1401. When the processor 1401 executes the computer program, it implements the aforementioned HUD processing method.

[0097] In some examples, such as Figure 15As shown, vehicles can employ the aforementioned head-up display processing method or integrate the aforementioned head-up display processing equipment. The image to be displayed is projected onto the vehicle's windshield. Viewed from inside the cockpit, the corresponding display area on the windshield shows the vehicle's status information and navigation guidance information superimposed on the real-world scene outside the windshield. Because image distortion is corrected before projection, the projection effect meets viewing requirements, and the oblique projection also meets the requirement of being close to the ground. Furthermore, drivers do not need to look down at the traditional instrument panel while driving, improving driving safety. It should be noted that vehicles are not limited to this... Figure 15 The vehicles shown can also include buses, trucks, excavators, motorcycles, trains, high-speed trains, ships, yachts, airplanes, spacecraft, etc. The projected windshield is not limited to the windshield of a car; it can also be a transparent surface in other locations.

[0098] In some examples, a computer-readable storage medium stores a computer program that, when executed by a processor, implements the head-up display processing method described above.

[0099] Based on the above examples, the technical solutions involved in this application can be directly embodied in hardware, software modules executed by a control unit, or a combination of both, i.e., one or more steps and / or combinations of one or more steps. These can correspond to various software modules in a computer program flow, or to various hardware modules, such as ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any suitable combination thereof. For ease of description, the above description divides the functions into various modules and describes them separately. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware components.

[0100] Through the above description of examples, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution involved in this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This software is executed by a microcontroller unit and, depending on the required configuration, can include one or more microcontroller units of any type, including but not limited to microcontroller 8, microcontrollers, DSPs (Digital Signal Processors), or any combination thereof. The software is stored in memory, such as volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory, flash memory), or any combination thereof.

[0101] In summary, this application divides the display area projected onto the windshield by the HUD display device into a ground-level oblique projection area and a non-ground-level orthographic projection area. Different distortion correction methods are applied to the oblique and orthographic projection areas respectively, accurately determining the impact of production and installation deviations on image distortion for each vehicle. This application can improve the distortion correction effect of multi-projection displays, and its implementation is simple, requiring no complex algorithms.

[0102] It should be understood that although this specification includes some examples, none of these examples constitutes a single, independent technical solution. This descriptive style is merely for clarity. Those skilled in the art should consider this specification as a whole, and the technical solutions in the examples can be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0103] The detailed descriptions listed above are merely specific descriptions of feasible implementation methods of this application and are not intended to limit the scope of protection of this application. All equivalent implementation methods or modifications that do not depart from the teachings of this application should be included within the scope of protection of this application.

Claims

1. A head-up display processing method, characterized in that, include: The HUD display device projects a display area onto the windshield of the vehicle. The display area includes a first virtual image portion displayed close to the ground and a second virtual image portion displayed on the front. The first virtual image portion and the second virtual image portion are located on the same display focal plane, and the first virtual image portion and the second virtual image portion do not overlap. The first virtual image portion is calibrated according to the oblique projection standard diagram to achieve three-dimensional style correction of near-large and far-small based on the oblique projection standard diagram; The second virtual image portion is calibrated for display based on the orthographic projection standard diagram.

2. The head-up display processing method according to claim 1, characterized in that, The first virtual image portion is used to display navigation guidance information to indicate the driver's vehicle's direction of travel in the next moment through visual changes from near to far and from large to small. The second virtual image portion is used to display vehicle status information.

3. The head-up display processing method according to claim 1, characterized in that, The projection position of the second virtual image portion is at the bottom of the windshield, and the first virtual image portion is located above the second virtual image portion.

4. The head-up display processing method according to claim 1, characterized in that, The step of displaying and calibrating the first virtual image portion according to the oblique projection standard diagram includes: The second virtual image portion is masked according to the sub-region mask, and the first virtual image portion is separately calibrated with the oblique projection standard map; The step of displaying and calibrating the second virtual image portion according to the orthographic projection standard diagram includes: The first virtual image portion is masked according to the sub-region mask, and the second virtual image portion is separately calibrated with the orthographic projection standard image.

5. The head-up display processing method according to claim 1, characterized in that, The oblique projection standard map contains a standard image that is close to the actual road and is in a normal, undistorted display shape, with the near side appearing larger and the far side smaller, forming a close-to-the-ground pattern.

6. A head-up display processing method, characterized in that, include: HUD display devices project a display area onto the windshield of a vehicle; The display area includes a first virtual image portion displayed close to the ground and a second virtual image portion displayed on the front. The first virtual image portion and the second virtual image portion are located on the same display focal plane, and the first virtual image portion and the second virtual image portion do not overlap. The first virtual image portion is distorted according to the first distortion parameter calibrated by the display, so as to present a stereoscopic visual effect of near-large and far-small on the same display focal plane; The second virtual image portion is distorted according to the second distortion parameter specified by the display, so as to present a planar visual effect of front display on the same display focal plane.

7. The head-up display processing method according to claim 6, characterized in that, The distortion correction of the first virtual image portion according to the first distortion parameter calibrated by the display includes: The second virtual image portion is masked according to the sub-region mask, and the first virtual image portion is individually corrected according to the first distortion parameter; The distortion correction of the second virtual image portion according to the second distortion parameter calibrated by the display includes: The first virtual image portion is masked according to the sub-region mask, and the second virtual image portion is individually corrected according to the second distortion parameter.

8. A head-up display processing device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the head-up display processing method according to any one of claims 1-7.

9. A means of transportation, characterized in that, The device includes a head-up display (HUD) device having a cooperating memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the head-up display processing method according to any one of claims 6-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the head-up display processing method according to any one of claims 1-7.

Images