Brightness adjusting method of head-up display and head-up display system

Abstract

The invention provides a brightness adjusting method of a head-up display and a head-up display system. The head-up display comprises a backlight source and a display screen, the display screen is configured to generate a projection display image, and the brightness adjusting method comprises the following steps: acquiring ambient illuminance of a virtual image background area through an illuminance sensor; adjusting the brightness of the backlight source according to the ambient illuminance; acquiring a first image of the virtual image background area through the camera; determining relative brightness distribution of a virtual image background area according to the first image; and correcting the color value of each pixel point of the projection display image according to the relative brightness distribution of the virtual image background area. The brightness of the backlight source and the color value of each pixel point of the projection display image are adjusted, pixel-level adjustment of the brightness of the virtual image is achieved, and it is guaranteed that a driver can clearly see all information in the virtual image in a complex illumination environment.

Description

Technical Field

[0001] The present invention generally relates to the technical field of brightness adjustment of a head-up display, and in particular to a brightness adjustment method of a head-up display and a head-up display system. Background Art

[0002] Head-up display (HUD), also known as head-up display, was originally designed as a display system for military fighter jets to help pilots view key information at any time during flight. In recent years, automotive head-up display technology has gradually matured, and can project important driving information such as speed and navigation directly onto the windshield in front of the driver. This design allows drivers to obtain the required information without lowering their heads, effectively reducing cognitive load and significantly improving driving safety.

[0003] During the driving process, the change of sunlight angle, entering and exiting shadows or tunnels / basements will also cause the driver's perceived ambient brightness to change continuously. If the brightness of the virtual image of the head-up display is too low, the driver may have difficulty seeing the information; if the brightness is too high, it may cause glare. Therefore, in order to ensure that the head-up display can provide a comfortable visual experience in different environments, the virtual image brightness of the head-up display needs to be automatically adjusted according to the ambient brightness.

[0004] At present, the brightness adjustment of the head-up display generally relies on the illumination sensor to detect the average brightness of the environment in front of the vehicle, and accordingly adjusts the virtual image brightness of the head-up display as a whole (for example, by controlling the luminous power of the backlight module for overall adjustment). This adjustment method causes the virtual image brightness of the head-up display to be uniformly distributed, which is difficult to adapt to the complex and changeable virtual image background area. Specifically, the virtual image background area may include objects of various colors and brightness such as roads, markings, vehicles, trees, buildings and shadows. When the brightness of the virtual image background area is unevenly distributed, the virtual image contrast of the head-up display is low in the high-brightness background area, making the virtual image difficult to recognize; in the low-brightness background area, the virtual image contrast is high and may be too prominent. This mismatch in brightness and contrast can cause problems in certain situations, such as when a vehicle is about to drive into the shadow of a building. Figure 1 As shown, the virtual image of the head-up display covers the boundary of the shadow. The virtual image icons located in the shadow will appear dazzling due to the high brightness, and the virtual image icons located in the illuminated area will be difficult to see due to the low brightness, which affects the driver's judgment and safety.

[0005] The contents of the background technology section are merely the technologies known to the inventors and do not necessarily represent the prior art in the field. Summary of the invention

[0006] In view of one or more defects in the prior art, the present invention provides a brightness adjustment method for a head-up display, wherein the head-up display comprises a backlight source and a display screen, wherein the display screen is configured to generate a projected display image, and the brightness adjustment method comprises:

[0007] The ambient light intensity of the virtual image background area is obtained by using a light intensity sensor;

[0008] According to the ambient light illumination, adjusting the brightness of the backlight source;

[0009] Acquire a first image of the virtual image background area through a camera;

[0010] Determine the relative brightness distribution of the virtual image background area according to the first image; and

[0011] According to the relative brightness distribution of the virtual image background area, the color value of each pixel of the projected display image is corrected.

[0012] According to one aspect of the present invention, the step of adjusting the brightness of the backlight source according to the ambient light illumination of the virtual image background area comprises:

[0013] Determining the brightness of the backlight source according to the ambient light illumination and the first relationship;

[0014] Determining the input power of the backlight light source according to the brightness of the backlight light source and the second relationship; and

[0015] driving the backlight source with the input power to adjust the brightness of the backlight source;

[0016] The first relationship is the corresponding relationship between the ambient light illumination and the brightness of the backlight light source; and the second relationship is the corresponding relationship between the brightness of the backlight light source and the input power of the backlight light source.

[0017] According to one aspect of the present invention, the light illumination sensor acquires the ambient light illumination of the virtual image background area at a preset time interval;

[0018] The step of adjusting the brightness of the backlight source according to the ambient light illumination of the virtual image background area comprises:

[0019] Determine the effective ambient light intensity at the current moment according to the ambient light intensity obtained at multiple moments;

[0020] Determining the brightness of the backlight source according to the effective ambient light illumination and a third relationship;

[0021] Determining the input power of the backlight light source according to the brightness of the backlight light source and the second relationship; and

[0022] driving the backlight source with the input power to adjust the brightness of the backlight source;

[0023] The second relationship is the corresponding relationship between the brightness of the backlight light source and the input power of the backlight light source; and the third relationship is the corresponding relationship between the effective ambient light illumination and the brightness of the backlight light source.

[0024] According to one aspect of the present invention, the step of determining the relative brightness distribution of the virtual image background area according to the first image comprises:

[0025] generating a second image according to the first image, wherein the second image is an image of a virtual image background area in an eye coordinate system;

[0026] Determine, according to the second image, the brightness of each pixel of the second image;

[0027] Determining an average brightness of the second image according to the brightness of each pixel of the second image; and

[0028] According to the brightness of each pixel of the second image and the average brightness of the second image, the relative brightness of each pixel of the second image is determined as the relative brightness distribution of the virtual image background area.

[0029] According to one aspect of the present invention, the step of determining the brightness of a pixel point of the second image comprises:

[0030] Determining the gamma space brightness of a pixel in the second image based on the grayscale value of the pixel; and

[0031] Based on the gamma space brightness of the pixel, the linear space brightness of the pixel is determined as the brightness of the pixel.

[0032] According to one aspect of the present invention, the average brightness of the second image is obtained by performing weighted average calculation on the brightness of each pixel of the second image.

[0033] According to one aspect of the present invention, the projection display image adopts RGBA color format, and the step of correcting the color value of each pixel of the projection display image includes: correcting the image transparency of each pixel of the projection display image.

[0034] According to one aspect of the present invention, the image transparency of each pixel of the projected display image is corrected by the following formula:

[0035]

[0036] in, is the image transparency of the pixel with coordinates (u, v) in the corrected projection display image, A hud img(u,v) is the image transparency of the pixel with coordinates (u,v) in the projection display image before correction, B′ background (u, v) is the relative brightness of the pixel with coordinates (u, v) in the second image, and γ is the gamma correction coefficient.

[0037] According to one aspect of the present invention, the projection display image adopts the HSL color format, and the step of correcting the color value of each pixel of the projection display image includes: correcting the brightness of each pixel of the projection display image.

[0038] According to one aspect of the present invention, the brightness of each pixel of the projected display image is corrected by the following formula:

[0039]

[0040] in, is the brightness of the pixel with coordinates (u, v) in the corrected projection display image, L hud img(u,v) is the brightness of the pixel with coordinates (u,v) in the projection display image before correction, B′ background (u, v) is the relative brightness of the pixel with coordinates (u, v) in the second image, and γ is the gamma correction coefficient.

[0041] The present invention also provides a head-up display system, comprising:

[0042] a display screen configured to generate a projected display image;

[0043] a backlight source configured to project light onto the display screen;

[0044] an illumination sensor configured to detect an ambient light illumination of a virtual image background area;

[0045] A camera configured to capture a first image of a virtual image background area;

[0046] A control unit is coupled to the display screen, the backlight source, the illuminance sensor, and the camera, respectively. The control unit is configured to adjust the brightness of the backlight source according to the ambient light illumination. The control unit is also configured to determine the relative brightness distribution of the virtual image background area according to the first image, and correct the color value of each pixel of the projected display image according to the relative brightness distribution of the virtual image background area.

[0047] According to one aspect of the present invention, the control unit is configured as:

[0048] Determining the brightness of the backlight source according to the ambient light illumination and the first relationship;

[0049] Determining the input power of the backlight light source according to the brightness of the backlight light source and the second relationship;

[0050] driving the backlight source with the input power to adjust the brightness of the backlight source;

[0051] The first relationship is the corresponding relationship between the ambient light illumination and the brightness of the backlight light source; and the second relationship is the corresponding relationship between the brightness of the backlight light source and the input power of the backlight light source.

[0052] According to one aspect of the present invention, the light illumination sensor is configured to obtain the ambient light illumination of the virtual image background area at a preset time interval; and the control unit is configured to:

[0053] The step of adjusting the brightness of the backlight source according to the ambient light illumination of the virtual image background area comprises:

[0054] Determine the effective ambient light intensity at the current moment according to the ambient light intensity obtained at multiple moments;

[0055] Determining the brightness of the backlight source according to the effective ambient light illumination and a third relationship;

[0056] Determining the input power of the backlight light source according to the brightness of the backlight light source and the second relationship; and

[0057] driving the backlight source with the input power to adjust the brightness of the backlight source;

[0058] The second relationship is the corresponding relationship between the brightness of the backlight light source and the input power of the backlight light source; and the third relationship is the corresponding relationship between the effective ambient light illumination and the brightness of the backlight light source.

[0059] According to one aspect of the present invention, the control unit is configured as:

[0060] generating a second image according to the first image, wherein the second image is an image of a virtual image background area in an eye coordinate system;

[0061] Determine, according to the second image, the brightness of each pixel of the second image;

[0062] Determining an average brightness of the second image according to the brightness of each pixel of the second image;

[0063] According to the brightness of each pixel of the second image and the average brightness of the second image, the relative brightness of each pixel of the second image is determined respectively as the relative brightness distribution of the virtual image background area.

[0064] According to one aspect of the present invention, the control unit is configured as:

[0065] determining a gamma space brightness of a pixel in the second image based on a grayscale value of the pixel;

[0066] Based on the gamma space brightness of the pixel, the linear space brightness of the pixel is determined as the brightness of the pixel.

[0067] According to one aspect of the present invention, the control unit is configured to: perform weighted average calculation on the brightness of each pixel of the second image to obtain an average brightness of the second image.

[0068] According to one aspect of the present invention, the projected display image adopts an RGBA color format, and the control unit is configured to correct the image transparency of each pixel of the projected display image according to the relative brightness distribution of the virtual image background area.

[0069] According to one aspect of the present invention, the projected display image adopts the HSL color format, and the control unit is configured to correct the brightness of each pixel of the projected display image according to the relative brightness distribution of the virtual image background area.

[0070] Compared with the prior art, the embodiments of the present invention provide a head-up display brightness adjustment method and a head-up display system, which implements pixel-level adjustment of virtual image brightness by adjusting the brightness of the backlight source and the color value of each pixel of the projected display image, thereby ensuring that the driver can clearly see all information in the virtual image even in a complex lighting environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0072] Figure 1 A schematic diagram of an application scenario of an existing head-up display is shown;

[0073] Figure 2 A schematic diagram of a head-up display according to an embodiment of the present invention is shown;

[0074] Figure 3 A schematic diagram showing a brightness adjustment method of a head-up display according to an embodiment of the present invention is shown;

[0075] Figure 4 A schematic diagram of an application scenario of a head-up display according to an embodiment of the present invention is shown;

[0076] Figure 5 A schematic diagram showing a virtual image background area according to an embodiment of the present invention is shown;

[0077] Figure 6 A schematic diagram showing a first image and an image of a virtual image background area observed by a user.

[0078] Figure 7 A schematic diagram of a head-up display system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0079] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0080] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0081] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0082] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature is lower in level than the second feature.

[0083] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0084] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0085] Figure 2 FIG. 1 shows a schematic diagram of a head-up display 100 according to an embodiment of the present invention. Figure 2 As shown, the head-up display 100 may include a backlight source 110 and a display screen 120. The backlight source 110 is configured to project light onto the display screen 120, and its brightness can be adjusted as needed. The display screen 120 is configured to generate a projected display image. For example, the display screen 120 may be a thin film transistor liquid crystal display (TFT-LCD). The thin film transistor liquid crystal display can generate a projected display image, and can control the arrangement of liquid crystal molecules by controlling the voltage of each pixel, thereby controlling the passage of light to achieve the projection display of the projected display image. In addition, the thin film transistor liquid crystal display also has a color management function, which can adjust the grayscale or color of the projected display image at the pixel level to ensure the richness and accuracy of the display effect. The head-up display 100 may also include one or more reflectors 130 for reflecting the projection of the projected display image to the outside of the head-up display 100. Those skilled in the art will understand that the actual brightness of the virtual image (i.e., the virtual image brightness B hudimg) is affected by the brightness of the backlight source 110 (or the input power P backlight ) and the color value Gray of each pixel of the projected image TFT The common influence of img(u,v) (B hud img=f(P backlight ,Gray TFT img(u,v))). Therefore, the brightness of the backlight source 110 and the color value of each pixel of the projected display image can be adjusted to achieve pixel-level adjustment of the virtual image brightness, ensuring that the driver can clearly see all the information in the virtual image even in a complex lighting environment.

[0086] Figure 3 A schematic diagram of a method 200 for adjusting brightness of a head-up display according to an embodiment of the present invention is shown, wherein the head-up display comprises a backlight source and a display screen, the backlight source is configured to project light onto the display screen, and the display screen is configured to generate a projected display image. Figure 3 As shown, the brightness adjustment method 200 includes the following steps, which are described in detail below.

[0087] In step S210: the ambient light illumination of the virtual image background area is acquired by using a light illumination sensor.

[0088] Figure 4 FIG. 4 shows a schematic diagram of an application scenario of a head-up display according to an embodiment of the present invention. Figure 4 As shown, in the vehicle scene, a head-up display is provided below the front windshield, which is used to project a virtual image to a specific area in front of the driver, such as vehicle speed, navigation information, etc. A light sensor is also provided in the vehicle, which is used to sense the ambient light illumination outside the vehicle, such as sensing the ambient light illumination of the virtual image background area of ​​the head-up display. The light sensor can be provided independently of the head-up display, or can be integrated in the head-up display. Figure 5 FIG. 4 is a schematic diagram showing a virtual image background area according to an embodiment of the present invention. Figure 5 As shown, the virtual image background area of ​​the head-up display can be understood as a perspective projection quadrangular pyramid area formed by the observation point and the far plane. Among them, the observation point refers to the spatial position of the user (driver)'s eyes when observing the virtual image projected by the head-up display. The far plane refers to the area corresponding to the virtual image projected by the head-up display at the farthest display distance.

[0089] In a specific embodiment, Figure 4 As shown, the light sensor faces the center of the virtual image (the center point of the virtual image projected by the head-up display), which enables the light sensor to accurately obtain the ambient light illumination of the virtual image background area to optimize the display effect.

[0090] In step S220: the brightness of the backlight source is adjusted according to the ambient light illumination.

[0091] In a specific implementation, the backlight source brightness can be adjusted according to the time interval Δt1, and the time interval Δt1 is, for example, 0.1 second, 0.2 second, 0.5 second, 1 second, 2 seconds or a longer time interval. Correspondingly, the light intensity sensor can sense the ambient light intensity of the virtual image background area according to a fixed time interval Δt1', and Δt1' can be the same as Δt1 or different, and the present invention is not limited thereto.

[0092] In some embodiments, the brightness of the backlight source can be determined based on the ambient light illumination and the first relationship. The first relationship refers to the correspondence between the ambient light illumination and the brightness of the backlight source. For example, the brightness of the backlight source can be represented by a brightness level. Each brightness level of the backlight source corresponds to a specific ambient light illumination range. The ambient light illumination range can be a point value or an interval range. Whenever the light sensor senses the ambient light illumination I0 of the virtual image background area, the brightness (brightness level) of the backlight source corresponding to the ambient light illumination I0 can be determined by querying the corresponding relationship (first relationship).

[0093] The first relationship can be determined experimentally, for example: j Under the lighting conditions of , the tester observes the virtual image projected by the head-up display and gradually adjusts the brightness of the backlight source. When the tester thinks that the virtual image is clearly visible and does not cause visual fatigue or discomfort, it can be determined that the brightness of the backlight source at this time is consistent with the ambient light illuminance I j Those skilled in the art will appreciate that, in order to improve accuracy, multiple testers may perform tests separately, and perform statistical and analytical processing (for example, calculating the average or median) on the test results, so as to more accurately match the brightness of the backlight source with the ambient light illumination.

[0094] In other embodiments, the effective ambient light illumination at the current moment can be calculated based on the ambient light illumination of the virtual image background area sensed by the light illumination sensor at the current moment t0 and one or more previous moments (for example, moment t0-△t1, moment t0-2△t1, ..., moment t0-n△t1). For example, the mean or median of the ambient light illumination of the virtual image background area sensed at these moments (moment t0, moment t0-△t1, ..., moment t0-n△t1) can be used as the effective ambient light illumination at the current moment. There is a preset corresponding relationship (third relationship, the third relationship can be the same as the first relationship) between the effective ambient light illumination and the brightness of the backlight source. For example, each brightness level of the backlight source corresponds to a specific effective ambient light illumination range, which can be a point value or an interval range. After determining the effective ambient light illumination I′0 at the current moment, the brightness (brightness level) of the backlight source corresponding to the effective ambient light illumination I′0 can be determined by querying the corresponding relationship (third relationship).

[0095] The input power of the backlight source can be determined based on the brightness of the backlight source and the second relationship. The second relationship refers to the corresponding relationship between the brightness (brightness level) of the backlight source and the input power of the backlight source. For example, each brightness level of the backlight source corresponds to a specific input power. In a specific implementation, the brightness (brightness level) of the backlight source is determined. Afterwards, the brightness of the backlight source can be determined by querying the second relationship The corresponding input power P0 of the backlight source. Those skilled in the art will appreciate that, in other embodiments, a corresponding relationship between the ambient light illumination (or effective ambient light illumination) and the input power of the backlight source may also be established, so that after the ambient light illumination I0 (or effective ambient light illumination I′0) of the virtual image background area is determined, the input power P0 of the backlight source may be determined by querying the corresponding relationship.

[0096] In a specific implementation, after the input power P0 of the backlight source is determined, the input power P0 can be used to drive the backlight source, thereby achieving brightness adjustment of the backlight source. It should be noted that the input power of the backlight source can be adjusted to the input power P0 instantaneously to quickly respond to environmental changes; the input power of the backlight source can also be gradually and smoothly adjusted to the input power P0 to avoid visual discomfort.

[0097] In the above, we have described in detail how to adjust the brightness of the backlight source. Next, we will discuss the process of adjusting the color value of each area or each pixel in the projected display image. It is worth noting that the adjustment of the brightness of the backlight source and the adjustment of the color value of each pixel in the projected display image can be performed simultaneously or at different times, and the present invention is not limited to this.

[0098] In step S230: a first image of the virtual image background area is acquired through a camera.

[0099] like Figure 4 As shown, in a vehicle scene, the vehicle is equipped with a camera, and the camera can be located on the top of the vehicle or other positions to facilitate taking pictures or recording videos of a target area (e.g., a virtual image background area). In some embodiments, the camera can take pictures of the virtual image background area at a fixed time interval △t2. The pictures taken by the camera can be used as the first image of the virtual image background area. In other embodiments, the camera can also record a video of the virtual image background area. The latest image frame can be captured from the video at a fixed time interval △t2, and the captured image frame can be used as the first image of the virtual image background area. The time interval △t2 is, for example, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds or a longer time interval.

[0100] In step S240: according to the first image, the relative brightness distribution of the virtual image background area is determined.

[0101] In the first image, each pixel contains the brightness information of the virtual image background area. Specifically, the area with higher brightness in the virtual image background area has higher color values ​​in the first image, and the area with lower brightness in the virtual image background area has lower color values ​​in the first image. Therefore, the first image can be used to analyze and determine the relative brightness distribution of the virtual image background area. For example, based on the distribution of color values ​​in the first image, high-brightness areas and low-brightness areas in the virtual image background area can be identified (areas with color values ​​higher than a preset threshold are high-brightness areas, and areas with color values ​​lower than the preset threshold are low-brightness areas).

[0102] In the vehicle scenario, Figure 4 As shown in FIG. 1 , the position and installation angle of the camera usually do not coincide with the observation point and viewing direction of the user (driver) (the direction of the user's gaze when observing the center of the virtual image projected by the head-up display). Therefore, there may be a certain deviation between the first image captured by the camera and the image of the virtual image background area observed by the user, for example Figure 6A schematic diagram of a first image captured by a camera and an image of a virtual image background area observed by a user is shown, in which the deviation is schematically shown, and the difference between the first image captured by the camera and the image of the virtual image background area actually observed by the user is compared. This deviation can be taken into account in the brightness adjustment of the head-up display to ensure the accuracy and reliability of the brightness adjustment, thereby providing the user with the best visual effect.

[0103] In some embodiments, a second image can be generated based on the first image, wherein the first image is an image of the virtual image background area in the camera coordinate system, and the second image is an image of the virtual image background area in the eye coordinate system, and the size of the second image is the same as the size of the projected display image (the same number of pixel rows and the same number of pixel columns). Figure 4 As shown, the camera coordinate system (x camera ,y camera , z camera ) and the eye coordinate system (x eye ,y eye , z eye ) are all three-dimensional Cartesian coordinate systems. Camera coordinate system (x camera ,y camera , z camera ) is a coordinate system established with the camera position as the origin. Optionally, the z coordinate system of the camera camera The axis coincides with the optical axis of the camera. eye ,y eye , z eye ) is a coordinate system with the observation point as the origin. Optionally, the z coordinate system of the eye eye The axis coincides with the user's viewing direction.

[0104] In some embodiments, a homography transformation method may be used to transform the first image into the second image. This transformation process may be implemented, for example, by the following formula:

[0105]

[0106] Where s is the size factor of the homogeneous coordinates, is the pixel with coordinates (u, v) in the second image, H is the homography matrix, is the pixel with coordinates (u′, v′) in the first image.

[0107] For ease of understanding, the derivation process of the above formula is given below:

[0108] According to the camera imaging principle, we can get the first formula:

[0109]

[0110] Among them, Acamera is the intrinsic parameter matrix of the camera, Represents the rotation and translation transformation matrix from the world coordinate system to the camera coordinate system, is the coordinate of a point in the world coordinate system, is the coordinate of the point in the first image, and ρ is the scale factor of the homogeneous coordinates.

[0111] Assuming that the virtual image background is on a plane in world coordinates, the first formula above can be simplified to:

[0112]

[0113] Where [r1|r2|t] camera It is a matrix composed of the first two columns r1 and r2 of the rotation transformation matrix and the translation vector t. is the coordinate of the point in the world coordinate system.

[0114] Similarly, for the virtual image (second image) in the eye coordinate system, we can get the second formula:

[0115]

[0116] in, is the coordinate of the certain point in the virtual image (second image), A HUD is the equivalent internal parameter matrix of the head-up display, and σ is the size factor of the homogeneous coordinates.

[0117] Based on the first and second formulas, we can get:

[0118]

[0119] H=A HUD [r1|r2|t] eye (A camera [r1|r2|t] camera ) -1 ,s=ρ / σ.

[0120] It should be noted that in the process of converting the first image into the second image through the homography matrix H, in addition to the coordinate transformation, the brightness and color information in the first image can also be supplemented into the second image, so that the second image can accurately reflect the relative brightness distribution of the virtual image background area, that is, the relative brightness distribution of the virtual image background area can be calculated based on the color values ​​of each pixel point of the second image.

[0121] In a specific embodiment, the brightness of each pixel of the second image can be determined based on the second image. In some embodiments, the brightness of each pixel of the second image can be represented by the linear space brightness of the corresponding pixel, the gamma space brightness of each pixel can be represented by the color value (such as grayscale value) of the corresponding pixel, and the linear space brightness of each pixel can be calculated by the gamma space brightness of the corresponding pixel, wherein the linear space brightness refers to the brightness in the linear color space without gamma correction, wherein the brightness and light intensity have a linear relationship; the gamma space brightness refers to the brightness in the color space after gamma correction, wherein the brightness and light intensity have a nonlinear relationship, which is generally consistent with the human eye's perception of brightness.

[0122] The following is a detailed description using the example of determining the brightness of a pixel (u, v):

[0123] The gamma space brightness of a pixel (u, v) can be represented by the grayscale value of the pixel (u, v), that is:

[0124] B sRGB (u,v)=Gray background (u,v)=0.299R+0.587G+0.114B;

[0125] Among them, B sRGB (u,v) is the gamma space brightness of the pixel (u,v), Gray background (u,v) is the grayscale value of the pixel (u,v), and R, G, and B are the red, green, and blue channel values ​​of the pixel (u,v), respectively.

[0126] The linear space brightness of a pixel (u,v) can be determined by the following formula:

[0127] B linear (u,v)=B sRGB (u,v) 1 / γ ;

[0128] Among them, B linear (u,v) is the linear space brightness of the pixel (u,v), and γ is the gamma correction coefficient.

[0129] In a specific implementation, the average brightness of the second image can be determined based on the brightness of each pixel of the second image. Specifically, the brightness of each pixel of the second image can be weighted averaged to obtain the average brightness of the second image. During the calculation process, all pixels can be given the same weight, or different weights can be given to pixels in different areas as required (for example, pixels in the center area of ​​the second image can be set with higher weights, and pixels in the edge area can be set with lower weights). Optionally, the average brightness of the second image can be determined by the following formula:

[0130]

[0131] in, is the average brightness of the second image, H is the number of pixel rows in the second image, and W is the number of pixel columns in the second image.

[0132] In a specific implementation, the relative brightness of each pixel of the second image can be determined based on the brightness of each pixel of the second image and the average brightness of the second image, thereby obtaining the relative brightness distribution of the virtual image background area. The relative brightness of a pixel of the second image is the ratio of the brightness of the pixel to the average brightness of the second image, for example: the relative brightness of the pixel (u, v)

[0133]

[0134] In some embodiments, high-brightness areas (areas with relative brightness greater than 1) and low-brightness areas (areas with relative brightness less than 1) in the virtual image background area can be identified based on the relative brightness of each pixel of the second image.

[0135] In step S250: the color value of each pixel of the projected display image is corrected according to the relative brightness distribution of the virtual image background area.

[0136] In a specific embodiment, for a virtual image in a high brightness area displayed in the virtual image background area, the color value of the pixel points in the corresponding area in the projected display image can be increased to enhance its brightness. For a virtual image in a low brightness area displayed in the virtual image background area, the color value of the pixel points in the corresponding area in the projected display image can be reduced.

[0137] In some embodiments, the projected display image adopts the RGBA color format, which is a color representation method that defines colors through four channels: red (Red), green (Green), blue (Blue) and transparency (Alpha). In these embodiments, the pixel transparency of each pixel of the projected display image can be corrected according to the relative brightness distribution of the virtual image background area to achieve pixel-level adjustment of the virtual image brightness, ensuring that the driver can clearly see all the information in the virtual image even in a complex lighting environment. Optionally, the image transparency of each pixel of the projected display image can be corrected by the following formula:

[0138]

[0139] in, is the image transparency of the pixel with coordinates (u, v) in the corrected projection display image, A hud img (u,v) is the image transparency of the pixel with coordinates (u,v) in the projection display image before correction.

[0140] In other embodiments, the projected display image adopts the HSL color format, which is a color representation method based on human perception of color, and defines color through three channels: hue, saturation, and lightness. In these embodiments, the brightness of each pixel of the projected display image can be corrected according to the relative brightness distribution of the virtual image background area, so as to achieve pixel-level adjustment of the virtual image brightness, and ensure that the driver can clearly see all the information in the virtual image even in a complex lighting environment. Optionally, the brightness of each pixel of the projected display image can be corrected by the following formula:

[0141]

[0142] in, is the brightness of the pixel with coordinates (u, v) in the corrected projection display image, L hud img (u, v) is the brightness of the pixel with coordinates (u, v) in the projection display image before correction, B′ background (u, v) is the relative brightness of the pixel with coordinates (u, v) in the second image, and γ is the gamma correction coefficient.

[0143] Compared with the prior art, the embodiments of the present invention provide a method for adjusting the brightness of a head-up display, which implements pixel-level adjustment of the virtual image brightness by adjusting the brightness of the backlight source and the color value of each pixel of the projected display image, thereby ensuring that the driver can clearly see all information in the virtual image even in a complex lighting environment.

[0144] Figure 7 A schematic diagram of a head-up display system 300 according to an embodiment of the present invention is shown below. Figure 7 Describe in detail.

[0145] like Figure 7 As shown, the head-up display system 300 may include a display screen 310, a backlight source 320, an illuminance sensor 330, a camera 340, and a control unit 350. The display screen 310 is configured to generate a projection display image, for example, the display screen 310 may be a thin film transistor liquid crystal display (TFT-LCD). The backlight source 320 is configured to project light to the display screen 310, and its brightness can be adjusted as needed. The illuminance sensor 330 is configured to detect the ambient light illumination of the virtual image background area. The camera 340 is configured to capture a first image of the virtual image background area. The control unit 350 is coupled to the display screen 310, the backlight source 320, the illuminance sensor 330, and the camera 340, respectively. The control unit 350 is configured to adjust the brightness of the backlight source 320 according to the ambient light illumination. The control unit 350 is also configured to determine the relative brightness distribution of the virtual image background area according to the first image, and to correct the color value of each pixel of the projection display image according to the relative brightness distribution of the virtual image background area.

[0146] According to one embodiment of the present invention, Figure 7 As shown, the control unit 350 can be configured to determine the brightness of the backlight source 320 based on the ambient light illumination and the first relationship. The first relationship refers to the corresponding relationship between the ambient light illumination and the brightness of the backlight source 320. For example, the brightness of the backlight source 320 can be represented by a brightness level, and each brightness level of the backlight source 320 corresponds to a specific ambient light illumination range, which can be a point value or an interval range. Whenever the light sensor 330 senses the ambient light illumination I0 of the virtual image background area, the brightness (brightness level) of the backlight source 320 corresponding to the ambient light illumination I0 can be determined by querying the corresponding relationship (first relationship). In other embodiments, the light sensor 330 may be configured to sense the ambient light illumination of the virtual image background area at a fixed time interval Δt1. The control unit 350 may be configured to: determine the effective ambient light illumination at the current moment according to the ambient light illumination acquired at multiple moments; determine the brightness of the backlight source 320 according to the effective ambient light illumination and the third relationship, where the third relationship is the corresponding relationship between the effective ambient light illumination and the brightness of the backlight source 320.

[0147] The control unit 350 is further configured to determine the input power of the backlight light source 320 according to the brightness of the backlight light source 320 and the second relationship; drive the backlight light source 320 with the input power to adjust the brightness of the backlight light source 320. The second relationship refers to the corresponding relationship between the brightness (brightness level) of the backlight light source 320 and the input power of the backlight light source 320, for example: each brightness level of the backlight light source 320 corresponds to a specific input power. In a specific embodiment, the brightness (brightness level) of the backlight light source 320 is determined. Afterwards, the brightness of the backlight source 320 can be determined by querying the second relationship. The corresponding input power P0 of the backlight light source 320. After the input power P0 of the backlight light source 320 is determined, the input power P0 can be used to drive the backlight light source 320, thereby achieving brightness adjustment of the backlight light source 320.

[0148] According to one embodiment of the present invention, Figure 7 As shown, the control unit 350 can also be configured to generate a second image based on the first image, wherein the second image is an image of the virtual image background area in the eye coordinate system; determine the brightness of each pixel of the second image based on the second image; determine the average brightness of the second image based on the brightness of each pixel of the second image; determine the relative brightness of each pixel of the second image based on the brightness of each pixel of the second image and the average brightness of the second image, as the relative brightness distribution of the virtual image background area.

[0149] According to one embodiment of the present invention, Figure 7 As shown, the control unit 350 can also be configured to: determine the gamma space brightness of a pixel point in the second image based on the grayscale value of the pixel point; determine the linear space brightness of the pixel point based on the gamma space brightness of the pixel point as the brightness of the pixel point.

[0150] According to one embodiment of the present invention, Figure 7 As shown, the control unit 350 may also be configured to: perform weighted average calculation on the brightness of each pixel of the second image to obtain the average brightness of the second image.

[0151] According to one embodiment of the present invention, the projected display image may adopt the RGBA color format, and accordingly, the control unit 350 may be configured to correct the image transparency of each pixel of the projected display image according to the relative brightness distribution of the virtual image background area. In other embodiments, the projected display image adopts the HSL color format, and accordingly, the control unit 350 may be configured to correct the brightness of each pixel of the projected display image according to the relative brightness distribution of the virtual image background area.

[0152] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for adjusting the brightness of a head-up display, the head-up display comprising a backlight source and a display screen, the display screen being configured to generate a projected display image, the brightness adjustment method comprising: The ambient light intensity of the virtual image background area is obtained by using a light intensity sensor; According to the ambient light illumination, adjusting the brightness of the backlight source; Acquire a first image of the virtual image background area through a camera; Determine the relative brightness distribution of the virtual image background area according to the first image; and According to the relative brightness distribution of the virtual image background area, the color value of each pixel of the projected display image is corrected.

2. The brightness adjustment method according to claim 1, wherein: The step of adjusting the brightness of the backlight source according to the ambient light illumination of the virtual image background area comprises: Determining the brightness of the backlight source according to the ambient light illumination and the first relationship; Determining the input power of the backlight light source according to the brightness of the backlight light source and the second relationship; and driving the backlight source with the input power to adjust the brightness of the backlight source; The first relationship is the corresponding relationship between the ambient light illumination and the brightness of the backlight light source; and the second relationship is the corresponding relationship between the brightness of the backlight light source and the input power of the backlight light source.

3. The brightness adjustment method according to claim 1, wherein: The light intensity sensor acquires the ambient light intensity of the virtual image background area at a preset time interval; The step of adjusting the brightness of the backlight source according to the ambient light illumination of the virtual image background area comprises: Determine the effective ambient light intensity at the current moment according to the ambient light intensity obtained at multiple moments; Determining the brightness of the backlight source according to the effective ambient light illumination and a third relationship; Determining the input power of the backlight light source according to the brightness of the backlight light source and the second relationship; and driving the backlight source with the input power to adjust the brightness of the backlight source; The second relationship is the corresponding relationship between the brightness of the backlight light source and the input power of the backlight light source; and the third relationship is the corresponding relationship between the effective ambient light illumination and the brightness of the backlight light source.

4. The brightness adjustment method according to claim 1, wherein: The step of determining the relative brightness distribution of the virtual image background area according to the first image comprises: generating a second image according to the first image, wherein the second image is an image of a virtual image background area in an eye coordinate system; Determine, according to the second image, the brightness of each pixel of the second image; Determining an average brightness of the second image according to the brightness of each pixel of the second image; and According to the brightness of each pixel of the second image and the average brightness of the second image, the relative brightness of each pixel of the second image is determined as the relative brightness distribution of the virtual image background area.

5. The brightness adjustment method according to claim 4, wherein: The step of determining the brightness of a pixel of the second image comprises: Determining the gamma space brightness of a pixel in the second image based on the grayscale value of the pixel; and Based on the gamma space brightness of the pixel, the linear space brightness of the pixel is determined as the brightness of the pixel.

6. The brightness adjustment method according to claim 4, wherein: The average brightness of the second image is obtained by performing weighted average calculation on the brightness of each pixel of the second image.

7. The brightness adjustment method according to claim 1, wherein: The projection display image adopts RGBA color format, and the step of correcting the color value of each pixel of the projection display image includes: correcting the image transparency of each pixel of the projection display image.

8. The brightness adjustment method according to claim 7, wherein: The image transparency of each pixel of the projected display image is corrected by the following formula: in, is the image transparency of the pixel with coordinates (u, v) in the corrected projection display image, A hud img(u,v) is the image transparency of the pixel with coordinates (u,v) in the projection display image before correction, B ′ background (u, v) is the relative brightness of the pixel with coordinates (u, v) in the second image, and γ is the gamma correction coefficient.

9. The brightness adjustment method according to claim 1, wherein: The projection display image adopts the HSL color format, and the step of correcting the color value of each pixel of the projection display image includes: correcting the brightness of each pixel of the projection display image.

10. The brightness adjustment method according to claim 9, wherein: The brightness of each pixel of the projected display image is corrected by the following formula: in, is the brightness of the pixel with coordinates (u, v) in the corrected projection display image, L hud img(u,v) is the brightness of the pixel with coordinates (u,v) in the projection display image before correction, B ′ background (u, v) is the relative brightness of the pixel with coordinates (u, v) in the second image, and γ is the gamma correction coefficient.

11. A head-up display system, comprising: a display screen configured to generate a projected display image; a backlight source configured to project light onto the display screen; an illumination sensor configured to detect an ambient light illumination of a virtual image background area; A camera configured to capture a first image of a virtual image background area; A control unit is coupled to the display screen, the backlight source, the illuminance sensor, and the camera, respectively. The control unit is configured to adjust the brightness of the backlight source according to the ambient light illumination. The control unit is also configured to determine the relative brightness distribution of the virtual image background area according to the first image, and correct the color value of each pixel of the projected display image according to the relative brightness distribution of the virtual image background area.

12. The head-up display system according to claim 11, wherein: The control unit is configured as follows: Determining the brightness of the backlight source according to the ambient light illumination and the first relationship; Determining the input power of the backlight light source according to the brightness of the backlight light source and the second relationship; driving the backlight source with the input power to adjust the brightness of the backlight source; Wherein, the first relationship is the corresponding relationship between the ambient light illumination and the brightness of the backlight light source; The second relationship is a corresponding relationship between the brightness of the backlight light source and the input power of the backlight light source.

13. The head-up display system according to claim 11, wherein: The light sensor is configured to obtain the ambient light illumination of the virtual image background area at a preset time interval; the control unit is configured to: The step of adjusting the brightness of the backlight source according to the ambient light illumination of the virtual image background area comprises: Determine the effective ambient light intensity at the current moment according to the ambient light intensity obtained at multiple moments; Determining the brightness of the backlight source according to the effective ambient light illumination and a third relationship; Determining the input power of the backlight light source according to the brightness of the backlight light source and the second relationship; and driving the backlight source with the input power to adjust the brightness of the backlight source; The second relationship is the corresponding relationship between the brightness of the backlight light source and the input power of the backlight light source; and the third relationship is the corresponding relationship between the effective ambient light illumination and the brightness of the backlight light source.

14. The head-up display system according to claim 11, wherein: The control unit is configured as follows: generating a second image according to the first image, wherein the second image is an image of a virtual image background area in an eye coordinate system; Determine, according to the second image, the brightness of each pixel of the second image; Determining an average brightness of the second image according to the brightness of each pixel of the second image; According to the brightness of each pixel of the second image and the average brightness of the second image, the relative brightness of each pixel of the second image is determined respectively as the relative brightness distribution of the virtual image background area.

15. The head-up display system according to claim 14, wherein: The control unit is configured as follows: determining a gamma space brightness of a pixel in the second image based on a grayscale value of the pixel; Based on the gamma space brightness of the pixel, the linear space brightness of the pixel is determined as the brightness of the pixel.

16. The head-up display system according to claim 14, wherein: The control unit is configured to perform weighted average calculation on the brightness of each pixel of the second image to obtain an average brightness of the second image.

17. The head-up display system according to claim 11, wherein: The projected display image adopts RGBA color format, and the control unit is configured to correct the image transparency of each pixel of the projected display image according to the relative brightness distribution of the virtual image background area.

18. The head-up display system according to claim 11, wherein: The projected display image adopts the HSL color format, and the control unit is configured to correct the brightness of each pixel of the projected display image according to the relative brightness distribution of the virtual image background area.

Images