Virtual image brightness adjusting method of head-up display and head-up display system

Abstract

The invention provides a virtual image brightness adjusting method of a head-up display and the head-up display. The virtual image brightness adjusting method comprises the following steps: acquiring effective ambient illuminance of a virtual image background area; determining the virtual image brightness of the head-up display according to the effective ambient illuminance of the virtual image background area and a preset virtual image brightness adjustment relation, and projecting a virtual image to the target area; obtaining user-defined virtual image brightness input by a user; determining a plurality of effective ambient illuminances and corresponding virtual image brightness corrections; according to the plurality of effective ambient illuminances and the corresponding virtual image brightness corrections, a user specificity relation is obtained based on a preset virtual image brightness adjustment relation producer; and taking the user specificity relation as a new virtual image brightness adjustment relation. The user specificity relationship fits the physiological characteristics and visual preferences of the user, and virtual image brightness adjustment is performed based on the user specificity relationship, so that the driving interference caused by brightness discomfort can be effectively reduced, and the safety in the driving process is remarkably enhanced.

Description

Technical Field

[0001] The present invention generally relates to the technical field of brightness adjustment of a head-up display, and in particular to a virtual image brightness adjustment method of a head-up display and a head-up display system. Background Art

[0002] Head-up display (HUD), also known as head-up display, was originally designed as a display system for military fighter jets to help pilots view key information at any time during flight. In recent years, automotive head-up display technology has gradually matured, and can project important driving information such as speed and navigation directly onto the windshield in front of the driver. This design allows drivers to obtain the required information without lowering their heads, effectively reducing cognitive load and significantly improving driving safety.

[0003] During the driving process, the change of sunlight angle, entering and exiting shadows or tunnels / basements will also cause the driver's perceived ambient brightness to change continuously. If the brightness of the virtual image of the head-up display is too low, the driver may have difficulty seeing the information; if the brightness is too high, it may cause glare. Therefore, in order to ensure that the head-up display can provide a comfortable visual experience in different environments, the virtual image brightness of the head-up display needs to be automatically adjusted according to the ambient brightness.

[0004] At present, the brightness adjustment of the head-up display mainly relies on the standard ambient light illumination-optimal virtual image brightness curve to determine the target virtual image brightness under different ambient light illuminations, and adjusts the virtual image brightness to the target value through backlight adjustment. The standard ambient light illumination-optimal virtual image brightness curve is a standard model established based on a large amount of user data using statistical methods. However, due to differences in individual perception of brightness and color, the relationship curve between the ambient light illumination and the optimal virtual image brightness of each user (user-specific curve) may deviate from the standard ambient light illumination-optimal virtual image brightness curve (standard brightness adjustment curve). This means that relying solely on the standard brightness adjustment curve for brightness adjustment may not meet the needs of all users. For example Fig.15 A schematic diagram of a standard brightness adjustment curve and a user-specific curve is shown, wherein the horizontal axis represents the ambient light illuminance / l ux, and the vertical axis represents the relative virtual image brightness as a percentage of the maximum virtual image brightness supported by the hardware (for example, when the vertical axis is "1", the backlight module will adjust the virtual image brightness to the maximum).

[0005] In the related art, the user-specific curve is usually generated based on the standard brightness adjustment curve. Specifically, the system will shift the standard brightness adjustment curve as a whole according to the user's optimal virtual image brightness setting under a specific ambient light intensity to generate a user-specific curve. The limitation of this method is that the generated user-specific curve may be completely above or below the standard brightness adjustment curve, and cannot meet the user's personalized needs for brightness adjustment under different lighting conditions. For example, if the standard brightness adjustment curve corresponds to a relative virtual image brightness of 0.83 under an ambient light intensity of 3000l ux, and a user feels that a relative virtual image brightness of 0.75 is more suitable under the same light intensity, the system will shift the standard brightness adjustment curve downward as a whole. This adjustment may result in the user-specific curve being unable to meet the user's actual needs under an ambient light intensity of 4000l ux, making the virtual image too dim.

[0006] The contents of the background technology section are merely the technologies known to the inventors and do not necessarily represent the prior art in the field. Summary of the invention

[0007] In view of one or more defects in the prior art, the present invention provides a method for adjusting the brightness of a virtual image of a head-up display, comprising:

[0008] S110: Obtaining the effective ambient light illumination of the virtual image background area;

[0009] S120: determining the virtual image brightness of the head-up display according to the effective ambient light illumination of the virtual image background area and a preset virtual image brightness adjustment relationship, and projecting the virtual image to the target area, wherein the virtual image brightness adjustment relationship is a corresponding relationship between the effective ambient light illumination and the virtual image brightness of the head-up display;

[0010] S130: Obtaining the custom virtual image brightness input by the user;

[0011] S140: Determine a plurality of effective ambient light illuminations and corresponding virtual image brightness correction values;

[0012] S150: obtaining a user-specific relationship based on the plurality of effective ambient light illuminations and the corresponding virtual image brightness correction amounts and based on a preset virtual image brightness adjustment relationship generator, wherein the user-specific relationship is a corresponding relationship between the effective ambient light illumination and the virtual image brightness of the head-up display applicable to a specific user;

[0013] S160: Using the user-specific relationship as a new virtual image brightness adjustment relationship.

[0014] According to one aspect of the present invention, the effective ambient light illumination is represented by an ambient light illumination level.

[0015] According to one aspect of the present invention, the preset virtual image brightness adjustment relationship is a user-specific relationship or a standard virtual image brightness adjustment relationship, and the standard virtual image brightness adjustment relationship is a corresponding relationship between the effective ambient light illumination and the standard virtual image brightness of the head-up display.

[0016] According to one aspect of the present invention, step S130 includes:

[0017] The customized virtual image brightness and its corresponding effective ambient light illumination are updated into the virtual image brightness adjustment database.

[0018] According to one aspect of the present invention, step S130 further includes:

[0019] The oldest custom virtual image brightness and its corresponding effective ambient light illumination in the virtual image brightness adjustment database are deleted.

[0020] According to one aspect of the present invention, in step S140,

[0021] If the effective ambient light illumination has one or more corresponding custom virtual image brightnesses in the virtual image brightness adjustment database, the difference between the latest custom virtual image brightness and the standard virtual image brightness corresponding to the effective ambient light illumination is used as the virtual image brightness correction amount corresponding to the effective ambient light illumination;

[0022] If the effective ambient light illumination has no corresponding custom virtual image brightness in the virtual image brightness adjustment database, the virtual image brightness correction amount corresponding to the effective ambient light illumination is set to zero.

[0023] According to one aspect of the present invention, step S150 includes:

[0024] Generate an input matrix according to the multiple effective ambient light illuminations and corresponding virtual image brightness correction amounts;

[0025] The input matrix is ​​input into the virtual image brightness adjustment relationship generator, and the user-specific relationship is determined based on the output of the virtual image brightness adjustment relationship generator.

[0026] According to one aspect of the present invention, the virtual image brightness adjustment method further includes:

[0027] A virtual image brightness adjustment relationship generator is established based on a recurrent neural network.

[0028] According to one aspect of the present invention, the step of establishing a virtual image brightness adjustment relationship generator based on a recurrent neural network includes:

[0029] Acquire a training data set, wherein each set of training data in the training data set includes a plurality of preset effective ambient light illuminations and a virtual image brightness and a virtual image brightness correction value applicable to a specific user corresponding to each effective ambient light illumination;

[0030] The training data set is input into the recurrent neural network to establish a virtual image brightness adjustment relationship generator.

[0031] The present invention also provides a head-up display system, comprising:

[0032] Imaging unit;

[0033] a light sensor configured to measure ambient light levels;

[0034] A human-computer interaction interface configured to output a customized virtual image brightness in response to a user operation; and

[0035] A control unit is coupled to the imaging unit, the light illuminance sensor and the human-computer interaction interface respectively, and the control unit is configured to: determine the effective ambient light illuminance of the virtual image background area based on the ambient light illuminance measured by the light illuminance sensor; determine the virtual image brightness of the head-up display according to the effective ambient light illuminance of the virtual image background area and a preset virtual image brightness adjustment relationship, and control the imaging unit to project the virtual image to the target area; obtain the custom virtual image brightness input by the user; determine multiple effective ambient light illuminances and corresponding virtual image brightness correction amounts; obtain a user-specific relationship based on the multiple effective ambient light illuminances and the corresponding virtual image brightness correction amounts and based on a preset virtual image brightness adjustment relationship generator, and use the user-specific relationship as a new virtual image brightness adjustment relationship;

[0036] Among them, the virtual image brightness adjustment relationship is the corresponding relationship between the effective ambient light illumination and the virtual image brightness of the head-up display; the user-specific relationship is the corresponding relationship between the effective ambient light illumination and the virtual image brightness of the head-up display suitable for a specific user.

[0037] According to one aspect of the present invention, the preset virtual image brightness adjustment relationship is a user-specific relationship or a standard virtual image brightness adjustment relationship, and the standard virtual image brightness adjustment relationship is a corresponding relationship between the effective ambient light illumination and the standard virtual image brightness of the head-up display.

[0038] According to one aspect of the present invention, the control unit is configured to update the custom virtual image brightness and its corresponding effective ambient light illumination into a virtual image brightness adjustment database.

[0039] According to one aspect of the present invention, the control unit is configured to delete the oldest custom virtual image brightness and its corresponding effective ambient light illumination in the virtual image brightness adjustment database.

[0040] According to one aspect of the present invention, the control unit is configured to use the difference between the latest custom virtual image brightness and the standard virtual image brightness corresponding to the effective ambient light illumination as the virtual image brightness correction amount corresponding to the effective ambient light illumination if the effective ambient light illumination has one or more corresponding custom virtual image brightnesses in the virtual image brightness adjustment database; if the effective ambient light illumination has no corresponding custom virtual image brightness in the virtual image brightness adjustment database, the virtual image brightness correction amount corresponding to the effective ambient light illumination is set to zero.

[0041] According to one aspect of the present invention, the control unit is configured to generate an input matrix according to the multiple effective ambient light illuminations and the corresponding virtual image brightness correction amounts;

[0042] The input matrix is ​​input into the virtual image brightness adjustment relationship generator, and the user-specific relationship is determined based on the output of the virtual image brightness adjustment relationship generator.

[0043] Compared with the prior art, the embodiments of the present invention provide a head-up display virtual image brightness adjustment method and head-up display system. In which, based on the preset virtual image brightness adjustment relationship generator, a user-specific relationship is obtained, and the user-specific relationship is more in line with the user's physiological characteristics and visual preferences. Based on the user-specific relationship, the virtual image brightness of the head-up display is adjusted, which can effectively reduce driving interference caused by inappropriate brightness and significantly enhance safety during driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0045] Figure 1 A flow chart showing a method for adjusting virtual image brightness of a head-up display according to an embodiment of the present invention is shown;

[0046] Figure 2 A schematic diagram of an application scenario of a head-up display according to an embodiment of the present invention is shown;

[0047] Figure 3 A schematic diagram showing a virtual image background area according to an embodiment of the present invention is shown;

[0048] Figure 4 A schematic diagram of a brightness adjustment database according to an embodiment of the invention is shown;

[0049] Figure 5a A schematic diagram of an input matrix according to an embodiment of the present invention is shown;

[0050] Figure 5bA schematic diagram of an output matrix according to an embodiment of the present invention is shown;

[0051] Figure 6 A schematic diagram showing a standard brightness adjustment curve and a user-specific curve according to an embodiment of the present invention;

[0052] Figure 7 A flow chart of establishing a virtual image brightness adjustment relationship generator based on a recurrent neural network according to an embodiment of the present invention is shown;

[0053] Figure 8 A schematic diagram showing training data according to an embodiment of the present invention is shown;

[0054] Fig. 9 A schematic diagram showing a directed graph representation of a sequence of virtual image brightness adjustment relationships (user-specific relationships);

[0055] Fig.10 A schematic diagram of a single-layer recurrent neural network is shown;

[0056] Fig.11 A schematic diagram of a multi-layer recurrent neural network according to an embodiment of the present invention is shown;

[0057] Fig.12 A flowchart showing an experimental calibration of user-specific relationships for a specific user according to an embodiment of the present invention is shown;

[0058] Fig.13 A schematic diagram showing a method of dragging a standard brightness adjustment curve to generate a user-specific curve based on virtual image brightness correction amounts under multiple effective ambient light illuminations according to an embodiment of the present invention;

[0059] Fig.14 A schematic diagram of a head-up display system according to an embodiment of the present invention is shown;

[0060] Fig.15 A schematic diagram showing a standard brightness adjustment curve and a user-specific curve. DETAILED DESCRIPTION

[0061] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0062] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0063] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0064] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature is lower in level than the second feature.

[0065] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0066] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0067] Figure 1 FIG. 1 is a flow chart showing a method 100 for adjusting virtual image brightness of a head-up display according to an embodiment of the present invention. Figure 1 As shown, the virtual image brightness adjustment method 100 includes the following steps, and each step is described in detail below.

[0068] In step S110: the effective ambient light illumination of the virtual image background area is obtained.

[0069] Figure 2 FIG. 4 shows a schematic diagram of an application scenario of a head-up display according to an embodiment of the present invention. Figure 2 As shown, in the vehicle scene, a head-up display is provided inside the vehicle to project a virtual image to a target area in front of the driver, such as vehicle speed, navigation information, etc. A light sensor is also provided in the vehicle, which can be provided independently of the head-up display or integrated in the head-up display. The light sensor is used to sense the ambient light illumination outside the vehicle, such as sensing the ambient light illumination of the background area of ​​the virtual image. Figure 3 FIG. 4 is a schematic diagram showing a virtual image background area according to an embodiment of the present invention. Figure 3 As shown, the virtual image background area of ​​the head-up display can be understood as a perspective projection quadrangular pyramid area formed by the observation point and the far plane. Among them, the observation point refers to the spatial position of the user (driver)'s eyes when observing the virtual image projected by the head-up display. The far plane refers to the area corresponding to the virtual image projected by the head-up display at the farthest display distance.

[0070] The light sensor has a preset detection range [I min , I max ], where I min is the minimum ambient light intensity that the light sensor can sense, I maxis the maximum ambient light intensity that can be sensed by the light intensity sensor. min , I max ] can be divided into multiple ambient light illumination levels in geometric proportions, arithmetic progressions, etc., and the effective ambient light illumination can be represented by these ambient light illumination levels. For example, the detection interval [I min , I max ] is divided into N+1 ambient light illumination levels (I0, I1, ..., I N ), therefore, the effective ambient light intensity can be obtained through I0, I1, …, I N These effective ambient light illuminances (ambient light illuminance levels) form a geometric progression [I0, I1, ..., I N ], let the common ratio of the geometric sequence be a, then the kth effective ambient light illumination (ambient light illumination level) I k =I0a k At the same time, the effective ambient light intensity I k It also represents the ambient light illumination range [I0a k , I0a k+1 ). In some embodiments, the common ratio a can be set to (1+α), then the geometric progression [I0, I1, …, I N The kth effective ambient light illuminance (ambient light illuminance level) I k =I0(1+

[0071] α) k , the effective ambient light intensity I k The representative ambient light illumination interval is [I0(1+α) k , I0(1+α) k+1 ). Among them, α is the sensitivity coefficient, which refers to the minimum rate of change of ambient light intensity that the human eye can perceive at the current brightness, that is, the sensitivity coefficient Where I is the ambient light intensity, ΔI is the minimum brightness change that can be perceived by the human eye under ambient light intensity I, and the sensitivity coefficient can be determined experimentally. Using (1+α) as the common ratio can ensure that in [I0, I1, …, I N ], the human eye cannot perceive the change of ambient light illumination.

[0072] In a specific implementation, the light sensor may sense the ambient light illumination of the virtual image background area at a fixed time interval Δt, where the time interval Δt is, for example, 0.1 second, 0.2 second, 0.5 second, 1 second, 2 seconds or a longer time interval.

[0073] In some embodiments, whenever the light sensor senses the ambient light illumination of the virtual image background area, the ambient light illumination level corresponding to the ambient light illumination can be determined, and the ambient light illumination level can be used as the effective ambient light illumination of the virtual image background area. The ambient light illumination level corresponding to the ambient light illumination can be the ambient light illumination level that is closest to and greater than the ambient light illumination. For example, when the ambient light illumination of the virtual image background area collected by the light sensor is I cur When I N-1

[0074] <I cur <I N , then the ambient light intensity I cur The corresponding ambient light illumination level is I N , then the effective ambient light illumination of the virtual image background area is I N In other embodiments, the ambient light illumination level corresponding to the ambient light illumination may also be the ambient light illumination level that is closest to and smaller than the ambient light illumination. cur When I N-1

[0075] <I cur <I N , then the ambient light intensity I cur The corresponding ambient light illumination level is I N-1 , then the effective ambient light illumination of the virtual image background area is I N-1 .

[0076] In some other embodiments, the practical ambient light illumination at the current moment can be calculated based on the ambient light illumination of the virtual image background area sensed by the light illumination sensor at the current moment t0 and one or more previous moments (for example: moment t0-△t1, moment t0-2△t1, ..., moment t0-n△t1). For example: the mean or median of the ambient light illumination of the virtual image background area sensed at these moments (moment t0, moment t0-△t1, ..., moment t0-n△t1) can be used as the practical ambient light illumination at the current moment. Subsequently, the ambient light illumination level corresponding to the practical ambient light illumination can be determined, and the ambient light illumination level can be used as the effective ambient light illumination of the virtual image background area. Among them, the ambient light illumination level corresponding to the practical ambient light illumination can be the ambient light illumination level that is closest to and greater than the practical ambient light illumination. For example: when the practical ambient light illumination is I ′ cur When I N <I ′ cur <I N , then the practical ambient light intensity I ′cur The corresponding ambient light illumination level is I N , then the effective ambient light illumination of the virtual image background area is I N In other embodiments, the ambient light illumination level corresponding to the practical ambient light illumination may also be the ambient light illumination level that is closest to and smaller than the practical ambient light illumination. ′ cur When I N <I ′ cur <I N , then the practical ambient light intensity I ′ cur The corresponding ambient light illumination level is I N-1 , then the effective ambient light illumination of the virtual image background area is I N-1 .

[0077] In step S120: the virtual image brightness of the head-up display is determined according to the effective ambient light illumination of the virtual image background area and the preset virtual image brightness adjustment relationship, and the virtual image is projected to the target area.

[0078] The virtual image brightness adjustment relationship is the correspondence between the effective ambient light illumination and the virtual image brightness of the head-up display. Specifically, the virtual image brightness adjustment relationship can adopt a user-specific relationship or a standard virtual image brightness adjustment relationship, wherein the user-specific relationship is the correspondence between the effective ambient light illumination and the virtual image brightness of the head-up display applicable to a specific user obtained by the virtual image brightness adjustment method 100 (described in detail later); the standard virtual image brightness adjustment relationship is a standard model established based on a large amount of user data using a statistical method, reflecting the correspondence between the effective ambient light illumination and the standard virtual image brightness of the head-up display, and the standard virtual image brightness adjustment relationship can be represented by a standard brightness adjustment curve (e.g. Fig.15 It should be noted that before the virtual image brightness adjustment method 100 is used to obtain the user-specific relationship, the virtual image brightness adjustment relationship adopts the standard virtual image brightness adjustment relationship. After the virtual image brightness adjustment method 100 is used to obtain the user-specific relationship, the virtual image brightness adjustment relationship adopts the latest user-specific relationship.

[0079] In a specific implementation, after the effective ambient light illumination of the virtual image background area is determined, the virtual image brightness of the head-up display can be determined using the virtual image brightness adjustment relationship, and the head-up display can be controlled to project the virtual image toward the target.

[0080] In the above, we have elaborated on how to automatically adjust the virtual image brightness of the head-up display according to the ambient light. Next, we will discuss how to create a new user-specific relationship that is more in line with the user's physiological characteristics and visual preferences when the current virtual image brightness adjustment relationship cannot meet the user's needs.

[0081] like Figure 2 As shown, a human-machine interaction interface is also provided in the vehicle, and the human-machine interaction interface can be a button, a knob, a touch screen, etc. The human-machine interaction interface can output a custom virtual image brightness in response to a user operation. During the use of the head-up display, if the user feels that the virtual image brightness of the head-up display is excessive or too dark, the user can input a custom virtual image brightness through the human-machine interaction interface to adjust the virtual image brightness of the head-up display. When the user is satisfied with the virtual image brightness of the head-up display and believes that it meets personal visual preferences, the virtual image brightness of the head-up display will be adopted as the custom virtual image brightness input by the user.

[0082] In step S130: obtaining the user-defined virtual image brightness input by the user.

[0083] In some embodiments, the custom virtual image brightness input by the user may be directly received from the human-computer interaction interface. In other embodiments, the custom virtual image brightness input by the user is stored in a storage module of the head-up display, and thus the custom virtual image brightness input by the user may be obtained by reading the storage module.

[0084] In a specific implementation, after obtaining the custom virtual image brightness input by the user, the custom virtual image brightness input by the user and its corresponding effective ambient light illumination can be updated to the virtual image brightness adjustment database. Optionally, the oldest custom virtual image brightness and its corresponding effective ambient light illumination in the virtual image brightness adjustment database can also be deleted. Among them, "the effective ambient light illumination corresponding to the custom virtual image brightness" refers to: when the user inputs the custom virtual image brightness, the effective ambient light illumination of the virtual image background area. The virtual image brightness adjustment database can be a data queue or a data set, and the following is explained by taking the data queue as an example. Figure 4 FIG. 4 shows a schematic diagram of a brightness adjustment database according to an embodiment of the invention. Figure 4As shown, the data queue (i.e., virtual image brightness adjustment database) has a preset length limit. When the data stored in the data queue (i.e., the custom virtual image brightness and its corresponding effective ambient light illumination) does not reach the preset length limit, a new custom virtual image brightness and its corresponding effective ambient light illumination can be inserted into the end of the data queue. When the data stored in the data queue reaches the preset length, a new custom virtual image brightness and its corresponding effective ambient light illumination can be inserted into the end of the data queue, and the custom virtual image brightness and its corresponding effective ambient light illumination at the head of the data queue are deleted. In this way, the virtual image brightness adjustment database can be prevented from being too bloated, while ensuring that the data in the virtual image brightness adjustment database always reflects the user's latest preferences.

[0085] In step S140: a plurality of effective ambient light illuminations and corresponding virtual image brightness correction values ​​are determined.

[0086] In some embodiments, the multiple effective ambient light illuminances are I0, I1, ..., I N-1 ,I N For each effective ambient light illuminance, the corresponding virtual image brightness correction amount refers to the difference between the custom virtual image brightness and the standard virtual image brightness under the effective ambient light illuminance.

[0087] Those skilled in the art can understand that under some effective ambient light illumination, the user has actively adjusted the virtual image brightness once or more times, and these adjustments are recorded in the virtual image brightness adjustment database. This means that these effective ambient light illuminations are associated with one or more custom virtual image brightnesses in the virtual image brightness adjustment database. For any of these effective ambient light illuminations, the virtual image brightness correction amount corresponding to it refers to the difference between the latest custom virtual image brightness and the standard virtual image brightness under the effective ambient light illumination.

[0088] Under other effective ambient light illuminances, the user may not have actively adjusted the virtual image brightness, or may have adjusted it but it has not been recorded in the virtual image brightness adjustment database. This means that these effective ambient light illuminances do not have corresponding custom virtual image brightness in the virtual image brightness adjustment database. For any of these effective ambient light illuminances, we can assume that the corresponding custom virtual image brightness is the same as the standard virtual image brightness, so the corresponding virtual image brightness correction amount can be set to zero.

[0089] In other embodiments, the multiple effective ambient light illuminations may also be effective ambient light illuminations recorded in the virtual image brightness adjustment database. For any of these effective ambient light illuminations, the virtual image brightness correction value corresponding to it refers to the difference between the latest custom virtual image brightness and the standard virtual image brightness under the effective ambient light illumination.

[0090] In step S150: according to the multiple effective ambient light illuminations and the corresponding virtual image brightness correction amounts, based on a preset virtual image brightness adjustment relationship generator, a user-specific relationship is obtained.

[0091] The virtual image brightness adjustment relationship generator can be established based on a recurrent neural network, a convolutional neural network, etc., and the present invention is not limited here.

[0092] A recurrent neural network is a deep neural network that can be trained on sequential data to create a machine learning model that can process sequential data input and convert it into a specific sequential data output. The sequential data can include words, sentences, or time series data, etc., in which the data points are related to each other according to complex semantic and grammatical rules. In some embodiments, we can use a large amount of user data to train the deep neural network to obtain a virtual image brightness adjustment relationship generator that can generate user-specific relationships that conform to the user's physiological characteristics and visual preferences (described in detail later).

[0093] In a specific implementation, an input matrix may be generated according to a plurality of effective ambient light illuminations and corresponding virtual image brightness correction amounts. Figure 5a FIG. 4 shows a schematic diagram of an input matrix according to an embodiment of the present invention. Figure 5a As shown, the input matrix may be a matrix of (N+1) rows and 2 columns, wherein each row includes the effective ambient light illumination and the corresponding virtual image brightness correction value.

[0094] The input matrix may be input into a virtual image brightness adjustment relationship generator, and based on the output of the virtual image brightness adjustment relationship generator, the user-specific relationship may be determined. In some embodiments, the output of the virtual image brightness adjustment relationship generator may be presented in the form of an output matrix. For example Figure 5b FIG. 4 shows a schematic diagram of an output matrix according to an embodiment of the present invention. Figure 5b As shown, the output matrix is ​​a matrix with (N+1) rows and 1 column, where each row corresponds to the virtual image brightness under an effective ambient light illumination that meets the user's physiological characteristics and visual preferences. N-1 ,I N Each has a corresponding virtual image brightness B0, B1, B2 that conforms to the user's physiological characteristics and visual preferences. N , based on which, the user-specific relationship can be constructed. Optionally, the user-specific relationship can be represented by a user-specific curve, Figure 6 A schematic diagram of a standard brightness adjustment curve and a user-specific curve according to an embodiment of the present invention is shown, wherein the abscissa represents the ambient light illuminance / l ux and the ordinate represents the percentage of the relative virtual image brightness to the maximum virtual image brightness supported by the hardware.

[0095] In step S160: the user-specific relationship is used as a new virtual image brightness adjustment relationship.

[0096] According to one embodiment of the present invention, steps S110-S120 may be performed at a preset time interval Δt' (that is, steps S110-S120 are performed once every Δt', where Δt' and Δt may be the same or different) to achieve dynamic adjustment of virtual image brightness, ensuring that the virtual image brightness of the head-up display can adapt to changes in ambient light and provide users with a better visual experience. When the user inputs a custom virtual image brightness, steps S130-S160 are performed to generate a new user-specific relationship that is more in line with the user's physiological characteristics and visual preferences, ensuring that the brightness adjustment of the head-up display can not only respond to changes in ambient light, but also meet the user's personalized needs, further improving the user's satisfaction with the use and the comfort of the display effect.

[0097] Before being applied to adjust the virtual image brightness of a head-up display, the virtual image brightness adjustment method 100 further includes establishing a virtual image brightness adjustment relationship generator based on a recurrent neural network.

[0098] Figure 7 FIG. 4 shows a flow chart of establishing a virtual image brightness adjustment relationship generator based on a recurrent neural network according to an embodiment of the present invention. Figure 7 As shown, establishing a virtual image brightness adjustment relationship generator based on a recurrent neural network includes the following steps, and each step is described in detail below.

[0099] In step S210: a training data set is obtained.

[0100] In a specific embodiment, the training data set may include multiple sets of training data. Figure 8 FIG. 4 shows a schematic diagram of training data according to an embodiment of the present invention. Figure 8 As shown, the training data includes an “effective ambient light illumination” sequence, a “virtual image brightness correction amount” sequence and a “virtual image brightness” sequence.

[0101] The “effective ambient light illuminance” sequence can be composed of effective ambient light illuminance I0, I1, …, I N-1 ,I N constitute.

[0102] The "virtual image brightness" sequence is composed of the virtual image brightness applicable to a specific user corresponding to each effective ambient light illumination. The virtual image brightness applicable to a specific user corresponding to the effective ambient light illumination can be determined based on the user-specific relationship of the user. The user-specific relationship of the user refers to the correspondence between the effective ambient light illumination and the virtual image brightness applicable to the user. The user-specific relationship of the user can be calibrated experimentally (the calibration process is described in detail later).

[0103] The "virtual image brightness correction" sequence is composed of the virtual image brightness correction corresponding to each effective ambient light illumination. Among them, the virtual image brightness correction corresponding to a part of the effective ambient light illumination can be determined based on the user-specific relationship and the standard virtual image brightness adjustment relationship. Specifically, the virtual image brightness correction corresponding to the effective ambient light illumination is the difference between the virtual image brightness applicable to the user and the standard virtual image brightness under the effective ambient light illumination. The virtual image brightness correction corresponding to another part of the effective ambient light illumination is set to zero. Those skilled in the art will easily understand that the standard virtual image brightness adjustment relationship is a standard model established based on a large amount of user data using statistical methods, which reflects the corresponding relationship between the effective ambient light illumination and the standard virtual image brightness of the head-up display. Therefore, for some users, their user-specific relationship is the same as the standard virtual image brightness adjustment relationship. In the training data provided by them, the "effective ambient light illumination" sequence can be composed of effective ambient light illuminations I0, I1, ..., I N-1 ,I N The "virtual image brightness" sequence is composed of the standard virtual image brightness corresponding to each effective ambient light illumination, and each element in the "virtual image brightness correction amount" sequence is zero.

[0104] In step S220: the training data set is input into the recurrent neural network to establish a virtual image brightness adjustment relationship generator.

[0105] The recurrent neural network is a directed graph in terms of model structure. This application uses a sequence to represent the "virtual image brightness adjustment relationship (user-specific relationship)", and this sequence can also be represented by a directed graph, for example Fig. 9 A schematic diagram of a directed graph representation of a sequence of "virtual image brightness adjustment relationship (user-specific relationship)" is shown. A person skilled in the art can know that a recurrent neural network is composed of a basic recurrent neural network unit (RNN Cell). In the simplest single-layer recurrent neural network, a recurrent neural network unit represents a node. The reference sequence structure can be established. Fig.10 The single-layer recurrent neural network shown in Figure 1 can be expanded to a multi-layer recurrent neural network as needed to enhance the learning ability and prediction accuracy of the model, for example Fig.11 A schematic diagram of a multi-layer recurrent neural network according to an embodiment of the present invention is shown.

[0106] In a specific implementation, the training data set can be input into the multi-layer recurrent neural network for training. During the training process, the input of node k is the effective ambient illumination level I in the training data. k and virtual image brightness correction, the output is the effective ambient illumination level I in the training data k The corresponding virtual image brightness. Fig.11As shown, the multi-layer recurrent neural network consists of N+1 nodes, the inputs of these nodes constitute the input of the recurrent neural network, and their outputs constitute the output of the recurrent neural network. In this way, the recurrent neural network can learn and generate a virtual image brightness adjustment relationship generator that can generate user-specific relationships that conform to the user's physiological characteristics and visual preferences.

[0107] The virtual image brightness adjustment relationship generator can receive a matrix of (N+1) rows and 1 column as an input matrix, wherein each row includes the effective ambient light illumination and the corresponding virtual image brightness correction amount. The output of the virtual image brightness adjustment relationship generator is an output matrix of (N+1) rows and 1 column, wherein each row corresponds to a virtual image brightness that meets the user's physiological characteristics and visual preferences under an effective ambient light illumination, and accordingly, a user-specific relationship that meets the user's physiological characteristics and visual preferences can be constructed.

[0108] Before starting to build a training dataset, it is necessary to obtain user-specific relationships of different users in advance. Fig.12 FIG. 1 is a flowchart of experimentally calibrating a user-specific relationship of a specific user according to an embodiment of the present invention. The process of experimentally calibrating a user-specific relationship of a specific user may be performed in FIG. Figure 2 The following will be implemented in the application scenario of the head-up display shown in Fig.12 A detailed description of each step shown in .

[0109] In step S310: the effective ambient light illumination of the virtual image background area is obtained.

[0110] In some embodiments, the implementation of step S310 is substantially the same as step S110 in the virtual image brightness adjustment method 100 . To simplify the description and avoid redundancy in content, the details of step S310 will not be described repeatedly here.

[0111] In step S320: the virtual image brightness of the head-up display is determined according to the effective ambient light illumination of the virtual image background area and a preset standard brightness adjustment relationship, and the virtual image is projected to the target area.

[0112] The standard virtual image brightness adjustment relationship is a standard model established based on a large amount of user data using statistical methods, which reflects the corresponding relationship between the effective ambient light illumination and the standard virtual image brightness of the head-up display. The standard virtual image brightness adjustment relationship can be represented by a standard brightness adjustment curve (e.g. Fig.15 shown).

[0113] In a specific implementation, after the effective ambient light illumination of the virtual image background area is determined, the virtual image brightness of the head-up display can be determined by referring to the virtual image brightness adjustment relationship, and the head-up display can be controlled to project the virtual image to the target according to the virtual image brightness.

[0114] In step S330: the user-defined virtual image brightness inputted by the user is obtained, and the virtual image brightness correction amount is determined.

[0115] In a specific embodiment, if the user feels that the virtual image brightness of the head-up display is too bright or too dark, the user can input the custom virtual image brightness they expect through the human-computer interaction interface to adjust the virtual image brightness of the head-up display. When the user is satisfied with the virtual image brightness of the head-up display and believes that it meets the personal visual preference, the virtual image brightness of the head-up display will be adopted as the custom virtual image brightness input by the user. By subtracting the custom virtual image brightness input by the user from the standard virtual image brightness, the virtual image brightness correction amount corresponding to the effective ambient light illumination of the current virtual image background area can be obtained.

[0116] S340: For a plurality of effective ambient light illuminations in the virtual image background area, respectively determine corresponding virtual image brightness correction amounts.

[0117] In a specific implementation, steps S310 - S330 may be repeated under different ambient light illuminations to determine virtual image brightness correction amounts under multiple effective ambient light illuminations.

[0118] S350: Determine a user-specific relationship based on the standard brightness adjustment relationship and the virtual image brightness correction amounts under multiple effective ambient light illuminations, wherein the user-specific relationship is a correspondence between the ambient light illumination level applicable to a specific user and the virtual image brightness of the head-up display.

[0119] In a specific embodiment, the standard brightness adjustment relationship can be represented by a standard brightness adjustment curve, and correspondingly, the user-specific relationship can be represented by a user-specific curve. Fig.13 As shown, based on the virtual image brightness correction amount under multiple effective ambient light illuminations, the standard brightness adjustment curve can be dragged to generate a user-specific curve.

[0120] According to an embodiment of the present invention, after the user-specific curve is generated, the virtual image brightness of the head-up display can be adjusted based on the user-specific curve, and the user can evaluate the virtual image brightness of the head-up display. If the user believes that the virtual image brightness of the head-up display adjusted based on the user-specific curve cannot meet his needs, a new user-specific curve is regenerated. The above process is repeated until the new user-specific curve passes the user evaluation.

[0121] Fig.14 A schematic diagram of a head-up display system 400 according to an embodiment of the present invention is shown below. Fig.14 Describe in detail.

[0122] like Fig.14As shown, the head-up display system includes an imaging unit 410, an illuminance sensor 420, a human-computer interaction interface 430 and a control unit 440, wherein the imaging unit 410, the illuminance sensor 420 and the human-computer interaction interface 430 are respectively coupled to the control unit 440. The imaging unit 410 is configured to project a virtual image to a target area, and the virtual image includes, for example, vehicle speed, navigation information, etc. The illuminance sensor 420 is configured to measure the ambient light illuminance of the virtual image background area. The control unit 440 is configured to determine the effective ambient light illuminance of the virtual image background area based on the ambient light illuminance measured by the illuminance sensor 420; the virtual image brightness of the head-up display can be determined according to the effective ambient light illuminance of the virtual image background area and the preset virtual image brightness adjustment relationship, and the imaging unit 410 can be controlled to project the virtual image to the target area, wherein the virtual image brightness adjustment relationship is the corresponding relationship between the effective ambient light illuminance and the virtual image brightness of the head-up display. The control unit 440 is also configured to obtain the user-defined virtual image brightness input by the user; determine multiple effective ambient light illuminances and corresponding virtual image brightness corrections; obtain a user-specific relationship based on the multiple effective ambient light illuminances and corresponding virtual image brightness corrections, based on a preset virtual image brightness adjustment relationship generator, and use the user-specific relationship as a new virtual image brightness adjustment relationship. The user-specific relationship is a corresponding relationship between the effective ambient light illuminance applicable to a specific user and the virtual image brightness of the head-up display.

[0123] According to an embodiment of the present invention, the virtual image brightness adjustment relationship may adopt a user-specific relationship or a standard virtual image brightness adjustment relationship. The standard virtual image brightness adjustment relationship is a standard model established based on a large amount of user data using a statistical method, reflecting the corresponding relationship between the effective ambient light illumination and the standard virtual image brightness of the head-up display. The standard virtual image brightness adjustment relationship may be represented by a standard brightness adjustment curve (e.g. Fig.15 shown).

[0124] According to one embodiment of the present invention, the virtual image brightness adjustment database may be a data queue or a data set. Fig.14 As shown, the control unit 440 can be configured to update the user-defined virtual image brightness and its corresponding effective ambient light illumination into the virtual image brightness adjustment database. Optionally, the control unit 440 is also configured to delete the oldest customized virtual image brightness and its corresponding effective ambient light illumination in the virtual image brightness adjustment database.

[0125] According to one embodiment of the present invention, Fig.14As shown, the control unit 440 can also be configured to, if the effective ambient light illumination has one or more corresponding custom virtual image brightnesses in the virtual image brightness adjustment database, use the difference between the latest custom virtual image brightness and the standard virtual image brightness corresponding to the effective ambient light illumination as the virtual image brightness correction amount corresponding to the effective ambient light illumination; if the effective ambient light illumination has no corresponding custom virtual image brightness in the virtual image brightness adjustment database, set the virtual image brightness correction amount corresponding to the effective ambient light illumination to zero.

[0126] According to one embodiment of the present invention, Fig.14 As shown, the control unit 440 can also be configured to generate an input matrix based on multiple effective ambient light illuminations and corresponding virtual image brightness corrections; input the input matrix into a virtual image brightness adjustment relationship generator, and determine a user-specific relationship based on the output of the virtual image brightness adjustment relationship generator.

[0127] Compared with the prior art, the embodiment of the present invention provides a head-up display virtual image brightness adjustment method and head-up display system 400. Among them, based on the preset virtual image brightness adjustment relationship generator, a user-specific relationship is obtained, and the user-specific relationship is more in line with the user's physiological characteristics and visual preferences. Adjusting the virtual image brightness of the head-up display based on the user-specific relationship can effectively reduce driving interference caused by inappropriate brightness and significantly enhance safety during driving.

[0128] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for adjusting the brightness of a virtual image of a head-up display, comprising: S110: Acquire the effective ambient light illumination of the virtual image background area; S120: determining the virtual image brightness of the head-up display according to the effective ambient light illumination of the virtual image background area and a preset virtual image brightness adjustment relationship, and projecting the virtual image to the target area, wherein the virtual image brightness adjustment relationship is a corresponding relationship between the effective ambient light illumination and the virtual image brightness of the head-up display; S130: Obtaining the custom virtual image brightness input by the user; S140: Determine a plurality of effective ambient light illuminations and corresponding virtual image brightness correction values; S150: obtaining a user-specific relationship based on the plurality of effective ambient light illuminations and the corresponding virtual image brightness correction amounts and based on a preset virtual image brightness adjustment relationship generator, wherein the user-specific relationship is a corresponding relationship between the effective ambient light illumination and the virtual image brightness of the head-up display applicable to a specific user; S160: Using the user-specific relationship as a new virtual image brightness adjustment relationship.

2. The virtual image brightness adjustment method according to claim 1, wherein: The effective ambient light illumination is represented by an ambient light illumination level.

3. The virtual image brightness adjustment method according to claim 1, wherein: The preset virtual image brightness adjustment relationship is a user-specific relationship or an adjustment relationship, and the adjustment relationship is a corresponding relationship between the effective ambient light illumination and the standard virtual image brightness of the head-up display.

4. The virtual image brightness adjustment method according to claim 1, wherein: Step S130 includes: The customized virtual image brightness and its corresponding effective ambient light illumination are updated into the virtual image brightness adjustment database.

5. The virtual image brightness adjustment method according to claim 4, wherein: Step S130 also includes: The oldest custom virtual image brightness and its corresponding effective ambient light illumination in the virtual image brightness adjustment database are deleted.

6. The virtual image brightness adjustment method according to claim 4, wherein: In step S140, If the effective ambient light illumination has one or more corresponding custom virtual image brightnesses in the virtual image brightness adjustment database, the difference between the latest custom virtual image brightness and the standard virtual image brightness corresponding to the effective ambient light illumination is used as the virtual image brightness correction amount corresponding to the effective ambient light illumination; If the effective ambient light illumination has no corresponding custom virtual image brightness in the virtual image brightness adjustment database, the virtual image brightness correction amount corresponding to the effective ambient light illumination is set to zero.

7. The virtual image brightness adjustment method according to claim 1, wherein: Step S150 includes: Generate an input matrix according to the multiple effective ambient light illuminations and corresponding virtual image brightness correction amounts; The input matrix is ​​input into the virtual image brightness adjustment relationship generator, and the user-specific relationship is determined based on the output of the virtual image brightness adjustment relationship generator.

8. The virtual image brightness adjustment method according to claim 1, wherein: The virtual image brightness adjustment method further comprises: A virtual image brightness adjustment relationship generator is established based on a recurrent neural network.

9. The virtual image brightness adjustment method according to claim 8, wherein: The steps of establishing a virtual image brightness adjustment relationship generator based on a recurrent neural network include: Acquire a training data set, wherein each set of training data in the training data set includes a plurality of preset effective ambient light illuminations and a virtual image brightness and a virtual image brightness correction value applicable to a specific user corresponding to each effective ambient light illumination; The training data set is input into the recurrent neural network to establish a virtual image brightness adjustment relationship generator.

10. A head-up display system, comprising: Imaging unit; a light sensor configured to measure ambient light levels; A human-computer interaction interface configured to output a custom virtual image brightness in response to a user operation; and A control unit is coupled to the imaging unit, the light intensity sensor and the human-computer interaction interface respectively, and the control unit is configured to: determine the effective ambient light intensity of the virtual image background area based on the ambient light intensity measured by the light intensity sensor; determine the virtual image brightness of the head-up display according to the effective ambient light intensity of the virtual image background area and a preset virtual image brightness adjustment relationship, and control the imaging unit to project the virtual image to the target area; and obtain the custom virtual image brightness input by the user; A plurality of effective ambient light illuminations and corresponding virtual image brightness correction values ​​can be determined; a user-specific relationship can be obtained based on the plurality of effective ambient light illuminations and corresponding virtual image brightness correction values ​​and a preset virtual image brightness adjustment relationship generator, and the user-specific relationship can be used as a new virtual image brightness adjustment relationship; Wherein, the virtual image brightness adjustment relationship is the corresponding relationship between the effective ambient light illumination and the virtual image brightness of the head-up display; The user-specific relationship is a corresponding relationship between the effective ambient light illumination applicable to a specific user and the virtual image brightness of the head-up display.

11. The head-up display system according to claim 10, wherein: The preset virtual image brightness adjustment relationship is a user-specific relationship or a standard virtual image brightness adjustment relationship, and the standard virtual image brightness adjustment relationship is a corresponding relationship between the effective ambient light illumination and the standard virtual image brightness of the head-up display.

12. The head-up display system according to claim 10, wherein: The control unit is configured to update the user-defined virtual image brightness and its corresponding effective ambient light illumination into a virtual image brightness adjustment database.

13. The head-up display system according to claim 12, wherein: The control unit is configured to delete the oldest custom virtual image brightness and its corresponding effective ambient light illumination in the virtual image brightness adjustment database.

14. The head-up display system according to claim 12 or 13, wherein: The control unit is configured to use the difference between the latest custom virtual image brightness and the standard virtual image brightness corresponding to the effective ambient light illumination as the virtual image brightness correction amount corresponding to the effective ambient light illumination if the effective ambient light illumination has one or more corresponding custom virtual image brightnesses in the virtual image brightness adjustment database; and to set the virtual image brightness correction amount corresponding to the effective ambient light illumination to zero if the effective ambient light illumination has no corresponding custom virtual image brightness in the virtual image brightness adjustment database.

15. The head-up display system according to claim 10, wherein: The control unit is configured to generate an input matrix according to the multiple effective ambient light illuminations and corresponding virtual image brightness correction amounts; The input matrix is ​​input into the virtual image brightness adjustment relationship generator, and the user-specific relationship is determined based on the output of the virtual image brightness adjustment relationship generator.

Images