A HUD backlight display system and method

Abstract

The present invention discloses a HUD backlight display system and method, which specifically relates to the field of HUD backlight display technology. A micro-light sensor array is used to monitor the light environment within the driver's field of view in real time to obtain a real-time ambient display light distribution map. A prediction model constructed based on a recurrent neural network and a long short-term memory network is integrated with historical driving scene data, real-time vehicle status and changes in the ambient light ahead to accurately predict future light intensity distribution. The current light intensity value of the gaze area is extracted from the ambient display light distribution map, and the target backlight brightness of the gaze area in the next prediction step is calculated in combination with the predicted light intensity distribution of the next prediction step. The backlight display system is globally pre-adjusted according to backlight pre-adjustment parameters. Light intensity prediction and gaze area optimization are achieved to solve the problem of dynamic mismatch between display brightness and ambient light in traditional HUD systems.

Description

Technical Field

[0001] The present invention relates to the field of HUD backlight display technology, and more specifically, to a HUD backlight display system and method. Background Art

[0002] In complex dynamic lighting scenarios, automotive head-up display (HUD) systems need to adjust backlight brightness in real time to balance display clarity and visual comfort. In such scenarios, ambient light is highly non-uniform and sudden; on nighttime urban roads, light sources such as streetlights, headlights, and advertising screens create high-frequency flickering and localized highlights. Traditional HUD systems rely on single-point light sensors or static threshold adjustment strategies, which can only respond to changes in the global mean light intensity and cannot capture the local light intensity gradient distribution within the driver's field of view (such as direct sunlight on a specific area of ​​the windshield or interference from the high beams of oncoming vehicles). This results in a dynamic mismatch between backlight brightness and ambient light. When the displayed content is too dark, the readability of the information is reduced, while when it is too bright, it causes glare, seriously threatening driving safety.

[0003] Existing technologies face the following core issues: 1) Data from a single ambient light sensor alone cannot perceive differences in light intensity distribution within the driver's field of view; 2) Dynamic prediction and adjustment lag: The fixed-frequency light intensity prediction model is insufficiently updated in non-steady-state scenarios (light intensity change rate >10% / s) and cannot respond promptly to sudden changes in light intensity gradients; At the same time, brightness adjustment relies on a linear compensation algorithm that does not incorporate a spatiotemporal attention mechanism, resulting in poor adaptability to multi-scale light and shadow patterns (such as headlight spots and tunnel gradient light). Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides a HUD backlight display system and method. By building a prediction model combining a recurrent neural network and a long short-term memory network, integrating historical driving scene data, real-time vehicle status and changes in the ambient light ahead, light intensity prediction and gaze area optimization are achieved to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a HUD backlight display system, comprising:

[0006] The ambient display light collection module integrates multiple micro light sensors around the windshield. The micro light sensors cover key areas within the driver's field of view and output a real-time ambient display light distribution map.

[0007] The light intensity trend prediction module builds a light intensity prediction model that combines a recurrent neural network and a long short-term memory network. The light intensity prediction model is pre-trained based on historical driving scene data. It inputs a real-time ambient light distribution map, current vehicle status data, and the ambient light intensity trend of the vehicle ahead, and outputs a predicted light intensity distribution.

[0008] The brightness pre-adjustment module generates corresponding backlight pre-adjustment parameters based on the predicted light intensity distribution; extracts the current light intensity value of the gaze area from the ambient display light distribution map, and calculates the target backlight brightness of the gaze area at the next prediction step based on the predicted light intensity distribution of the next prediction step; and performs global pre-adjustment of the backlight display system based on the backlight pre-adjustment parameters;

[0009] The sight area optimization module obtains eye tracking data through an integrated eye tracking device, determines the driver's current gaze area based on the eye tracking data, and monitors the coordinates of the driver's gaze point in real time. It uses a Mini-LED array to divide the display area into several intervals, supporting independent brightness adjustment for each interval. It is equipped with a fast-response controller to process interval backlight adjustment commands, and the brightness of the current gaze area has the highest priority.

[0010] Preferably, the process of building the light intensity prediction model includes:

[0011] Data preprocessing and frequency domain decomposition: Input the ambient light distribution map, vehicle status data, and weather information, and through frequency domain decomposition and enhancement and time-space alignment, output the frequency domain enhanced light intensity feature map;

[0012] Multi-scale spatiotemporal feature extraction is performed based on multi-scale convolution fusion and spatiotemporal attention mechanism. The light intensity feature map after frequency domain enhancement is input and the multi-scale spatiotemporal joint feature vector is output, which includes local texture, global distribution and long-distance gradient features.

[0013] Input multi-scale spatiotemporal joint feature vector and historical light intensity prediction error, and output predicted light intensity distribution map and dynamically optimized LSTM parameters;

[0014] During the training process, the loss function value is calculated in real time. When the loss value does not decrease significantly after multiple consecutive training cycles or reaches the preset threshold, the model is judged to have converged. The trained model parameters are saved, and a trained light intensity prediction model is generated and deployed in the HUD backlight display system.

[0015] Preferably, the multi-scale convolution fusion includes:

[0016] Parallel convolution branch: 1x1, 3x3, and 5x5 convolution kernels are used to extract three sets of outputs in parallel, namely local high-frequency texture, mid-range light intensity distribution, and global light intensity distribution pattern;

[0017] Feature map splicing: splice the three sets of outputs along the channel dimension to form a multi-scale fusion feature map;

[0018] The spatiotemporal attention mechanism includes: calculating the spatiotemporal attention weights of the real state Re and the virtual state Im respectively through multi-head self-attention, and outputting the attention weight matrix; using the vehicle acceleration as the gating signal to adjust the distribution ratio of the attention weight in the time dimension.

[0019] Preferably, the target backlight brightness is: any one of the first target backlight brightness, the second target backlight brightness, and the third target backlight brightness; the first target backlight brightness is obtained based on the preliminary target brightness and the glare threshold constraint; the second target backlight brightness is obtained based on forced correction of contrast to ensure that visibility of displayed content is prioritized; the third target backlight brightness is obtained based on dynamic balance between multi-objective optimization function and reinforcement learning.

[0020] Preferably, the first target backlight brightness is obtained in the following manner:

[0021] Input the average ambient light intensity Ea and HUD reflectivity R of the current viewing area HUD , predict the light intensity change ΔL pred , the initial target backlight brightness L is obtained by the following formula prel ,

[0022]

[0023] Among them, k s is the safety factor, which is adjusted dynamically according to vehicle speed;

[0024] Glare threshold constraint, input preliminary target backlight brightness L prel and glare threshold L X , the first target backlight brightness AL is obtained through the following model target :AL target =min(L prel ,L X ).

[0025] Preferably, the second target backlight brightness is obtained in the following manner:

[0026] Enter the initial target backlight brightness AL target , background brightness (average light intensity in the non-fixated area) and contrast threshold;

[0027] Verify whether the preliminary target backlight brightness meets the contrast standard: if the ratio of the preliminary target backlight brightness to the background brightness is less than the preset value, a forced correction is triggered;

[0028] Directly correct the target backlight brightness to 3 times the background brightness and output the second target backlight brightness BL target ;

[0029] If the corrected target backlight brightness is greater than the glare threshold, it is processed according to priority:

[0030] Gaze zone: temporarily allows the target backlight brightness to exceed the glare threshold, but starts gradual attenuation;

[0031] Non-attention area: Force the target backlight brightness to be less than half of the glare threshold to disperse the interference of high-brightness areas on the driver.

[0032] Preferably, the multi-objective adjustment optimization module divides the display area into several intervals based on physical hardware mapping; inputs the second target backlight brightness and the multi-objective fitness function, and outputs the third target backlight brightness with the multi-objective fitness function as a constraint, where the third target backlight brightness is a set of brightness of each interval.

[0033] Preferably, the operation process of the multi-objective adjustment optimization module includes the following steps:

[0034] Based on the physical hardware mapping, the display area is divided into several intervals, i represents the interval index, N represents the total number of intervals; based on the time sequence, the time step index t is set, and T represents the time window length;

[0035] The weight coefficient is set based on the user's gaze ratio, denoted as w t，i ;

[0036] Obtain the clarity index R1, energy consumption index R2, and comfort index R3 of the HUD backlight display system;

[0037]

[0038] Among them, I tar,i is the target backlight brightness of the i-th interval (obtained by the light intensity prediction model outputting the predicted light intensity distribution), I real,i is the actual brightness after adjustment, I max To allow maximum brightness, I min To allow the minimum brightness;

[0039]

[0040] Among them, kh is the hardware power consumption coefficient calibrated by experiments;

[0041]

[0042] Among them, UGB represents the unified glare index, λ p is the smoothing coefficient, Li is the interval brightness, φ i is the solid angle, L b is the background brightness, P i is the position factor;

[0043] The multi-objective fitness function LR of multi-objective optimization correction is obtained through the following model:

[0044] LR=w1·R1+w2·R2+w3·R3

[0045] Wherein, w1, w2, and w3 represent the weight coefficients of each item, and w1+w2+w3=1;

[0046] The Actor network is used to generate a brightness adjustment action corresponding to the third target backlight brightness, realize backlight brightness adjustment, and control the brightness adjustment amount to follow Gaussian distribution.

[0047] Preferably, the multi-objective fitness function drives the Actor network and optimizes the Critic network, which is optimized by the state value function V(s t ) Evaluate the long-term comprehensive benefits of implementing brightness adjustment actions; combine the immediate reward signal r t Quantify the overall strategic value from the current state, based on potential future benefits (e.g., long-term comfort);

[0048] r t =LR=w1·R1+w2·R2+w3·R3

[0049] The function of the Critic network is: input the current state s t , output state value V(s t ); the state value function satisfies the following formula:

[0050]

[0051] Where E(·) represents the expected operation function, which reflects the weighted average of all possible future paths after performing the brightness adjustment action in state s; k is the index of the future time step starting from the current time step t; γ represents the discount factor, which controls the degree of attenuation of future rewards and ranges from 0 to 1;

[0052] The evaluation results of the Critic network are fed back to the Actor network to guide its parameter adjustment; by adjusting the brightness adjustment action ΔIi, V(s t ), so that the long-term cumulative reward is optimal.

[0053] Preferably, an ambient display light preprocessing module is included after the ambient display light acquisition module and before the light intensity change trend prediction module. Each sensor collects ambient light intensity data in real time to form a discrete time series data set; a spatial interpolation algorithm is used to convert the discrete time series data set into a continuous light intensity distribution map on the windshield; the light intensity distribution map is smoothed (for example, Gaussian filtering) to filter out noise and generate a smooth ambient display light distribution map.

[0054] Preferably, the ambient display light preprocessing module includes an anomaly rejection unit that cleans the discrete time series data set collected by the light sensor based on a dynamic neighborhood distance threshold method; by calculating the mean and standard deviation of the distance between each light intensity data point and its adjacent points, outliers (such as sudden flash interference) are dynamically rejected to improve data reliability;

[0055] Multiple micro-light sensors (photodiode arrays) integrated around the windshield collect discrete time series data sets within the driver's field of view to form a time series light intensity matrix;

[0056] For each data point in the light intensity matrix, calculate the mean and standard deviation of its Euclidean distance to its k adjacent data points. Set the dynamic threshold to the weighted sum of the mean and three standard deviations of the Euclidean distance. (For normally distributed data, approximately 99.7% of the data points fall within the range of the mean ± three standard deviations; extreme values ​​outside this range are considered outliers.)

[0057] If the mean neighborhood distance of a data point exceeds the dynamic threshold, it is considered an outlier and removed.

[0058] Preferably, the quantification method of real-time lighting complexity is:

[0059] Calculate the percentage of light intensity change per unit time, recorded as ΔI rate ;

[0060] Calculate frequency domain energy distribution: Apply short-time Fourier transform to the light intensity time series data, decompose it into frequency domain real state Re and virtual state, and calculate the high-frequency energy proportion:

[0061]

[0062] Where f represents frequency, and fc is the cutoff frequency, which is used to distinguish high-frequency noise from low-frequency gradients;

[0063] When the high-frequency energy ratio Ehigh>30%, it indicates the presence of a high-frequency interference source, and the high-frequency suppression factor η needs to be enabled to reduce the noise weight;

[0064] Calculate the variance of brightness differences:

[0065]

[0066] Brightness difference variance σ gradient The higher the value, the more uneven the spatial light intensity distribution is, and the multi-scale convolution fusion module needs to be activated to enhance local feature extraction.

[0067] Dynamic scene classification: The light and shadow complexity of the scene is divided into several levels based on the light intensity change rate and the proportion of high-frequency energy; in one possible embodiment, (ΔI rate +1)(Ehigh +1)(σ gradient +1) quantifies the degree of light and shadow complexity.

[0068] To achieve the above object, the present invention provides the following technical solution: a HUD backlight display method, comprising the following steps:

[0069] Step 1: Light field acquisition: Multiple micro-light sensors integrated around the windshield collect ambient light intensity data within the driver's field of view in real time, generating a two-dimensional ambient light distribution map.

[0070] Step 2: Light Intensity Prediction: Build a light intensity prediction model that integrates a recurrent neural network and a long short-term memory network. Input the ambient light distribution map, vehicle status data, and the light intensity trend of the vehicle ahead, and output the predicted light intensity distribution for the next 0.5 seconds. The light intensity prediction model uses multi-scale convolution to extract local high-frequency textures, mid-range gradient light, and global light intensity patterns. Combined with a spatiotemporal attention mechanism, it dynamically assigns weights to improve adaptability to complex scenarios.

[0071] Step 3: Optimize the visual area: Obtain eye tracking data through integrated eye tracking equipment, determine the driver's current gaze area based on the eye tracking data, and monitor the coordinates of the driver's gaze point in real time; use a Mini-LED array to divide the display area into several intervals, and support independent brightness adjustment for each interval; be equipped with a fast-response controller to process interval backlight adjustment commands, and the brightness of the current gaze area has the highest priority.

[0072] Technical effects and advantages of the present invention:

[0073] (1) The HUD backlight display system provided by the present invention monitors the light environment within the driver's field of view in real time through a micro-light sensor array and generates a dynamic light distribution map; a prediction model constructed based on a recurrent neural network and a long short-term memory network integrates historical driving scene data, real-time vehicle status and changes in the ambient light ahead to accurately predict future light intensity distribution; light intensity prediction and gaze area optimization are achieved to solve the problem of blurred HUD display under complex light conditions (such as flashing lights at night), improve visibility and comfort, and reduce driver distraction; through multi-objective optimization, clarity, energy consumption and comfort are balanced to solve the failure problem of single-objective optimization (such as contrast only) under sudden environmental changes, and adapt to diverse driving scenarios.

[0074] (2) The HUD backlight display system provided by the present invention generates backlight parameters based on the prediction model, combines the light intensity value of the gaze area tracked by the eye tracker, calculates the target brightness and globally pre-adjusts the backlight system; adopts Mini-LED zone light control technology, and prioritizes the brightness of the gaze area through a fast response controller to achieve dynamic zone brightness adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 This is a structural block diagram of the initial HUD backlight display system of the present invention.

[0076] Figure 2 Build a flow chart for the light intensity prediction model of the present invention.

[0077] Figure 3 This is a flow chart of backlight brightness adjustment based on multi-objective fitness function of the present invention. DETAILED DESCRIPTION

[0078] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0079] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0080] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0081] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0082] Example 1, see Figure 1 The initial HUD backlight display system structure block diagram, the present invention provides the following Figure 1 A HUD backlight display system shown includes:

[0083] The ambient display light collection module integrates multiple micro light sensors (such as a photodiode array, with at least 5-7 sampling points and a sampling frequency of ≥100Hz) around the windshield. The micro light sensors cover key areas within the driver's field of view and output a real-time ambient display light distribution map, reflecting changes in light intensity within the driver's field of view.

[0084] The specific implementation process includes arranging 7 groups of light-sensitive sensor arrays in an arc shape along the lower edge of the windshield, and the spatial layout meets the following requirements:

[0085] a) Covering the driver's primary field of view with a horizontal field of view angle of 120° and a vertical field of view angle of 40°;

[0086] b) The distance between sensors is ≤ 15 cm, ensuring that adjacent detection areas have a 10% overlap rate;

[0087] The light intensity trend prediction module builds a light intensity prediction model that combines a recurrent neural network (RNN) and a long short-term memory network (LSTM). The light intensity prediction model is pre-trained based on historical driving scene data. It inputs a real-time ambient light distribution map, current vehicle status data (such as speed, acceleration, GPS location), and the ambient light intensity trend of the vehicle ahead, and outputs a predicted light intensity distribution.

[0088] It is necessary to further explain in the embodiments of the present invention that LSTM is used to process time series changes, and CNN is used to extract spatial light and shadow pattern features to improve adaptability to complex scenes (such as flashing lights in cities at night); the update frequency of the light intensity prediction model is dynamically adjusted according to the real-time light and shadow complexity (in stable scenes, it is updated according to the base frequency; in unstable scenes, the light intensity change rate is positively correlated with the update frequency; for example, when the light intensity change rate is greater than 10% / s, the update frequency is increased to 10Hz);

[0089] The brightness pre-adjustment module generates corresponding backlight pre-adjustment parameters based on the predicted light intensity distribution; extracts the current light intensity value of the gaze area from the ambient display light distribution map, combines it with the predicted light intensity distribution of the next prediction step (such as 0.5 seconds), and calculates the target backlight brightness of the gaze area in the next prediction step; and performs global pre-adjustment of the backlight display system based on the backlight pre-adjustment parameters;

[0090] The sight area optimization module obtains eye tracking data through an integrated eye tracking device, determines the driver's current gaze area based on the eye tracking data, and monitors the driver's gaze point coordinates (a rectangular area centered on the gaze point) in real time; it uses a Mini-LED array to divide the display area into several intervals (the display area is the possible full set of gaze areas, and the dynamic constraints of light and text ensure that the gaze area is always within the display area), supports independent brightness adjustment of each interval; is equipped with a fast-response controller to process interval backlight adjustment instructions; the brightness of the current gaze area has the highest priority (prioritizes optimization of the interval backlight brightness of the current gaze area).

[0091] It is necessary to further explain in the embodiments of the present invention that Figure 2 The light intensity prediction model construction process includes:

[0092] Step S11, data preprocessing and frequency domain decomposition: Input the ambient display light distribution map, vehicle status data and weather information, and output the frequency domain enhanced light intensity feature map (including the real state Re and the virtual state Im) through frequency domain decomposition (applying short-time Fourier transform to the light intensity time series data, based on the high-frequency suppression factor of the environment) and enhancement and time-space alignment (aligning the frequency domain features with the vehicle status data through timestamps to generate a time-space joint input matrix); specifically including:

[0093] The photodiode array around the windshield collects two-dimensional time-series data of light intensity within the driver's field of view to form an ambient light distribution map. In one possible embodiment, dynamic thresholds are set based on driving scenarios (e.g., nighttime urban roads, highways), outliers (e.g., sudden high-beam interference) are removed, and bilinear interpolation is used to fill in data gaps.

[0094] The light intensity time series is subjected to a short-term Fourier transform (STFT) and decomposed into the frequency domain real state Re (amplitude) and imaginary state Im (phase). A high-frequency suppression factor η is introduced to dynamically adjust the frequency domain sensitivity based on weather data to dynamically suppress high-frequency noise (such as raindrop reflections).

[0095] Explanation: The virtual state Im represents the phase component of the light intensity signal in the frequency domain, reflecting the time offset relationship between different frequency components. For example, a phase jump may correspond to the instantaneous interference of a sudden strong light source (such as the high beam of an oncoming vehicle); phase continuity corresponds to the long-term influence of a stable light source (such as a street lamp); its functions include: dynamic interaction modeling (in the spatiotemporal attention mechanism, the virtual state Im interacts with the real state Re to capture the spatiotemporal correlation of light intensity changes), motion compensation (when the vehicle acceleration is >0.3g, the virtual state Im corrects the phase offset through Kalman filtering to avoid distortion of light intensity prediction caused by sudden acceleration / deceleration of the vehicle); anomaly detection (drastic changes in the phase mutation Im can trigger abnormal event markers such as sudden glare, assisting the model to respond quickly);

[0096] Step S12: extract multi-scale spatiotemporal features based on multi-scale convolution fusion and spatiotemporal attention mechanism, input the intensity feature map after frequency domain enhancement, and output a multi-scale spatiotemporal joint feature vector, which includes local texture, global distribution and long-distance gradient features. In the specific implementation:

[0097] The multi-scale convolution fusion includes:

[0098] Parallel convolution branch: 1x1, 3x3, and 5x5 convolution kernels are used to extract three sets of outputs in parallel: local high-frequency texture (such as car light spots), mid-range light intensity distribution (such as gradient light in a tunnel), and global light intensity distribution pattern (such as uniform lighting on a highway).

[0099] Feature map splicing: splice the three sets of outputs along the channel dimension to form a multi-scale fusion feature map;

[0100] The spatiotemporal attention mechanism includes: calculating the spatiotemporal attention weights for the real state Re and the virtual state Im respectively through multi-head self-attention, focusing on dynamically changing areas (such as flashing traffic lights), and outputting an attention weight matrix; using vehicle acceleration as a gating signal to adjust the distribution ratio of attention weights in the time dimension;

[0101] Step S13, LSTM dynamic parameter optimization and training: input the multi-scale spatiotemporal joint feature vector and the historical light intensity prediction error (mean square error MSE), output the predicted light intensity distribution map (time step Δt = 0.5s), and dynamically optimized LSTM parameters (learning rate, number of hidden units). The implementation methods include:

[0102] RNN-LSTM hybrid architecture:

[0103] First layer: Bidirectional RNN: captures short-term temporal dependencies (<1s) and outputs preliminary temporal features;

[0104] Second layer: LSTM network: processes long-term time series (>1s) and dynamically suppresses low-frequency noise through forget gates;

[0105] Dynamic parameter optimization: Agent parameter mapping: Mapping LSTM learning rate (lr) and number of hidden units (n_units) to agent individual parameters;

[0106] Two-stage optimization strategy: Dynamic optimization is performed using a balance strategy between individual learning and collective learning stages. The individual learning stage involves agents adjusting parameters based on historical optimal solutions to accelerate convergence. The collective learning stage involves introducing a diversity maintenance mechanism to avoid falling into local optimality.

[0107] Model training and validation: The historical dataset is divided into a training set (70%) and a test set (30%), preserving temporal continuity. The Adam optimizer is used for training, and the mean square error and model response speed are monitored to ensure a balance between prediction accuracy and real-time performance.

[0108] Step S14, model convergence and deployment: Calculate the loss function value (such as the mean square value of the prediction error) in real time during the training process. When the loss value does not decrease significantly for multiple consecutive training cycles (such as 10 cycles) or reaches a preset threshold, the model is determined to have converged. Save the trained model parameters, generate a trained light intensity prediction model and deploy it in the HUD backlight display system, and output the next prediction step (such as 0.5 seconds) to predict the light intensity distribution. Test the performance of the light intensity prediction model in real driving scenarios, including typical scenarios such as urban nighttime (high-frequency flickering), highways (low-frequency gradient), and tunnel entry and exit (light intensity mutation). Fine-tune the model parameters according to the test results, such as adjusting the multi-objective weights or optimizing the attention mechanism to ensure generalization ability.

[0109] It should be further explained in the embodiment of the present invention that the target backlight brightness is any one of the first target backlight brightness, the second target backlight brightness, and the third target backlight brightness; the first target backlight brightness is obtained as follows:

[0110] Step S21: Input the average ambient light intensity Ea and HUD reflectivity R of the current viewing area.HUD (value range is 0.2~0.3), predict the change of light intensity ΔL pred , the initial target backlight brightness L is obtained by the following formula prel ,

[0111]

[0112] Among them, k s is the safety factor (ranging from 0.6 to 0.8), which is dynamically adjusted according to vehicle speed (the higher the speed, the smaller the safety factor);

[0113] The prediction compensation logic is as follows: if the predicted light intensity change is positive, the brightness is pre-increased to offset the future increase in ambient light (such as when exiting a tunnel); if the predicted light intensity change is negative, the brightness is pre-reduced to avoid overexposure (such as when entering a shadowed area). HUD reflectivity refers to the efficiency with which the windshield or HUD optical system reflects display light, directly affecting the visibility of displayed content in ambient light. If the HUD reflectivity is lower than 0.2, the displayed content has difficulty competing with the ambient light, resulting in blurred information. If it is higher than 0.3, it is prone to glare, especially at night or in low-light environments.

[0114] Step S22: Glare threshold constraint, input preliminary target backlight brightness L prel , current day / night mode (from GPS time or light intensity sensor) and glare threshold L X , output the first target backlight brightness; the first target backlight brightness AL is obtained through the following model target :AL target =min(L prel ,L X );

[0115] Different glare thresholds are set according to the day and night modes; in the embodiment of the present invention, L X =500cd / m 2 , L in night mode X =350cd / m 2 ;

[0116] In a possible embodiment, in order to ensure a balance between display clarity and visual comfort, the following steps are included: Step S23, forced correction of contrast: inputting a preliminary target backlight brightness AL target, background brightness (average light intensity in the non-gaze area) and contrast threshold; verify whether the preliminary target backlight brightness meets the contrast threshold: if the ratio of the preliminary target backlight brightness to the background brightness is less than the contrast threshold (based on actual conditions, the embodiment of the present invention is set to 3. When the ratio of the preliminary target backlight brightness to the background brightness is greater than 3, the displayed content is always visible; when the ratio is less than 3, the time required for the driver to read the HUD information increases), a forced correction is triggered and a second target backlight brightness BL is output. target Explanation: In the embodiment of the present invention, the contrast threshold is set to 3, and the target backlight brightness is directly corrected to 3 times the background brightness; if the corrected target backlight brightness is greater than the glare threshold, it is processed according to priority:

[0117] Gaze zone: temporarily allows the target backlight brightness to exceed the glare threshold, but starts a gradual attenuation (e.g., reducing by 5% every 0.5 seconds);

[0118] Non-attention area: Force the target backlight brightness to be less than half of the glare threshold to disperse the interference of high-brightness areas on the driver.

[0119] The explanation states that forced correction may conflict with glare suppression; for example, when the ambient light is extremely high, higher brightness may be required according to contrast requirements, but this may exceed the glare threshold and cause discomfort to the driver; therefore, the system needs to find a balance between contrast requirements and glare suppression; solutions include dynamically adjusting the maximum brightness threshold, or giving priority to meeting contrast requirements in extreme cases, while using other means (such as reducing the brightness of non-gaze areas) to alleviate glare.

[0120] Background: Single-objective optimization (such as contrast or glare suppression alone) can easily lead to strategy failure due to sudden environmental changes. Multi-objective optimization enhances adaptability through comprehensive indicators. To improve overall system performance and balance display clarity, energy consumption, and visual comfort (such as glare suppression) in brightness adjustment, the display area is divided into several intervals based on physical hardware mapping, enabling refined management of each interval. Reinforcement learning is used to optimize the backlighting of each interval in real time. Based on this, Example 2 is set up.

[0121] Example 2: The difference between this embodiment of the present invention and Example 1 is that it also includes

[0122] The multi-objective adjustment optimization module divides the display area into several intervals based on physical hardware mapping; inputs the second target backlight brightness and the multi-objective fitness function, and outputs the third target backlight brightness with the multi-objective fitness function as a constraint. The third target backlight brightness is the brightness set of each interval;

[0123] See Figure 3 The backlight brightness adjustment flow chart based on the multi-objective fitness function is shown in the figure. The operation process of the multi-objective adjustment optimization module includes the following steps:

[0124] Step S31: Divide the display area into several intervals based on the physical hardware mapping, with i representing the interval index and N representing the total number of intervals; set the time step index t based on the time sequence, with T representing the time window length;

[0125] Step S32: Set a weight coefficient based on the user's gaze ratio for the interval, denoted as w t，i ;

[0126] Specifically: the frequency of the user's vision falling into the i-th interval is recorded as f s，i , through the formula Calculate the weight coefficient;

[0127] Step S33: obtaining a clarity index R1, an energy consumption index R2, and a comfort index R3 of the HUD backlight display system;

[0128] Step 3.1

[0129]

[0130] Among them, I tar,i is the target backlight brightness of the i-th interval (obtained by the light intensity prediction model outputting the predicted light intensity distribution), I real,i is the actual brightness after adjustment, I max To allow maximum brightness, I min To allow the minimum brightness;

[0131] Step 3.2

[0132]

[0133] Among them, kh is the hardware power consumption coefficient calibrated through experiments, including:

[0134] Set up the experimental environment: Fix the HUD display area in a dark room and use a high-precision power meter to measure the actual power consumption at different brightness levels.

[0135] Data fitting: Assume that the power consumption P and brightness I satisfy the following relationship:

[0136] P=kh·I 2 +b·I+c

[0137] The experimental data were fitted by the least square method, and kh was extracted as the dominant coefficient;

[0138] Step 3.3

[0139]

[0140]

[0141] Among them, UGB represents the unified glare index, λ p is the smoothing coefficient (usually 0.1 to 0.3), Li is the interval brightness, φ i is the solid angle, L b is the background brightness, P i is the position factor; |I t -I t-1 | represents the absolute value of the brightness difference between adjacent time sequences.

[0142] Explanation: Solid angle refers to the spatial coverage of the light source in the driver's field of view. The calculation formula is the light source luminous area Ai (unit: m 2 ) to the square of the distance di (unit: m) from the light source to the eye; the position factor quantifies the effect of the light source position on glare perception and is defined as Where θi is the angle between the center of the light source and the driver's line of sight. The larger the angle (the further the light source deviates from the center of sight), the larger the position factor value is, and the glare effect is weakened.

[0143] Explanation: The human eye's perception of brightness is nonlinear. For example, in high-brightness environments, the sensitivity to changes in light intensity is low, while in low-brightness environments, small differences in brightness can cause significant discomfort. The background brightness in the driving environment (such as strong light during the day and weak light at night) has a significant impact on glare perception. The formula enhances the responsiveness to low-background-brightness scenes, ensuring that potential glare can be identified and suppressed in low-light conditions such as at night or in tunnels. The position of the light source (such as near the center or edge of the line of sight) and its physical size (solid angle) contribute differently to glare. The formula reasonably evaluates the impact of light sources in different areas by dynamically adjusting the weights of the position factor and solid angle. For example, the glare effect of edge light sources is appropriately weakened, while interfering light sources near the center of the line of sight are focused on to avoid evaluation distortion caused by position deviation.

[0144] Step S34: Obtain the multi-objective fitness function LR of the multi-objective optimization correction through the following model:

[0145] LR=w1·R1+w2·R2+w3·R3

[0146] Wherein, w1, w2, and w3 represent weight coefficients of each item respectively. The weight coefficients are dynamically adjusted according to the driving scenario, w1+w2+w3=1. The driving scenario includes at least: urban night scene, highway scene, and tunnel entry and exit scene.

[0147] Step S35: Generate a brightness adjustment action corresponding to the third target backlight brightness based on the Actor network to achieve backlight brightness adjustment.

[0148] In the embodiments of the present invention, it is necessary to further explain that an interval brightness adjustment strategy that meets multiple objective trade-offs is generated through the Actor network, and the brightness adjustment amount is controlled to follow a Gaussian distribution. The interval brightness adjustment strategy needs to balance the following objectives: display clarity, energy consumption, and visual comfort. The brightness adjustment amount is output in the form of a probability distribution (such as a Gaussian distribution), which retains a certain degree of randomness to explore a better strategy, and constrains the adjustment range by the mean and standard deviation to avoid extreme operation.

[0149] In the embodiment of the present invention, it is necessary to further explain that the multi-objective fitness function drives the Actor network and optimizes the Critic network. The Critic network is optimized by the state value function V(s t ) Evaluate the long-term comprehensive benefits of implementing brightness adjustment actions, including:

[0150] Cumulative reward prediction: combining immediate reward signal r t Quantify the overall strategic value from the current state by comparing the current regulatory effect with the potential future benefits (such as long-term comfort).

[0151] Reward signal r t Directly taken from the multi-objective fitness function value, r t =LR=w1·R1+w2·R2+w3·R3

[0152] The function of the Critic network is: input the current state s t (such as ambient light distribution, vehicle speed, driver's gaze area), output state value V(s t ); the state value function satisfies the following formula:

[0153]

[0154] Here, E(·) represents the expected value function, reflecting the weighted average of all possible future paths after executing the brightness adjustment action in state s. This explains that in a dynamic environment, multiple action options may be available under the same state (the brightness adjustment action has multiple execution instructions), and environmental feedback (such as light intensity changes and driver behavior) is random. By calculating the expectation, the state-value function can integrate all possible future paths and evaluate the long-term average performance of the strategy, rather than relying on a single random result.

[0155] k is the index of the future time step starting from the current time step t (k=0 corresponds to the current time step t, k=1 corresponds to the index of the next prediction step, and so on);

[0156] γ represents the discount factor, which controls the degree of decay of future rewards and ranges from 0 to 1; k ·r t+kIndicates that the reward for the kth step in the future will be attenuated; by accumulating the rewards of all future time steps, the Critic network can evaluate the long-term comprehensive effect of the current strategy; by reducing the weight of short-term brightness mutations, it encourages smooth adjustment strategies and reduces interference with the driver's attention;

[0157] In the embodiment of the present invention, it is necessary to further explain that the evaluation results of the Critic network are fed back to the Actor network to guide its parameter adjustment; by adjusting the brightness adjustment action ΔIi, V(s t ) to optimize long-term cumulative rewards. For example, if a certain adjustment causes the glare index to increase, the Critic network will lower the value score of the strategy, prompting the Actor network to avoid similar actions in subsequent decisions; dynamically adjust the weights of multiple objectives based on driving scenarios (such as city nighttime, highways, and rainy and foggy weather); for example, in tunnel scenarios, prioritize ensuring the smoothness of brightness transitions (comfort weight is increased); increase the energy consumption control weight in low-power mode to limit backlight power consumption;

[0158] What needs to be further explained in the embodiments of the present invention is that feedback data (such as actual brightness and driver eye movement indicators) is collected in real time through on-board sensors to continuously optimize the Actor and Critic networks to adapt to personalized driving habits and environmental changes.

[0159] Example 3: The difference between this embodiment of the present invention and Examples 1 and 2 is that:

[0160] In one possible embodiment, an ambient display light preprocessing module is included after the ambient display light acquisition module and before the light intensity change trend prediction module, and each sensor collects ambient light intensity data in real time to form a discrete time series data set; a spatial interpolation algorithm (such as Kriging or inverse distance weighted method) is used to convert the discrete time series data set into a continuous light intensity distribution map on the windshield; the light intensity distribution map is smoothed (for example, Gaussian filtering) to filter out noise and generate a smooth ambient display light distribution map.

[0161] In one possible embodiment, the ambient display light preprocessing module includes an anomaly rejection unit that cleans the discrete time series data set collected by the light sensor based on a dynamic neighborhood distance threshold method. By calculating the mean and standard deviation of the distance between each light intensity data point and its adjacent points, outliers (such as sudden flash interference) are dynamically rejected to improve data reliability.

[0162] Explanation: Burst flashes are instantaneous and non-representative, and cannot reflect the long-term trend of light intensity within the driver's field of view. Retaining such outliers will distort the light intensity distribution map, affecting the input quality of subsequent light intensity prediction models. Adjusting backlight brightness based on burst flash data may cause the HUD display to flicker frequently or experience sudden changes in brightness, disrupting the driver's field of view. The implementation process of the dynamic neighborhood distance threshold method includes:

[0163] Multiple micro-light sensors (photodiode arrays) integrated around the windshield collect discrete time series data sets within the driver's field of view to form a time series light intensity matrix;

[0164] For each data point in the light intensity matrix, calculate the mean and standard deviation of its Euclidean distance to its k adjacent data points. Set the dynamic threshold to the weighted sum of the mean and three standard deviations of the Euclidean distance. (For normally distributed data, approximately 99.7% of the data points fall within the range of the mean ± three standard deviations; extreme values ​​outside this range are considered outliers.)

[0165] If the mean neighborhood distance of a data point exceeds the dynamic threshold, it is considered an outlier and removed;

[0166] An interpolation algorithm (such as bilinear interpolation) is used to fill in the data gaps after the removal to generate a cleaned two-dimensional light intensity distribution map.

[0167] It is necessary to further explain in the embodiment of the present invention that the quantification method of real-time light and shadow complexity is:

[0168] Calculate the percentage of light intensity change per unit time, recorded as ΔI rate ;

[0169] Calculate frequency domain energy distribution: Apply short-time Fourier transform to the light intensity time series data, decompose it into frequency domain real state Re and virtual state, and calculate the high-frequency energy proportion:

[0170]

[0171] Where fc is the cutoff frequency (e.g., 50 Hz), which is used to distinguish high-frequency noise (raindrop reflections, traffic light flashes) from low-frequency gradients (natural light, tunnel transitions).

[0172] When the high-frequency energy ratio Ehigh>30%, it indicates the presence of a high-frequency interference source, and the high-frequency suppression factor η needs to be enabled to reduce the noise weight;

[0173] Calculate the variance of brightness differences:

[0174]

[0175] Brightness difference variance σ gradientThe higher the value, the more uneven the spatial light intensity distribution (e.g., direct sunlight in some areas and overlapping shadows), and the multi-scale convolution fusion module needs to be activated to enhance local feature extraction.

[0176] Dynamic scene classification: Classifies the scene's light and shadow complexity into several levels based on the light intensity change rate and the proportion of high-frequency energy;

[0177] In one possible embodiment, (ΔI rate +1)(E high +1)(σ gradient +1) quantifies the degree of light and shadow complexity;

[0178] For example, if the light intensity change rate ΔLrate is less than 10% / s and the high-frequency energy accounts for less than 10%, it is classified as a low-complexity scene (such as driving at a constant speed on a highway); if the light intensity change rate ΔLrate is ≥30% / s and the high-frequency energy accounts for ≥30%, it is classified as a high-complexity scene (such as in the city at night or entering or exiting a tunnel).

[0179] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A HUD backlight display system, characterized in that: include: The ambient display light collection module uses a micro light sensor to collect real-time ambient display light distribution maps covering the driver's field of view; The light intensity trend prediction module builds a light intensity prediction model that combines a recurrent neural network and a long short-term memory network. The light intensity prediction model is pre-trained based on historical driving scene data. It inputs a real-time ambient light distribution map, current vehicle status data, and the ambient light intensity trend of the vehicle ahead, and outputs a predicted light intensity distribution. The brightness pre-adjustment module generates corresponding backlight pre-adjustment parameters based on the predicted light intensity distribution; extracts the current light intensity value of the gaze area from the ambient display light distribution map, and calculates the target backlight brightness of the gaze area in the next prediction step based on the predicted light intensity distribution of the next prediction step; Perform global pre-adjustment of the backlight display system according to the backlight pre-adjustment parameters; The target backlight brightness is: any one of the first target backlight brightness, the second target backlight brightness, and the third target backlight brightness; The first target backlight brightness is obtained based on the preliminary target brightness and the glare threshold constraint; The second target backlight brightness is obtained based on forced contrast correction to ensure visibility of displayed content. The third target backlight brightness is obtained based on the dynamic balance of multi-objective optimization function and reinforcement learning; The sight area optimization module acquires eye tracking data through an integrated eye tracking device, determines the driver's current gaze area based on this data, and monitors the coordinates of the driver's gaze point in real time. It uses a Mini-LED array to divide the display area into several zones, supporting independent brightness adjustment for each zone. A fast-response controller processes zone backlight adjustment commands, giving the current gaze area the highest brightness priority. The multi-objective adjustment optimization module inputs the second target backlight brightness and the multi-objective fitness function, takes the multi-objective fitness function as a constraint, and outputs the third target backlight brightness, where the third target backlight brightness is the brightness set of each interval; The operation process of the multi-objective adjustment optimization module includes: Based on the physical hardware mapping, the display area is divided into several intervals, i represents the interval index, N represents the total number of intervals; based on the time sequence, the time step index t is set, and T represents the time window length; The weight coefficient is set based on the user's gaze ratio, which is expressed as ; Get the clarity index of the HUD backlight display system , energy consumption index and comfort index ; Joint Clarity Index , energy consumption index and comfort index , the multi-objective fitness function LR of multi-objective optimization correction is obtained through the following model: ； in, Represent the weight coefficients of each item, and ; The Actor network is used to generate a brightness adjustment action corresponding to the third target backlight brightness, realize backlight brightness adjustment, and control the brightness adjustment amount to follow Gaussian distribution.

2. The HUD backlight display system according to claim 1, characterized in that: The process of building the light intensity prediction model includes: Data preprocessing and frequency domain decomposition: Input the ambient light distribution map, vehicle status data, and weather information, and through frequency domain decomposition and enhancement and time-space alignment, output the frequency domain enhanced light intensity feature map; Multi-scale spatiotemporal feature extraction is performed based on multi-scale convolution fusion and spatiotemporal attention mechanism. The light intensity feature map after frequency domain enhancement is input and the multi-scale spatiotemporal joint feature vector is output, which includes local texture, global distribution and long-distance gradient features. Input multi-scale spatiotemporal joint feature vector and historical light intensity prediction error, and output predicted light intensity distribution map and dynamically optimized LSTM parameters; During the training process, the loss function value is calculated in real time. When the loss value does not decrease significantly after multiple consecutive training cycles or reaches the preset threshold, the model is judged to have converged. The trained model parameters are saved, and a trained light intensity prediction model is generated and deployed.

3. The HUD backlight display system according to claim 2, characterized in that: The multi-scale convolution fusion includes: Parallel convolution branch: 1x1, 3x3, and 5x5 convolution kernels are used to extract three sets of outputs in parallel, namely local high-frequency texture, mid-range light intensity distribution, and global light intensity distribution pattern; Feature map splicing: splice the three sets of outputs along the channel dimension to form a multi-scale fusion feature map; The spatiotemporal attention mechanism includes: calculating the spatiotemporal attention weights of the real state Re and the virtual state Im respectively through multi-head self-attention, and outputting the attention weight matrix; using the vehicle acceleration as the gating signal to adjust the distribution ratio of the attention weight in the time dimension.

4. The HUD backlight display system according to claim 1, characterized in that: The first target backlight brightness is obtained as follows: Enter the average ambient light intensity Ea and HUD reflectivity of the current viewing area , predict the change in light intensity , the initial target backlight brightness is obtained by the following formula , ； in, is the safety factor, which is adjusted dynamically according to vehicle speed; Glare threshold constraint, input preliminary target backlight brightness and glare threshold , the first target backlight brightness is obtained through the following model : .

5. The HUD backlight display system according to claim 4, characterized in that: The second target backlight brightness is obtained as follows: Enter the preliminary target backlight brightness , background brightness and contrast thresholds; Verify whether the preliminary target backlight brightness meets the contrast standard: if the ratio of the preliminary target backlight brightness to the background brightness is less than the preset value, a forced correction is triggered; Directly correct the target backlight brightness to 3 times the background brightness and output the second target backlight brightness ; If the corrected target backlight brightness is greater than the glare threshold, it is processed according to priority: Gaze zone: temporarily allows the target backlight brightness to exceed the glare threshold, but starts gradual attenuation; Non-attention area: Force the target backlight brightness to be less than half of the glare threshold to disperse the interference of high-brightness areas on the driver.

6. The HUD backlight display system according to claim 1, characterized in that: The multi-objective fitness function drives the Actor network and optimizes the Critic network, which is optimized by the state value function Evaluate the long-term comprehensive benefits of implementing brightness adjustment actions; combine immediate reward signals Quantify the overall strategic value from the current state and potential future returns; ； The function of the Critic network is: input the current state , output state value ; The state value function satisfies the following formula: ； in, represents the expected operation function, which reflects the weighted average result of all possible future paths after performing the brightness adjustment action in state s; k is the index of the future time step starting from the current time step t; γ represents the discount factor, which controls the degree of attenuation of future rewards and ranges from 0 to 1; The evaluation results of the Critic network are fed back to the Actor network to guide its parameter adjustment; by adjusting the brightness adjustment action ΔIi, the maximum , making the long-term cumulative reward optimal.

7. A HUD backlight display method, used to implement the system of claim 1, characterized in that: The following steps are involved: Step 1: Light field acquisition: Multiple micro-light sensors integrated around the windshield collect ambient light intensity data within the driver's field of view in real time, generating a two-dimensional ambient light distribution map. Step 2: Light Intensity Prediction: Build a light intensity prediction model that integrates a recurrent neural network and a long short-term memory network. Input the ambient light distribution map, vehicle status data, and the light intensity trend of the vehicle ahead, and output the predicted light intensity distribution for the next 0.5 seconds. The light intensity prediction model uses multi-scale convolution to extract local high-frequency textures, mid-range gradient light, and global light intensity patterns. Combined with a spatiotemporal attention mechanism, it dynamically assigns weights to improve adaptability to complex scenarios. Step 3: Optimize the visual area: Obtain eye tracking data through integrated eye tracking equipment, determine the driver's current gaze area based on the eye tracking data, and monitor the coordinates of the driver's gaze point in real time; use a Mini-LED array to divide the display area into several intervals, and support independent brightness adjustment for each interval; be equipped with a fast-response controller to process interval backlight adjustment commands, and the brightness of the current gaze area has the highest priority.

Images