Augmented reality head-up display system and display method thereof

Abstract

The present invention discloses an augmented reality head-up display system and a display method thereof, which relate to the field of assisted driving technology. In response to complex road conditions and diversified driving needs, the system achieves a comprehensive integrated analysis of the driving environment and driver status by sequentially executing four steps: environmental perception and multi-source data acquisition, situational analysis and information complexity determination, dynamic content screening and display strategy generation, and real-time monitoring and adaptive feedback closed loop. First, the multi-dimensional sensor data is labeled and normalized to form environmental factors and driving load factors, and the current information complexity is determined by a nonlinear algorithm. Then, the most critical content is screened from the information pool according to priority, and the display position and style are finely arranged. If the monitoring detects a HUD configuration deviation, a dynamic strategy update is executed, so that the AR-HUD always guarantees the accuracy and conciseness of the displayed information in different scenarios, thereby improving driving safety and user experience.

Description

Technical Field

[0001] The present invention relates to the field of assisted driving technology, and in particular to an augmented reality head-up display system and a display method thereof. Background Art

[0002] With the continuous evolution of vehicle intelligence and human-computer interaction technology, augmented reality head-up display (AR-HUD) has gradually become one of the important means to improve driving safety and comfort. By superimposing driving navigation, road signs, traffic warnings and vehicle status information on the driver's front field of view, AR-HUD can significantly reduce the driver's visual switching frequency of the instrument or central control panel, so that the human eye is always focused on the road ahead, thereby maintaining a higher degree of concentration in various scenarios such as highways, urban main roads, tunnels, rainy and snowy weather or night driving. However, the current road traffic environment is showing a trend of high complexity and dynamism: vehicles have to face suburban ring roads, congested ramps, mountain bends, extreme climates and various individual driver habits (such as novice, experienced drivers, special vision needs, etc.), which makes it difficult for traditional static or low-frequency information display modes to meet high real-time and diversified needs. In order to achieve more flexible and accurate assistance effects, an AR-HUD solution that can work together in multi-source data perception, situational analysis and adaptive display has gradually become a research hotspot in the industry.

[0003] The Chinese invention patent application, publication number CN118560516A, provides a HUD adaptive display method, system, device, and vehicle. This solution obtains vehicle environmental information and driving information; performs environmental recognition processing on the vehicle environmental information to obtain a display scene; responsive to a display mode, performs theme matching processing on the display scene according to the display mode to obtain a display theme; determines a driving scene based on the driving information, and adjusts the brightness of the display theme based on the driving scene to obtain a display image; and superimposes driving assistance information on the display image to obtain a fused image, which is then displayed. The system can adaptively adjust the theme and brightness of the HUD display, thereby improving the user experience and driving safety, and can be widely applied in the field of assisted driving technology.

[0004] However, considering the above practical application scenarios and existing technologies, there is still a prominent technical bottleneck:

[0005] When the external road environment and the driver's internal state change suddenly (such as a surge in rainfall causing a sharp drop in visibility, or the driver's frequent continuous operations causing excessive load), existing AR-HUD systems are often unable to quickly and accurately identify and respond to their differentiated display needs.

[0006] Specifically, if information delivery logic is still based on simple, fixed rules or low-frequency judgments, it is highly likely to cause information mismatch in extreme situations. For example, a large amount of secondary information may remain in the central visual field, while key warning elements may not be highlighted in a timely manner. This can cause driver delays due to visual overload or missing critical information, which not only degrades the driving experience but also increases the risk of traffic accidents. This is especially true on highways or in congested urban areas, where drivers must constantly manage complex road conditions and unexpected situations. If important AR-HUD information, such as navigation arrows and collision warnings, is not effectively addressed due to a mismatch between the actual situation, driver reaction time can be significantly prolonged, potentially leading to vehicle collisions or even more serious accidents. Clearly, this model, based on traditional display strategies or static information configuration, is no longer safe and targeted. A technical solution is urgently needed that can integrate multi-source sensory data, understand the driver's workload, and dynamically generate or refresh the HUD information layout in real time to balance information richness and cognitive load, truly meeting the diverse needs of today's changing road environments and personalized demands.

[0007] To this end, the present invention provides an augmented reality head-up display system and a display method thereof. Summary of the Invention

[0008] (1) Technical problems solved

[0009] To address the shortcomings of existing technologies, the present invention provides an augmented reality head-up display system and display method. This system, designed to address complex road conditions and diverse driving needs, achieves a comprehensive integrated analysis of the driving environment and driver status by sequentially executing four steps: environmental perception and multi-source data acquisition, contextual analysis and information complexity determination, dynamic content screening and display strategy generation, and real-time monitoring and adaptive feedback. The system first labels and normalizes multidimensional sensor data to generate environmental factors and driving load factors, and uses a nonlinear algorithm to determine the current information complexity. Next, it prioritizes the most critical content from the information pool and meticulously arranges it for display location and style. If monitoring detects HUD configuration deviations, it executes dynamic strategy updates, ensuring that the AR-HUD consistently displays accurate and concise information in different scenarios, thus resolving the technical issues discussed in the background art.

[0010] (2) Technical solution

[0011] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0012] An augmented reality head-up display method, comprising:

[0013] When the vehicle is powered on and the sensor array outputs an activation signal, the sensor data is cleaned, normalized, time-aligned, and labeled on the collected multi-source data input, and the processed data is aggregated into a traceable and multi-dimensional environmental comprehensive value F E (t), establish an index field to quickly associate vehicle locations with time periods;

[0014] When it is detected that the environmental comprehensive value F E (t) and driving load value D L (t) Summarize, perform nonlinear fusion and threshold classification, retain intermediate calculation logs for subsequent backtracking, and then output the target information complexity label, ultimately laying a precise scene guide for subsequent content screening and improving dynamic adaptation efficiency;

[0015] When it is detected that the target information complexity label is ready, the content screening engine performs priority scoring P on all information items in the AR-HUD information pool Ω. j (t) Set a more eye-catching font and center position for high-priority elements, retain only the outline or hide low-priority elements, and record the reasons for layout decisions and configuration logs. Finally, dynamically generate the HUD layout configuration vector and trigger on-demand display;

[0016] When the HUD configuration vector G(t) is detected and the expected configuration G * (t) The deviation measurement function E fb (t) exceeds the deviation threshold θ fb , perform time-integrated analysis and anomaly detection on the interface status and external data, and refresh the display strategy locally or globally, record deviation correction logs for long-term analysis, and ultimately achieve adaptive correction and ensure information fit in a dynamic environment;

[0017] When it is detected that the archiving of the evaluation data has been completed, the user preference vector H u , historical feedback and physiological characteristics to perform multi-dimensional matrix aggregation, and based on the reinforcement learning dynamic reward function R r1 (t, a(t)) iteratively updates the UI configuration strategy, ultimately forming a humanized HUD display solution that self-evolves with usage time.

[0018] Preferably, the vehicle-mounted sensor array outputs multi-dimensional raw measurement values ​​in the time series t; the external data source synchronously provides global / regional environmental parameters; and the environmental comprehensive value F is defined. E (t), define the vector:

[0019] M=diag(λ1,λ2,…,λ n )

[0020] Where: Θ i(τ) represents the normalized measurement value of the i-th sensor or external data source at time τ; λ i Represents the weight coefficient of the i-th sensor measurement value;

[0021] diag(·) represents the diagonal matrix construction operation, M is an n×n weight diagonal matrix, and n is the total number of sensors / data sources;

[0022]

[0023] Where: Θ(τ) is a column vector of all sensor measurements collected at time, and each component Θ i (τ) are all normalized and mapped to [0, 1] as defined previously; M is the weight diagonal matrix, λ i Located on the main diagonal;

[0024] T is the lookback time window, which represents the time range before the current time t;

[0025] k is the exponential decay coefficient, k>0; ||·||2 is the Euclidean norm operation;

[0026] Environmental comprehensive value F E The larger the value of (t), the more complex or tense the environment at time t is, and vice versa.

[0027] Preferably, the environmental comprehensive value F E (t) and each original measurement value Θ i (t) Further tagging is performed, and timestamp, vehicle location coordinates, and data source identifier are added; additional scene type fields are added to form a multidimensional index after classification;

[0028] Preferably, data sources related to the driver are combined, including eye tracking Φ eye (t), gesture frequency Φ gesture (t), operation switching frequency Φ switch (t), defines the driving load value D L (t):

[0029]

[0030] Among them: m (t) represents the mth normalized index related to driver behavior, ω m is the importance weight of the corresponding indicator;

[0031] Preferably, the following formula is introduced to obtain the comprehensive situation score C score (t), let the vector comprehensive context vector X(τ):

[0032]

[0033] Where: α∈[0,1] represents the balance weight between environment complexity and driving load;

[0034] In the time interval [tT, t], the comprehensive situation vector X(τ) is exponentially decayed and the p-norm is taken. Then, combined with the power exponent β and the logarithmic mapping, the comprehensive situation score C at the current time t is obtained. score (t):

[0035]

[0036] Where: T is the lookback time window, T>0, e is a natural constant; k is the exponential decay coefficient, k>0; ‖·‖ p , is the p-norm; β is the power exponent, β>0;

[0037] C scorc (t) is compared with the predefined complexity threshold interval {θ1, θ2}: When C scorc (t)≤θ1, it is judged as simple; when θ1<C scorc (t)≤θ2, it is judged as moderate; when C score (t)>θ2, judged as rich;

[0038] The target complexity label γ(t) obtained is recorded and output together with the timestamp t;

[0039] Preferably, all information items currently available for display are obtained from the AR-HUD information pool Ω maintained by the AR-HUD overall device, and are recorded as Ω(t)={ω1, ω2, ...ω n};

[0040] According to the target information complexity label γ(t) and the current situation score C score (t), for each piece of information ω i Calculate content consistency R i (t): Define a Logistic mapping to measure the information item ω i Matching with the current complexity label γ(t):

[0041]

[0042] Where: target information complexity Γ(ω i ) represents information ω i The pre-calibrated value in the complexity dimension, μ is the curve steepness coefficient, μ>0;

[0043] Set the threshold ρ∈(0,1) and set the content matching degree R i Items with (t)≥ρ are included in the preliminary selection set S init(t), the remaining information can be temporarily excluded;

[0044] Preferably, for those who have entered the preliminary selection set S init Each piece of information ω j (j=1, 2, ..., |S init |), calculate the priority score P j (t):

[0045] Let each information item to be evaluated ω j Two key attributes are generated over time τ; U j (τ) represents each piece of information ω j The urgency or security level of j (τ) represents ω j User preference or personal interest value; for unified measurement, define the priority feature vector X j (τ):

[0046]

[0047] And let W p is its weight diagonal matrix;

[0048] W p =diag(ω U ,ω Q )

[0049] The weight coefficient ω U ≥0,ω Q ≥0; introduce time window [tT p , t] and exponential decay coefficient κ p >0, defines the decay priority accumulation vector Z j (t):

[0050]

[0051] Where: T P >0 is the length of the lookback time for the priority score; The closer the measurement to the current time τ≈t is, the more attenuated the priority accumulation vector Z is. j (t) The greater the contribution; introduce the p-norm (r ≥ 1) and power δ > 0:

[0052]

[0053] in, and Represents the decay priority accumulation vector Z j (t) The first and second components of the vector, and then the norm result is power mapped

[0054] Define the priority score P j (t):

[0055]

[0056] Among them, λ p >0,λ p is the steepness adjustment coefficient of the Logistic curve;

[0057] According to the priority score P j (t) Sort from high to low and map them to the corresponding areas of the HUD interface:

[0058] High-priority information is placed in the core area of ​​the driver's field of vision (such as the center of the windshield) and given higher brightness or larger font size; medium-priority information can be placed at the edge of the field of vision or presented in a semi-transparent manner; low-priority information can be temporarily hidden or displayed in abbreviated form as an icon to prevent overcrowding of the interface;

[0059] The position, size, transparency and other parameter tags of the layout information in the HUD are retained and associated with the timestamp t to form the UI presentation layout;

[0060] Preferably, a trigger threshold is set for the generated high-priority content;

[0061] If the environment or driver data is detected to meet the trigger conditions, the corresponding information will be switched from a semi-transparent or hidden state to a full-screen / highlight state, accompanied by visual effects or sound reminders;

[0062] For medium priority information, it can be displayed in a gradual in / out manner according to the real-time fluctuation of the environment complexity γ(t).

[0063] The time point of triggering / folding action, triggering reason and corresponding information identification are recorded to form an event log;

[0064] Preferably, continuously obtain the latest measurements from onboard sensors and external data sources, as well as the generated UI presentation layout and event logs;

[0065] By defining the deviation metric function E fb (t), monitors the consistency between the previous decision and the current new observation data, where:

[0066] Let: G(t) represents the current actual HUD configuration vector; G * (t) represents the expected configuration vector obtained based on the latest environmental information or fast estimation; ΔG(τ) = G(τ) - G * (τ) represents the configuration difference at time τ; d,t] observe the continuous deviation between the HUD configuration and the expectation, and define the configuration deviation measurement function E fb (t):

[0067]

[0068] Where: is a vector of length m; T d To monitor the backtracking time window, T d >0;κ d is the exponential decay coefficient, κ d >0,; is the p-norm (p d ≥1); α d is a power operation, α d >0;

[0069] When the configuration deviation metric function E fb (t) If it is too high, it indicates that there is a significant deviation between the current HUD strategy and the latest external environment (or driver status), and adjustment is needed;

[0070] If the configuration deviation metric function E fb (t) is lower than the preset deviation threshold θ fb , then the current configuration is considered to be basically consistent with the external data and does not need to be refreshed; if the configuration deviation measurement function E fb (t) is higher than the preset deviation threshold θ fb , then re-evaluate the situation and update the display strategy;

[0071] Preferably, when the configuration deviation metric function E fb (t)≥θ fb When the

[0072] Based on the contextual reassessment results, if only some information elements are deviated, the local display elements can be corrected by triggering logic on demand, without having to refresh all HUD content globally.

[0073] The updated new HUD configuration vector G(t new ) Write the AR-HUD overall device record and mark the specific reason for the change;

[0074] Preferably, the trigger time t, the trigger reason, the new HUD configuration vector G(t new ) and the driver's response are structured and stored; the feedback evaluation function F is defined eval The impact of the updated strategy on driver behavior or overall AR-HUD device efficiency is measured as follows:

[0075] F eval (t new)=exp(-[φ1Λ op (t new )+φ2Λ eye (t new )])

[0076] Where: Λ op (t new ) represents the driver's operating load after the new strategy takes effect; Λ eye (t new ) represents the driver’s attention to the HUD or gaze dispersion in the eye movement data; φ1, φ2 ≥ 0 are adjustment coefficients;

[0077] Using the accumulated multi-dimensional feedback data {G(t), E fb (t), F eval (t),...},periodic or online updates of machine learning models or expert rules;

[0078] Preferably, after detecting the driver's login identity, all the user's past evaluation events and operation preference data (including HUD interface operation and trigger records, AR-HUD overall device evaluation information when the event is triggered, driver's personalized preferences and historical interaction data, and evaluation indicators and feedback data) will be retrieved from the archived historical records to form a user preference vector H u ;

[0079] Combined with the feedback evaluation function results, the previous UI interface layout configuration strategies and the corresponding driver load indicators are quantitatively aggregated to obtain the personalized preference matrix P u ;

[0080] Maintain user preference vector H respectively u and personalized preference matrix P u And automatically switch when logging in; according to the user preference vector H u and personalized preference matrix P u , output basic personalized configuration in the current situation

[0081] Preferably, the state vector S is defined as rl (t), including: HUD current layout and style; environment complexity F E (t) and driving load information D L (t), user preference vector H u The most relevant entry in the current period; action set A rl ;

[0082] Take an action a(t)∈A at time t rl After obtaining the new HUD configuration, the dynamic reward function R is calculated at the next moment t+Δ based on the returned feedback data. τl(t, a(t)):

[0083] Let: Λ op (τ) and Λ eye (τ) represents the driver's operating load and eye movement dispersion at time τ, respectively;

[0084] F eval (t) represents the comprehensive evaluation result of the AR-HUD device at time t;

[0085] L(τ)=[Λ op (τ),Λ eye (τ)] T is the negative impact vector; M r =diag(μ op , μ eye ), used to assign differential weights to negative factors; T r >0 is the lookback time window, κ r >0 is the exponential decay coefficient, is the p-norm (p r ≥1); α r is the power term, α r >0;β + , β - ≥0 indicates the adjustment coefficient for positive and negative terms, respectively;

[0086] Based on the above definition, the dynamic reward function R r1 (t, a(t)) is:

[0087]

[0088] Where: Weight coefficient μ op , μ eye , located in M r The diagonal of is used to distinguish the relative weight of operation load and eye movement load in penalty calculation. If a scene is particularly sensitive to eye movement deviation, the weight coefficient μ can be increased. eye ;

[0089] p r , α r are the p-norm exponent and power correction coefficient respectively;

[0090] T r , κ r are the lookback time window, exponential decay coefficient, and κ r The value is greater than 0;

[0091] Reinforcement learning algorithm is based on the dynamic reward function R τl (t, a(t)) learns the long-term benefits of different actions and gradually converges to the optimal or suboptimal strategy after multiple iterations;

[0092] After multiple rounds of interaction, the AR-HUD device will learn the UI configuration strategy And when the same or similar state S is detected again rl (t) automatically adopts the strategy for UI layout; at the same time, the learned UI configuration strategy will be regularly Integrate with the basic personalized configuration and make adjustments if certain preferences conflict;

[0093] Augmented reality head-up display system, including:

[0094] The multi-source data acquisition unit, when the vehicle is powered on and detects the sensor array output activation signal, performs sensor data cleaning, normalization, time alignment and labeling operations on the collected multi-source data input, and aggregates the processed data into a traceable and multi-dimensional environmental comprehensive value F E (t), establish an index field to quickly associate vehicle locations with time periods;

[0095] The information complexity judgment unit detects that the environmental comprehensive value F has been completed. E (t) and driving load value D L (t) Summarize, perform nonlinear fusion and threshold classification, retain intermediate calculation logs for subsequent backtracking, and then output the target information complexity label, ultimately laying a precise scene guide for subsequent content screening and improving dynamic adaptation efficiency;

[0096] When the strategy generation module detects that the target information complexity label is ready, the content screening engine performs priority scoring P on all information items in the AR-HUD information pool Ω. j (t) Set a more eye-catching font and center position for high-priority elements, retain only the outline or hide low-priority elements, and record the reasons for layout decisions and configuration logs. Finally, dynamically generate the HUD layout configuration vector and trigger on-demand display;

[0097] Adaptive feedback module, when monitoring the HUD configuration vector G(t) and the expected configuration G * (t) The deviation measurement function E fb (t) exceeds the deviation threshold θ fb , perform time-integrated analysis and anomaly detection on the interface status and external data, and refresh the display strategy locally or globally, while recording deviation correction logs for long-term analysis, ultimately achieving adaptive correction and ensuring information fit in a dynamic environment;

[0098] The personalized configuration module detects that the archive of the evaluation data has been completed and generates the user preference vector H u , historical feedback and physiological characteristics to perform multi-dimensional matrix aggregation, and based on the reinforcement learning dynamic reward function Rr1 (t, a(t)) iteratively updates the UI configuration strategy, ultimately forming a humanized HUD display solution that self-evolves with usage time.

[0099] (3) Beneficial effects

[0100] The present invention provides an augmented reality head-up display system and a display method thereof, which have the following beneficial effects:

[0101] Utilizes onboard sensors and external data sources to generate comprehensive environmental values, which are then normalized and labeled according to AR-HUD requirements to ensure subsequent data quality and high-precision fusion of multi-source data.

[0102] Through multi-factor situational analysis, the driving load and environmental comprehensive values ​​are integrated to accurately determine the target information complexity. Based on this, the most critical content is filtered from the information pool. Combined with priority scoring and UI layout optimization strategies, key information is highlighted and less important information is deemphasized. This achieves multi-factor synergy, forms a situation classification and driving load information, and guides the selection and arrangement of HUD content.

[0103] Real-time monitoring is performed using a deviation measurement function. Once a threshold is exceeded, context reassessment and display refresh are triggered, ensuring that the HUD configuration always meets the driver's needs in dynamic environments. Deviation detection and timely correction of mismatches are implemented. The introduction of personalized preference vectors and preference matrices, combined with a reinforcement learning reward function, allows for adaptive iteration based on driver operating behavior and safety indicators accumulated over long-term use, gradually forming the most suitable HUD configuration and information display strategy for the individual.

[0104] This allows it to maintain high availability and scalability under a variety of complex road conditions and multiple user preferences, realizing the intelligent presentation and optimization of AR-HUD information. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] Figure 1 This is a flow chart of the augmented reality head-up display method of the present invention;

[0106] Figure 2 This is a schematic diagram of the structure of the augmented reality head-up display system of the present invention. DETAILED DESCRIPTION

[0107] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0108] See also Figure 1The present invention provides an augmented reality head-up display method, comprising:

[0109] Step 1: When the vehicle is powered on and the sensor array outputs an activation signal, the sensor data is cleaned, normalized, time-aligned, and labeled for the collected multi-source data input (vehicle speed, light, rainfall, external road conditions, etc.), and the processed data is aggregated into a traceable and multi-dimensional environmental comprehensive value F. E (t), establish an index field to quickly associate vehicle locations with time periods;

[0110] The step 1 includes the following:

[0111] Step 101: Multi-sensor data acquisition and preliminary quantification of environmental factors

[0112] The onboard sensor array (rain sensor, light sensor, speed sensor, steering angle sensor, in-car camera, etc.) outputs multi-dimensional raw measurement values ​​in time series t. External data sources (such as navigation API, weather API, road condition API, etc.) simultaneously provide global / regional environmental parameters. All sensor and external data are initially aligned at timestamp t to ensure that subsequent processing is integrated on the same time basis.

[0113] In order to measure the comprehensive state of the environment at the current moment, the comprehensive value of the environment F is defined E (t) is used to provide a basic quantitative reference for subsequent scenario analysis. To be compatible with the input / output naming conventions of the previous steps, the vector is defined as:

[0114] M=diag(λ1,λ2,…,λ n )

[0115] Where: Θ i (τ) represents the normalized measurement value of the i-th sensor or external data source at time τ, that is, the original measurement value; λ i represents the weight coefficient for the i-th sensor measurement value, and

[0116] diag(·) represents the diagonal matrix construction operation, M is an n×n weight diagonal matrix used to assign differentiated weights to different sensor vector components, and n is the total number of sensors / data sources;

[0117] In the time range [tT, t], the historical data is weightedly integrated using the exponential decay kernel, and the result is taken as the Euclidean norm ||·||2 and then logarithmically mapped to finally obtain the environmental comprehensive value F at the time moment E (t):

[0118]

[0119] Where: Θ(τ) is a column vector of all sensor measurements collected at time, and each component Θ i (τ) are all normalized and mapped to [0, 1] as defined previously; M is the weight diagonal matrix, λ i Located on the main diagonal, it reflects the contribution of each sensor to the complexity of the environment and can be adjusted according to actual driving needs or importance during the deployment and tuning of the AR-HUD overall device;

[0120] T is the lookback time window, which represents the time range before the current time t. It is used to consider the impact of short-term historical environmental changes on the comprehensive factor at the current time. The specific value can be set by the vehicle driving scenario.

[0121] k is the exponential decay coefficient, k>0; ||·||2 is the Euclidean norm operation, if the vector x=[x1,...,x m ] T ,but Map the vector value obtained by integration into a non-negative scalar to facilitate subsequent logarithmic operations and to determine the complexity of the environment;

[0122] Environmental comprehensive value F E The larger the value of (t), the more complex or tense the environment at time t is, and vice versa.

[0123] Aggregate the information from multiple sensors and multiple external data sources into a single quantitative indicator environmental comprehensive value F E (t), providing input that is easy to compare and calculate, so that the comprehensive value of this environment can be expanded in data dimension and parameter self-learning (later it can be combined with machine learning model to automatically tune λ i ) is more flexible and has more potential.

[0124] Step 102: Data marking and aligned storage

[0125] Environmental comprehensive value F E (t) and each original measurement value Θ i (t) further tagging and adding information such as timestamp, vehicle location coordinates, and data source identifier to the data structure. The tagged data is uniformly stored in a hierarchical database or memory, and indexed with time as the primary key. During the storage process, a scene type field (such as urban roads, suburban highway underground parking, etc.) can be added to form a multidimensional index after preliminary classification.

[0126] Through the comprehensive value of the environment F E (t) and the original measurement value Θ i(t) and other key variables are uniformly labeled, and the environmental comprehensive values ​​and raw sensor data are packaged in the same index system, which facilitates subsequent rapid retrieval and correlation analysis in different time windows (such as short time / long time) or specific events, reducing the complexity of data preparation.

[0127] Step 2: When the environmental comprehensive value F is detected E (t) and driving load value D L (t) Summary, comprehensive environmental value F E (t) and driving load value D L (t) Perform nonlinear fusion and threshold classification, retain intermediate calculation logs for subsequent backtracking, and then output the target information complexity label, ultimately laying a precise scene guide for subsequent content screening and improving dynamic adaptation efficiency;

[0128] The second step includes the following:

[0129] Step 201: Multi-factor situation label generation and driving load quantification

[0130] The obtained environmental comprehensive value F E (t) and other marking information (such as scene type label), it is used to identify the external environment conditions (such as highway, urban road, etc.) in which the vehicle is located, and provide a numerical reference for measuring the dynamic complexity of the external environment;

[0131] Incorporating driver-related data sources, including eye tracking eye (t), gesture frequency Φ gesture (t), operation switching frequency Φ switch (t), etc., define the driving load value D L (t) is used to quantify the driver's current attention and stress. The following formula can be used to fuse the multi-dimensional driving load signals:

[0132]

[0133] Among them: m (t) represents the mth normalized index related to driver behavior (such as Φ eye (t), Φ gesture (t), Φ switch (t), etc.), whose value range has been determined as [0, 1] in the marking stage of step 1;

[0134] ω m is the importance weight of the corresponding indicator, And ω m ≥0;

[0135] When used, the exponential and logarithmic operations are combined to synthesize multiple driver behavior signals while suppressing the excessive impact of extreme high values ​​on the overall result. The final driving load value D L If (t) is larger, it means that the current driver's operation frequency or attention load is higher, and multiple driving-related signals are uniformly mapped to |D L (t), which can be subsequently compared with F E (t) Perform simple coupling to greatly reduce the complexity of dynamic calculations, ω m In actual deployment, machine learning algorithms can be automatically tuned to enhance adaptability to different driver behavior patterns.

[0136] Step 202: Information complexity determination and target complexity level

[0137] When obtaining the environmental comprehensive value F E (t) (Environmental complexity measurement) and driving load value D L (t), the following formula is introduced to obtain the comprehensive situation score C score (t), let the vector comprehensive context vector X(τ):

[0138]

[0139] Among them: F E (τ) is the comprehensive environmental value at time τ; D L (τ) is the driving load value at time τ; α∈[0,1] represents the balance weight between environmental complexity and driving load; the first component of the comprehensive situation vector X(τ) is αF E (τ), the second component is (1-α)D L (τ);

[0140] In the time interval [tT, t], the comprehensive situation vector X(τ) is exponentially decayed and the p-norm is taken. Then, combined with the power exponent β and the logarithmic mapping, the comprehensive situation score C at the current time t is obtained. score (t):

[0141]

[0142] Where: X(τ) integrates the environmental factors and the driver load factors into the same vector through weighted integration, reflecting the comprehensive level of the external environment and internal state at a given time τ;

[0143] T is the lookback time window, T>0, indicating that the data are cumulatively analyzed within the time interval [tT, t], and e is a natural constant;

[0144] k is the exponential decay coefficient, k>0, which is used to reduce the impact of historical data far from the current time on the score; ||·||p , is the p-norm, (p≥1), if Then ||v|| p =(v1| p +|v2| p ) 1 / p

[0145] Different p values ​​can control the sensitivity between environmental factors and driving load values. For example, p = 1 tends to be linearly additive, while p = 2 tends to be balanced. p > 2 can amplify large-value components more strongly (for example, when a particular factor is particularly high, the impact is more significant).

[0146] β is the power exponent. When β>0, the gate is used to strengthen or suppress the output of the p-norm. When β>1, the weight of high-value scenarios is increased, and when β<1, the influence of extreme values ​​is relatively smoothed. β, α, and p can all be adjusted through actual measurement of the AR-HUD device or data analysis to meet the needs of different driving environments.

[0147] Target complexity level judgment: C scorc (t) is compared with the predefined complexity threshold interval {θ1, θ2} to determine the target information complexity of the HUD: scorc (t)≤θ1, it is judged as simple; when θ1<C scorc (t)≤θ2, it is judged as moderate; when C score (t)>θ2, judged as rich;

[0148] The target complexity label γ(t) (e.g., concise, moderately rich) is recorded and output together with the timestamp t;

[0149] When used, the classification method in the form of threshold intervals is simple and clear, easy to call quickly in the next step, and easily compatible with more subdivision levels (such as adding minimalist or super rich). The environmental factors and driving load values ​​are added nonlinearly and then subjected to a logarithmic function. It not only realizes the flexible coupling between multiple factors in terms of numerical value, but also provides an adjustable parameter entry for subsequent machine learning. It can automatically optimize β, α, and p based on real driving data. This comprehensive scoring mechanism solves the problem of insufficient distinction between extreme scenarios by traditional simple superposition methods, and is more conducive to coping with changing road and driver conditions.

[0150] Through multi-factor situational label generation and driving load quantification, as well as the organic coordination of information complexity judgment and target complexity level, a comprehensive assessment of environmental factors and driver load is achieved, and the results are converted into target information complexity output to drive the subsequent HUD dynamic display strategy, significantly improving the adaptability of the AR-HUDAR-HUD overall device in complex road environments and diverse driver conditions.

[0151] Step 3: When it is detected that the target information complexity label is ready, the content screening engine performs priority scoring P on all information items in the AR-HUD information pool Ω. j (t) High-priority elements are given a more prominent font and central position, while low-priority elements are retained as outlines or hidden. The reasons for layout decisions and configuration logs are recorded, and ultimately, the HUD layout configuration vector is dynamically generated and triggered to be displayed on demand, maintaining display simplicity and ensuring the driver's core visual area.

[0152] The step three includes the following:

[0153] Step 301: Content pool screening and initial selection set formation

[0154] All information items currently available for display (such as navigation prompts, ADAS warnings, surrounding points of interest, vehicle status, etc.) are obtained from the AR-HUD information pool Ω maintained by the AR-HUD overall device, and are recorded as Ω(t) = {ω1, ω2, ...ω n};

[0155] According to the target information complexity label γ(t) and the current situation score C score (t), for each piece of information ω i Calculate content consistency R i (t):

[0156] Define a Logistic mapping to measure the information item ω i Matching with the current complexity label γ(t):

[0157]

[0158] Where: target information complexity Γ(ω i ) represents information ω i Pre-calibrated value in the complexity dimension; target information complexity Γ target By complexity label γ(t) or comprehensive context score C score (t) is derived and used to convert the target information complexity into the same metric space; μ is the steepness coefficient of the curve, μ>0;

[0159] If the content matches R i (t) is close to 1, indicating that the information item w i Very consistent with current requirements;

[0160] If the content matches R i (t) is close to 0, indicating that it does not match the current situation; set the threshold ρ∈(0, 1) and set the content matching degree R i Items with (t)≥ρ are included in the preliminary selection set S init(t), the remaining information can be temporarily excluded;

[0161] When in use, the nonlinear mapping of the Logistic type of conformity can provide a certain buffer when there is a slight deviation between the information and the complexity of the situation, rather than simply excluding and screening out only the content that is more in line with the current situation, which can reduce the burden of subsequent processing.

[0162] Step 302: Priority allocation and interface layout adjustment

[0163] For those who have entered the initial selection set S init Each piece of information ω j (j=1, 2, ..., |S init |), calculate the priority score P according to its urgency, importance and security level j (t), the following formula can be introduced:

[0164] Let each information item to be evaluated ω j Two key attributes are generated over time τ; U j (τ) represents each piece of information ω j The urgency or security level of j (τ) represents ω j The user preferences or personal interests of can be formed based on historical interaction records, user manual preference settings, etc. To unify the measurement, define the priority feature vector X j (τ):

[0165]

[0166] And let W p Its weight diagonal matrix is ​​used to differentially amplify or suppress urgency and preference in the vector dimension;

[0167] W p =diag(ω U ,ω Q )

[0168] The weight coefficient ω U ≥0,ω Q ≥0 and can be tuned during AR-HUD overall device deployment or online learning; in order to take into account the impact of recent history on current priority evaluation, a time window [tT p , t] and exponential decay coefficient κ p >0, defines the decay priority accumulation vector Z j (t):

[0169]

[0170] Where: T p>0 is the length of the lookback time for the priority score; The closer the measurement to the current time τ≈t is, the more attenuated the priority accumulation vector Z is. j (t) The greater the contribution; the cumulative vector Z of the attenuation priority j (t) is transformed into a recognizable scalar metric, introducing the p-norm (r ≥ 1) and the power δ > 0:

[0171]

[0172] in, and Represents the decay priority accumulation vector Z j (t) The first and second components of the vector, and then the norm result is power mapped Flexibility to enhance (δ>1) or smooth (δ<1) sensitivity to extreme values;

[0173] The power norm output is used as an independent variable, and the Logistic function is used to implement the mapping from (-∞, +∞) to (0, 1), so that it can be directly interpreted as the priority weight in the subsequent layout or trigger mechanism, and the priority score P is defined. j (t):

[0174]

[0175] Among them, λ p >0,λ p is the steepness adjustment coefficient of the Logistic curve;

[0176] when When larger (indicates each piece of information ω j Recently, there are continuous high values ​​in urgency or preference), priority score P j (t) approaches 1; when Near 0 o'clock, or lower (depending on λ p Size), description ω j There is no significant urgency or value of attention in the near future;

[0177] According to the priority score P j (t) Sort from high to low and map them to the corresponding areas of the HUD interface:

[0178] High-priority information is placed in the core area of ​​the driver's field of vision (such as the center of the windshield) and given higher brightness or larger font size; medium-priority information can be placed at the edge of the field of vision or presented in a semi-transparent manner; low-priority information can be temporarily hidden or displayed in abbreviated form as an icon to prevent overcrowding of the interface;

[0179] The position, size, transparency and other parameter tags of the layout information in the HUD are retained and associated with the timestamp t to form the UI presentation layout;

[0180] During use, information is prioritized through a nonlinear combination of safety urgency and user preferences, balancing the safety requirements of the AR-HUD as a whole with personalized needs. Dynamic adjustments to the UI layout ensure that drivers can quickly capture the most important information at critical moments while also ensuring they don't miss any auxiliary prompts. This approach not only meets strict constraints on safety factors but also maintains flexible support for user preferences. The layout results are promptly updated into the data structure and bound to a timestamp, providing a traceable basis for adaptive optimization.

[0181] Step 303: On-demand triggering logic and local dynamic display

[0182] Set trigger thresholds for generating high-priority content, such as collision warnings or emergency information;

[0183] If it is detected that the environment or driver data meets the trigger conditions (for example, vehicle speed ≥ v crit , following distance ≤ d crit ), the corresponding information will be switched from a semi-transparent or hidden state to a full-screen / highlighted state, accompanied by visual effects or sound reminders;

[0184] For medium-priority information, it can be displayed in a gradual in / out manner based on the real-time fluctuations of the environmental complexity γ(t). For example, when the environmental complexity γ(t) gradually increases to a medium-high level and the driver's idleness is high, the POI list can be automatically expanded. Conversely, when γ(t) drops or in an emergency, the information can be reduced and made semi-transparent to reduce visual interference.

[0185] The time point of triggering / retracting action, triggering reason (environmental factors or driver status) and corresponding information identification are recorded to form an event log for adaptive feedback learning or troubleshooting;

[0186] During use, on-demand triggering brings critical information to the forefront of the driver's field of view only when truly needed, significantly reducing the driver's cognitive load. Simultaneously, the dynamic and smooth presentation of intermediate information on the interface meets real-time changes without excessively disrupting the driver's focus. Combining the AR-HUD's contextual analysis with dynamic UI effects, the system goes beyond a binary on / off display and enables gradual transitions based on thresholds and environmental fluctuations.

[0187] Step 4: When the HUD configuration vector G(t) is detected and the expected configuration G * (t) The deviation measurement function E fb (t) exceeds the deviation threshold θ fb, perform time-integrated analysis and anomaly detection on the interface status and external data, and refresh the display strategy locally or globally, while recording deviation correction logs for long-term analysis, ultimately achieving adaptive correction and ensuring information fit in a dynamic environment;

[0188] The step 4 includes the following contents:

[0189] Step 401: Real-time data capture and deviation detection

[0190] Continuously obtain the latest measurement values ​​from vehicle sensors and external data sources (such as environmental comprehensive value F E (t), driving load value D L (t), navigation API updates, etc.), as well as generated UI presentation layouts, event logs, etc.;

[0191] By defining the deviation metric function E fb (t), monitor the consistency between previous decisions (including situation judgment and UI configuration) and the current new observation data, where:

[0192] Let: G(t) represents the current actual HUD configuration vector, such as size, position, transparency and other comprehensive states; G * (t) represents the expected configuration vector obtained based on the latest environmental information or fast estimation; ΔG(τ) = G(τ) - G * (τ) represents the configuration difference at time τ; d , t], and assign greater weight to the difference closer to the current moment. The exponential decay time integral and p-norm operation are introduced to define the configuration deviation measurement function E fb (t):

[0193]

[0194] Where: is a vector of length m (m is the number of HUD-related dimensions, such as navigation information, ADAS warnings, font size, transparency, etc.). ΔG(τ) strictly matches the name and length of the HUD configuration vector.

[0195] T d To monitor the backtracking time window, T d >0;κ d is the exponential decay coefficient, κ d >0, the larger the value, the more emphasis is placed on the current moment difference, and the smaller the value, the more traces of historical differences are retained; is the p-norm (p d ≥1), used to map the vector obtained by integration into a non-negative scalar:

[0196]

[0197] By choosing different p d You can adjust the sensitivity to large or uniform deviations in the components; for example, |p d =2 is more balanced, p d When >2, the extreme value sensitivity is stronger;

[0198] α d is a power operation, α d >0, used to amplify the norm result again (α d >1) or smooth (α d <1) For large deviations; the response flexibility is high and can be compared with k d 、p d Collaborative optimization to adapt to different vehicle operating conditions or user groups;

[0199] When the configuration deviation metric function E fb (t) If it is too high, it indicates that there is a significant deviation between the current HUD strategy and the latest external environment (or driver status), and adjustment is needed;

[0200] By configuring the deviation measurement function E fb Timely measurement of (t) can capture potential mismatches when there are subtle changes in the environment or driver situation, thereby deciding whether to trigger re-analysis in the next step, where:

[0201] If the configuration deviation metric function E fb (t) is lower than the preset deviation threshold θ fb , then the current configuration is considered to be basically consistent with the external data and does not need to be refreshed; if the configuration deviation measurement function E fb (t) is higher than the preset deviation threshold θ fb , then re-evaluate the situation and update the display strategy;

[0202] When in use, the UI configuration, priority information and environmental factors are encapsulated into the actual configuration vector G(t), and the overall consistency is measured by the vector norm. This breaks the limitations of previous simple binary judgments of missing information or incorrect display, and provides multi-dimensional deviation measurement. By adjusting sub-items, it can flexibly amplify / suppress differences in certain specific dimensions (such as different tolerance levels when ADAS warning items are inconsistent with general navigation information), thereby more delicately managing the importance differences of different information in the HUD.

[0203] Step 402: Context reassessment and dynamic strategy adjustment

[0204] When the configuration deviation metric function E fb (t)≥θ fbWhen the user updates the image, the core processes of step 2 (context analysis and information complexity determination) and step 3 (dynamic content screening and display strategy generation) are called for rapid re-determination. For example, only the sensor information with a large update amplitude is analyzed, without having to repeat all algorithms, thus shortening the overall response delay of the AR-HUD device.

[0205] Based on the situational re-determination results, if only some information elements are deviated (such as a sudden change in weather causing the navigation priority to be increased), the on-demand triggering logic in sub-step 303 can be used to correct the local display elements without having to globally refresh all HUD content.

[0206] The updated new HUD configuration vector G(t new ) Write the AR-HUD overall device record and mark the specific reason for the change (such as a surge in rain sensor or abnormal driver gestures, etc.);

[0207] Reapplying the previous steps two and three to real-time scenarios reduces the redundancy of fixed-period polling or full-page refreshes in traditional AR-HUD devices, ensuring precise adjustments only when needed; the idea of ​​local configuration correction improves the dynamic performance efficiency of the UI, which can respond to emergencies without frequent large-scale changes that interfere with the driver; combining deviation-triggered situational re-determination with partial layout refreshes avoids triggering large-scale recalculations due to minor environmental changes, thereby significantly improving performance while ensuring safety. It supports multiple update modes (full / incremental) and is compatible with deployment scenarios under different vehicle hardware computing powers.

[0208] Step 403: Historical data archiving and AR-HUD overall device model optimization

[0209] The trigger time t, trigger reason (such as vehicle speed ≥ v crit 、Rainfall>r crit etc.), the new HUD configuration vector G(t new ) and driver responses (such as operation delay, frequency of sight shift, etc.) are stored in a structured manner;

[0210] Define the feedback evaluation function F eval The impact of the updated strategy on driver behavior or overall AR-HUD device efficiency is measured as follows:

[0211] F eval (t new )=exp(-[φ1Λ op (t new )+φ2Λ eye (t new )])

[0212] Where: Λ op (t new) represents the driver's operating load after the new strategy takes effect (which can be used to count the frequency of operation switching, probability of gesture mistouch, etc.); A eve (t new ) represents the driver’s attention to the HUD or gaze dispersion in the eye movement data; φ1, φ2 ≥ 0 are adjustment coefficients used to balance the operating load and eye gaze attention;

[0213] When the feedback evaluation function F eval The higher the value, the lower the driver load and the more appropriate the attention concentration of the AR-HUD device after the strategy update, indicating that the strategy effect is better.

[0214] Using the accumulated multi-dimensional feedback data {G(t), E fb (t), F eval (t), ...}, etc., to regularly or online update the machine learning model or expert rules, and adopt methods such as incremental training of neural networks and fine-tuning of rule parameters to make better judgment and layout generation strategies when the next trigger is triggered;

[0215] During use, through event archiving and feedback evaluation, the AR-HUD overall device can continuously learn which scenarios and configurations better meet the driver's needs, forming an adaptive iteration; the actual use effect of the dynamic HUD configuration G(t) (measured by driver behavior indicators) is closed-loop integrated with the machine learning or rule-based AR-HUD overall device, which is more real-time and automated compared to the common configuration parameter adjustment or fixed-time manual update mode. It can quickly respond to sudden changes in the environment or driver status in the short term, and can also continuously accumulate experience in the long term to provide a basis for incremental improvement of the machine learning model or expert rules of the AR-HUD overall device, so that the AR-HUD can gradually enhance its adaptability throughout its life cycle and truly achieve the goal of self-adaptation. Through multi-dimensional deviation detection, local refresh strategy and quantitative evaluation of driver behavior, the AR-HUD overall device shows higher flexibility and intelligence while maintaining safety and usability, forming a more forward-looking and creative technical system compared to traditional static HUD solutions.

[0216] Step 5: When it is detected that the evaluation data has been archived, the user preference vector H u , historical feedback and physiological characteristics to perform multi-dimensional matrix aggregation, and based on the reinforcement learning dynamic reward function R r1 (t, a(t)) Iteratively updates the UI configuration strategy, ultimately forming a humanized HUD display solution that evolves with usage time.

[0217] The step five includes the following:

[0218] Step 501: Multi-user model initialization and personalized preference aggregation

[0219] After the driver's identity is detected and logged in (or through face recognition, fingerprint verification, etc.), all the evaluation events and operation preference data generated by the user in step 4 in the past (including HUD interface operation and trigger records, AR-HUD overall device evaluation information when the event is triggered, driver personalized preferences and historical interaction data, evaluation indicators and feedback data) will be retrieved from the archived historical records to form the user preference vector H u ;

[0220] Combined with the feedback evaluation function results (such as feedback evaluation function F eval (t new )), quantitatively aggregate the previous multiple UI interface layout configuration strategies and the corresponding driver load indicators to obtain the personalized preference matrix P u ;

[0221] P u Comprehensive scores for various UI elements (such as font size, background transparency, and color blindness adaptation level) can be stored for quick recall during subsequent training or inference.

[0222] If multiple drivers share a car, they will maintain the user preference vector H u and personalized preference matrix P u And automatically switch when logging in; according to the user preference vector H u and personalized preference matrix P u , outputting basic personalized configurations (such as default font style, UI layout ratio, color theme) in the current context, and injecting them into the reinforcement learning initial state required in the subsequent step 502;

[0223] When using it, by summarizing historical data, when generating personalized solutions for the first time, we can base our decisions on previous user behaviors and feedback, avoiding starting from scratch and significantly improving practicality and user experience. By using a clear user preference vector H u and personalized preference matrix P u , each driver can get differentiated interface customization, while providing rich feature input for subsequent self-learning algorithms; mapping multi-dimensional feedback data to the personalized preference matrix P u , so that personalized initialization not only relies on simple static configuration, but also can be combined with the interaction results during actual driving; by integrating multiple visual / interaction elements in matrix or vector form, the AR-HUD overall device can flexibly balance personalization and safety requirements.

[0224] Regarding the comprehensive score, it should be noted that: during the operation of the AR-HUD, all the driver's operating behaviors on the UI configuration (such as font size, background transparency, color blindness adaptation level, etc.) are continuously recorded, and relevant monitoring data (such as eye tracking focus, false touch rate, operation delay) are collected regularly; when the driver stably uses a specific configuration without multiple switches or quick abandonment, the AR-HUD overall device will give a positive weighted score; if a significant increase in negative indicators is detected, the corresponding point deduction correction will be performed; finally, after the operation is completed or a preset interval, the comprehensive score result will be stored in the preference matrix as input for the next strategy evaluation, so as to accumulate and gradually optimize the HUD personalized configuration over the long term.

[0225] Step 502: Adaptive configuration update based on reinforcement learning

[0226] Define the state vector S rl (t), which may include:

[0227] The current layout and style of the HUD (such as font size, information card position, transparency, etc.) can be derived from the basic personalized configuration; the complexity of the environment F E (t) and driving load information D L (t), user preference vector H u The most relevant items for the current time period (such as color blindness adaptation requirements);

[0228] Action Set A rl : Multiple operations that can be adjusted in a given context (such as switching font schemes, modifying layout density, updating color themes, etc.);

[0229] Take an action a(t)∈A at time t rl After obtaining the new HUD configuration, it will be combined with the returned feedback data (such as the driver's operating load Λ op 、Eye focus Λ eye , evaluation function F eval etc.) to calculate the dynamic return function R τl (t, a(t)):

[0230] Let: Λ op (τ) and Λ eye (τ) represents the driver's operating load and eye movement dispersion at time τ, respectively;

[0231] F eval (t) represents the comprehensive evaluation result of the AR-HUD device at time t (including the quantification of driver comfort and safety indicators after UI adjustment);

[0232] L(τ)=[Λ op (τ),Λ eye (τ)T is the negative impact vector; M r =diag(μ op , μ eye ), used to assign differential weights to negative factors; T r >0 is the lookback time window, κ r >0 is the exponential decay coefficient, is the p-norm (p r ≥1); α r is the power term, α r >0 is used to amplify (or smooth) extremely high loads; β + , β _ ≥0 represents the adjustment coefficient for positive and negative terms, respectively, to balance the intensity of rewards and penalties of the AR-HUD overall device;

[0233] Based on the above definition, the dynamic reward function R r1 (t, a(t)) is:

[0234]

[0235] Where: Adjustment coefficient β - Determines the impact of negative accumulation on the final return; adjustment coefficient β + Determines the bonus effect of positive evaluations;

[0236] Weight coefficient μ op , μ eye , located in M r The diagonal of is used to distinguish the relative weight of operation load and eye movement load in penalty calculation. If a scene is particularly sensitive to eye movement deviation, the weight coefficient μ can be increased. eye ;

[0237] p r , α r are the p-norm exponent and power correction coefficient, respectively, which are used to adjust the sensitivity of multi-dimensional load superposition:

[0238] When p r When α > 2, the extremely high load component will be additionally amplified; r >1 will also further strengthen the large load. The combination of these two can implement strong penalties in high-load scenarios.

[0239] T r , κ r are the lookback time window, exponential decay coefficient, and κ r The value is greater than 0, which determines the degree of memory of historical load. r The larger the value, the longer the recording time; T The larger the value, the faster the decay, and more attention is paid to the latest load data, which is suitable for flexible parameter adjustment in rapidly changing driving scenarios;

[0240] Reinforcement learning algorithms (such as Q-learning, Actor-Critic, etc.) are based on the dynamic reward function R τl (t, a(t)) learns the long-term benefits of different actions and gradually converges to the optimal or suboptimal strategy after multiple iterations;

[0241] After multiple rounds of interaction, the AR-HUD device will learn a UI configuration strategy that balances driving safety, user preferences, and environmental complexity. And when the same or similar state S is detected again rl (t) automatically adopts this strategy for UI layout;

[0242] At the same time, the UI configuration strategies learned will be regularly Integrating with basic personalized configurations, if conflicts are detected (e.g., between safety requirements and user preferences), fine-tuning of rules can be performed manually or semi-automatically to ensure that the AR-HUD ultimately adheres to safety guidelines.

[0243] When used, through the dynamic return function R r1 (t, a(t)), the AR-HUD device can continuously try different UI combinations and select the solution that best meets the driver's comfort and safety indicators during long-term use; the iterative process of reinforcement learning can adaptively respond to changing driving habits and environmental backgrounds, so that the HUD configuration can truly achieve better matching with users over time; using a dynamic reward function R r1 (t, a(t)) expresses the reward, which not only retains significant penalties for extremely high loads or low evaluation values, but also provides greater positive incentives for good states, forming a more flexible self-learning mechanism;

[0244] It should be noted that: by integrating the driver's physiological and behavioral data (such as operating load Λ op (t), eye movement discrete Λ eye (t)), AR-HUD display status data (such as the current interface configuration G(t) or the actual trigger event), and the monitoring results of the external environment and vehicle driving safety by the on-board sensors. These multi-source information are weighted or aggregated using a specific nonlinear evaluation model (such as exponential mapping or p-norm fusion algorithm), and finally a numerical comprehensive evaluation value F is output at time t. eval (t), the higher the evaluation value, the more the current interface configuration is in line with the driver's current state and driving environment, which can ensure both safety and user experience. Otherwise, it means that it needs to be optimized or adjusted in subsequent steps.

[0245] See also Figure 2 The present invention provides an augmented reality head-up display system, comprising:

[0246] The multi-source data acquisition unit, when the vehicle is powered on and detects the sensor array output activation signal, performs sensor data cleaning, normalization, time alignment and labeling operations on the collected multi-source data input, and aggregates the processed data into a traceable and multi-dimensional environmental comprehensive value F E (t), establish an index field to quickly associate vehicle locations with time periods;

[0247] The information complexity judgment unit detects that the environmental comprehensive value F has been completed. E (t) and driving load value D L (t) Summarize, perform nonlinear fusion and threshold classification, retain intermediate calculation logs for subsequent backtracking, and then output the target information complexity label, ultimately laying a precise scene guide for subsequent content screening and improving dynamic adaptation efficiency;

[0248] When the strategy generation module detects that the target information complexity label is ready, the content screening engine performs priority scoring P on all information items in the AR-HUD information pool Ω. j (t) Set a more eye-catching font and center position for high-priority elements, retain only the outline or hide low-priority elements, and record the reasons for layout decisions and configuration logs. Finally, dynamically generate the HUD layout configuration vector and trigger on-demand display;

[0249] Adaptive feedback module, when monitoring the HUD configuration vector G(t) and the expected configuration G * (t) The deviation measurement function E fb (t) exceeds the deviation threshold θ fb , perform time-integrated analysis and anomaly detection on the interface status and external data, and refresh the display strategy locally or globally, while recording deviation correction logs for long-term analysis, ultimately achieving adaptive correction and ensuring information fit in a dynamic environment;

[0250] The personalized configuration module detects that the archive of the evaluation data has been completed and generates the user preference vector H u , historical feedback and physiological characteristics to perform multi-dimensional matrix aggregation, and based on the reinforcement learning dynamic reward function R r1 (t, a(t)) iteratively updates the UI configuration strategy, ultimately forming a humanized HUD display solution that self-evolves with usage time.

[0251] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0252] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0253] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is only for some logical functions. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0254] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0255] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An augmented reality head-up display method, characterized in that: include, When the vehicle starts and completes sensor initialization, it gathers the environmental comprehensive value and performs denoising, normalization and labeling. At the same time, it establishes a feature vector index mapping to generate a multi-dimensional perception data structure. Among them, the environmental comprehensive value F is defined as E (t), define the vector: Where: Θ i (τ) represents the normalized measurement value of the i-th sensor or external data source at time τ; λ i represents the weight coefficient for the i-th sensor measurement value; diag(·) represents the diagonal matrix construction operation, M is an n×n weight diagonal matrix, and n is the total number of sensors / data sources; Where: Θ(τ) is a column vector of all sensor measurements collected at time, and each component Θ i (τ) are all normalized and mapped to [0, 1] as defined previously; M is the weight diagonal matrix, λ i Located on the main diagonal; T is the lookback time window, which represents the time range before the current time t; k is the exponential decay coefficient, k>0; ||·||2 is the Euclidean norm operation; When the environmental comprehensive value and driving load value data are detected and available, the scenario analysis engine performs nonlinear fusion and compares the two with corresponding thresholds. It also associates historical operation logs to identify the complexity of target information, outputs scenario-level labels, and clarifies the direction of subsequent content screening; When receiving the scene level label, it extracts optional items from the information pool and calculates the priority score, simultaneously enhancing the visibility of high-priority information and weakening the visibility of low-priority information to generate the HUD layout vector; When the deviation between the monitored HUD configuration vector and the expected vector exceeds the deviation threshold, the UI layout and external environment are searched and anomaly identified, the HUD layout record is updated, and a local refresh is triggered; Define the configuration deviation measurement function E fb (t): Where: is a vector of length m; T d To monitor the backtracking time window, T d >0;κ d is the exponential decay coefficient, κ d >0, ;||·|| pd is the p-norm (p d ≥1); α d is a power operation, α d >0; After completing the evaluation data archiving, load the user preference vector and preference matrix and apply the reinforcement learning reward function to dynamically adjust the UI layout strategy, retain the best configuration log, and form a personalized HUD display solution; The defined state vector includes: the current HUD layout and style, the environment complexity and driving load value, the most relevant entry in the user preference vector for the current time period, and the action set; After performing an action to obtain a new HUD configuration, the dynamic reward function is calculated in the next moment based on the feedback data returned. The reinforcement learning algorithm learns the long-term benefits of different actions based on the dynamic reward function and gradually converges to the optimal or suboptimal strategy after multiple iterations. After multiple rounds of interaction, the AR-HUD device will learn the UI configuration strategy and automatically adopt the strategy for UI layout when the same or similar state is detected again; at the same time, it will regularly integrate the learned UI configuration strategy with the basic personalized configuration, and adjust it if some preference conflicts are found.

2. The augmented reality head-up display method according to claim 1, characterized in that: The vehicle-mounted sensor array outputs multi-dimensional raw measurement values ​​in time series, and external data sources simultaneously provide global / regional environmental parameters; define the comprehensive environmental value; The comprehensive environmental values ​​and each original measurement value are marked, and timestamp, vehicle location coordinates, data source identification are added, and scene type field classification is attached to form a multidimensional index.

3. The augmented reality head-up display method according to claim 2, characterized in that: By combining driver-related data sources, including eye tracking, gesture frequency, and operation switching frequency, a driving load value is defined to construct a comprehensive situational score. The comprehensive situational score is then compared with a pre-defined complexity threshold range: When the comprehensive context score is less than or equal to the first complexity threshold, it is judged as concise; when the comprehensive context score is between the first and second complexity thresholds, it is judged as moderate; when the comprehensive context score is greater than the second complexity threshold, it is judged as rich; the obtained target complexity label is recorded and output together with the timestamp.

4. The augmented reality head-up display method according to claim 3, characterized in that: Obtain all information items currently available for display from the information pool maintained by the AR-HUD overall device; According to the target information complexity label and the current situation score, the content consistency is calculated for each piece of information; the items with content consistency greater than or equal to the consistency are included in the preliminary selection set, and the remaining information is temporarily excluded.

5. The augmented reality head-up display method according to claim 4, characterized in that: For each piece of information that has entered the preliminary selection set, calculate the priority score, sort it from high to low according to the priority score, and map it to the corresponding area of ​​the HUD interface; High-priority information is placed in the core area of ​​the driver's field of vision and given higher brightness or larger font size; Medium-priority information can be placed at the edge of the line of sight or presented in a semi-transparent manner; low-priority information can be temporarily hidden or displayed in thumbnail form as icons; the parameter tags of the laid-out information in the HUD are retained and associated with the timestamp to form a UI presentation layout.

6. The augmented reality head-up display method according to claim 5, characterized in that: Set trigger thresholds for generated high-priority content. If the environment or driver data is detected to meet the trigger conditions, the corresponding information will be switched from a semi-transparent or hidden state to a full-screen / highlight state, accompanied by visual effects or sound reminders; For medium-priority information, it can be displayed in a fade-in / fade-out manner according to the real-time fluctuations in the complexity of the environment, and the time point of the triggering / folding action, the triggering reason and the corresponding information identification are recorded to form an event log.

7. The augmented reality head-up display method according to claim 6, characterized in that: Continuously obtain the latest measurements from onboard sensors and external data sources, as well as the generated UI layout and event logs, and monitor the consistency between previous decisions and current new observations by defining a deviation measurement function; When the configuration deviation metric function is too high, it indicates that there is a significant deviation between the current HUD strategy and the latest external environment, and adjustment is required; if the configuration deviation metric function is lower than the preset deviation threshold, no refresh is required; if the configuration deviation metric function is higher than the preset deviation threshold, the situation is re-evaluated and the display strategy is updated.

8. The augmented reality head-up display method according to claim 7, characterized in that: When the configuration deviation measurement function is greater than or equal to the deviation threshold, a quick re-determination is performed; Based on the results of the situational re-judgment, if only some information elements are deviated, the local display elements are corrected by triggering the logic on demand, without having to globally refresh all HUD content; the updated new HUD configuration vector is written into the AR-HUD overall device record, and the specific reason for the change is marked.

9. The augmented reality head-up display method according to claim 8, characterized in that: The triggering time, triggering reason, new HUD configuration vector and driver response are stored in a structured manner; A feedback evaluation function is defined to measure the impact of strategy updates on driver behavior or AR-HUD overall device efficiency, and the accumulated multi-dimensional feedback data is used to periodically or online update the machine learning model or expert rules.

10. The augmented reality head-up display method according to claim 9, characterized in that: After detecting the driver's login, all past evaluation events and operation preference data of the user are retrieved from the archived historical records to form a user preference vector. Combined with the results of the feedback evaluation function, the previous multiple UI interface layout configuration strategies and the corresponding driver load indicators are quantitatively aggregated to obtain a personalized preference matrix. Maintain the user preference vector and personalized preference matrix separately and automatically switch them at login. According to the user preference vector and personalized preference matrix, output the basic personalized configuration in the current context.

11. An augmented reality head-up display system, applying the display method according to any one of claims 1 to 10, characterized in that: include, The multi-source data acquisition unit, when the vehicle starts and completes sensor initialization, aggregates environmental comprehensive values ​​and performs denoising, normalization, and labeling. It also establishes a feature vector index mapping to generate a multi-dimensional perception data structure. The information complexity judgment unit detects that the environmental comprehensive value and driving load value data are available. The situational analysis engine performs nonlinear fusion and compares the two with the corresponding thresholds. It also associates historical operation logs to identify the target information complexity, outputs a scene level label, and clarifies the direction of subsequent content screening. The strategy generation module, upon receiving the scene level label, extracts optional items from the information pool and calculates the priority score, simultaneously enhancing the visibility of high-priority information and weakening the visibility of low-priority information, and generates the HUD layout vector; The adaptive feedback module searches and identifies anomalies in the UI layout and external environment when it detects that the deviation between the HUD configuration vector and the expected vector exceeds the deviation threshold, updates the HUD layout record, and triggers a local refresh. After completing the archiving of the evaluation data, the personalized configuration module loads the user preference vector and preference matrix and applies the reinforcement learning reward function to dynamically adjust the UI layout strategy, retain the excellent configuration log, and form a personalized HUD presentation solution.

Images