Vehicle-mounted scene-based HUD adaptive display method and system

Abstract

This invention discloses an adaptive display method and system for in-vehicle HUDs, relating to the field of in-vehicle head-up display technology. It includes acquiring multi-source perception data to predict the evolution sequence of driving scenarios within a future time window; acquiring a heatmap of the driver's attention spatial distribution; determining the presentation priority weight of each information element based on the scenario prediction confidence and attention distribution density; adaptively adjusting the HUD's display content and position according to the priority weight and the driving scenario evolution sequence; allocating display resources for each information element according to its priority weight when multiple scenarios overlap; and smoothly controlling changes in the display layout. This invention, by integrating driving scenario prediction and driver visual attention perception, achieves proactive predictive adjustment of the HUD display layout and intelligent resource allocation under multi-scenario conflicts, improving the efficiency of acquiring key information and driving safety.

Description

Technical Field

[0001] This invention relates to the field of vehicle head-up display technology, specifically to a vehicle-mounted scene-based adaptive HUD display method and system. Background Technology

[0002] With the rapid development of automotive intelligence and connectivity technologies, head-up displays (HUDs) have become an important component of modern smart cockpits. HUDs project key information such as vehicle speed, navigation guidance, and intelligent driving assistance status onto the driver's field of vision, allowing the driver to obtain necessary information while keeping their eyes on the road ahead. This effectively reduces the time the driver's gaze deviates from the road, playing a crucial role in improving driving safety.

[0003] Currently, mainstream HUD interface display solutions in the industry are relatively fixed, typically displaying information such as vehicle speed, navigation, and driver assistance systems in a preset layout. This type of HUD display method has the following shortcomings:

[0004] First, the content displayed on the HUD cannot automatically change according to the current driving scenario of the vehicle. It often displays the same set of information in all scenarios, lacking scenario adaptability.

[0005] Second, the HUD display position cannot be adaptively adjusted according to the combination of different information content. The screen position of information elements is relatively fixed and cannot be dynamically arranged according to the importance of the information and the driver's attention.

[0006] Third, when driving scenarios such as driving, intelligent driving, navigation, and traffic jams switch or overlap, the HUD interface lacks a corresponding dynamic layout mechanism. When multiple scenarios exist simultaneously, information is presented in a mixed manner, making it difficult for drivers to quickly extract key information.

[0007] Fourth, because the displayed content and location are fixed, the efficiency of drivers in obtaining key information in different driving scenarios is low. When the scenario changes suddenly, drivers need to spend a long time searching for the required information in the fixed display area, which reduces interaction efficiency and driving safety.

[0008] Fifth, existing HUD display adjustment solutions typically only respond passively based on the current function switch status, such as deciding whether to display navigation information based solely on whether navigation is on, without considering the dynamic evolution of driving scenarios, resulting in a lag in the switching of display layout.

[0009] Sixth, the existing solution does not incorporate the driver's visual attention distribution characteristics, and cannot dynamically adjust the information layout according to the driver's current focus area, making it difficult to achieve truly personalized human-machine collaborative optimization.

[0010] In summary, existing HUD display solutions generally suffer from core technical problems such as passive and delayed scene response, lack of predictability in layout adjustments, and information conflicts that lead to a surge in driver cognitive load when multiple scenes are superimposed. Summary of the Invention

[0011] To overcome the shortcomings of existing technologies, this invention proposes an in-vehicle scene-based adaptive HUD display method and system, aiming to solve the technical problems of passive lag in scene response, lack of predictability in layout adjustment, failure to consider the driver's visual attention distribution characteristics, and excessive cognitive load when multiple scenes are superimposed in existing HUD display methods.

[0012] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0013] In a first aspect, the present invention provides an adaptive display method for a vehicle-mounted HUD, comprising:

[0014] Acquire multi-source perception data and predict the evolution sequence of driving scenarios within a future time window based on the multi-source perception data;

[0015] Obtain a heatmap of the spatial distribution of driver attention;

[0016] Based on the scenario prediction confidence of the driving scenario evolution sequence and the attention distribution density of the driver attention spatial distribution heatmap in the display area of ​​each information element, the presentation priority weight of each information element is determined.

[0017] Based on the presentation priority weight and the driving scenario evolution sequence, the display content and display position of the HUD are adaptively adjusted.

[0018] When multiple scenarios overlap in the driving scenario evolution sequence, the display resources of each information element are allocated according to the presentation priority weight of each information element.

[0019] Smooth transition control is implemented for the changes in display layout during the adaptive adjustment and the allocation of display resources.

[0020] Optionally, predicting the evolution sequence of driving scenarios within a future time window based on the multi-source perception data includes:

[0021] Acquire the multi-source perception data; the multi-source perception data includes vehicle dynamics parameters, navigation path planning data, and high-precision map data;

[0022] The multi-source perception data is input into the driving scene prediction network, and the driving scene prediction network outputs a sequence of scene type probability distributions within the future time window with a preset time interval as the step size.

[0023] The scenario types include driving scenarios, intelligent driving scenarios, navigation scenarios, congestion scenarios, ramp scenarios, and lane changing scenarios.

[0024] Optionally, obtaining the spatial distribution heatmap of driver attention includes:

[0025] Acquire a driver's facial image, and extract key eye information from the facial image. The key eye information includes at least the pupil center coordinates, the gaze direction vector, and the eyelid opening degree.

[0026] The driver's head posture is estimated to obtain head posture parameters, which include at least the head yaw angle.

[0027] The position of the gaze point is calculated based on the pupil center coordinates and the gaze direction vector. The position of the gaze point is then processed by Gaussian kernel diffusion to generate a heat map of the spatial distribution of the driver's attention.

[0028] The kernel bandwidth parameter of the Gaussian kernel is adaptively adjusted based on the eyelid opening and closing degree to reflect changes in attention concentration.

[0029] Optionally, determining the presentation priority weight of each information element includes:

[0030] Obtain the prediction confidence level corresponding to each scenario type in the driving scenario evolution sequence;

[0031] Obtain the attention distribution density of the driver's attention spatial distribution heatmap in each information element display area, and calculate the normalized ratio of the attention distribution density to the total attention distribution density in the HUD display area;

[0032] Obtain the current timeliness and urgency of each information element;

[0033] The presentation priority weight of each information element is obtained by weighting and summing the scenario prediction confidence, the normalized ratio, and the information timeliness urgency.

[0034] Optionally, the adaptive adjustment of the HUD's display content and position includes:

[0035] Analyze the geometric information of the navigation path and calculate the rate of curvature change of the navigation path within a preset distance range ahead;

[0036] When the rate of change of curvature is detected to be greater than the preset curvature threshold, it is determined that there is a curve ahead, triggering a predictive layout adjustment;

[0037] The advance time is calculated based on the vehicle speed, the rate of curvature change, and the preset safe viewing angle threshold.

[0038] According to the aforementioned advance time, the navigation steering prompt information is gradually migrated towards the driver's central field of vision at a preset distance from the curve entrance, and the migration trajectory is smoothly controlled using a cubic Bézier curve.

[0039] Optionally, the adaptive adjustment of the HUD's display content and position further includes:

[0040] In response to the navigation function activation signal, a first layout adjustment command is generated. The first layout adjustment command is used to control the vehicle speed information and the intelligent driving logo to move horizontally to the left by a first preset offset on the HUD display plane.

[0041] A second layout adjustment command is generated, which is used to control the display of navigation-related information on the right side of the HUD display plane;

[0042] A third layout adjustment instruction is generated, which is used to control the overall display area of ​​the vehicle speed information, the intelligent driving logo and the navigation related information to be centered on the HUD display plane.

[0043] Optionally, when multiple scenes overlap in the driving scene evolution sequence, allocating display resources for each information element according to its presentation priority weight includes:

[0044] Obtain all scenario types existing at the current moment in the driving scenario evolution sequence, and construct scenario combinations;

[0045] Calculate the scene compatibility index between any two scene types in the scene combination;

[0046] When the scenario compatibility index is less than the preset compatibility threshold, it is determined that there is a conflict between multiple scenarios, and the information time-sharing reuse strategy is triggered.

[0047] Based on the presentation priority weight of each information element, the display duration and space occupation ratio of each information element are dynamically adjusted.

[0048] Secondary information is mapped to the color channel for display, while key information is mapped to the luminance and motion channels for display.

[0049] Optionally, the smooth transition control of display layout changes during the adaptive adjustment and display resource allocation process includes:

[0050] Obtain the layout parameter vector at the current moment and the layout parameter vector at the previous moment, and calculate the change of the layout parameter vector at the current moment relative to the layout parameter vector at the previous moment.

[0051] Obtain the spatial position of each information element under the layout parameter vector at the current moment, and calculate the distance between the spatial position of each information element and the optimal attention landing point position of the corresponding information element;

[0052] A first cost term is constructed based on the change amount, and a second cost term is constructed based on the weighted sum of the distance and the presentation priority weight. The combination of the first cost term and the second cost term is used as the objective function.

[0053] A predictive control method is used to solve the minimization problem of the objective function in the prediction time domain, outputting the layout parameter sequence in the control time domain, and generating a continuous and smooth layout migration trajectory.

[0054] Further, obtaining the spatial position of each information element under the layout parameter vector at the current moment, and calculating the distance between the spatial position of each information element and the optimal attention landing point position of the corresponding information element includes:

[0055] The optimal attention landing point position corresponding to each information element is determined based on the heat map of the driver's attention spatial distribution.

[0056] Based on the layout parameter vector at the current moment, the spatial coordinates of each information element on the HUD display plane are analyzed;

[0057] Calculate the Euclidean distance between the spatial coordinates of each information element and the optimal attention point location.

[0058] Secondly, this invention provides an in-vehicle scene-based adaptive HUD display system, comprising:

[0059] The first acquisition module is used to acquire multi-source perception data and predict the evolution sequence of driving scenarios within a future time window based on the multi-source perception data.

[0060] The second acquisition module is used to acquire a heat map of the spatial distribution of driver attention;

[0061] The determination module is used to determine the presentation priority weight of each information element based on the scenario prediction confidence of the driving scenario evolution sequence and the attention distribution density of the driver attention spatial distribution heatmap in the display area of ​​each information element.

[0062] The adjustment module is used to adaptively adjust the display content and display position of the HUD according to the presentation priority weight and the driving scenario evolution sequence;

[0063] The priority allocation module is used to allocate display resources for each information element according to the presentation priority weight of each information element when there are multiple overlapping scenarios in the driving scenario evolution sequence.

[0064] The control module is used to smoothly control the changes in display layout during the adaptive adjustment and the allocation of display resources.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The present invention provides an adaptive HUD display method and system for in-vehicle scenarios. By introducing a driving scenario prediction mechanism, it can predict the evolution sequence of driving scenarios within a future time window based on multi-source perception data, realize the proactive predictive adjustment of the HUD display layout, effectively avoid the lag in information presentation when switching scenarios, and significantly reduce the cognitive load jump of the driver at the moment of scenario switching.

[0067] This invention deploys a driver visual attention perception module to calculate the driver's attention spatial distribution heatmap on the HUD display plane in real time. This allows the layout of information elements to be dynamically optimized to match the driver's visual attention distribution characteristics, significantly increasing the probability that key information falls into the driver's optimal visual area and achieving human-machine collaborative optimization tailored to the individual.

[0068] This invention establishes a dynamic priority calculation model for information elements, and calculates the presentation priority weight of each information element by comprehensively considering three dimensions: scene prediction confidence, attention distribution density, and information timeliness urgency, so that the information presentation order is more in line with the actual needs of the current driving scenario.

[0069] This invention analyzes the geometric features of the navigation path in a navigation scenario and triggers layout adjustments in advance when a curve is detected ahead. This allows navigation steering prompts to be moved to the driver's central field of vision before entering the curve, effectively improving the timeliness of obtaining key navigation information.

[0070] This invention establishes a scene conflict resolution mechanism, dynamically allocates display resources based on scene compatibility judgment and priority weight when multiple scenes are superimposed, and uses visual channel separation technology to map information of different priorities to different display channels, effectively reducing information interference and driver cognitive load when multiple scenes are superimposed.

[0071] This invention establishes a display layout smooth transition controller to generate a continuous and smooth layout transition trajectory with the goal of minimizing the driver's visual search cost. This effectively suppresses visual disturbances caused by sudden layout changes and improves the human-computer interaction comfort during the display switching process. Attached Figure Description

[0072] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0073] Figure 1 This is a flowchart of an in-vehicle scene-based adaptive HUD display method provided by the present invention;

[0074] Figure 2 This is a schematic diagram of the structure of an in-vehicle scene-based adaptive HUD display system provided in an embodiment of the present invention;

[0075] Figure 3 A schematic diagram of the overall control flow of the vehicle-mounted scene-based HUD adaptive display method is provided for embodiments of the present invention;

[0076] Figure 4 This is an internal structure diagram of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0077] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of the present invention and are therefore merely examples, and should not be construed as limiting the scope of protection of the present invention.

[0078] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0079] This invention provides a method and system for adaptive HUD display in vehicle scenarios. The embodiments of this invention are described below with reference to the accompanying drawings.

[0080] Example 1: As Figure 1 As shown, Embodiment 1 of the present invention provides a vehicle-mounted scene-based adaptive HUD display method, which specifically includes the following steps:

[0081] Step S101: In response to the mute request signal, determine the target mute channel data from multiple audio channels based on the pre-acquired vehicle status data and channel attribute data.

[0082] In this step, the system continuously monitors mute trigger signals from virtual buttons on the central control screen, physical buttons on the steering wheel, or the voice recognition engine. When any trigger source generates a valid mute request signal, the system determines the target mute channel data that can be muted from multiple audio channels based on pre-acquired vehicle status data and channel attribute data of each audio channel. The vehicle status data includes at least vehicle speed data, gear status data, and driving mode data, which can be obtained from the vehicle's electronic control unit via the CAN bus. The channel attribute data includes at least the functional domain classification label and safety level label of each channel, which are pre-stored in the system's channel configuration table.

[0083] The total length of the aforementioned extended fields does not exceed the maximum payload length of a BLE broadcast packet and is fully compatible with the standard BLE protocol. Native BLE devices that do not support this extended field will automatically ignore unknown content in the vendor-defined data segment when scanning for this broadcast packet, and will not generate parsing errors or anomalies.

[0084] In one embodiment, such as Figure 1 As shown, the overall forwarding process of this invention includes the following steps:

[0085] The original service data is encrypted to generate an encrypted service message; the encrypted service message and an extended field containing the remaining hop count are encapsulated into a BLE broadcast packet, and the remaining hop count is set as a preset initial hop count, and the BLE broadcast packet is sent on the broadcast channel;

[0086] Listen to the broadcast channel, receive the BLE broadcast packet, and then verify the validity of the BLE broadcast packet;

[0087] Step S101: Acquire multi-source perception data, and predict the evolution sequence of driving scenarios within a future time window based on the multi-source perception data;

[0088] In this step, predicting the evolution sequence of driving scenarios within a future time window based on the multi-source perception data includes:

[0089] Acquire the multi-source perception data; the multi-source perception data includes vehicle dynamics parameters, navigation path planning data, and high-precision map data;

[0090] The multi-source perception data is input into the driving scene prediction network, and the driving scene prediction network outputs a sequence of scene type probability distributions within the future time window with a preset time interval as the step size.

[0091] The scenario types include driving scenarios, intelligent driving scenarios, navigation scenarios, congestion scenarios, ramp scenarios, and lane changing scenarios.

[0092] In this embodiment, the multi-source sensing data is acquired by establishing a real-time communication link with the vehicle CAN bus, navigation ECU, high-precision map service and in-vehicle image acquisition device, and at a preset sampling frequency.

[0093] Then, the evolution sequence of driving scenarios within the future time window is predicted based on the multi-source perception data. In this embodiment, a driving scenario prediction network is constructed, preferably using a temporal encoder based on the Transformer architecture. The acquired multi-source perception data is input into the driving scenario prediction network, which extracts temporal and spatial features from the data and outputs a probability distribution sequence of scenario types within the future time window with a preset time interval as the step size.

[0094] The future time window can be set according to the actual application scenario, for example, a 5-second window in the future; the preset time interval can be set to 100ms. The scenario types include, but are not limited to, driving scenarios, intelligent driving scenarios, navigation scenarios, congestion scenarios, ramp scenarios, and lane-changing scenarios. The probability distribution of each scenario type can be encoded as a multi-dimensional vector, for example, a 6-dimensional one-hot vector, with each dimension corresponding to the probability value of a scenario type. Through the above steps, the system can quantitatively predict the evolution trend of driving scenarios over a future period at the current moment, providing a data foundation for proactive adjustments to the subsequent display layout.

[0095] Step S102: Obtain a heatmap of the spatial distribution of driver attention;

[0096] This step involves obtaining a heatmap of the spatial distribution of driver attention, including:

[0097] Acquire a driver's facial image, and extract key eye information from the facial image. The key eye information includes at least the pupil center coordinates, the gaze direction vector, and the eyelid opening degree.

[0098] The driver's head posture is estimated to obtain head posture parameters, which include at least the head yaw angle.

[0099] The position of the gaze point is calculated based on the pupil center coordinates and the gaze direction vector. The position of the gaze point is then processed by Gaussian kernel diffusion to generate a heat map of the spatial distribution of the driver's attention.

[0100] The kernel bandwidth parameter of the Gaussian kernel is adaptively adjusted based on the eyelid opening and closing degree to reflect changes in attention concentration.

[0101] Specifically, firstly, an in-vehicle image acquisition device is used to acquire the driver's facial image. The in-vehicle image acquisition device is preferably an IR camera deployed inside the A-pillar to ensure that clear facial images can still be acquired at night or in poor lighting conditions.

[0102] Key eye information is extracted from the acquired driver's facial images. In this embodiment, key eye detection can be implemented using a lightweight neural network (such as MobileNetV3). This network outputs the pupil center coordinates, gaze direction vector, and eyelid opening / closing degree in real time. The pupil center coordinates are used to determine the position of the eyeball in space, the gaze direction vector is used to determine the gaze direction, and the eyelid opening / closing degree is used to reflect the driver's level of concentration.

[0103] The driver's head attitude is estimated to obtain head attitude parameters, which include at least the head yaw angle. Head attitude estimation can employ a 6-DOF pose calculation method to determine the driver's head rotation angles (pitch, yaw, and roll angles) in three-dimensional space. To improve the stability of head attitude estimation in dynamic scenes, IMU (Inertial Measurement Unit) data can be fused for compensation and correction.

[0104] The gaze point position is calculated based on the pupil center coordinates and the gaze direction vector. This gaze point position is the driver's fixation point position on the HUD display plane. Because the gaze point position at a single moment contains jitter and noise, Gaussian kernel diffusion processing is applied to the gaze point position to smooth the discrete gaze point data into a continuously distributed driver attention spatial distribution heatmap A(x,y,t), where (x,y) are the two-dimensional coordinates on the HUD display plane, and t is the time index. The numerical values ​​on this heatmap represent the driver's attention distribution density at the corresponding spatial location.

[0105] Simultaneously, the kernel bandwidth parameter of the Gaussian kernel is adaptively adjusted based on the eyelid opening and closing degree. When the eyelid opening and closing degree is large, it indicates that the driver's attention is relatively concentrated. In this case, the kernel bandwidth parameter is reduced to make the attention heatmap more resolution and more concentrated. When the eyelid opening and closing degree is small, it indicates that the driver's attention is relatively scattered or there are signs of fatigue. In this case, the kernel bandwidth parameter is increased to make the attention heatmap distribution smoother and more diffuse, so as to more accurately reflect the actual attention distribution.

[0106] Step S103: Determine the presentation priority weight of each information element based on the scene prediction confidence of the driving scene evolution sequence and the attention distribution density of the driver attention spatial distribution heatmap in the display area of ​​each information element.

[0107] In one embodiment, determining the presentation priority weight of each information element includes:

[0108] Obtain the prediction confidence level corresponding to each scenario type in the driving scenario evolution sequence;

[0109] Obtain the attention distribution density of the driver's attention spatial distribution heatmap in each information element display area, and calculate the normalized ratio of the attention distribution density to the total attention distribution density in the HUD display area;

[0110] Obtain the current timeliness and urgency of each information element;

[0111] The presentation priority weight of each information element is obtained by weighting and summing the scenario prediction confidence, the normalized ratio, and the information timeliness urgency.

[0112] Specifically, the prediction confidence level for each scenario type in the driving scenario evolution sequence is first obtained. For example, when predicting the next 2 seconds from the current moment, the confidence level for the driving scenario is 0.9, the confidence level for the navigation scenario is 0.7, and the confidence level for the intelligent driving scenario is 0.3, etc. This prediction confidence level reflects the probability of a specific scenario occurring at a certain future moment and is one of the important bases for determining the priority of information elements.

[0113] Secondly, the attention distribution density of the driver's attention spatial distribution heatmap within the display area of ​​each information element is obtained. For each information element i to be displayed (such as vehicle speed information, navigation arrow, intelligent driving status indicator, etc.), the display area Ri of the information element in the current layout is determined, the integral value of the attention heatmap in area Ri is calculated as ∫Ri A(x,y,t)dxdy, and the normalized ratio of this integral value to the total attention distribution density ∫HUD A(x,y,t)dxdy in the entire HUD display area is calculated to obtain the attention index.

[0114] Then, the urgency level Ui(t) of each information element at the current moment is obtained. The urgency level reflects the degree of urgency of the information in the time dimension. For example, for navigation turn prompts, their urgency level can be defined as a function that increases exponentially with decreasing distance from the turn point.

[0115] Ui(t)=exp(-d turn ·v ego / τ urgent )

[0116] Where d turn v represents the distance from the vehicle's current position to the turning point. ego For the vehicle's speed, τ urgentThis is the preset urgency time constant. For intelligent driving status prompts, when the system detects that the driver needs to take over, the urgency level is set to a higher value; when the system degrades, the urgency level is set to a medium value.

[0117] Finally, the scene prediction confidence, the normalized ratio, and the information urgency are weighted and summed to obtain the presentation priority weight wi(t) of each information element. The specific calculation formula is as follows:

[0118] wi(t)=α·Pi(t)+β·Ai(t)+γ·Ui(t)

[0119] Where Pi(t) is the scene prediction confidence, Ai(t) is the normalized attention distribution density ratio, and Ui(t) is the information urgency; α, β, and γ are calibrable weight coefficients, satisfying α + β + γ = 1. These weight coefficients can be calibrated through driving simulator experiments. A higher priority weight for an information element indicates greater importance of that information in the current moment and scenario, and its display quality should be prioritized.

[0120] Step S104: Adaptively adjust the display content and display position of the HUD according to the presentation priority weight and the driving scene evolution sequence;

[0121] This step involves adaptively adjusting the content and position of the HUD display, including:

[0122] Analyze the geometric information of the navigation path and calculate the rate of curvature change of the navigation path within a preset distance range ahead;

[0123] When the rate of change of curvature is detected to be greater than the preset curvature threshold, it is determined that there is a curve ahead, triggering a predictive layout adjustment;

[0124] The advance time is calculated based on the vehicle speed, the rate of curvature change, and the preset safe viewing angle threshold.

[0125] According to the aforementioned advance time, the navigation steering prompt information is gradually migrated towards the driver's central field of vision at a preset distance from the curve entrance, and the migration trajectory is smoothly controlled using a cubic Bézier curve.

[0126] Optionally, adaptively adjusting the displayed content and position of the HUD also includes:

[0127] In response to the navigation function activation signal, a first layout adjustment command is generated. The first layout adjustment command is used to control the vehicle speed information and the intelligent driving logo to move horizontally to the left by a first preset offset on the HUD display plane.

[0128] A second layout adjustment command is generated, which is used to control the display of navigation-related information on the right side of the HUD display plane;

[0129] A third layout adjustment instruction is generated, which is used to control the overall display area of ​​the vehicle speed information, the intelligent driving logo and the navigation related information to be centered on the HUD display plane.

[0130] In a specific embodiment, when the navigation function is enabled, the adaptive adjustment includes two levels: static layout adjustment and predictive layout adjustment.

[0131] 1. Static layout adjustment:

[0132] In response to the navigation function activation signal, multiple layout adjustment commands are generated. A first layout adjustment command is generated, which controls the vehicle speed information and the intelligent driving indicator to shift horizontally to the left by a preset offset on the HUD display plane to reserve display space for navigation-related information. A second layout adjustment command is generated, which controls the navigation-related information to be displayed in the right-hand area of ​​the HUD display plane, creating a spatial partition between the navigation information and vehicle speed information. A third layout adjustment command is generated, which controls the overall display area of ​​the vehicle speed information, the intelligent driving indicator, and the navigation-related information to be centrally located on the HUD display plane, ensuring that the information combination is within the driver's central field of vision.

[0133] 2. Predictive layout adjustment (curving scenario):

[0134] After the navigation function is activated, the system analyzes the geometric information of the navigation path and calculates the rate of curvature change κ'(s) of the navigation path within a preset distance range ahead (e.g., within 500m ahead), where s is the arc length parameter along the path. The rate of curvature change reflects the degree of change in the curvature of the path.

[0135] When the rate of change of curvature is detected to be greater than a preset curvature threshold (e.g., κ' > 0.001m) -1 When the vehicle detects a curve ahead, a predictive layout adjustment is triggered. At this point, based on the vehicle's speed v... ego The curvature change rate κ'(s) and the preset safe viewing angle threshold θ safe Calculate the advance time Δt. This advance time is used to determine how far in advance the navigation turn signal should enter the driver's central field of vision. In a preferred embodiment, the advance time can be calculated as Δt = (κ(s)·v ego +ε) / (v ego ·θ safeThe calculation is performed, where ε is a very small positive number to prevent division by zero.

[0136] Based on the aforementioned advance time, the navigation steering prompt information gradually migrates towards the driver's central field of vision at a preset distance from the curve entrance. For example, the migration begins approximately 150 meters from the curve entrance, with the migration trajectory smoothly controlled using a cubic Bézier curve, causing the navigation steering arrow to move smoothly from the right-hand information area to the central field of vision. The migration process can be set to last 2 seconds, with a 3-second stable display time reserved to allow the driver sufficient time to read the information before entering the curve.

[0137] Step S105: When multiple scenes are superimposed in the driving scene evolution sequence, the display resources of each information element are allocated according to the presentation priority weight of each information element, including:

[0138] Obtain all scenario types existing at the current moment in the driving scenario evolution sequence, and construct scenario combinations;

[0139] Calculate the scene compatibility index between any two scene types in the scene combination;

[0140] When the scenario compatibility index is less than the preset compatibility threshold, it is determined that there is a conflict between multiple scenarios, and the information time-sharing reuse strategy is triggered.

[0141] Based on the presentation priority weight of each information element, the display duration and space occupation ratio of each information element are dynamically adjusted.

[0142] In one specific embodiment, when multiple scenes are superimposed in the driving scene evolution sequence, the display resources of each information element are allocated according to the presentation priority weight of each information element.

[0143] Specifically, firstly, all scenario types existing at the current moment in the driving scenario evolution sequence are obtained, and scenario combinations are constructed. For example, if both "navigation scenario" and "intelligent driving scenario" exist at the current moment, then the scenario combination is {navigation, intelligent driving}.

[0144] Calculate the scene compatibility index C(S) between any two scene types in the scene combination. i ,S j The scene compatibility index reflects whether there are display conflicts between the corresponding information elements when two scenes exist simultaneously. The value range of the scene compatibility index can be preset through experiments, for example, the value range is [0,1], and the larger the value, the better the compatibility.

[0145] When the scenario compatibility index is less than a preset compatibility threshold (e.g., θ) compatWhen the value is 0.5, it is determined that there is a conflict between multiple scenarios, and the information time-sharing reuse strategy is triggered.

[0146] Based on the presentation priority weight of each information element, the display duration and space occupation ratio of each information element are dynamically adjusted. Information elements with higher presentation priority weights receive longer display durations and larger display areas; information elements with lower presentation priority weights have correspondingly reduced display durations and space occupations.

[0147] Meanwhile, visual channel separation technology is employed to reduce the information overlay effect. Lower-priority information (secondary information) is mapped to the color channel for display, for example, presented as semi-transparent, low-contrast, or cool tones; higher-priority information (critical information) is mapped to the brightness and motion channels for display, for example, presented as high brightness, high contrast, flickering, or dynamic effects. Through this multi-channel separation strategy, drivers can quickly capture critical information without being overly distracted by secondary information, effectively reducing cognitive load when multiple scenarios overlap.

[0148] Secondary information is mapped to the color channel for display, while key information is mapped to the luminance and motion channels for display.

[0149] Step S106: Perform smooth transition control on the display layout changes during the adaptive adjustment and the allocation of display resources.

[0150] In this step, smooth transition control includes:

[0151] Obtain the layout parameter vector at the current moment and the layout parameter vector at the previous moment, and calculate the change of the layout parameter vector at the current moment relative to the layout parameter vector at the previous moment.

[0152] Obtain the spatial position of each information element under the layout parameter vector at the current moment, and calculate the distance between the spatial position of each information element and the optimal attention landing point position of the corresponding information element;

[0153] A first cost term is constructed based on the change amount, and a second cost term is constructed based on the weighted sum of the distance and the presentation priority weight. The combination of the first cost term and the second cost term is used as the objective function.

[0154] A predictive control method is used to solve the minimization problem of the objective function in the prediction time domain, outputting the layout parameter sequence in the control time domain, and generating a continuous and smooth layout migration trajectory.

[0155] In a preferred embodiment of the present invention, after receiving a BLE broadcast packet, the vehicle-mounted main control device 300 further extracts the BLE relay forwarding device identification information from the BLE broadcast packet and records the path information. This path information may include the device identification sequence of each level of BLE relay forwarding device traversed by the broadcast packet. Based on the path information, the vehicle-mounted main control device 300 calculates the position information of the terminal source device relative to the vehicle (e.g., determining whether the terminal source device is located in the driver's side near zone, the passenger side near zone, the trunk area, or the far zone outside the vehicle), and selects the corresponding vehicle control strategy based on this position information. For example, when it is determined that the terminal source device is located in the driver's side near zone, the door unlocking and welcoming operations are performed; when it is determined that the terminal source device is located in the trunk area, the trunk opening operation is performed.

[0156] During the aforementioned adaptive adjustment and display resource allocation process, the display layout may undergo significant changes. To avoid visual disturbance to the driver caused by abrupt layout changes, a smooth transition control is implemented for the display layout changes during the adaptive adjustment and display resource allocation process.

[0157] Specifically, the layout parameter vector Lt at the current time t and the layout parameter vector Lt-1 at the previous time t-1 are obtained. The layout parameter vector encodes the display attributes of each information element on the HUD display plane, such as screen coordinates, scaling ratio, transparency, and motion speed.

[0158] Calculate the change in the layout parameter vector at the current time relative to the layout parameter vector at the previous time: ||Lt-Lt-1|| 2 This change reflects the degree of abrupt change in the layout at adjacent time points.

[0159] The spatial position of each information element under the layout parameter vector at the current moment is obtained, and the distance di(Lt) between the spatial position of each information element and the optimal attention focus position of the corresponding information element is calculated. The optimal attention focus position is determined based on the driver's attention spatial distribution heatmap, specifically the coordinate position corresponding to the area with the highest attention distribution density. This distance reflects whether the information element is located in the area of ​​greatest driver interest under the current layout.

[0160] Construct a first cost term λ1·||Lt-Lt-1|| based on the change. 2 The first cost term is used to suppress abrupt changes in layout parameters and ensure the continuity of layout changes. A second cost term, λ2·Σwi(t)·di(Lt), is constructed based on a weighted sum of the distance and the presentation priority weights. This second cost term is used to guide high-priority information elements closer to the area where the driver's attention is focused.

[0161] The combination of the first cost term and the second cost term is taken as the objective function J: J=Σ[λ1·||Lt -Lt-1||² +λ2·Σwi(t)·di(Lt)]

[0162] Here, λ1 and λ2 are weighting coefficients used to balance layout stability and attention matching. In a preferred embodiment, λ1=0.7 and λ2=0.3 to preferentially suppress layout abrupt changes.

[0163] A predictive control method (e.g., Model Predictive Control, MPC) is employed to minimize the objective function within the prediction time domain Np, outputting a sequence of layout parameters within the control time domain Nc, generating a continuous and smooth layout transition trajectory. The solution algorithm can employ the interior-point method, with a single-step solution time of less than 5ms, meeting the real-time requirements of the in-vehicle system. The HUD's display layout is then controlled to transition frame-by-frame according to the generated layout transition trajectory, achieving a smooth, seamless display switching effect.

[0164] Example 2: Based on the same technical concept, Example 2 of the present invention also provides a data processing system based on disassembly spectra and dose control, such as... Figure 2 As shown, it includes: a basic subject generation module 210, a threat subject generation module 220, a map construction module 230, a counterfactual attribution module 240, and a path recommendation module 250, wherein:

[0165] The first acquisition module 210 is used to acquire multi-source perception data and predict the evolution sequence of driving scenarios within a future time window based on the multi-source perception data.

[0166] The second acquisition module 220 is used to acquire a heat map of the spatial distribution of driver attention;

[0167] The determination module 230 is used to determine the presentation priority weight of each information element based on the scene prediction confidence of the driving scene evolution sequence and the attention distribution density of the driver attention spatial distribution heatmap in the display area of ​​each information element.

[0168] The adjustment module 240 is used to adaptively adjust the display content and display position of the HUD according to the presentation priority weight and the driving scenario evolution sequence;

[0169] The priority allocation module 250 is used to allocate display resources for each information element according to the presentation priority weight of each information element when there are multiple overlapping scenes in the driving scene evolution sequence.

[0170] The control module 260 is used to perform smooth transition control on the display layout changes during the adaptive adjustment and the allocation of display resources.

[0171] The technical effects of the vehicle-mounted scene-based adaptive HUD display method of the present invention will be further explained below with reference to two embodiments applied in real-world scenarios.

[0172] Example 3: Taking the application of this method to a passenger vehicle as an example, the specific implementation process is as follows:

[0173] like Figure 3 As shown, after the vehicle is powered on, the HUD initialization is completed, and the system obtains gear position information via the CAN bus. When the gear is P (Parking), the HUD displays the parking gear information.

[0174] The driver presses the brake pedal and shifts the gear to D (drive), the vehicle starts, and the HUD switches to display vehicle speed information.

[0175] The driver activates the intelligent driving function via the buttons on the steering wheel, and the intelligent driving logo is displayed below the vehicle speed information display area on the HUD.

[0176] When the driver sets a navigation destination and activates navigation through the vehicle's infotainment system, the system triggers adaptive adjustments. Vehicle speed information and the intelligent driving icon move to the left, while navigation-related information is displayed on the right side of the HUD, with the overall display area centered.

[0177] During driving, the system continuously performs driving scenario prediction and attention heatmap generation. When a curve appears ahead of the navigation path, the system begins to smoothly shift the navigation steering arrow towards the center of the field of vision 150m from the curve entrance, ensuring that the driver receives steering prompts before entering the curve.

[0178] If, during the approach to a curve, the intelligent driving system simultaneously detects a blind spot collision risk—that is, the "navigation scenario" and the "blind spot warning scenario" overlap—the system calculates a low compatibility index, triggering conflict resolution. The blind spot warning information, due to its higher priority, occupies the main central field of vision as a bright red flashing light; the navigation arrow information is downgraded to a semi-transparent static display and shifted to the edge area. After 3 seconds, the blind spot warning is deactivated, and the navigation arrow automatically returns to its bright state and returns to the central field of vision, completing the orderly delivery of information.

[0179] All layout parameter changes during the above adjustment process are handled by a smooth transition controller, and the movement trajectory of each information element is continuous and smooth, so as not to cause sudden visual shock to the driver.

[0180] Example 4: In one embodiment, embodiment 4 of the present invention also provides an electronic device, which may be a terminal, and its internal structure diagram may be as follows. Figure 4As shown. The electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the vehicle-mounted scene-based HUD adaptive display method described in any one of steps S101 to S106. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0181] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0182] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0184] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0185] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that, when performed on the computer or other programmable apparatus, provide for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0186] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A vehicle-mounted scene-based adaptive HUD display method, characterized in that, include: Acquire multi-source perception data and predict the evolution sequence of driving scenarios within a future time window based on the multi-source perception data; Obtain a heatmap of the spatial distribution of driver attention; Based on the scenario prediction confidence of the driving scenario evolution sequence and the attention distribution density of the driver attention spatial distribution heatmap in the display area of ​​each information element, the presentation priority weight of each information element is determined. Based on the presentation priority weight and the driving scenario evolution sequence, the display content and display position of the HUD are adaptively adjusted. When multiple scenarios overlap in the driving scenario evolution sequence, the display resources of each information element are allocated according to the presentation priority weight of each information element. Smooth transition control is implemented for the changes in display layout during the adaptive adjustment and the allocation of display resources.

2. The method according to claim 1, characterized in that, The step of predicting the evolution sequence of driving scenarios within a future time window based on the multi-source perception data includes: Acquire the multi-source perception data; the multi-source perception data includes vehicle dynamics parameters, navigation path planning data, and high-precision map data; The multi-source perception data is input into the driving scene prediction network, and the driving scene prediction network outputs a sequence of scene type probability distributions within the future time window with a preset time interval as the step size. The scenario types include driving scenarios, intelligent driving scenarios, navigation scenarios, congestion scenarios, ramp scenarios, and lane changing scenarios.

3. The method according to claim 1, characterized in that, The acquisition of the driver attention spatial distribution heatmap includes: Acquire a driver's facial image, and extract key eye information from the facial image. The key eye information includes at least the pupil center coordinates, the gaze direction vector, and the eyelid opening degree. The driver's head posture is estimated to obtain head posture parameters, which include at least the head yaw angle. The position of the gaze point is calculated based on the pupil center coordinates and the gaze direction vector. The position of the gaze point is then processed by Gaussian kernel diffusion to generate a heat map of the spatial distribution of the driver's attention. The kernel bandwidth parameter of the Gaussian kernel is adaptively adjusted based on the eyelid opening and closing degree to reflect changes in attention concentration.

4. The method according to claim 1, characterized in that, The determination of the presentation priority weights for each information element includes: Obtain the prediction confidence level corresponding to each scenario type in the driving scenario evolution sequence; Obtain the attention distribution density of the driver's attention spatial distribution heatmap in each information element display area, and calculate the normalized ratio of the attention distribution density to the total attention distribution density in the HUD display area; Obtain the current timeliness and urgency of each information element; The presentation priority weight of each information element is obtained by weighting and summing the scenario prediction confidence, the normalized ratio, and the information timeliness urgency.

5. The method according to claim 1, characterized in that, The adaptive adjustment of the HUD's display content and position includes: Analyze the geometric information of the navigation path and calculate the rate of curvature change of the navigation path within a preset distance range ahead; When the rate of change of curvature is detected to be greater than the preset curvature threshold, it is determined that there is a curve ahead, triggering a predictive layout adjustment; The advance time is calculated based on the vehicle speed, the rate of curvature change, and the preset safe viewing angle threshold. According to the aforementioned advance time, the navigation steering prompt information is gradually migrated towards the driver's central field of vision at a preset distance from the curve entrance, and the migration trajectory is smoothly controlled using a cubic Bézier curve.

6. The method according to claim 1, characterized in that, The adaptive adjustment of the HUD's display content and position also includes: In response to the navigation function activation signal, a first layout adjustment command is generated. The first layout adjustment command is used to control the vehicle speed information and the intelligent driving logo to move horizontally to the left by a first preset offset on the HUD display plane. A second layout adjustment command is generated, which is used to control the display of navigation-related information on the right side of the HUD display plane; A third layout adjustment instruction is generated, which is used to control the overall display area of ​​the vehicle speed information, the intelligent driving logo and the navigation related information to be centered on the HUD display plane.

7. The method according to claim 1, characterized in that, When multiple scenarios overlap in the driving scenario evolution sequence, the display resources of each information element are allocated according to its presentation priority weight, including: Obtain all scenario types existing at the current moment in the driving scenario evolution sequence, and construct scenario combinations; Calculate the scene compatibility index between any two scene types in the scene combination; When the scenario compatibility index is less than the preset compatibility threshold, it is determined that there is a conflict between multiple scenarios, and the information time-sharing reuse strategy is triggered. Based on the presentation priority weight of each information element, the display duration and space occupation ratio of each information element are dynamically adjusted. Secondary information is mapped to the color channel for display, while key information is mapped to the luminance and motion channels for display.

8. The method according to claim 1, characterized in that, The smooth transition control of display layout changes during the adaptive adjustment and display resource allocation process includes: Obtain the layout parameter vector at the current moment and the layout parameter vector at the previous moment, and calculate the change of the layout parameter vector at the current moment relative to the layout parameter vector at the previous moment. Obtain the spatial position of each information element under the layout parameter vector at the current moment, and calculate the distance between the spatial position of each information element and the optimal attention landing point position of the corresponding information element; A first cost term is constructed based on the change amount, and a second cost term is constructed based on the weighted sum of the distance and the presentation priority weight. The combination of the first cost term and the second cost term is used as the objective function. A predictive control method is used to solve the minimization problem of the objective function in the prediction time domain, outputting the layout parameter sequence in the control time domain, and generating a continuous and smooth layout migration trajectory.

9. The method according to claim 8, characterized in that, The step of obtaining the spatial position of each information element under the layout parameter vector at the current moment and calculating the distance between the spatial position of each information element and the optimal attention landing point position of the corresponding information element includes: The optimal attention landing point position corresponding to each information element is determined based on the heat map of the driver's attention spatial distribution. Based on the layout parameter vector at the current moment, the spatial coordinates of each information element on the HUD display plane are analyzed; Calculate the Euclidean distance between the spatial coordinates of each information element and the optimal attention point location.

10. A vehicle-mounted scene-based adaptive HUD display system, characterized in that, include: The first acquisition module is used to acquire multi-source perception data and predict the evolution sequence of driving scenarios within a future time window based on the multi-source perception data. The second acquisition module is used to acquire a heat map of the spatial distribution of driver attention; The determination module is used to determine the presentation priority weight of each information element based on the scenario prediction confidence of the driving scenario evolution sequence and the attention distribution density of the driver attention spatial distribution heatmap in the display area of ​​each information element. The adjustment module is used to adaptively adjust the display content and display position of the HUD according to the presentation priority weight and the driving scenario evolution sequence; The priority allocation module is used to allocate display resources for each information element according to the presentation priority weight of each information element when there are multiple overlapping scenarios in the driving scenario evolution sequence. The control module is used to smoothly control the changes in display layout during the adaptive adjustment and the allocation of display resources.

Images