Vehicle-mounted augmented reality head-up display control system and method

Abstract

The invention provides a vehicle-mounted augmented reality head-up display control system and method. The method comprises the following steps: determining a reference weight of each piece of augmented reality information in a vehicle-mounted display according to a driving safety level and a scene emergency index in vehicle driving; arbitration threshold values of various types of augmented reality information in the information priority rule set are dynamically adjusted based on the cognitive load level of the driving state parameters in the vehicle driving process, then the reference weight value of each piece of augmented reality information is compared with the arbitration threshold value corresponding to the category of the augmented reality information, and a to-be-displayed information set of the vehicle-mounted displayer is screened out; and performing spatial layout optimization on the display position of each piece of to-be-displayed information in the to-be-displayed information set through the sight focus position of the driving state parameter in vehicle driving to obtain the optical display attribute of the vehicle-mounted display. According to the scheme, spatial layout optimization of each piece of augmented reality information in the vehicle-mounted display can be realized, so that the man-machine interaction safety of information display in the vehicle-mounted display can be improved.

Description

Technical Field

[0001] This application relates to the field of vehicle display technology, and more specifically, to a vehicle-mounted augmented reality head-up display control system and method. Background Technology

[0002] The in-vehicle display uses optical projection technology to project driving information onto the windshield in real time. Augmented reality technology is used to blend virtual information with the real road, displaying key data such as vehicle speed, navigation, and safety warnings. This display system effectively reduces the frequency of driver eye movement, improving driving safety and interaction efficiency, and serves as the core human-machine interface of the modern smart cockpit.

[0003] Traditional fixed-position display modes, due to their static display architecture, cannot dynamically adjust information priority based on real-time risks in dynamic driving environments. This can lead to non-critical information obscuring safety warnings; the fixed display area causes information overlay interference, forcing drivers to perform visual separation operations; and the lack of perception of the driving environment prevents critical safety information from triggering display reconstruction in emergency situations. All of these factors combined result in drivers needing to continuously expend additional cognitive resources to filter information, increasing cognitive load and potentially delaying critical safety decisions, thus posing a driving safety hazard. Therefore, optimizing the spatial layout of various augmented reality information items in in-vehicle displays to improve the human-machine interaction safety of information display in in-vehicle displays has become a challenge for the industry. Summary of the Invention

[0004] This application provides a vehicle-mounted augmented reality head-up display control system and method, which can optimize the spatial layout of various augmented reality information in the vehicle-mounted display, thereby improving the human-computer interaction safety of information display in the vehicle-mounted display.

[0005] Firstly, this application provides a method for controlling an in-vehicle augmented reality head-up display, including: The system monitors the driver's driving status parameters while the vehicle is in motion and obtains the information priority rule set of the in-vehicle display. The driving status parameters include cognitive load level and gaze focus position. Registration information for augmented reality display is determined based on vehicle status data and environmental perception data during vehicle operation. Multiple augmented reality information to be displayed in the vehicle display is generated through the registration information. Then, the baseline weight of each augmented reality information in the vehicle display is determined based on the driving safety level and emergency indicators of the scene during vehicle operation. Based on the cognitive load level in the driving state parameters, the arbitration threshold of various types of augmented reality information in the information priority rule set is dynamically adjusted, and then the baseline weight of each augmented reality information is compared with the arbitration threshold corresponding to its category to filter out the set of information to be displayed on the vehicle display. By optimizing the spatial layout of the display positions of each piece of information to be displayed in the set of information to be displayed based on the gaze focus position in the driving state parameters, the optical display attributes of the vehicle display are obtained, and then the optimized augmented reality information is projected onto the vehicle display according to the optical display attributes.

[0006] In some embodiments, determining the registration information for augmented reality display based on vehicle status data and environmental perception data during vehicle operation specifically includes: The vehicle speed, steering angle, acceleration and gear information are obtained through the vehicle bus to obtain vehicle status data during driving. By acquiring road geometry information, traffic participant information, and traffic sign information through environmental perception sensors, environmental perception data of vehicles in motion can be obtained. The vehicle status data and the environmental perception data are spatially registered and fused to obtain registration information for augmented reality display.

[0007] In some embodiments, generating multiple augmented reality information items to be displayed on the in-vehicle display using the registration information specifically includes: The registration information is categorized and processed to generate three types of augmented reality information: navigation guide lines, lane departure warning, and forward collision warning. Set information attributes for each type of augmented reality information, including information type identifier, spatial coordinate data, and display timeliness; By calling the display template of the vehicle display according to various information attributes, standardized augmented reality display elements are generated, resulting in multiple augmented reality information to be displayed on the vehicle display.

[0008] In some embodiments, determining the baseline weight of each piece of augmented reality information in the in-vehicle display based on the driving safety level and scene emergency indicators while the vehicle is in motion specifically includes: Determine the driving safety level of the vehicle while it is in motion, and calculate the emergency indicators for the driving scenario. Obtain the safety impact weight and scene impact weight of each augmented reality information in the vehicle display; The driving safety level and the emergency indicators of the scenario are integrated into a baseline weight for each piece of augmented reality information in the vehicle display by using various safety impact weights and scenario impact weights.

[0009] In some embodiments, dynamically adjusting the arbitration thresholds for various types of augmented reality information in the information priority rule set based on the cognitive load level in the driving state parameters specifically includes: For each type of augmented reality information in the information priority rule set, obtain the arbitration threshold for the augmented reality information; The cognitive load level in the driving state parameters is compared with the load safety threshold of augmented reality information to obtain the sensitivity coefficient and adjustment strategy of augmented reality information. The arbitration threshold is dynamically adjusted by the sensitivity coefficient and the adjustment strategy, thereby achieving dynamic adjustment of the arbitration threshold for various types of augmented reality information in the information priority rule set.

[0010] In some embodiments, comparing the baseline weight of each augmented reality information with the arbitration threshold corresponding to its category to filter out the set of information to be displayed on the in-vehicle display specifically includes: For each augmented reality information, obtain the arbitration threshold for the corresponding category of the augmented reality information; The baseline weight of augmented reality information is compared with the arbitration threshold to obtain the information weight of augmented reality information, and then the information weight of each augmented reality information is obtained. The set of information to be displayed on the vehicle display is selected by filtering out each information weight.

[0011] In some embodiments, the spatial layout optimization of the display positions of each piece of information to be displayed in the set of information to be displayed is performed based on the gaze focus position in the driving state parameters, resulting in the optical display attributes of the vehicle display specifically including: Obtain the gaze focus position from the driving status parameters; Using the weights of each piece of information in the set of information to be displayed as constraints, and minimizing the line-of-sight deflection distance as the objective function, a spatial layout model is established. The location of the focal point is used as the visual focal point parameter in the spatial layout model; The spatial layout model is used to optically optimize the display position of each piece of information to be displayed in the set of information to be displayed, so as to obtain the optical attribute value of each piece of information to be displayed. The optical display properties of the vehicle display are determined by all optical property values.

[0012] Secondly, this application provides an in-vehicle augmented reality head-up display control system, including a display unit, the display unit comprising: The acquisition module is used to monitor the driver's driving state parameters while the vehicle is in motion and acquire the information priority rule set of the in-vehicle display. The driving state parameters include cognitive load level and gaze focus position. The processing module is used to determine the registration information for augmented reality display based on the vehicle status data and environmental perception data during vehicle operation, generate multiple augmented reality information to be displayed in the vehicle display through the registration information, and then determine the benchmark weight of each augmented reality information in the vehicle display based on the driving safety level and scene emergency indicators during vehicle operation. The processing module is also used to dynamically adjust the arbitration threshold of various types of augmented reality information in the information priority rule set based on the cognitive load level in the driving state parameters, and then compare the benchmark weight of each augmented reality information with the arbitration threshold corresponding to its category to filter out the set of information to be displayed on the vehicle display. The execution module is used to optimize the spatial layout of the display positions of each piece of information to be displayed in the set of information to be displayed by using the line-of-sight focus position in the driving state parameters, to obtain the optical display attributes of the vehicle display, and then project the optimized augmented reality information into the vehicle display according to the optical display attributes.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described vehicle-mounted augmented reality head-up display control method.

[0014] Fourthly, this application provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the aforementioned vehicle-mounted augmented reality head-up display control method.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: This application provides a vehicle-mounted augmented reality head-up display control system and method, which monitors the driver's driving state parameters while the vehicle is in motion and obtains an information priority rule set for the vehicle-mounted display. The driving state parameters include cognitive load level and gaze focus position. Registration information for augmented reality display is determined based on vehicle state data and environmental perception data during vehicle operation. Multiple augmented reality information items to be displayed on the vehicle-mounted display are generated using the registration information. A baseline weight for each augmented reality information item on the vehicle-mounted display is determined based on the vehicle's driving safety level and scene urgency indicators. Based on the cognitive load level in the driving state parameters, the arbitration thresholds for various types of augmented reality information in the information priority rule set are dynamically adjusted. The baseline weight of each augmented reality information item is compared with the arbitration threshold corresponding to its category to filter out a set of information items to be displayed on the vehicle-mounted display. The display position of each information item in the set of information items to be displayed is spatially optimized using the gaze focus position in the driving state parameters to obtain the optical display attributes of the vehicle-mounted display. The optimized augmented reality information is then projected onto the vehicle-mounted display based on these optical display attributes.

[0016] Therefore, in this application, the display positions of each piece of information to be displayed in the set of information to be displayed are spatially optimized by using the focal position of the line of sight in the driving state parameters, thereby obtaining the optical display attributes of the vehicle display. Then, optimized augmented reality information is projected onto the vehicle display based on these optical display attributes. First, determining the baseline weights provides a quantitative basis for the display priority of each piece of augmented reality information. By integrating driving safety levels and scenario urgency indicators, multi-dimensional environmental parameters are transformed into a unified weight measurement standard, ensuring that highly critical information receives priority in layout. This guarantees the display priority of safety-related information from the data source, enabling the layout algorithm to distinguish between different levels of information such as safety warnings and navigation guidance. By establishing a mapping relationship between weights and display requirements, hazard warning information can be automatically allocated to the optimal field of vision area, fundamentally improving human-computer interaction safety from the perspective of display priority. The algorithm first determines the set of information to be displayed, then obtains the effective information cluster after arbitration screening, thereby optimizing the allocation efficiency of visual space resources. The arbitration threshold mechanism enables precise filtering of information display, effectively controlling the number of simultaneously displayed information and avoiding unnecessary information from occupying visual space. The information cluster after threshold arbitration has clear quantity boundaries and weight distribution characteristics, allowing the layout algorithm to perform in-depth optimization for a limited number of key information items. By establishing a matching relationship between the number of information items and the display area, it ensures that each displayed information item receives sufficient visual presentation space, reducing information overlap and visual interference. This significantly improves the efficiency of information acquisition and safety perception performance in driving scenarios from the perspective of information density control. In summary, based on the above scheme, the spatial layout optimization of various augmented reality information items in the vehicle display can be achieved, thereby improving the human-computer interaction safety of information display in the vehicle display. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is an exemplary flowchart of an in-vehicle augmented reality head-up display control method according to some embodiments of this application; Figure 2 This is a flowchart illustrating the process of determining the set of information to be displayed according to some embodiments of this application; Figure 3 This is a schematic diagram of the structure of a display unit according to some embodiments of this application; Figure 4This is a schematic diagram of the structure of a computer device for implementing an in-vehicle augmented reality head-up display control method according to some embodiments of this application. Detailed Implementation

[0019] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] refer to Figure 1 The figure is an exemplary flowchart of an in-vehicle augmented reality head-up display control method according to some embodiments of this application. The in-vehicle augmented reality head-up display control method mainly includes the following steps: In step 101, the driver's driving state parameters during vehicle operation are monitored, and the information priority rule set of the vehicle display is obtained. The driving state parameters include cognitive load level and line-of-sight focus position.

[0021] It should be noted that, in this application, driving state parameters are quantitative indicators used to characterize the driver's physiological and psychological state during vehicle operation. These driving state parameters include cognitive load level and visual focus position. Cognitive load level is the degree of information processing burden on the driver's brain during driving, and visual focus position is the specific location where the driver's gaze is focused during driving. The information priority rule set is a set of rules governing the priority order of various types of information displayed on the in-vehicle display. In specific implementation, firstly, monitoring the driver's driving state parameters while the vehicle is in motion can be achieved in the following way: using a capacitive pressure sensor array installed on the inner side of the steering wheel rim, continuously monitoring the grip force distribution data of the driver's hands at a sampling frequency of 100 Hz, and using the grip force variance value obtained after processing the voltage signal in the grip force distribution data output by the capacitive pressure sensor array through Kalman filtering as the cognitive load level during vehicle operation; simultaneously, using a near-infrared eye tracker located above the dashboard to track the driver's pupil movement trajectory at a frequency of 60 frames per second, and using the average value of three consecutive stable pupil center coordinates as the focal point position of the gaze; then, obtaining the information priority rule set of the in-vehicle display can be achieved in the following way: reading predefined display information management rules from the non-volatile memory of the in-vehicle infotainment system, which contains the display priority and adjustment threshold of various types of information, and using the predefined display information management rules from the non-volatile memory of the in-vehicle infotainment system as the information priority rule set of the in-vehicle display.

[0022] In step 102, registration information for augmented reality display is determined based on vehicle status data and environmental perception data during vehicle operation. Multiple augmented reality information to be displayed in the vehicle display is generated using the registration information. Then, the baseline weight of each augmented reality information in the vehicle display is determined based on the driving safety level and scene emergency indicators during vehicle operation.

[0023] In some embodiments, determining the registration information for augmented reality display based on vehicle status data and environmental perception data during vehicle operation can be achieved through the following steps: The vehicle speed, steering angle, acceleration and gear information are obtained through the vehicle bus to obtain vehicle status data during driving. By acquiring road geometry information, traffic participant information, and traffic sign information through environmental perception sensors, environmental perception data of vehicles in motion can be obtained. The vehicle status data and the environmental perception data are spatially registered and fused to obtain registration information for augmented reality display.

[0024] It should be noted that, in this application, the registration information is a set of augmented reality display elements with precise spatial location coordinates and semantic tags; the vehicle status data is a set of physical quantities reflecting the vehicle's own motion state; and the environmental perception data is a set of information reflecting the characteristics of the environment surrounding the vehicle.

[0025] In specific implementation, firstly, vehicle speed, steering angle, acceleration, and gear information are acquired via the vehicle bus. This vehicle status data can be obtained in the following ways: speed data calculated by the four-wheel speed sensors is obtained from the anti-lock braking system module; steering angle data measured by the steering wheel angle sensor is obtained from the electric power steering system module; longitudinal and lateral acceleration data measured by the three-axis inertial sensors are obtained from the electronic stability program system module; and gear information indicating the current gear ratio transition is obtained from the transmission control module. The verified and decoded speed, steering angle, acceleration, and gear status values ​​are then packaged into a single standardized dataset containing speed, steering angle, acceleration, and gear information to form the vehicle status data. Next, road geometry, traffic participant information, and traffic sign information are acquired via environmental perception sensors. This environmental perception data can be obtained in the following ways: solid-state LiDAR is used to scan the road point cloud data ahead to extract lane curvature and road boundary features; millimeter-wave radar is used to detect the distance, speed, and azimuth information of surrounding vehicles; and... The forward-facing camera uses a deep learning model to identify traffic sign content and traffic participant types. It then performs time-stamp alignment and coordinate unification on the road geometry parameters obtained from LiDAR processing, the traffic participant motion parameters tracked by millimeter-wave radar, and the traffic sign semantic information obtained from visual recognition. Finally, it uses an environmental description set containing road geometry information, traffic participant information, and traffic sign information as environmental perception data. Finally, it spatially registers and fuses the vehicle state data and the environmental perception data to obtain registration information for augmented reality display. This can be achieved by: establishing a local coordinate system with the vehicle's centroid as the origin; projecting and mapping the vehicle speed, steering angle, acceleration, and gear information from the vehicle state data with the road geometry information, traffic participant information, and traffic sign information from the environmental perception data using a coordinate transformation matrix; then spatiotemporally aligning the position information of the vehicle state data with the position information of the environmental perception data; and finally, matching the movement trajectory of traffic participants with the road geometry structure topologically. The resulting multi-source dataset, after coordinate unification, spatiotemporal alignment, and semantic association, is used as registration information for augmented reality display.

[0026] In some embodiments, generating multiple augmented reality information items to be displayed on the in-vehicle display using the registration information can be achieved through the following steps: The registration information is categorized and processed to generate three types of augmented reality information: navigation guide lines, lane departure warning, and forward collision warning. Set information attributes for each type of augmented reality information, including information type identifier, spatial coordinate data, and display timeliness; By calling the display template of the vehicle display according to various information attributes, standardized augmented reality display elements are generated, resulting in multiple augmented reality information to be displayed on the vehicle display.

[0027] It should be noted that, in this application, the augmented reality information to be displayed is a standardized set of graphic elements that meet the rendering requirements of in-vehicle displays; the information attributes are a set of parameters used to describe the display characteristics of augmented reality information; and the three types of augmented reality information are sets of information with clear functional categories, including navigation guide lines, lane departure warning, and forward collision warning.

[0028] In specific implementation, firstly, the registration information is classified and processed to generate three types of augmented reality information: navigation guidance lines, lane departure warnings, and forward collision warnings. This can be achieved in the following ways: Road geometry information is extracted from the registration information to generate path guidance lines according to preset mapping rules; the vehicle's pose is simulated using the vehicle's current speed, steering angle, acceleration, and gear information from the registration information, and the relative positional relationship between the simulated vehicle pose and the lane lines is used as the lane departure warning; the forward collision warning is generated using the vehicle's current speed, steering angle, acceleration, and gear information from the registration information combined with traffic participant information. This process integrates structured datasets of navigation guide lines, lane departure warnings, and forward collision warnings into a classification result for augmented reality information. Then, information attributes are set for each type of augmented reality information, including information type identifiers, spatial coordinate data, and display duration. This can be achieved as follows: For each type of augmented reality information, a unique information type identifier is assigned; the world coordinate system coordinates of the augmented reality information are obtained from the multi-source dataset of registered information and converted into display coordinate system coordinates; different display duration parameters are set according to the information category characteristics. This process integrates information type identifiers, spatial coordinate data, and display duration... The set of effectiveness attributes serves as the information attributes of this type of augmented reality information. The information attributes of various types of augmented reality information can be obtained through the above methods. For example: assigning a type identifier "NL001" to navigation guide lines, "LDW002" to lane departure warning, and "FCW003" to forward collision warning; acquiring road geometry data in the world coordinate system, and calculating the two-dimensional display coordinates within the windshield display area through perspective projection transformation; setting the display effectiveness of navigation guide lines to permanent display, lane departure warning to a 5-second timed display, and forward collision warning to a 3-second emergency display according to safety display specifications; finally, through... The process of generating standardized augmented reality display elements by calling the display template of the vehicle display through various information attributes and obtaining multiple augmented reality information to be displayed on the vehicle display can be achieved in the following way: select the corresponding display template according to the information type identifier in the augmented reality information attributes, input the spatial coordinate data into the selected display template for position mapping, set the display time period of the element according to the display timeliness parameter, instantiate the configured display template into specific graphics rendering instructions, and use the set of all graphics rendering instructions as the augmented reality information to be displayed, thus obtaining multiple augmented reality information to be displayed on the vehicle display.

[0029] In some embodiments, determining the baseline weight of each augmented reality information in the in-vehicle display based on the driving safety level and scene emergency indicators while the vehicle is in motion can be achieved through the following steps: Determine the driving safety level of the vehicle while it is in motion, and calculate the emergency indicators for the driving scenario. Obtain the safety impact weight and scene impact weight of each augmented reality information in the vehicle display; The driving safety level and the emergency indicators of the scenario are integrated into a baseline weight for each piece of augmented reality information in the vehicle display by using various safety impact weights and scenario impact weights.

[0030] It should be noted that in this application, the benchmark weight is a quantitative indicator used to characterize the priority of display of various augmented reality information in the vehicle display; the driving safety level is a comprehensive score characterizing the overall danger level of the current vehicle driving; and the scenario emergency index is a quantitative indicator reflecting the urgency of the current vehicle driving danger.

[0031] In specific implementation, firstly, determining the driving safety level of the vehicle and calculating the scene emergency index can be achieved as follows: based on the vehicle speed, acceleration, and relative distance to the vehicle in front from the vehicle status data, the result of a weighted calculation according to a preset safety level calculation rule table is used as the driving safety level of the vehicle; simultaneously, based on the reciprocal of the collision time and lane departure rate from the environmental perception data, combined with the road adhesion coefficient, the output of the emergency level calculation model is used as the scene emergency index of the vehicle; then, obtaining the safety impact weight and scene impact weight of each augmented reality information in the in-vehicle display can be achieved as follows: obtaining the information weight configuration table of each augmented reality information from the in-vehicle storage... The safety impact weight and scene impact weight of augmented reality information are determined. Finally, the driving safety level and the scene emergency index are integrated into a benchmark weight for each augmented reality information in the vehicle display by using the various safety impact weights and scene impact weights. This can be achieved in the following way: For each augmented reality information in the vehicle display, the driving safety level value is multiplied by the safety impact weight of the augmented reality information to obtain a safety score component, and the scene emergency index value is multiplied by the scene impact weight of the augmented reality information to obtain an emergency score component. The sum of the safety score component and the emergency score component is normalized and used as the benchmark weight of the augmented reality information. The benchmark weight for each augmented reality information in the vehicle display can be obtained in the above way.

[0032] In step 103, based on the cognitive load level in the driving state parameters, the arbitration threshold of various types of augmented reality information in the information priority rule set is dynamically adjusted, and then the baseline weight of each augmented reality information is compared with the arbitration threshold corresponding to its category to filter out the set of information to be displayed on the vehicle display.

[0033] In some embodiments, dynamically adjusting the arbitration thresholds for various types of augmented reality information in the information priority rule set based on the cognitive load level in the driving state parameters can be achieved through the following steps: For each type of augmented reality information in the information priority rule set, obtain the arbitration threshold for the augmented reality information; The cognitive load level in the driving state parameters is compared with the load safety threshold of augmented reality information to obtain the sensitivity coefficient and adjustment strategy of augmented reality information. The arbitration threshold is dynamically adjusted by the sensitivity coefficient and the adjustment strategy, thereby achieving dynamic adjustment of the arbitration threshold for various types of augmented reality information in the information priority rule set.

[0034] It should be noted that, in this application, the arbitration threshold is a threshold used to determine whether various types of augmented reality information are qualified for display; the sensitivity coefficient represents the parameter adjustment range of the arbitration threshold for various types of augmented reality information; and the adjustment strategy refers to the method of controlling the arbitration threshold for various types of augmented reality information.

[0035] In specific implementation, firstly, for various types of augmented reality information in the information priority rule set, the arbitration threshold for augmented reality information can be obtained in the following way: The initial arbitration threshold parameters corresponding to navigation, safety, and warning augmented reality information are read from the rule configuration table of the vehicle's memory. These threshold parameters are preset based on ergonomic experimental data under normal driving conditions. Then, the cognitive load level in the driving state parameters is compared with the load safety threshold of the augmented reality information to obtain the sensitivity coefficient and adjustment strategy of the augmented reality information. This can be achieved in the following way: The difference between the cognitive load level value and the preset load safety threshold of the augmented reality information is calculated. The corresponding adjustment range is selected according to the different intervals of the difference, and a corresponding threshold adjustment method is selected from the adjustment strategy library. This adjustment range value is used as the sensitivity coefficient of the augmented reality information. The selected threshold adjustment method is used as the adjustment strategy for augmented reality information. For example, when the range of the difference is within a safe range, the adjustment strategy is to multiply the arbitration threshold of the augmented reality information by using the sensitivity coefficient as a factor. When the range of the difference is within an unsafe range, the adjustment strategy is to exponentially adjust the arbitration threshold of the augmented reality information by using the sensitivity coefficient as the exponent base. The safe and unsafe ranges can be preset based on historical experience. Finally, the arbitration threshold is dynamically adjusted by the sensitivity coefficient and the adjustment strategy to achieve the dynamic adjustment of the arbitration threshold of various types of augmented reality information in the information priority rule set. This can be achieved by using the sensitivity coefficient as the adjustment parameter in the adjustment strategy to achieve the dynamic adjustment of the arbitration threshold of the augmented reality information. The dynamic adjustment of the arbitration threshold of various types of augmented reality information in the information priority rule set can be achieved through the above method.

[0036] In some embodiments, the baseline weight of each augmented reality information is compared with the arbitration threshold corresponding to its category to filter out the set of information to be displayed on the in-vehicle display, with reference to... Figure 2 The diagram is a flowchart illustrating the process of determining the set of information to be displayed in some embodiments of this application. In this embodiment, determining the set of information to be displayed can be achieved using the following steps: In step 1031, for each augmented reality information, the arbitration threshold for the corresponding category of the augmented reality information is obtained; In step 1032, the baseline weight of the augmented reality information is compared with the arbitration threshold to obtain the information weight of the augmented reality information, and then the information weight of each augmented reality information is obtained. In step 1033, the set of information to be displayed on the vehicle display is selected by filtering out each information weight.

[0037] It should be noted that, in this application, the set of information to be displayed is the set of all augmented reality information that meets the display conditions; the information weight is a quantitative indicator describing the degree to which each augmented reality information needs to be displayed in the vehicle display.

[0038] In specific implementation, firstly, for each augmented reality (AR) information, obtaining the arbitration threshold for the corresponding category can be achieved as follows: For each AR information, the corresponding category number is searched in the dynamically updated information priority rule set using the AR information's information type identifier, and the currently effective threshold parameter for that category number is read from the arbitration threshold storage area; then, the baseline weight of the AR information is compared with the arbitration threshold to obtain the AR information's information weight. Furthermore, obtaining the information weight of each AR information can be achieved as follows: The difference between the baseline weight of the AR information and the arbitration threshold is calculated. When the baseline weight is greater than or equal to the arbitration threshold, the information weight is equal to the ratio of the baseline weight to the arbitration threshold; when the baseline weight is less than the arbitration threshold, the information weight is set to zero. The information weight of each AR information can be obtained through this method. Finally, the set of information to be displayed on the vehicle display can be filtered out using each information weight as follows: AR information with an information weight greater than 0 is selected as the information to be displayed, thus making the set of all information to be displayed the set of information to be displayed on the vehicle display.

[0039] In step 104, the display positions of each piece of information to be displayed in the set of information to be displayed are optimized by using the gaze focus position in the driving state parameters, so as to obtain the optical display attributes of the vehicle display, and then the optimized augmented reality information is projected into the vehicle display according to the optical display attributes.

[0040] In some embodiments, optimizing the spatial layout of the display positions of each piece of information to be displayed in the set of information to be displayed by using the gaze focus position in the driving state parameters to obtain the optical display properties of the vehicle display can be achieved by the following steps: Obtain the gaze focus position from the driving status parameters; Using the weights of each piece of information in the set of information to be displayed as constraints, and minimizing the line-of-sight deflection distance as the objective function, a spatial layout model is established. The location of the focal point is used as the visual focal point parameter in the spatial layout model; The spatial layout model is used to optically optimize the display position of each piece of information to be displayed in the set of information to be displayed, so as to obtain the optical attribute value of each piece of information to be displayed. The optical display properties of the vehicle display are determined by all optical property values.

[0041] It should be noted that, in this application, optical display attributes are a set of all parameters that control the visual presentation effect of augmented reality information, including display position coordinates, display size, transparency, and color parameters; the spatial layout model is a geometric arrangement algorithm model based on multi-objective optimization. This spatial layout model establishes a polar coordinate system with the focal point of the eye as the reference, transforms the weights of each piece of information into layout priority constraints, and takes minimizing the sum of the Euclidean distances between the human eye's gaze point and the display positions of each piece of information as the core optimization objective. This spatial layout model solves the problem of position allocation on the windshield projection surface. Under the premise of ensuring that high-weight information is in the optimal field of view, it comprehensively considers factors such as the length of the human eye's scanning path, the occlusion relationship between information, and the continuity of visual flow. It uses a gradient descent algorithm for iterative calculation and finally generates a three-dimensional spatial layout scheme that meets the requirements of human factors engineering. The three-dimensional spatial layout scheme output by this spatial layout model includes the display position coordinates, display size, transparency, and color parameters of each information element, providing the optimal spatial arrangement solution for augmented reality display. Thus, the set of display position coordinates, display size, transparency, and color parameters of each information element is used as the optical attribute value of the corresponding information to be displayed, and the set of all optical attribute values ​​is used as the optical display attribute of the vehicle display.

[0042] In some embodiments, the projection of optimized augmented reality information onto an in-vehicle display based on the optical display attributes can be achieved in the following manner: the display position coordinates in the optical attribute values ​​of each piece of information to be displayed are used as the precise position of the corresponding information element on the windshield projection surface; the visual size of the information element is adjusted according to the display size in the optical attribute values ​​of each piece of information to be displayed; the degree of integration between the information element and the real-world background is adjusted according to the transparency in the optical attribute values ​​of each piece of information to be displayed; the color representation of the information element is set in combination with the color parameters in the optical attribute values ​​of each piece of information to be displayed; all configured information elements are used to generate a corresponding image frame sequence through a graphics rendering engine; the image frame sequence is precisely projected onto a designated area of ​​the windshield via an optical projection system; and the set of visual information presented on the windshield after spatial layout optimization and optical attribute configuration is used as the optimized augmented reality information.

[0043] Furthermore, in another aspect of this application, in some embodiments, this application provides an in-vehicle augmented reality head-up display control system, which includes a display unit, with reference to... Figure 3 The figure is a schematic diagram of the structure of a display unit according to some embodiments of this application. The display unit includes: an acquisition module 201, a processing module 202, and an execution module 203, which are described below: The acquisition module 201 in this application is mainly used to monitor the driver's driving state parameters while the vehicle is in motion, and to acquire the information priority rule set of the vehicle display. The driving state parameters include cognitive load level and line of sight focus position. Processing module 202 in this application is used to determine registration information for augmented reality display based on vehicle status data and environmental perception data during vehicle operation, generate multiple augmented reality information to be displayed in the vehicle display through the registration information, and then determine the benchmark weight of each augmented reality information in the vehicle display based on the driving safety level and scene emergency index during vehicle operation. It should be noted that the processing module 202 is also used to dynamically adjust the arbitration threshold of various types of augmented reality information in the information priority rule set based on the cognitive load level in the driving state parameters, and then compare the benchmark weight of each augmented reality information with the arbitration threshold corresponding to its category to filter out the set of information to be displayed on the vehicle display. The execution module 203 in this application is mainly used to optimize the spatial layout of the display positions of each piece of information to be displayed in the set of information to be displayed by using the focus position of the line of sight in the driving state parameters, to obtain the optical display attributes of the vehicle display, and then to project the optimized augmented reality information into the vehicle display according to the optical display attributes.

[0044] The foregoing has detailed examples of the in-vehicle augmented reality head-up display control system and method provided in the embodiments of this application. It is understood that, in order to achieve the above functions, the corresponding device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0045] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described in-vehicle augmented reality head-up display control method.

[0046] In some embodiments, reference Figure 4 The dashed lines in the figure indicate that the unit or module is optional. This figure is a structural schematic diagram of a computer device for implementing an in-vehicle augmented reality head-up display control method according to an embodiment of this application. The in-vehicle augmented reality head-up display control method described in the above embodiments can... Figure 4 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a memory 302 and at least one communication unit 305. The computer device may be a terminal device, a server or a chip.

[0047] Processor 301 can be a general-purpose processor or a special-purpose processor. For example, processor 301 can be a central processing unit (CPU), which can be used to control computer devices, execute software programs, and process data from software programs. The computer device may also include a communication unit 305 for inputting (receiving) and outputting (transmitting) signals.

[0048] For example, the computer device may be a chip, and the communication unit 305 may be the input and / or output circuit of the chip, or the communication unit 305 may be the communication interface of the chip, which may be a component of a terminal device, network device or other device.

[0049] For example, the computer device may be a terminal device or a server, and the communication unit 305 may be a transceiver of the terminal device or the server, or the communication unit 305 may be a transceiver circuit of the terminal device or the server.

[0050] The computer device may include one or more memories 302 storing a program 304. The program 304 can be executed by a processor 301 to generate instructions 303, causing the processor 301 to execute the method described in the above method embodiments according to the instructions 303. Optionally, the memory 302 may also store data (such as a target audit model). Optionally, the processor 301 may also read data stored in the memory 302, which may be stored at the same storage address as the program 304, or it may be stored at a different storage address than the program 304.

[0051] The processor 301 and memory 302 can be configured separately or integrated together, for example, integrated on the system on chip (SOC) of the terminal device.

[0052] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 301. The processor 301 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gate, transistor logic devices, or discrete hardware components.

[0053] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0054] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the above-described vehicle-mounted augmented reality head-up display control method.

[0055] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0056] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for controlling an in-vehicle augmented reality head-up display, characterized in that, Includes the following steps: The system monitors the driver's driving status parameters while the vehicle is in motion and obtains the information priority rule set of the in-vehicle display. The driving status parameters include cognitive load level and gaze focus position. Registration information for augmented reality display is determined based on vehicle status data and environmental perception data during vehicle operation. Multiple augmented reality information to be displayed in the vehicle display is generated through the registration information. Then, the baseline weight of each augmented reality information in the vehicle display is determined based on the driving safety level and emergency indicators of the scene during vehicle operation. Based on the cognitive load level in the driving state parameters, the arbitration threshold of various types of augmented reality information in the information priority rule set is dynamically adjusted, and then the baseline weight of each augmented reality information is compared with the arbitration threshold corresponding to its category to filter out the set of information to be displayed on the vehicle display. By optimizing the spatial layout of the display positions of each piece of information to be displayed in the set of information to be displayed based on the gaze focus position in the driving state parameters, the optical display attributes of the vehicle display are obtained, and then the optimized augmented reality information is projected onto the vehicle display according to the optical display attributes.

2. The method as described in claim 1, characterized in that, The registration information used for augmented reality displays, determined based on vehicle status data and environmental perception data during vehicle operation, specifically includes: The vehicle speed, steering angle, acceleration and gear information are obtained through the vehicle bus to obtain vehicle status data during driving. By acquiring road geometry information, traffic participant information, and traffic sign information through environmental perception sensors, environmental perception data of vehicles in motion can be obtained. The vehicle status data and the environmental perception data are spatially registered and fused to obtain registration information for augmented reality display.

3. The method as described in claim 1, characterized in that, Generating multiple augmented reality information items to be displayed on the in-vehicle display using the registration information specifically includes: The registration information is categorized and processed to generate three types of augmented reality information: navigation guide lines, lane departure warning, and forward collision warning. Set information attributes for each type of augmented reality information, including information type identifier, spatial coordinate data, and display timeliness; By calling the display template of the vehicle display according to various information attributes, standardized augmented reality display elements are generated, resulting in multiple augmented reality information to be displayed on the vehicle display.

4. The method as described in claim 1, characterized in that, The baseline weights for each piece of augmented reality information in the in-vehicle display are determined based on the vehicle's driving safety level and the emergency indicators of the scenario, specifically including: Determine the driving safety level of the vehicle while it is in motion, and calculate the emergency indicators for the driving scenario. Obtain the safety impact weight and scene impact weight of each augmented reality information in the vehicle display; The driving safety level and the emergency indicators of the scenario are integrated into a baseline weight for each piece of augmented reality information in the vehicle display by using various safety impact weights and scenario impact weights.

5. The method as described in claim 1, characterized in that, Based on the cognitive load level in the driving state parameters, dynamically adjusting the arbitration thresholds for various types of augmented reality information in the information priority rule set specifically includes: For each type of augmented reality information in the information priority rule set, obtain the arbitration threshold for the augmented reality information; The cognitive load level in the driving state parameters is compared with the load safety threshold of augmented reality information to obtain the sensitivity coefficient and adjustment strategy of augmented reality information. The arbitration threshold is dynamically adjusted by the sensitivity coefficient and the adjustment strategy, thereby achieving dynamic adjustment of the arbitration threshold for various types of augmented reality information in the information priority rule set.

6. The method as described in claim 1, characterized in that, The baseline weight of each augmented reality information is compared with the arbitration threshold corresponding to its category to filter out the set of information to be displayed on the in-vehicle display. Specifically, this includes: For each augmented reality information, obtain the arbitration threshold for the corresponding category of the augmented reality information; The baseline weight of augmented reality information is compared with the arbitration threshold to obtain the information weight of augmented reality information, and then the information weight of each augmented reality information is obtained. The set of information to be displayed on the vehicle display is selected by filtering out each information weight.

7. The method as described in claim 1, characterized in that, By optimizing the spatial layout of the display positions of each piece of information to be displayed in the set of information to be displayed based on the gaze focus position in the driving state parameters, the optical display attributes of the vehicle display are obtained, specifically including: Obtain the gaze focus position from the driving status parameters; Using the weights of each piece of information in the set of information to be displayed as constraints, and minimizing the line-of-sight deflection distance as the objective function, a spatial layout model is established. The location of the focal point is used as the visual focal point parameter in the spatial layout model; The spatial layout model is used to optically optimize the display position of each piece of information to be displayed in the set of information to be displayed, so as to obtain the optical attribute value of each piece of information to be displayed. The optical display properties of the vehicle display are determined by all optical property values.

8. A vehicle-mounted augmented reality head-up display control system, the vehicle-mounted augmented reality head-up display control system including a display unit, characterized in that, The display unit includes: The acquisition module is used to monitor the driver's driving state parameters while the vehicle is in motion and acquire the information priority rule set of the in-vehicle display. The driving state parameters include cognitive load level and gaze focus position. The processing module is used to determine the registration information for augmented reality display based on the vehicle status data and environmental perception data during vehicle operation, generate multiple augmented reality information to be displayed in the vehicle display through the registration information, and then determine the benchmark weight of each augmented reality information in the vehicle display based on the driving safety level and scene emergency indicators during vehicle operation. The processing module is also used to dynamically adjust the arbitration threshold of various types of augmented reality information in the information priority rule set based on the cognitive load level in the driving state parameters, and then compare the benchmark weight of each augmented reality information with the arbitration threshold corresponding to its category to filter out the set of information to be displayed on the vehicle display. The execution module is used to optimize the spatial layout of the display positions of each piece of information to be displayed in the set of information to be displayed by using the line-of-sight focus position in the driving state parameters, to obtain the optical display attributes of the vehicle display, and then project the optimized augmented reality information into the vehicle display according to the optical display attributes.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory being used to store a computer program, and the processor being used to call and run the computer program from the memory, causing the computer device to perform the vehicle-mounted augmented reality head-up display control method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions or code that, when executed on a computer, cause the computer to implement the vehicle-mounted augmented reality head-up display control method as described in any one of claims 1 to 7.

Images