Head-up display method, device and equipment

Abstract

The embodiment of the invention provides a head-up display method, device and equipment. The method comprises the following steps: acquiring driving state information, navigation information and current environment information of a vehicle; determining a target element and behavior intention information of the target element based on the current environment information, the navigation information and the driving state information; determining a target icon and a target visual attribute of the target element based on the target element, the behavior intention information of the target element and a preset symbol library; the symbol library comprises a corresponding relation between elements and icons and a corresponding relation between behavior intentions and visual attributes; and sending the target icon and the target visual attribute to a head-up display device so as to project the target icon and the target visual attribute to a preset area for display. The method is used for achieving the effect of improving the driving safety.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a head-up display method, apparatus, and device. Background Technology

[0002] Head-up display (HUD) technology projects key information such as vehicle status, navigation guidance, and driver assistance warnings onto the driver's field of vision, achieving a head-up presentation of driving information. This reduces the safety hazards of driver eye shift, shortens the time of eye deviation, and eliminates the need for drivers to switch between real-world road conditions and screen information, significantly improving the convenience and safety of driving operations.

[0003] In related technologies, vehicle-mounted sensors are used to reconstruct the three-dimensional point cloud data of the vehicle's surrounding environment into a refined three-dimensional digital model. The digital model is then spatially aligned with the real scene, ultimately rendering a virtual overlay image that matches the size and position of real objects.

[0004] However, in the above methods, high-fidelity rendering will present a large number of redundant details in a limited display area, resulting in a large cognitive load for the driver, making it difficult for the driver to quickly identify the core risk sources, prolonging the decision-making reaction time, and affecting driving safety. Summary of the Invention

[0005] The head-up display method, apparatus, and device provided in this application are used to improve driving safety.

[0006] In a first aspect, embodiments of this application provide a head-up display method, including:

[0007] Obtain vehicle driving status information, navigation information, and current environmental information;

[0008] Based on the current environment information, the navigation information, and the driving status information, the target element and the target element's behavioral intent information are determined;

[0009] Based on the target element, the behavioral intent information of the target element, and a preset symbol library, the target icon and target visual attributes of the target element are determined; the symbol library includes the correspondence between elements and icons, and the correspondence between behavioral intent and visual attributes.

[0010] The target icon and target visual attributes are sent to the head-up display device for projection onto a preset area for display.

[0011] In one possible implementation, determining the target element and the target element's behavioral intent information based on the current environment information, the navigation information, and the driving status information includes:

[0012] The current environmental information is detected and processed to determine the initial element and its state information;

[0013] Based on the state information of the initial element and the driving state information, the initial element is filtered to obtain the target element;

[0014] Based on the state information of the target element and the driving state information, the behavioral intention information of the target element is determined.

[0015] In one possible implementation, the symbol library further includes element mapping conditions; the step of filtering the initial elements based on the initial element's state information and the driving state information to obtain the target element includes:

[0016] If the state information of the initial element and the driving state information satisfy the mapping conditions of the initial element in the symbol library, then the initial element is determined to be the target element.

[0017] In one possible implementation, determining the behavioral intent information of the target element based on the state information of the target element and the driving state information includes:

[0018] Based on a preset trajectory prediction algorithm, the behavioral intent information of the target element is determined according to the state information and driving state information of the target element.

[0019] In one possible implementation, determining the target icon and target visual attributes of the target element based on the target element, the behavioral intent information of the target element, and a preset symbol library includes:

[0020] Based on the target element and the preset symbol library, determine the target icon corresponding to the target element;

[0021] Based on the behavioral intent information of the target element, the target visual attributes of the target element are determined.

[0022] In one possible implementation, the target element has state information, and the method further includes:

[0023] Based on a preset detection model, the risk level of the target element is determined according to the state information and driving state information of the target element;

[0024] The symbol library also includes the correspondence between risk levels and visual attributes; determining the target visual attributes of the target element based on the behavioral intent information of the target element includes:

[0025] Based on the risk level of the target element, determine the basic visual attributes of the target element;

[0026] The basic visual attributes are adjusted based on the behavioral intent information of the target element to obtain the target visual attributes of the target element.

[0027] In one possible implementation, the method further includes:

[0028] Based on the current environmental information and the driving status information, the projection mode is determined;

[0029] Sending the target icon and target visual attributes to a head-up display device for projection onto a preset area includes:

[0030] According to the projection mode, the target icon and target visual attributes are projected to obtain the information to be projected;

[0031] The information to be projected is sent to the head-up display device for projection onto a preset area for display.

[0032] Secondly, embodiments of this application provide a head-up display device, comprising:

[0033] The acquisition module is used to acquire vehicle driving status information, navigation information, and current environmental information;

[0034] The first determining module is used to determine the target element and the target element's behavioral intent information based on the current environment information, the navigation information, and the driving status information;

[0035] The second determining module is used to determine the target icon and target visual attributes of the target element based on the target element, the behavioral intent information of the target element, and a preset symbol library; the symbol library includes the correspondence between elements and icons, and the correspondence between behavioral intent and visual attributes.

[0036] The sending module is used to send the target icon and target visual attributes to the head-up display device for projection onto a preset area for display.

[0037] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0038] The memory stores computer-executed instructions;

[0039] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0040] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0041] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0042] The head-up display method, apparatus, and device provided in this application acquire vehicle driving status information, navigation information, and current environmental information to determine target elements and behavioral intent information strongly related to driving decisions. Then, based on a preset symbol library, they match the target elements with corresponding target icons and visual attributes, and finally send the data to the head-up display device for projection display. This approach achieves several advantages: firstly, it replaces redundant 3D model rendering data with lightweight symbolic information, reducing computational load and hardware costs; secondly, it conveys the behavioral intent of target elements through intuitive icons and visual attributes, allowing drivers to quickly capture key driving information without analyzing complex images, reducing cognitive load, improving the timeliness and accuracy of driving decisions, and enhancing driving safety; thirdly, the target visual attributes intuitively reflect the decision intent of the vehicle's intelligent driving system, allowing drivers to clearly see the source of the system's decisions, eliminating confusion or distrust caused by unexplained braking or deceleration, effectively eliminating information gaps in human-computer interaction, increasing driver trust and acceptance of the intelligent driving system, reducing operational conflicts caused by unclear decision-making basis, and improving driving safety and comfort. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0044] Figure 1 Flowchart of the header display method provided in this application Figure 1 ;

[0045] Figure 2 Flowchart of the header display method provided in this application Figure 2 ;

[0046] Figure 3 Flowchart of the header display method provided in this application Figure 3 ;

[0047] Figure 4 A schematic diagram illustrating one application scenario provided in this application;

[0048] Figure 5 A schematic diagram of the structure of the head-up display device provided in this application;

[0049] Figure 6 A schematic diagram of the structure of the electronic device provided in this application.

[0050] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0052] In related technologies, after collecting 3D point cloud data of the vehicle's surrounding environment through onboard sensors, physical entities such as roads, vehicles, pedestrians, and traffic signs are reconstructed into detailed 3D digital models. These digital models are then spatially aligned with the real scene, ultimately rendering a virtual overlay image that matches the size, texture, and position of real-world objects. However, in this method, high-fidelity rendering presents a large amount of redundant details within a limited display area, such as the texture of distant vehicles and the outlines of roadside vegetation. This non-critical information significantly increases the driver's cognitive load, making it difficult for the driver to quickly identify core risk sources, thus prolonging decision-making reaction time and affecting driving safety.

[0053] Therefore, the inventors reasoned that the core need of drivers during driving is to quickly obtain key information that is strongly related to driving decisions, rather than to completely reconstruct the surrounding physical entities. Therefore, they abandoned the idea of ​​detailed 3D modeling and instead focused on extracting target elements in the surrounding environment that are directly related to driving decisions and their core behavioral intentions (such as the vehicle in front changing lanes, pedestrians crossing the road, etc.). This key information is then converted into lightweight icons and corresponding visual attributes. By simplifying the presentation of information, drivers can directly capture core driving information without having to analyze complex images, thereby significantly reducing cognitive load, ensuring the timeliness and accuracy of driving decisions, and improving driving safety.

[0054] The execution subject of this application embodiment can be an electronic device with processing capabilities, such as an in-vehicle terminal or a head-up display (HUD). For example, after executing the HUD display method provided in this application embodiment, the in-vehicle terminal can send instructions to the HUD device for projection display, thereby reducing the hardware cost and computational burden of the HUD device; alternatively, the HUD device can directly execute the HUD display method provided in this application embodiment and then perform projection display. In this case, the HUD device has processing capabilities and does not rely on the computing power support of the in-vehicle terminal. It should be noted that this application embodiment does not limit the execution subject.

[0055] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0056] Figure 1 Flowchart of the header display method provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0057] S101. Obtain vehicle driving status information, navigation information, and current environmental information.

[0058] For example, vehicle driving status information refers to a set of parameters that characterize the vehicle's own operating conditions, including but not limited to vehicle speed, acceleration, steering angle, braking status, current gear, throttle opening, and lighting mode.

[0059] Navigation information refers to relevant data used to guide a vehicle's driving route, such as the vehicle's current location coordinates, planned driving route, type of intersection ahead, and road speed limits.

[0060] The current environmental information refers to the perceived data of the road and traffic scene surrounding the vehicle, which may include road information, traffic signs, status information of surrounding vehicles and obstacles, real-time traffic congestion, etc. Status information may include, for example, speed, coordinate position, and direction angle. It should be noted that this application embodiment does not limit the method of collecting the current environmental information; it can be obtained through various channels such as onboard sensors, vehicle networks, and navigation application interfaces. Onboard sensors may include, for example, millimeter-wave radar, lidar, and cameras; this application embodiment does not limit the specific methods used.

[0061] In one example, the electronic device communicates with the vehicle controller via a communication bus to obtain the vehicle's driving status information, obtains navigation information via a navigation device, and receives current environmental information collected by cameras and millimeter-wave radar via a multi-sensor fusion unit.

[0062] S102. Based on the current environmental information, navigation information, and driving status information, determine the target element and the target element's behavioral intent information.

[0063] For example, target elements refer to dynamic traffic participants or obstacles selected from the vehicle's surrounding environment that are strongly relevant to driving decisions, such as pedestrians, food delivery robots, vehicles, etc.

[0064] Behavioral intent information refers to the predicted results that characterize the movement trend of a target element. For example, it may include the intent to cut into the vehicle's own lane, the intent to trespass into the lane, the intent to move normally, or the intent to remain stationary.

[0065] In one example, the electronic device can first identify the current environment information, identify dynamic elements in the environment, and filter the dynamic elements based on their speed and coordinate position to obtain target elements. Then, based on a preset trajectory prediction model, according to the target element's speed, coordinate position, and direction angle, as well as the vehicle's speed and navigation route planning, the device can output the behavioral intent information corresponding to each target element. For example, the behavioral intent of the food delivery robot can be determined as "moving normally in the current direction", and the behavioral intent of a pedestrian approaching the lane can be determined as "entering the lane".

[0066] S103. Based on the target element, the target element's behavioral intent information, and the preset symbol library, determine the target icon and target visual attributes of the target element.

[0067] For example, a symbol library refers to a set of preset mapping rules, which may include, for instance, the correspondence between elements and icons, and the correspondence between behavioral intent and visual attributes. Optionally, the symbol library may also include the correspondence between risk levels and visual attributes. It should be noted that the correspondence in the symbol library can be customized based on actual needs, and this embodiment of the application does not limit it.

[0068] A target icon is a visual graphic symbol used to represent the type of a target element. This can be understood as a lightweight, simplified visual symbol; for example, a pedestrian icon corresponds to a pedestrian, a triangle icon to a food delivery robot, and a diamond icon to a temporarily parked vehicle. It should be noted that there is a one-to-one correspondence between icons and element types; each type of target element corresponds to a unique icon.

[0069] Target visual attributes refer to visual feature parameters used to distinguish the behavioral intent of target elements and enhance the visual cues. These may include, for example, icon color, dynamic effects, display priority, and directional arrow indicators. It should be noted that this application does not limit the specific shape of the icon or the content and parameters of its visual attributes.

[0070] In one example, the electronic device first retrieves the corresponding target icon from the symbol library based on the type of the target element, such as matching a pedestrian icon for a pedestrian and a triangle icon for a food delivery robot; then, based on the behavioral intent information of the target element, it matches the corresponding visual attributes from the symbol library, such as matching a red, pulse-flashing visual attribute for a pedestrian determined to have "intended to enter the lane", and matching a green, directional arrow visual attribute for a food delivery robot determined to have "intended to move normally".

[0071] It is understandable that the decision-making intent of a vehicle's intelligent driving system can be generated based on the behavioral intent of the target element. Since the visual attributes of the target are determined by the behavioral intent of the target element, the visual attributes of the target can also intuitively reflect the decision-making intent of the intelligent driving system while mapping the behavioral intent of the target element. Specifically, the intelligent driving system analyzes the behavioral intent of the target element (such as a pedestrian crossing a lane or a robot moving normally) to deduce and determine the corresponding decision-making intent (such as emergency braking or maintaining a safe distance). A pre-set symbol library matches the corresponding visual attributes based on the behavioral intent of the target element. This presentation of visual attributes (such as a red flashing pulse or a green directional arrow) not only conveys the behavioral risk of the target element but also allows the driver to perceive the decision-making logic of the intelligent driving system through visual signals. For example, a red flashing pedestrian icon corresponds to the intelligent driving system's emergency braking decision, and a green robot icon with a directional arrow corresponds to the intelligent driving system's safe distance decision.

[0072] S104. Send the target icon and target visual attributes to the head-up display device for projection onto the preset area for display.

[0073] For example, a head-up display device refers to a device that projects virtual symbol information onto the windshield of a vehicle.

[0074] The preset area refers to the projection area within the driver's field of vision that does not obstruct the road conditions ahead, such as the area located in the lower middle part of the windshield.

[0075] In one example, the electronic device sends instruction data containing the target icon and its visual attributes to the head-up display (HUD) via an in-vehicle Ethernet network. Upon receiving the instruction, the HUD controls the projection module to project the target icon onto a preset area according to its corresponding visual attributes. It is understood that the instruction data includes projection position information for the target icon and its visual attributes, allowing projection onto the preset area based on this information. This embodiment does not limit the method for determining the projection position information.

[0076] The head-up display method provided in this application acquires vehicle driving status information, navigation information, and current environmental information to determine target elements and behavioral intent information strongly related to driving decisions. Then, it matches the target elements with corresponding target icons and visual attributes based on a preset symbol library, and finally sends the data to the head-up display device for projection display. This approach achieves several advantages: firstly, it replaces redundant 3D model rendering data with lightweight symbolic information, reducing computational load and hardware costs; secondly, it conveys the behavioral intent of target elements through intuitive icons and visual attributes, allowing drivers to quickly capture key driving information without analyzing complex images, reducing cognitive load, improving the timeliness and accuracy of driving decisions, and enhancing driving safety; thirdly, the target visual attributes intuitively reflect the decision intent of the vehicle's intelligent driving system, allowing drivers to clearly see the source of the system's decisions, eliminating confusion or distrust caused by unexplained braking or deceleration, effectively eliminating information gaps in human-computer interaction, increasing driver trust and acceptance of the intelligent driving system, reducing operational conflicts caused by unclear decision-making basis, and improving driving safety and comfort.

[0077] Figure 2 Flowchart of the header display method provided in this application Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 1 Based on the embodiments, the display method of the head-up display is described in detail, and the method includes:

[0078] S201. Obtain vehicle driving status information, navigation information, and current environmental information.

[0079] It should be noted that this step is similar to the aforementioned step S101, and will not be repeated here.

[0080] S202. Detect and process the current environmental information to determine the initial element and its state information.

[0081] For example, the initial element refers to the collective term for all traffic participants and obstacles identified from the environmental information, such as pedestrians, motor vehicles, non-motor vehicles, food delivery robots, roadblocks, etc. The state information of the initial element refers to the data set characterizing the motion and attribute features of the initial element, such as coordinate position, speed, direction angle, shape features, and behavioral indicators (e.g., vehicle brake light status). It is understood that the initial element can be obtained by detecting and processing the current environmental information based on a preset target detection model. This target detection model has the ability to identify multiple types of targets and can accurately distinguish different types of initial elements from complex environmental information, while extracting the corresponding state information. It should be noted that the specific algorithm type of the target detection model is not limited in the embodiments of this application.

[0082] In one example, an electronic device can call the YOLO model to detect and process the information of the current environment, identify initial elements such as pedestrians, food delivery robots, and temporarily parked vehicles from the current environment information, and extract the position coordinates, movement speed, movement direction angle of each initial element, as well as the status information such as the brake light status of the temporarily parked vehicle.

[0083] S203. Based on the state information and driving state information of the initial elements, the initial elements are filtered to obtain the target elements.

[0084] For example, target elements refer to dynamic objects selected from the initial elements that are strongly related to the vehicle's driving decisions. In other words, target elements are initial elements that have a potential impact on the vehicle's driving safety or require the driver's close attention. It is understandable that the core purpose of the filtering process is to eliminate redundant elements unrelated to driving decisions, thereby reducing the computational load on subsequent data processing.

[0085] In one example, an electronic device can use an initial element that satisfies the mapping conditions as the target element based on preset mapping conditions.

[0086] Specifically, if the state information and driving state information of the initial element satisfy the mapping conditions of the initial element in the symbol library, then the initial element is determined to be the target element.

[0087] The aforementioned symbol library also includes element mapping conditions. Element mapping conditions refer to the set of rules used to determine whether an initial element qualifies as a target element. Different types of initial elements correspond to specific mapping conditions. For example, the mapping conditions for pedestrians may include a relative distance threshold to the current lane, a movement speed threshold, and a direction threshold. The mapping conditions for motor vehicles may include brake light status and a relative speed threshold to the vehicle itself. Specific settings can be configured based on actual needs, and this application embodiment does not impose limitations. Table 1 shows an example of a symbol library provided in this application embodiment.

[0088] Table 1 Symbol Library

[0089]

[0090] For example, the electronic device retrieves the mapping conditions corresponding to various initial elements in the symbol library. Taking a pedestrian as an example, the state information of the pedestrian initial element and the driving state information of the vehicle are substituted into the pedestrian mapping conditions. It is determined that the pedestrian's relative distance to the current lane is 1.5m (less than the 2m threshold), the speed is 3m / s (greater than the 2m / s threshold), and the direction is towards the vehicle's lane. If the mapping conditions are met, the pedestrian initial element is determined as the target element. It should be noted that for initial elements that do not meet the mapping conditions, they can be processed according to the original rendering logic.

[0091] S204. Based on the target element's state information and driving state information, determine the target element's behavioral intent information.

[0092] Specifically, based on a preset trajectory prediction algorithm, the behavioral intent information of the target element is determined according to the target element's state information and driving state information.

[0093] The preset trajectory prediction algorithm can predict the trajectory of a target element over a future period of time by using its historical motion data and real-time status. It should be noted that this application does not limit the type of trajectory prediction algorithm used in its embodiments.

[0094] For example, taking the Long Short-Term Memory (LSTM) algorithm as a trajectory prediction algorithm, the algorithm is called with the target element's position coordinates, speed, and direction angle as input. It outputs the predicted trajectory for the future time period. Based on the predicted trajectory and the vehicle's speed and steering state, the target element's behavioral intent is determined. For instance, if the predicted trajectory shows the target element will cut into the vehicle's lane or cross its path, and the collision time is less than a preset safety threshold, it is determined to be an intention to cut in / cross; if the predicted trajectory shows the target element moving at a constant speed in the current direction without intersecting the vehicle's path, it is determined to be a normal movement intent; if the predicted trajectory shows no significant change in the target element's position coordinates, it is determined to be a temporary stopping intent.

[0095] S205. Based on the target element and the preset symbol library, determine the target icon corresponding to the target element.

[0096] For example, referring to the symbol library shown in Table 1, the electronic device can retrieve a preset symbol library and match it according to the type of the target element. For example, it can match a pedestrian icon for a target element of the pedestrian type, a triangle icon for a target element of the food delivery robot type, and a diamond icon for a target element of the vehicle type.

[0097] S206. Based on the behavioral intent information of the target element, determine the target visual attributes of the target element.

[0098] For example, referring to the symbol library shown in Table 1, the electronic device can retrieve the preset symbol library, match the corresponding visual attributes from the symbol library based on the behavioral intent information of the target element, configure the pedestrian icon with the intent to cross the lane with a red, pulse flashing visual attribute, configure the food delivery robot triangle icon with the intent to move normally with a blue visual attribute with a movement direction arrow, and configure the vehicle diamond icon with the intent to stay still with a yellow, static display visual attribute.

[0099] S207. Determine the projection mode based on the current environmental information and driving status information.

[0100] For example, projection mode refers to the perspective and presentation format of the virtual image displayed by the head-up display device, used to adapt to the information reading needs of different driving scenarios. For instance, it may include bird's-eye view mode, first-person view mode, and cross-sectional view mode. Bird's-eye view mode presents the 360° environment around the vehicle from a top-down perspective, facilitating the prediction of distance to the vehicle and its relationship with the surrounding environment. First-person view mode presents the road ahead from the driver's eye level, highlighting close-range guidance information. Cross-sectional view mode presents information related to road slope, tunnel entrances and exits, and other height differences from a longitudinal cross-sectional perspective. It is understandable that determining the projection mode requires combining environmental information and driving status information to ensure adaptation to the current driving scenario; for example, bird's-eye view mode is suitable for high-speed driving scenarios, while first-person view mode is suitable for complex urban scenarios.

[0101] In one example, the electronic device can determine the projection mode based on the road type and driving status information in the current environmental information. For instance, when it is determined that the vehicle is on a highway and the speed is greater than a preset threshold, the projection mode is determined to be a bird's-eye view mode; when it is determined that the vehicle is at a complex urban intersection and the turn signal is on, the projection mode is determined to be a first-person view mode; and when it is determined that the vehicle is at a tunnel entrance and there is a significant height difference, the projection mode is determined to be a cross-sectional view mode.

[0102] S208. According to the projection mode, perform projection processing on the target icon and target visual attributes to obtain the information to be projected.

[0103] For example, projection processing refers to a series of operations, such as perspective adjustment, coordinate calibration, and layer blending, performed on the target icon and its visual attributes according to a determined projection mode, to ensure accurate alignment between the virtual image and the real road scene. The information to be projected refers to the image data that, after projection processing, can be directly sent to the head-up display device; for example, it may include the adjusted target icon, visual attribute parameters, projection coordinates, and other information.

[0104] In one example, the electronic device can adjust the target icon to the presentation form of the determined projection mode, calibrate the projection coordinates of the target icon based on the vehicle's current position and road coordinates, ensure that the icon is accurately aligned with the position of real road objects (such as the vehicle in front and lane lines), and align the target visual attributes with the adjusted projection coordinates of the target icon to generate the information to be projected.

[0105] S209. Send the information to be projected to the head-up display device so that it can be projected onto the preset area for display.

[0106] For example, electronic devices send the information to be projected to a head-up display device via an in-vehicle Ethernet network. After receiving the information, the head-up display device analyzes the target icon, target visual attributes, and coordinate parameters in the information and projects the virtual image onto a preset area of ​​the windshield. This achieves accurate overlay of the target icon and the real road scene, allowing the driver to read key information without having to shift their gaze.

[0107] The head-up display method provided in this application acquires vehicle driving status information, navigation information, and environmental information. It detects and processes the environmental information to determine initial elements and status information, then filters target elements based on the initial element status information and vehicle driving status information. Subsequently, it determines the behavioral intent information of the target element, matches the corresponding target icon and target visual attributes based on a symbol library, determines the projection mode based on environmental information and driving status, and generates the information to be projected. Finally, it sends the information to the head-up display device for projection display. This method achieves several advantages: firstly, it replaces traditional 3D model rendering with lightweight symbolic display, significantly reducing computational load and hardware costs; secondly, by matching behavioral intent information with corresponding visual attributes, drivers can quickly determine the risk level of target elements without analyzing complex environmental data, simply through intuitive icons and visual features, significantly reducing cognitive load and improving the timeliness of driving decisions; thirdly, the target visual attributes, while conveying the behavioral intent information of the target element, can intuitively reflect the decision-making basis of the intelligent driving system, effectively eliminating information gaps in human-computer interaction, increasing driver trust in the intelligent driving system, reducing operational conflicts caused by unclear decisions, and, combined with dynamically adjusted projection modes, ensuring clear and stable display effects in different scenarios, thus improving driving safety and comfort.

[0108] Figure 3 Flowchart of the header display method provided in this application Figure 3 ,like Figure 3 As shown, in this embodiment... Figure 1 Based on the embodiments, the display method of the head-up display is described in detail, and the method includes:

[0109] S301. Obtain vehicle driving status information, navigation information, and current environmental information.

[0110] It should be noted that this step is similar to the aforementioned step S101, and will not be repeated here.

[0111] S302. Detect and process the current environmental information to determine the initial element and its state information.

[0112] It should be noted that this step is similar to the aforementioned step S202, and will not be repeated here.

[0113] S303. Based on the state information and driving state information of the initial elements, the initial elements are filtered to obtain the target elements.

[0114] It should be noted that this step is similar to the aforementioned step S203, and will not be repeated here.

[0115] S304. Based on a preset trajectory prediction algorithm, determine the behavioral intent information of the target element according to the target element's state information and driving state information.

[0116] It should be noted that this step is similar to the aforementioned step S204, and will not be repeated here.

[0117] S305. Based on a preset detection model, determine the risk level of the target element according to its state information and driving status information.

[0118] For example, as mentioned above, the initial element has state information, therefore, the target element after filtering also has state information.

[0119] Risk level refers to a classification standard used to characterize the degree of threat posed by a target element to the vehicle's driving safety. For example, it can be divided into three levels: high, medium, and low. It should be noted that the embodiments of this application do not limit the method of risk level classification.

[0120] The preset detection model refers to a risk assessment model trained based on a large amount of real-vehicle test data. This model can calculate the risk value of the target element and classify it into levels based on the state information of the target element and the driving state information of the vehicle. It can be understood that the classification of risk levels follows the principle that the degree of threat is positively correlated with the risk level. It should be noted that the embodiments of this application do not limit the type of detection model.

[0121] In one example, an electronic device can call a preset risk assessment model, input the status information of the target element and the driving status information of the vehicle, output the risk value of the target element, and then determine the risk level of the target element based on the preset correspondence between risk value and risk level.

[0122] S306. Based on the target element and the preset symbol library, determine the target icon corresponding to the target element.

[0123] It should be noted that this step is similar to the aforementioned step S205, and will not be repeated here.

[0124] S307. Determine the basic visual attributes of the target element based on its risk level.

[0125] For example, basic visual attributes refer to visual parameters used to convey the degree of risk, determined based on the risk level of the target element. These may include icon color, basic display brightness, etc. It is understood that the symbol library pre-determines the correspondence between risk levels and basic visual attributes: high-risk target elements correspond to strongly warning basic visual attributes, medium-risk elements correspond to suggestive basic visual attributes, and low-risk elements correspond to mild basic visual attributes. It should be noted that the configuration of basic visual attributes follows the principle of a positive correlation between risk level and visual warning intensity; for example, high risk corresponds to red, medium risk to green, and low risk to yellow, to ensure that drivers can quickly determine the risk level through color.

[0126] In one example, an electronic device can match the corresponding basic visual attributes from a symbol library based on the risk level of the target element. For example, it can configure the basic visual attributes of a high-risk pedestrian icon as red and bright, the basic visual attributes of a medium-risk food delivery robot icon as green and medium brightness, and the basic visual attributes of a low-risk temporary parking vehicle icon as yellow and normal brightness.

[0127] This method allows drivers to quickly distinguish threat levels simply by color, avoiding excessive focus on low-risk targets in complex road conditions.

[0128] S308. Adjust the basic visual attributes based on the behavioral intent information of the target element to obtain the target visual attributes of the target element.

[0129] For example, target visual attributes refer to the final set of visual parameters determined after dynamic adjustments based on basic visual attributes and the behavioral intent information of the target element. Besides basic color and brightness, these may also include flashing frequency, display priority, directional arrow indicators, etc. It can be understood that the adjustment rules are preset in a symbol library, and different behavioral intents correspond to different adjustment strategies. For example, a target element with a traversing intent has a high-frequency pulse flashing effect added to its red basic attribute, while a target element with a normal movement intent has a directional arrow indicator added to its green basic attribute.

[0130] In one example, the electronic device can dynamically adjust the basic visual attributes based on the behavioral intent information of the target element. For the high-risk pedestrian icon with the intent to cross the lane, a pulse high-frequency flashing effect is added to the red high brightness, and the highest display priority is set; for the medium-risk food delivery robot icon with the intent to move normally, an arrow mark consistent with the direction of movement is added to the green medium brightness; and for the low-risk vehicle icon with the intent to stand still, a static display effect with yellow normal brightness is maintained.

[0131] S309. Based on the current environmental information and driving status information, determine the projection mode.

[0132] It should be noted that this step is similar to the aforementioned step S207, and will not be repeated here.

[0133] S310. According to the projection mode, perform projection processing on the target icon and target visual attributes to obtain the information to be projected.

[0134] It should be noted that this step is similar to the aforementioned step S208, and will not be repeated here.

[0135] S311. Send the information to be projected to the head-up display device so that it can be projected onto the preset area for display.

[0136] It should be noted that this step is similar to the aforementioned step S209, and will not be repeated here.

[0137] The head-up display method provided in this application obtains vehicle driving status information, navigation information, and environmental information. It detects and processes the environmental information to determine initial elements and status information. Then, it filters target elements based on the initial element status information and vehicle driving status information. Subsequently, it calls a trajectory prediction algorithm to determine the behavioral intent information of the target elements, uses a risk assessment model to determine the risk level of the target elements, matches the corresponding target icons based on a symbol library, and determines basic visual attributes based on the risk level of the target elements. It adjusts the basic visual attributes in combination with behavioral intent information to obtain target visual attributes, determines the projection mode in combination with the environment and driving status, and generates information to be projected. Finally, it sends the information to the head-up display device for projection display. This approach employs a two-tiered configuration logic: risk level determines basic visual attributes, and behavioral intent information is dynamically adjusted. This allows drivers to quickly assess the risk level and behavioral trends of target elements using intuitive icons and visual features, without needing to analyze complex environmental data. This significantly reduces cognitive load and improves the timeliness of driving decisions. Furthermore, since target visual attributes are jointly determined by risk level and behavioral intent, they not only directly reflect the threat level of a target element to the vehicle but also convey its movement trends. This reduces the inability of a single visual attribute to distinguish different behavioral states at the same risk level, enhancing the differentiation and hierarchy of visual information, further reducing the driver's cognitive load, and enabling them to accurately capture key driving information in a short time. This improves the timeliness and accuracy of driving decisions, comprehensively ensuring driving safety.

[0138] Figure 4 A schematic diagram illustrating an application scenario provided in this application, such as... Figure 4 As shown, electronic devices can be divided into a data input layer, a core processing layer, and a display output layer.

[0139] Combination Figure 3 and Figure 4As shown, the data input layer is used to acquire environmental information of the vehicle's current location collected by onboard sensors such as cameras / LiDAR, navigation information from navigation devices, and vehicle driving status information from the vehicle controller.

[0140] The core processing layer comprises a neural perception network module, a planning and decision-making module, and a symbol library. The neural perception network module detects and processes environmental information based on a pre-defined target detection model to obtain initial elements. Based on the initial elements' state and driving status information, it filters these initial elements to obtain target elements. The planning and decision-making module, based on a pre-defined trajectory prediction algorithm, determines the target element's behavioral intent information according to its state and driving status information. It also determines the target element's risk level based on the pre-defined detection model and its state and driving status information. Finally, the core processing layer, in conjunction with the symbol library, transforms the target element, its risk level, and behavioral intent into visualized image information such as target icons and visual attributes.

[0141] The display output layer is used to render and project the received visual image information, such as target icons and target visual attributes, and finally send the processed symbolized image to the projection display device to project it onto a preset area of ​​the vehicle's windshield and present it to the driver.

[0142] Figure 5 This is a schematic diagram of the structure of the head-up display device provided in this application, as shown below. Figure 5 As shown, the head-up display device 400 provided in this embodiment includes:

[0143] The acquisition module 401 is used to acquire the vehicle's driving status information, navigation information, and current environmental information;

[0144] The first determining module 402 is used to determine the target element and the target element's behavioral intent information based on the current environmental information, navigation information, and driving status information;

[0145] The second determining module 403 is used to determine the target icon and target visual attributes of the target element based on the target element, the behavioral intent information of the target element, and a preset symbol library; the symbol library includes the correspondence between elements and icons, and the correspondence between behavioral intent and visual attributes.

[0146] The sending module 404 is used to send the target icon and target visual attributes to the head-up display device for projection onto a preset area for display.

[0147] In one possible implementation, the first determining module 402 is configured to:

[0148] The current environmental information is detected and processed to determine the initial elements and their state information;

[0149] Based on the initial element's state information and driving state information, the initial element is filtered to obtain the target element;

[0150] Based on the target element's state information and driving state information, determine the target element's behavioral intent information.

[0151] In one possible implementation, the symbol library also includes mapping conditions for elements; a first determining module 402 is configured to:

[0152] If the state information and driving state information of the initial element satisfy the mapping conditions of the initial element in the symbol library, then the initial element is determined to be the target element.

[0153] In one possible implementation, the first determining module 402 is configured to:

[0154] Based on a preset trajectory prediction algorithm, the behavioral intent information of the target element is determined according to the target element's state information and driving state information.

[0155] In one possible implementation, the second determining module 403 is configured to:

[0156] Based on the target element and the preset symbol library, determine the target icon corresponding to the target element;

[0157] Based on the behavioral intent information of the target element, determine the target visual attributes of the target element.

[0158] In one possible implementation, the target element has state information, and the second determining module 403 is used for:

[0159] Based on the preset detection model, the risk level of the target element is determined according to the target element's state information and driving state information;

[0160] The symbol library also includes the correspondence between risk levels and visual attributes; the second determining module 403 is used for:

[0161] Based on the risk level of the target element, determine the basic visual attributes of the target element;

[0162] The basic visual attributes of the target element are adjusted based on the behavioral intent information of the target element to obtain the target visual attributes of the target element.

[0163] In one possible implementation, the sending module 404 is configured to:

[0164] Based on the current environmental information and driving status information, determine the projection mode;

[0165] Sending module 404 is used for:

[0166] Based on the projection mode, the target icon and target visual attributes are processed for projection to obtain the information to be projected.

[0167] The information to be projected is sent to the head-up display device for projection onto a preset area for display.

[0168] The head-up display device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0169] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Figure 6 As shown, the electronic device 500 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the electronic device 500 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus. This electronic device can be a vehicle-mounted terminal or a head-up display device.

[0170] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0171] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0172] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0173] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0174] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0175] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0176] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0177] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0178] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0179] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0180] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0181] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0182] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0183] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0184] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A head-up display method, characterized by, The method comprises: obtaining driving state information, navigation information and current environment information of a vehicle; determining a target element and behavior intention information of the target element based on the current environment information, the navigation information and the driving state information; determining a target icon and a target visual attribute of the target element based on the target element, the behavior intention information of the target element and a preset symbol library; the symbol library comprises a corresponding relationship between an element and an icon, and a corresponding relationship between a behavior intention and a visual attribute; sending the target icon and the target visual attribute to a head-up display device to project and display in a preset area.

2. The method of claim 1, wherein, The determination of the target element and the behavior intention information of the target element based on the current environment information, the navigation information and the driving state information comprises: detecting and processing the current environment information to determine an initial element and state information of the initial element; screening the initial element based on the state information of the initial element and the driving state information to obtain the target element; determining the behavior intention information of the target element based on the state information of the target element and the driving state information.

3. The method of claim 2, wherein, The symbol library further comprises a mapping condition of an element; the screening of the initial element based on the state information of the initial element and the driving state information to obtain the target element comprises: if it is determined that the state information of the initial element and the driving state information satisfy the mapping condition of the initial element in the symbol library, the initial element is determined as the target element.

4. The method of claim 2, wherein, The determination of the behavior intention information of the target element based on the state information of the target element and the driving state information comprises: determining the behavior intention information of the target element based on a preset trajectory prediction algorithm according to the state information of the target element and the driving state information.

5. The method of claim 1, wherein, The determination of the target icon and the target visual attribute of the target element based on the target element, the behavior intention information of the target element and the preset symbol library comprises: determining the target icon corresponding to the target element based on the target element and the preset symbol library; determining the target visual attribute of the target element based on the behavior intention information of the target element.

6. The method of claim 5, wherein, The target element has state information, and the method further comprises: determining a risk level of the target element based on a preset detection model according to the state information of the target element and the driving state information; the symbol library further comprises a corresponding relationship between a risk level and a visual attribute; the determination of the target visual attribute of the target element based on the behavior intention information of the target element comprises: determining a basic visual attribute of the target element based on the risk level of the target element; adjusting the basic visual attribute based on the behavior intention information of the target element to obtain the target visual attribute of the target element.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: determining a projection mode based on the current environment information and the driving state information; the sending of the target icon and the target visual attribute to the head-up display device to project and display in the preset area comprises: According to the projection mode, the target icon and the target visual attribute are projected to obtain to-be-projected information; The to-be-projected information is sent to a head-up display device to be projected to a preset area for display.

8. A head-up display display device, characterized by, Comprise: An acquisition module is configured to acquire driving state information, navigation information, and current environment information of a vehicle; A first determination module is configured to determine a target element and behavior intention information of the target element based on the current environment information, the navigation information, and the driving state information; A second determination module is configured to determine a target icon and a target visual attribute of the target element based on the target element, the behavior intention information of the target element, and a preset symbol library; The symbol library comprises a corresponding relationship between elements and icons, and a corresponding relationship between behavior intentions and visual attributes; A sending module is configured to send the target icon and the target visual attribute to a head-up display device to be projected to a preset area for display.

9. An electronic device, comprising: Comprise: A memory and a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-7.

10. A computer readable storage medium / computer program product, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method according to any one of claims 1-7; or The computer program product comprises a computer program, and the computer program is executed by the processor to implement the method according to any one of claims 1-7.

Images