An information display method, device, apparatus, and storage medium

Abstract

This application discloses an information display method, apparatus, device, and storage medium, relating to the field of vehicle networking technology. The method includes: determining the gaze point position of a vehicle user; wherein the gaze point position includes: the gaze focus position of the vehicle user in the projection area of ​​a head-up display device, or the gaze focus position of the vehicle user on information about the external environment, and the projection area is located within a preset area of ​​the vehicle's windshield; and adjusting the display content within a target area of ​​the projection area associated with the gaze point position according to the gaze point position.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of vehicle networking technology, and particularly to an information display method, device, equipment, and storage medium. Background Technology

[0002] With the increasing number of human-machine interaction functions in the cockpit, the use of display devices is becoming more and more frequent. Currently, cockpit display devices mainly fall into two categories: in-vehicle displays and projection displays, with head-up display (HUD) systems being the most representative of projection displays.

[0003] Traditional HUD systems primarily project vehicle information onto the windshield for easy driver viewing. However, in terms of operation and interaction, they typically rely on manual input or limited preset commands to adjust the displayed content, making it difficult to accurately perceive the driver's natural behavior and intentions, resulting in poor display quality. Summary of the Invention

[0004] One objective of this invention is to provide at least one information display method, apparatus, device, and storage medium. Its advantage lies in that, while a vehicle user is driving and the projection area of ​​a head-up display (HUD) is displaying content, the computer device can flexibly adjust the display strategy of the target area within the projection area associated with the user's gaze point, based on the user's gaze point position. This alters the display effect, filters and presents the most effective information to the user, reduces driver distraction, improves driving safety and operational efficiency, enhances the intelligence of the HUD, and improves the interactive experience between the HUD and the user.

[0005] To achieve the above objectives, the technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide an information display method, the method comprising:

[0007] Determine the gaze point position of the vehicle user; wherein, the gaze point position includes: the gaze focus position of the vehicle user in the projection area of ​​the head-up display device, or the gaze focus position of the vehicle user on the external environment information, wherein the projection area is located within a preset area of ​​the vehicle's windshield.

[0008] Based on the gaze point position, adjust the displayed content within the target area of ​​the projection area associated with the gaze point position.

[0009] Secondly, embodiments of this application provide an information display device, the device comprising:

[0010] A determination module is used to determine the gaze point position of a vehicle user; wherein, the gaze point position includes: the gaze focus position of the vehicle user in the projection area of ​​the head-up display device, or the gaze focus position of the vehicle user on information about the external environment, and the projection area is located within a preset area of ​​the vehicle's windshield.

[0011] The processing module is used to adjust the displayed content within the target area of ​​the projection area associated with the gaze point position according to the gaze point position.

[0012] Thirdly, embodiments of this application provide a computer device including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the program to implement some or all of the steps in the method described in the first aspect.

[0013] Fourthly, embodiments of this application provide a storage medium storing one or more computer programs, which can be executed by one or more processors to implement some or all of the steps in the method described in the first aspect.

[0014] Fifthly, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement some or all of the steps in the method described in the first aspect.

[0015] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0017] Figure 1 A flowchart illustrating an optional information display method provided in an embodiment of this application;

[0018] Figure 2 A flowchart illustrating an optional information display method provided in an embodiment of this application;

[0019] Figure 3 A flowchart illustrating an optional information display method provided in an embodiment of this application;

[0020] Figure 4 A flowchart illustrating an optional information display method provided in an embodiment of this application;

[0021] Figure 5A flowchart illustrating an optional information display method provided in this application embodiment.

[0022] Figure 6 This is a schematic diagram of the structure of an optional information display device provided in an embodiment of this application;

[0023] Figure 7 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0027] HUD, also known as a head-up display, is a display device that projects images into the driver's field of vision. Generally, a HUD includes components such as a projector, a reflector (also called a secondary reflector), and a projection mirror (also called a primary reflector). The imaging principle of a HUD is similar to that of a slide projector; by projecting an image onto the windshield, the driver can obtain an image of what's in front of them. Specifically, light information is first emitted by the projector, reflected by the reflector onto the projection mirror, and then reflected by the projection mirror onto the windshield. The image seen by the human eye is a virtual image located approximately 2-2.5 meters in front of the eyes, giving the impression that the information is floating on the road ahead.

[0028] It should be noted that the image position projected onto the windshield is adjustable; typically, the image position can be adjusted by changing the angle of the projection mirror. Furthermore, it's understandable that because car windshields are curved, projecting an image directly onto the curved glass surface would cause image distortion. This necessitates a corrective measure; therefore, both the projection mirror and the reflector are often designed to be curved.

[0029] As mentioned above, HUDs utilize the principle of optical reflection to project and display information such as speeding warnings, vehicle status monitoring, fuel consumption, speed, and navigation instructions onto the windshield. This allows the driver to focus their attention on the road ahead, achieving proactive driving safety. It also reduces the delay and discomfort caused by the constant adjustment of eye focus.

[0030] Related technologies rely on manual adjustments or limited preset commands to modify the HUD's display content. This makes it difficult to accurately perceive the driver's natural behavior and intentions, resulting in an inability to adapt the HUD's display information to the complex and ever-changing road environment. Consequently, it fails to provide users with smooth, clear, and interference-free HUD information, leading to poor display quality. Furthermore, due to the complex and ever-changing road environment, manual adjustments cannot meet real-time requirements, degrading the driver's experience. Additionally, unclear and inconsistent HUD information can interfere with driver judgment and negatively impact driving mood.

[0031] To address the aforementioned problems, embodiments of this application provide an information display method, which can be executed by a processor of a computer device. The computer device can refer to a server, laptop computer, tablet computer, desktop computer, smart TV, set-top box, mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device), or other device with data processing capabilities. In some embodiments, the computer device can be a HUD device. In some embodiments, the computer device can be an in-vehicle terminal device. This in-vehicle terminal device can be a terminal device deployed in a vehicle, communicating with the vehicle, and can be used independently of the vehicle or integrated into the vehicle control system.

[0032] Reference Figure 1 , Figure 1 This application provides a schematic diagram of the implementation process of an information display method, applied to a computer device. Here, it will be combined with... Figure 1 The steps shown are explained.

[0033] Step 101: Determine the location of the vehicle user's gaze point.

[0034] The gaze point position includes: the position where the vehicle user's gaze is focused on the projection area of ​​the head-up display device, or the position where the vehicle user's gaze is focused on information about the external environment of the vehicle, and the projection area is located within a preset area of ​​the vehicle's windshield.

[0035] In this embodiment, the gaze point location can be understood as the specific location where the vehicle user's (also known as the driver or the user driving the vehicle) line of sight is focused on different environments or areas in a driving scenario. Specifically, the gaze point location can be the location where the vehicle user's line of sight is focused on the projection area of ​​the head-up display device, or it can be the location where the vehicle user's line of sight is focused on a specific piece of information in the external environment.

[0036] In this embodiment, the projection area is the specific range of information projected by the head-up display (HUD). The projection area is located within a preset area on the vehicle's windshield, i.e., a pre-planned specific area on the windshield. The content of the HUD is displayed within this specific area. It should be noted that the projection area can be adjusted based on the driver's height and / or posture to ensure the vehicle user can obtain the optimal viewing field without adjusting their head position when viewing HUD information. Typically, the projection area is displayed in front of the vehicle user's line of sight, allowing them to obtain important driving information, such as vehicle speed, navigation instructions, and traffic sign recognition, without having to look down at the dashboard or center console. This facilitates the driver seeing the projected information while simultaneously focusing on the road ahead.

[0037] In this embodiment of the application, the external environment information can be understood as various situations that exist outside the vehicle that can be seen and paid attention to by the driver, such as traffic signs on the road, the driving status of other vehicles, the position of pedestrians, and buildings on the roadside. Vehicle users can focus their attention on the information while driving to understand the surrounding road conditions and make correct driving decisions.

[0038] In some embodiments, determining the gaze point of a vehicle user can be based on eye-tracking technology, which is achieved by installing a high-precision eye tracker inside the vehicle. Eye trackers typically use optical methods (such as infrared light reflection) or electrical methods (such as electrooculography) to track eye movements. As the vehicle user's eyes look at different locations, the eye tracker can accurately capture parameters such as eye rotation angle and pupil position, thereby calculating the direction of the vehicle user's gaze and determining the gaze point.

[0039] In practical applications, image acquisition devices such as eye trackers are installed around the driver's seat. These eye trackers include multiple infrared cameras and sensors. The infrared cameras emit infrared light, and when this light shines on the driver's eyes, the movement trajectory of the driver's eyes is determined by analyzing the changes in the reflected light. If the driver's pupils are stable at a certain position within the projection area of ​​the head-up display, it can be determined that the driver is looking at that location. If the driver's gaze is directed towards information outside the vehicle, such as a traffic sign, the specific location of the gaze point in the external environment can also be calculated.

[0040] In some embodiments, determining the gaze point location of a vehicle user can also be achieved by combining vehicle system data and scene analysis. Vehicles themselves have various sensors, such as vehicle speed sensors, steering wheel angle sensors, and navigation systems; the data from these sensors can provide information about the vehicle's current driving status and environment. Simultaneously, by combining this with scene information surrounding the vehicle, such as road condition images acquired through onboard cameras, the possible gaze point location of the vehicle user can be inferred.

[0041] In practical applications, when a vehicle approaches an intersection, especially a multi-way intersection, the speed gradually decreases, and the navigation system indicates an impending turn. At this point, it can be inferred that the user's gaze may be focused on traffic lights and signs outside the vehicle, or on turn-related information displayed on the head-up display (HUD). Additionally, if the vehicle is traveling on a highway and the steering wheel angle remains stable for an extended period, the user's gaze may be more focused on the road and other vehicles ahead. By analyzing these scenarios, it's possible to roughly determine the user's gaze location within the external environmental information or on the speed display or other relevant information within the HUD's projection area.

[0042] Step 102: Adjust the displayed content within the target area of ​​the projection area associated with the gaze point position, based on the gaze point position.

[0043] In this embodiment, the target area is a portion of the projection area, and is associated with the gaze point position. If the gaze point position is the focal point of the vehicle user's gaze on the projection area of ​​the head-up display device, the target area can be one or more rows in the projection area where the gaze point position is located, or it can be a certain area in the projection area centered on the gaze point position. If the gaze point position is the focal point of the vehicle user's gaze on information about the external environment, the target area is the area in the projection area where the information associated with the external environment information is located. This application does not impose specific limitations on this.

[0044] In this embodiment of the application, after the computer device determines the gaze point position of the vehicle user, it can adjust the display content and / or display position within the target projection area associated with the gaze point position based on the real-time determined gaze point position.

[0045] In this embodiment, when a computer device adjusts the displayed content and / or display position within a target area of ​​a projection area associated with the gaze point position in real time, it can do so in two ways: Method 1: The computer device adjusts the size of the displayed content based on the gaze point position; Method 2: The computer device adjusts the position and size of the displayed content based on the current distance and current relative posture. For example, adjusting the position of the displayed content includes changing its display position within the projection area; adjusting the size of the displayed content includes shrinking or enlarging its size; the displayed content includes, but is not limited to, text content and navigation instructions; this content can be projected using a head-up display device.

[0046] In other embodiments, the format of the displayed content can also be adjusted based on the gaze point position. This adjustment may include bolding, highlighting, or dynamically flashing the displayed content. Thus, while a vehicle user is driving and the projection area of ​​the head-up display (HUD) is displaying content, the computer device can flexibly adjust the display strategy of the target area associated with the gaze point position based on the user's gaze point position. This alters the display effect, allowing the system to filter and present the most effective information to the user, reducing driver distraction, improving driving safety and operational efficiency, enhancing the intelligence of the HUD, and improving the interactive experience between the HUD and the user.

[0047] Reference Figure 2 , Figure 2 This is a schematic diagram illustrating the implementation process of an information display method provided in an embodiment of this application. Here, it will be combined with... Figure 2 The steps shown are explained.

[0048] Step 201: Obtain multi-frame eye images of the vehicle user.

[0049] In this embodiment, the eye image refers to an image containing the human eye captured by a preset image acquisition device. The eye image is used to analyze the eye movements and gaze direction of the vehicle user to determine the user's gaze point. For example, the image acquisition device can be an eye tracker, which can be installed anywhere inside the vehicle where an image of the vehicle user's eyes can be captured. The eye image of the vehicle user is captured by the optical system of the eye tracker and then sent to a computer device.

[0050] Step 202: Perform image analysis on each frame of the eye image to obtain the position of the vehicle user's pupil in each eye image.

[0051] In this embodiment, the pupil position is the geometric center point of the pupil, that is, the center position of the pupil region. In the eye image of the vehicle user, the pupil position can be identified and the center point of the pupil region can be located using image processing technology.

[0052] In this embodiment of the application, after acquiring multiple frames of eye images of the vehicle user in real time through an image acquisition device, image processing technology can be used to analyze and process each frame of eye images to identify and locate the position of the vehicle user's pupils in each eye image.

[0053] Step 203: Using the pupil positions corresponding to multiple frames of eye images, estimate the eye movement information of the vehicle user.

[0054] Among these, motion information includes the speed of eye movement.

[0055] In this embodiment, eye movement information may include eye movement speed and direction. Based on the eye movement speed, it can be determined whether to adjust the displayed content of the target area corresponding to the gaze point. Thus, by employing image processing and sensor fusion technology, highly accurate and stable eye tracking is achieved, adapting to various driving environments and individual differences among vehicle users.

[0056] Step 204: Obtain eye movement data based on motion information and / or pupil position.

[0057] The eye-tracking data includes: the fixation point position of the eyeball, the fixation duration corresponding to the fixation point position, and the number of fixations.

[0058] In this embodiment, the fixation duration at a fixation point is the length of time from when the eye enters a fixation state at a certain fixation point until the eye leaves that fixation state. Leaving the fixation state includes situations such as the eye movement speed exceeding a speed threshold or a significant change in pupil position. Fixation duration reflects the continuous attention a vehicle user gives to the fixation point.

[0059] In this embodiment, the number of fixations can be understood as the total number of times a condition meeting the criteria for determining an eye fixation state is detected during the entire analysis of multiple frames of eye images. These criteria include: eye movement speed being less than or equal to a speed threshold or pupil position being relatively stable. The number of fixations reflects the frequency with which a vehicle user focuses on different fixation points.

[0060] In this embodiment of the application, after the computer device estimates the movement information of the vehicle user's eyeballs by using the pupil positions corresponding to multiple frames of eye images, it can also determine the fixation point position, the fixation duration corresponding to the fixation point position, and the number of fixations based on the pupil positions and / or movement information corresponding to each frame of eye images.

[0061] In some embodiments, determining eye data based on motion information and / or pupil position can be achieved as follows: when the eye movement speed decreases to a first speed threshold and the pupil position remains relatively stable for a certain period of time, the eye can be considered to be in a fixation state, and the stable pupil position is the fixation point position of the eye; the duration of the eye being in this fixation state is counted to obtain the fixation duration corresponding to the fixation point position, and the frequency of attention to the fixation point position within the target duration corresponding to multiple frames of eye images is counted to obtain the statistical frequency.

[0062] As mentioned above, by accurately knowing eye movement data such as the position of the eye's gaze point, the duration of gaze, and the number of gazes, we can understand the information that vehicle users are paying attention to during driving. This allows us to optimize the layout of various in-vehicle displays, such as head-up displays, and highlight the information that corresponds to the position of the vehicle user's gaze point, making it more eye-catching and easier to see. This reduces the time the driver spends looking away, improves the timeliness and accuracy of driving operations, and further ensures driving safety.

[0063] Step 205: Determine whether the parameters corresponding to the gaze point position meet the adjustment conditions. If so, adjust the display content in the target area of ​​the projection area associated with the gaze point position.

[0064] The adjustment conditions include at least one of the following: when moving to the fixation point, the eye movement speed is less than the speed threshold, the fixation duration is greater than the duration threshold, and the number of fixations is greater than the number of fixations threshold.

[0065] In this embodiment of the application, after obtaining eye movement data including the fixation point position of the eyeball, the fixation duration corresponding to the fixation point position, and the number of fixations based on motion information and / or pupil position, the computer device can predict the vehicle user's intention based on whether one or more of the eye movement speed, fixation duration at the fixation point position, and number of fixations meet adjustment conditions.

[0066] For example, it can be determined whether the eye movement speed when moving to the fixation point is less than at least one of the following thresholds: speed threshold, fixation duration threshold, or fixation count threshold. If the eye movement speed when moving to the fixation point is greater than or equal to the speed threshold, and the fixation duration is less than or equal to the duration threshold (or the fixation count is less than or equal to the count threshold), it is determined that the adjustment conditions are not met. This indicates that the vehicle user is only quickly scanning the target area corresponding to the fixation point, and the current displayed content can remain unchanged or only a slight prompt can be given. If the eye movement speed when moving to the fixation point is less than the speed threshold, and the fixation duration is greater than the duration threshold (or the fixation count is greater than the count threshold), it is determined that the adjustment conditions are met. This indicates that the vehicle user has a deeper need for attention to the information in the target area corresponding to the fixation point, and the computer device adjusts the displayed content in the target area associated with the fixation point, such as providing more detailed information or interactive options accordingly.

[0067] As described above, by accurately acquiring information such as fixation point location, fixation duration, and fixation frequency from eye-tracking data, it is possible to gain a detailed understanding of the specific direction and allocation of a vehicle user's attention during driving. For example, it is possible to clearly know the degree to which a driver focuses on different traffic lights, traffic signs, and surrounding vehicles when approaching an intersection. This allows for the reasonable adjustment of the layout and presentation of various information displays within the vehicle. Specifically, important information that the vehicle user is concerned about can be magnified or moved to a more prominent and visually pleasing position, avoiding the need for the vehicle user to spend excessive time searching for and focusing on a particular driving prompt. This reduces the time the driver spends shifting their gaze, enabling them to obtain key information more conveniently and quickly, thus optimizing the driving experience and improving driving efficiency.

[0068] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0069] Reference Figure 3 , Figure 3 This is a schematic diagram illustrating the implementation process of an information display method provided in an embodiment of this application. Here, it will be combined with... Figure 3 The steps shown are explained.

[0070] Step 301: Determine the location of the vehicle user's gaze point.

[0071] The gaze point position includes: the position where the vehicle user's gaze is focused on the projection area of ​​the head-up display device, or the position where the vehicle user's gaze is focused on information about the external environment of the vehicle, and the projection area is located within a preset area of ​​the vehicle's windshield.

[0072] Step 302: If the gaze point is the location where the vehicle user's gaze is focused on the external environment information, obtain the detailed information associated with the external environment information.

[0073] Among them, the external environment information includes at least one of the following: external intersection turn instruction information, external public place information, and external service area information.

[0074] In this embodiment, the external intersection turn instruction information can be understood as the relevant visual signs or signals located around the intersection when the vehicle is driving to the road intersection, used to prompt the vehicle user to perform what kind of turning operation (such as left turn, right turn, straight, U-turn, etc.); wherein, the associated information of the external intersection turn instruction information includes, but is not limited to, information in traffic lights, signs set up at the intersection, etc. The external intersection turn instruction information can help drivers accurately determine how to drive at the intersection, so as to follow traffic rules and pass through the intersection smoothly, ensuring the orderly and smooth traffic at the intersection.

[0075] In this embodiment of the application, the information on public places outside the vehicle refers to information related to various places around the vehicle that are available for public use, gathering or activities during the vehicle's operation. These public places include, but are not limited to, shopping malls, supermarkets, hospitals, schools, libraries, parks, gas stations and parking lots. The detailed information associated with the information on public places outside the vehicle includes, but is not limited to, the distance to the public place, the operating hours of the public place, and the appearance characteristics of the public place.

[0076] In this embodiment, the service area information refers to the information about service areas available to vehicle users for rest, refueling, vehicle maintenance, etc., along driving routes such as highways and long-distance roads. Detailed information associated with the service area information includes, but is not limited to, the distance to the service area and the available services such as food, refueling, accommodation, restrooms, and charging stations. Drivers can use this information to decide whether to enter a service area for rest, refueling, etc., to rationally plan their driving, avoid fatigue driving, and ensure the vehicle can continue to operate normally.

[0077] Step 303: Determine the target area corresponding to the vehicle external environment information from the projection area, wherein the target area includes at least one of the navigation map area, public place area, and service area area;

[0078] Step 304: Zoom in to display detailed information related to the external environment information of the target area.

[0079] In this embodiment, if the gaze point is the location where the vehicle user focuses their gaze on the external environment information, detailed information associated with at least one of the following in the external environment information is obtained: external intersection turning instruction information, external public place information, and external service area information. Further, a target area corresponding to the external environment information is determined from the projection area, and the target area includes at least one of the navigation map area, public place area, and service area area. Finally, the detailed information associated with the external environment information of the target area is enlarged and displayed.

[0080] In one feasible scenario, the HUD device displays navigation information. During the chaotic traffic conditions of the morning rush hour, when the driver's gaze is focused on the turn signal sign at the intersection ahead in the external environment, the device determines the turn area in the navigation area corresponding to the turn signal sign from the projection area. It then automatically zooms in and highlights the navigation display and map details of that turn area, showing one or more more detailed road information, intersection names and lane instructions, and intersection turn prompts. The driver does not need to struggle to distinguish complex road signs and can easily follow the navigation guidance, ensuring that the driver does not miss the correct turning direction.

[0081] In another feasible scenario, the HUD device displays map information. While the vehicle is driving on city roads, when the user's gaze is fixed on public places outside the vehicle, such as restaurants, gas stations, or parking lots, the system identifies the corresponding public place area from the projection area, retrieves its detailed information, and enlarges it to display this information, including but not limited to the place's name, features, and prices. This allows the user to quickly decide whether to eat, refuel, or park there without being distracted by checking their phone or other devices, improving decision-making efficiency and ensuring driving safety.

[0082] In another feasible scenario, the HUD device displays navigation map information. When the vehicle is traveling on a highway or long-distance road, and the user's gaze is directed to the service area information or service area icon on the navigation map, the system determines the service area corresponding to the service area information or icon from the projection area, obtains detailed information about the service area, and enlarges the display of this detailed information, including but not limited to the locations of gas stations, restaurants, rest areas, and whether there are charging stations. This allows the user to plan in advance whether to rest or refuel at the service area and avoid missing a suitable rest stop.

[0083] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0084] Reference Figure 4 , Figure 4 This is a schematic diagram illustrating the implementation process of an information display method provided in an embodiment of this application. Here, it will be combined with... Figure 4 The steps shown are explained.

[0085] Step 401: Determine the location of the vehicle user's gaze point.

[0086] The gaze point position includes: the position where the vehicle user's gaze is focused on the projection area of ​​the head-up display device, or the position where the vehicle user's gaze is focused on information about the external environment of the vehicle, and the projection area is located within a preset area of ​​the vehicle's windshield.

[0087] Step 402: If the gaze point is the focal point of the vehicle user's gaze in the projection area of ​​the head-up display device, determine the target area in the projection area associated with the gaze point.

[0088] Step 403: If the target area corresponding to the gaze point is a vehicle information area, zoom in and / or highlight the vehicle information, and further display the assisted driving information related to the vehicle information.

[0089] In this embodiment, the vehicle information area is an area for displaying the vehicle's status information, which includes, but is not limited to, vehicle speed information, fuel consumption information, and fault information.

[0090] In some embodiments, if the vehicle information area is a vehicle speed display area in step 403, the vehicle speed information in the vehicle speed display area is magnified and / or highlighted; the speed limit information in the driving scenario is obtained, and the vehicle speed information and speed limit information are dynamically displayed in the vehicle speed display area.

[0091] Understandably, while driving, vehicle users will periodically check vehicle status information, such as speed, fuel consumption, tire pressure, and engine status. When the target area corresponding to the user's gaze point is the vehicle information area, and that area is the speed display area, the computer device can enlarge and / or highlight the speed figure in the speed display area. Furthermore, it can also obtain the speed limit information for the current road and dynamically display the current speed and the speed limit information in the speed display area; if the speed is close to the speed limit, the speed information will turn red and flash, reminding the driver to control their speed. In this way, by comparing the speed information with the speed limit information, vehicle users can clearly understand whether their driving is compliant.

[0092] In some embodiments, if the vehicle information area is a fuel consumption display area, the fuel consumption information in the fuel consumption display area is enlarged in step 403; the mileage that the vehicle can drive is determined based on the fuel consumption information and the remaining fuel of the vehicle; if the mileage that can drive is less than the mileage threshold, the nearest gas station to the vehicle's current location is marked in the navigation map area; wherein, the target area includes the navigation map area.

[0093] Understandably, during driving, especially long-distance travel, it's crucial to closely monitor a vehicle's various statuses. Drivers frequently check information such as speed, fuel consumption, tire pressure, and engine status. The HUD (Head-Up Display) shows navigation map information. When the user's gaze falls on a vehicle information area that also displays fuel consumption, the computer can magnify and / or highlight the speedometer reading. Furthermore, based on current fuel consumption and remaining fuel, the system calculates the vehicle's remaining mileage in real-time. If the remaining mileage is below a threshold, it automatically searches for the nearest gas station and marks it on the navigation map. This helps drivers plan refueling times and reminds them to refuel promptly.

[0094] Step 404: If the target area corresponding to the gaze point is the navigation map area, zoom in to display simplified road sign information of the navigation map area.

[0095] The displayed content includes simplified road guidance information, which includes one or more of the following: lane instructions, road names, and lane turning instructions. The target area includes the navigation map area.

[0096] Understandably, HUD devices display navigation map information. When the target area corresponding to the user's gaze point is the navigation map area while the vehicle is in motion, the device enlarges the display to show simplified road guidance information, such as lane directions, road names, and lane turn signals, or one or more of these. Key lane directions, lane turn signal arrows, and road names are displayed in larger fonts and brighter colors to ensure the driver is traveling in the correct direction. At the same time, the device highlights the current location and route, making it easier for the driver to obtain navigation-related information.

[0097] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0098] Reference Figure 5 , Figure 5 This is a schematic diagram illustrating the implementation process of an information display method provided in an embodiment of this application. Here, it will be combined with... Figure 5The steps shown are explained.

[0099] Step 501: In response to a vehicle malfunction, a warning message is displayed in the vehicle information area of ​​the projection area.

[0100] In this embodiment of the application, the abnormality of the vehicle can be understood as an abnormality caused by some abnormal condition of the vehicle itself. For example, a key component of the vehicle malfunctions, such as engine failure, braking system failure, abnormal tire pressure, etc., or the vehicle's operating parameters exceed the normal range, such as excessively high water temperature, abnormal oil temperature, abnormal vehicle speed sensor data, etc. It may also be that the vehicle detects potential dangers in the external environment that affect the vehicle itself and determines it to be abnormal, such as being too close to the vehicle in front and having an unreasonable relative speed, detecting obstacles on the road but the vehicle's own obstacle avoidance system being unable to handle them properly, etc.

[0101] In this embodiment, the vehicle information area displays information closely related to vehicle operation, such as vehicle speed information, vehicle status icons, and fault prompts.

[0102] Understandably, when a vehicle malfunctions, a warning message will be displayed in the vehicle information area. The content of the warning message will vary depending on the specific malfunction. For example, if there is an engine malfunction, a red engine malfunction icon will be displayed; if there is an abnormal tire pressure, tire pressure information will be displayed, etc.

[0103] Step 502: Determine the location of the vehicle user's gaze point;

[0104] The gaze point position includes: the position where the vehicle user's gaze is focused on the projection area of ​​the head-up display device, and the projection area is located within a preset area of ​​the vehicle's windshield.

[0105] Step 503: If the target area corresponding to the gaze point is the area where the warning information is located, determine the execution unit that is in an abnormal state and the cause of the fault that caused the execution unit to be abnormal.

[0106] Step 504: Zoom in on the icon corresponding to the execution unit and display the cause of the fault.

[0107] Understandably, HUD devices display vehicle status information. During driving, various vehicle statuses require close monitoring, and users frequently check information such as speed, fuel consumption, tire pressure, and engine status. When the user's gaze point corresponds to the vehicle information area, and this area also contains warning messages, the system identifies the execution unit experiencing the abnormal state, the cause of the malfunction, or the displayed error message. For example, if the gaze point corresponds to the engine malfunction icon, the system identifies the engine as the execution unit experiencing the abnormal state, the cause of the malfunction, or the displayed error message. Furthermore, the system enlarges the engine malfunction icon and displays the cause or error message. This allows users to quickly and intuitively obtain key information about vehicle malfunctions without excessive eye strain, enabling them to make timely and appropriate decisions, such as safely stopping the vehicle or contacting roadside assistance, thus ensuring driving safety and preventing more serious consequences from abnormal situations.

[0108] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0109] Reference Figure 6 , Figure 6 This is a schematic diagram of an optional information display device provided in an embodiment of this application. The information display device 6 includes:

[0110] The determining module 601 is used to determine the gaze point position of the vehicle user; wherein, the gaze point position includes: the gaze focus position of the vehicle user in the projection area of ​​the head-up display device, or the gaze focus position of the vehicle user on the external environment information, and the projection area is located in a preset area of ​​the vehicle's windshield.

[0111] The processing module 602 is used to adjust the displayed content in the target area of ​​the projection area associated with the gaze point position according to the gaze point position.

[0112] This application provides a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0113] This application provides a storage medium that stores one or more computer programs, which can be executed by one or more processors to implement some or all of the steps in the above-described method. The storage medium can be transient or non-transient.

[0114] This application provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0115] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0116] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0117] Figure 7 This application provides a hardware entity diagram of a computer device as an embodiment of the present application, such as... Figure 7 As shown, the hardware entity of the computer device 7 includes a processor 701 and a memory 702, wherein the memory 702 stores a computer program that can run on the processor 701, and the processor 701 executes the program to perform the following steps.

[0118] Determine the gaze point position of the vehicle user; wherein, the gaze point position includes: the gaze focus position of the vehicle user in the projection area of ​​the head-up display device, or the gaze focus position of the vehicle user on the external environment information, and the projection area is located in a preset area of ​​the vehicle's windshield.

[0119] Adjust the displayed content within the target area of ​​the projection area associated with the gaze point location, based on the gaze point location.

[0120] The memory 702 stores computer programs that can run on the processor. The memory 702 is configured to store instructions and applications that can be executed by the processor 701. It can also cache data to be processed or already processed by the processor 701 and various modules in the computer device 7 (e.g., image data, audio data, voice communication data and video communication data). It can be implemented by flash memory or random access memory (RAM).

[0121] The processor 701 executes the program to implement any of the above-mentioned information display methods. The processor 701 typically controls the overall operation of the computer device 7.

[0122] This application provides a computer storage medium that stores one or more programs, which can be executed by one or more processors to implement the steps of the information display method as described in any of the above embodiments.

[0123] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0124] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0125] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0126] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0127] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0129] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0130] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0131] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0132] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an in-vehicle terminal (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0133] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An information display method characterized by comprising: Applied to a computer device, the method includes: Determine the gaze point position of the vehicle user; wherein, the gaze point position includes: the gaze focus position of the vehicle user in the projection area of ​​the head-up display device, or the gaze focus position of the vehicle user on the external environment information, wherein the projection area is located within a preset area of ​​the vehicle's windshield. Based on the gaze point position, adjust the displayed content within the target area of ​​the projection area associated with the gaze point position.

2. The method according to claim 1, characterized in that, The gaze point position is the focal point of the vehicle user's gaze on the external environment information. Adjusting the displayed content within the target area of ​​the projection area associated with the gaze point position, based on the gaze point position, includes: Obtain detailed information associated with external environmental information; wherein, the external environmental information includes at least one of external intersection turn instruction information, external public place information, and external service area information; A target area corresponding to the vehicle external environment information is determined from the projection area, wherein the target area includes at least one of a navigation map area, a public place area, and a service area area; The detailed information associated with the vehicle's external environment information in the target area is displayed in a magnified view.

3. The method according to claim 1, characterized in that, The gaze point position is the focal point of the vehicle user's gaze in the projection area of ​​the head-up display device. Adjusting the displayed content within a target area of ​​the projection area associated with the gaze point position, based on the gaze point position, includes: If the target area corresponding to the gaze point is a vehicle information area, the vehicle information is magnified and / or highlighted, and assisted driving information related to the vehicle information is further displayed.

4. The method of claim 3, wherein, The method includes: If the vehicle information area is a vehicle speed display area, zoom in and / or highlight the vehicle speed information in the vehicle speed display area; obtain the speed limit information in the driving scenario, and dynamically display the vehicle speed information and the speed limit information in the vehicle speed display area; If the vehicle information area is a fuel consumption display area, the fuel consumption information in the fuel consumption display area is enlarged; based on the fuel consumption information and the remaining fuel of the vehicle, the mileage of the vehicle is determined; if the mileage is less than a mileage threshold, the nearest gas station to the current location of the vehicle is marked in the navigation map area; wherein, the target area includes the navigation map area.

5. The method of claim 1, wherein, The gaze point position is the focal point of the vehicle user's gaze in the projection area of ​​the head-up display device. Adjusting the displayed content within a target area of ​​the projection area associated with the gaze point position, based on the gaze point position, includes: If the target area corresponding to the gaze point is a navigation map area, the simplified road indication information of the navigation map area is enlarged and displayed. The displayed content includes the simplified road guidance information, which includes one or more of lane guidance, road name, and lane turning guidance. The target area includes the navigation map area.

6. The method of claim 1, wherein, Before determining the gaze point position of the vehicle user, the method includes: In response to an abnormality in the vehicle, a warning message is displayed in the vehicle information area of ​​the projection area; Accordingly, adjusting the displayed content within the target area of ​​the projection region associated with the gaze point position according to the gaze point position includes: If the target area corresponding to the gaze point is the area where the warning information is located, determine the execution unit that is in an abnormal state and the cause of the failure that caused the execution unit to be abnormal; Enlarge the icon corresponding to the execution unit and display the cause of the fault.

7. The method according to any one of claims 1 to 6, characterized in that, Determining the gaze point location of the vehicle user includes: Obtain multiple frames of eye images of the vehicle user; Image analysis was performed on each frame of the eye image to obtain the position of the vehicle user's pupil in each of the eye images; Using the pupil positions corresponding to the multi-frame eye images, the eye movement information of the vehicle user is estimated, wherein the movement information includes the eye movement speed; Based on the motion information and / or the pupil position, eye movement data is obtained; the eye movement data includes: the fixation point position of the eyeball, the fixation duration corresponding to the fixation point position, and the number of fixations.

8. The method of claim 7, wherein, The step of adjusting the displayed content within the target area of ​​the projection area associated with the gaze point position according to the gaze point position includes: Determine whether the parameters corresponding to the gaze point position meet the adjustment conditions. If so, adjust the display content within the target area of ​​the projection area associated with the gaze point position. The adjustment conditions include at least one of the following: when moving to the fixation point, the speed of eye movement is less than a speed threshold, the fixation duration is greater than a duration threshold, and the number of fixations is greater than a number threshold.

9. An information display device, characterized by comprising: The device includes: A determination module is used to determine the gaze point position of a vehicle user; wherein, the gaze point position includes: the gaze focus position of the vehicle user in the projection area of ​​the head-up display device, or the gaze focus position of the vehicle user on information about the external environment, and the projection area is located within a preset area of ​​the vehicle's windshield. The processing module is used to adjust the displayed content within the target area of ​​the projection area associated with the gaze point position according to the gaze point position.

10. A computer device, comprising: The computer device includes: a memory and a processor. The memory stores computer programs that can run on the processor. When the processor executes the program, it implements the information display method as described in any one of claims 1 to 8.

11. A storage medium, characterized in that, The storage medium stores one or more computer programs, which can be executed by one or more processors to implement the information display method as described in any one of claims 1 to 8.

Images