Head-up display system control method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application disclose a head-up display system control method, device, equipment and medium. The method comprises: acquiring image information comprising eyes of a user, to obtain an eye region image based on the image information; acquiring an actual projection position of a head-up display system, to establish a line-of-sight point array corresponding to the eye region image based on the actual projection position; determining pupil position information of the user based on the line-of-sight point array, and acquiring a mapping relationship between the pupil position information and the actual projection position; if a change in the pupil position information is monitored, determining a target projection position of the head-up display system based on the changed pupil position information and the mapping relationship, and controlling a reflection angle of a mirror of the head-up display system based on the target projection position. Embodiments of the present application can continuously maintain the effectiveness and accuracy of information display by dynamically adjusting the projection position, adapt to various use scenarios and user states, improve the user experience, and improve driving safety.

Description

Technical Field

[0001] This application relates to the field of virtual image generation technology, specifically to a head-up display system control method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the development of technology, the main way to display information on the windshield is through head-up displays (HUDs). Existing augmented reality head-up displays (ARHUDs) assist drivers in driving by displaying information on the head-up display.

[0003] In current driving scenarios, head-up display (HUD) technology has been widely adopted in the automotive industry. It aims to improve driving safety and convenience by projecting key driving information onto the windshield, reducing the number of times drivers need to look down at the instrument panel. While the display position of a HUD system is typically fixed, which meets the basic needs of most drivers, different heights, seating positions, and driving environments can lead to some drivers not being able to clearly see the displayed information, thus affecting driving safety and convenience. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide a head-up display system control method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of the embodiments of this application, a head-up display system control method is provided, comprising: acquiring image information including a user's eyes, to acquire an eye region image based on the image information; acquiring the actual projection position of the head-up display system, to establish a gaze dot matrix corresponding to the eye region image based on the actual projection position; determining the user's pupil position information based on the gaze dot matrix, and acquiring a mapping relationship between the pupil position information and the actual projection position; if a change in the pupil position information is detected, determining a target projection position of the head-up display system based on the changed pupil position information and the mapping relationship, and controlling the reflection angle of the reflector of the head-up display system based on the target projection position.

[0006] According to one aspect of the embodiments of this application, the step of establishing a gaze dot matrix corresponding to the eye region image based on the actual projection position includes: establishing a first dot matrix coordinate system in the eye region image based on the actual projection position; extracting a target peri-eye region from the eye region image based on the first dot matrix coordinate system; establishing a second dot matrix coordinate system in the target peri-eye region based on the actual projection position, and adjusting the second dot matrix coordinate system based on the target dot matrix density to obtain the corresponding gaze dot matrix.

[0007] According to one aspect of the embodiments of this application, the step of extracting a target peri-eye region from the eye region image based on the first dot matrix coordinate system includes: obtaining a target human eye model, and determining candidate peri-eye regions from the eye region image based on the target human eye model; extracting multiple peri-eye coordinate data from the candidate peri-eye regions based on the first dot matrix coordinate system; and determining the target peri-eye region based on the multiple peri-eye coordinate data.

[0008] According to one aspect of the embodiments of this application, the method further includes: extracting pupil features in the target peri-ocular region; obtaining multiple peri-ocular coordinate data corresponding to the pupil features in the second dot matrix coordinate system; and determining pupil position information in the second dot matrix coordinate system based on the multiple peri-ocular coordinate data.

[0009] According to one aspect of the embodiments of this application, the method further includes: filtering the acquired multiple coordinate data to obtain filtered coordinate data; calculating the difference between the multiple coordinate data and the filtered coordinate data; and replacing the coordinate data in the multiple coordinate data whose difference with the filtered coordinate data reaches or exceeds a preset difference threshold with the filtered coordinate data.

[0010] According to one aspect of the embodiments of this application, before adjusting the second dot matrix coordinate system based on the target dot matrix density to obtain the corresponding line-of-sight dot matrix, the method further includes: acquiring vehicle operating condition information and driver information, the driver information including driver identity information and personalized information corresponding to the driver identity information; determining the target dot matrix density based on the operating condition information and the personalized information, so as to determine the second dot matrix coordinate system through the target dot matrix density.

[0011] According to one aspect of the embodiments of this application, the method further includes: acquiring a calibration sample set, the calibration sample set including multiple pupil position information and target projection positions corresponding to each of the multiple pupil position information; establishing a mapping relationship between the pupil position information and the target projection positions based on the multiple pupil position information and the multiple target projection positions; acquiring the user's operation information, the operation information including the user's operation information on the head-up display system; and correcting the mapping relationship based on the operation information to obtain a mapping relationship between the pupil position information and the actual projection position.

[0012] According to one aspect of the embodiments of this application, a head-up display system control device is provided, comprising: an acquisition module for acquiring image information including a user's eyes, and acquiring an eye region image based on the image information; an establishment module for acquiring the actual projection position of the head-up display system, and establishing a gaze dot matrix corresponding to the eye region image based on the actual projection position; a determination module for determining the user's pupil position information based on the gaze dot matrix, and acquiring a mapping relationship between the pupil position information and the actual projection position; and a control module for determining a target projection position of the head-up display system based on the changed pupil position information and the mapping relationship if a change in the pupil position information is detected, and controlling the reflection angle of the reflector of the head-up display system based on the target projection position.

[0013] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the head-up display system control method as described above.

[0014] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the head-up display system control method as described above.

[0015] According to one aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in the head-up display system control method as described above.

[0016] In the technical solutions provided in the embodiments of this application, by capturing and analyzing images of the user's eye region, especially pupil position information, the projection position of the head-up display system can be precisely adjusted. This makes the information display more in line with the user's viewing habits and comfort, providing a personalized driving or information display experience. Real-time monitoring of changes in pupil position information ensures that the displayed content of the head-up display system is always within the user's optimal field of vision, reducing distraction or information omissions caused by gaze deviation, thereby improving driving or operational safety. On the other hand, since the user's eye position may change due to head movement, posture changes, or physiological differences among different users, dynamically adjusting the projection position can continuously maintain the effectiveness and accuracy of information display, adapting to various usage scenarios and user states, enhancing the user experience, and improving driving safety.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0019] Figure 1 This is a schematic diagram illustrating an implementation environment for controlling a head-up display system during vehicle operation, as shown in an exemplary embodiment of this application.

[0020] Figure 2 This is a flowchart illustrating a head-up display system control method in an exemplary embodiment of this application;

[0021] Figure 3 This is a flowchart illustrating a head-up display system control method in another exemplary embodiment of this application;

[0022] Figure 4 This is a schematic diagram of a human eye image in a first dot matrix coordinate system, as shown in an exemplary embodiment.

[0023] Figure 5 This is a schematic diagram of a human eye image in a second dot matrix coordinate system, as shown in an exemplary embodiment.

[0024] Figure 6 This is a flowchart illustrating a head-up display system control method in another exemplary embodiment of this application;

[0025] Figure 7This is a flowchart illustrating a head-up display system control method in another exemplary embodiment of this application;

[0026] Figure 8 This is a flowchart illustrating a head-up display system control method in another exemplary embodiment of this application;

[0027] Figure 9 This is a schematic diagram illustrating data update after data filtering, as shown in an exemplary embodiment;

[0028] Figure 10 This is a flowchart illustrating a head-up display system control method in another exemplary embodiment of this application;

[0029] Figure 11 This is a flowchart illustrating a head-up display system control method in another exemplary embodiment of this application;

[0030] Figure 12 This is a simplified flowchart illustrating the control of a head-up display system in an exemplary application scenario.

[0031] Figure 13 This is a block diagram illustrating a head-up display system control device according to an exemplary embodiment of this application;

[0032] Figure 14 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0034] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0037] HUD (Head-Up Display) technology is a type of display technology. Its main function is to project key driving information (such as vehicle speed, navigation instructions, and warning messages) directly into the driver's line of sight, typically onto the windshield or a transparent display screen. This allows the driver to access this information without looking down at the dashboard or center console screen, significantly reducing the potential risks associated with shifting their gaze and improving driving safety.

[0038] HUD technology projects images or data onto a transparent interface in front of the driver's or pilot's line of sight using optical means, such as a car's windshield or an aircraft's cockpit window. This allows the driver or pilot to see the necessary information directly while maintaining a normal line of sight, without having to look down at instruments or displays. The core components of a HUD include: an image generator responsible for generating the image or data to be projected, typically using a projection source such as a liquid crystal display (LCD) or a digital light processing (DLP) display; an optical lens system that magnifies and zooms the image generated by the image generator to ensure it is clearly presented in front of the driver; this system usually includes a series of lenses and mirrors that work together to adjust the image's focal length, orientation, and size; a reflection unit that reflects the image processed by the optical lens system into the driver's line of sight; in automotive HUD systems, this is usually achieved through the windshield, and the reflection unit typically requires precise design and adjustment based on the windshield's characteristics to ensure clear image display; and a projection unit that projects the image generated by the image generator onto the optical lens system, typically using a projector or similar device.

[0039] Figure 1 This is a schematic diagram illustrating an implementation environment for controlling a head-up display system during vehicle operation, as shown in an exemplary embodiment of this application. Figure 1As shown, during vehicle operation, the image acquisition device corresponding to the smart terminal 110 acquires image information including the user's eyes, and obtains an image of the user's eye region based on this image information. Then, the server 120 acquires the actual projection position of the vehicle's head-up display system and obtains the eye region image on the smart terminal 110. Then, it establishes a gaze dot matrix corresponding to the eye region image based on the actual projection position. The server 120 can then determine the user's pupil position information based on the gaze dot matrix and obtain the mapping relationship between the pupil position information and the actual projection position. The server 120 continuously monitors the user's pupil position. If a change in the pupil position information is detected, the server determines the target projection position of the head-up display system based on the changed pupil position information and the mapping relationship. The server then controls the reflection angle of the head-up display system's mirror based on the target projection position, thereby achieving tracking of the user's pupil position and updating the projection position of the head-up display system.

[0040] in, Figure 1 The smart terminal 110 shown can be any terminal device that supports image acquisition and processing, such as a smartphone, in-vehicle computer, tablet computer, laptop computer, or wearable device, but is not limited to these. Figure 1 The server 120 shown is a server, which can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. No restrictions are placed on this. The smart terminal 110 can communicate with the server 120 via wireless networks such as 3G (third-generation mobile information technology), 4G (fourth-generation mobile information technology), and 5G (fifth-generation mobile information technology). No restrictions are placed on this as well.

[0041] In current driving scenarios, head-up display (HUD) technology has been widely adopted in the automotive industry. It aims to improve driving safety and convenience by projecting key driving information onto the windshield, reducing the number of times drivers need to look down at the instrument panel. While the display position of a HUD system is typically fixed, which meets the basic needs of most drivers, different heights, seating positions, and driving environments can lead to some drivers not being able to clearly see the displayed information, thus affecting driving safety and convenience.

[0042] The problems mentioned above are universally applicable in common travel scenarios. It can be seen that the projection position of existing head-up display systems is relatively fixed and cannot adapt to changes in the user's line of sight, resulting in a poor user experience. To solve these problems, embodiments of this application propose a head-up display system control method, a head-up display system control device, an electronic device, a computer-readable storage medium, and a computer program product, which will be described in detail below.

[0043] Please see Figure 2 , Figure 2 This is a flowchart illustrating a head-up display system control method in an exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown is specifically executed by server 120 within that implementation environment. It should be understood that this method can also be applied to other exemplary implementation environments and executed by devices in other implementation environments; this embodiment does not limit the implementation environment to which the method is applicable.

[0044] like Figure 2 As shown, in an exemplary embodiment, the head-up display system control method includes at least steps S210 to S240, which are described in detail below:

[0045] Step S210: Obtain image information including the user's eyes, so as to obtain an eye region image based on the image information.

[0046] Inside the vehicle cabin, high-resolution cameras are typically chosen, or images of the user's eyes can be acquired directly from the HUD system's built-in image acquisition device (camera). These cameras need to have good optical performance and image processing capabilities to ensure they can capture clear and accurate images of the eye area. The camera's installation location needs to be carefully selected to ensure it can capture the user's eye area while avoiding obstruction or interference from other objects in the cabin. Subsequently, image processing algorithms can extract the eye area from the captured image and perform further processing and analysis. These algorithms typically include image preprocessing, feature extraction, and object detection steps to achieve accurate identification of the eye area.

[0047] The process involves using cameras installed inside the vehicle cabin to capture images of the user's eyes. These images need to contain sufficient information for subsequent processing and analysis. Preprocessing of the captured images includes steps such as noise reduction, contrast enhancement, and brightness adjustment to improve image quality and clarity. Image processing algorithms are then applied to detect the eye region in the images. This typically involves using specific feature extraction algorithms, such as edge detection and circle fitting, to identify the shape and position of the eyes. After detecting the eye region, image processing techniques are used to extract it from the original image. This ensures that subsequent processing and analysis can focus on the eye region, improving accuracy and efficiency.

[0048] Step S220: Obtain the actual projection position of the head-up display system, and establish a gaze dot matrix corresponding to the eye region image based on the actual projection position.

[0049] First, ensure the HUD system is correctly installed and calibrated. This typically involves adjusting the HUD's height, brightness, and contrast, as well as performing horizontal and vertical calibrations to ensure the projected information corresponds to actual road conditions. These adjustments can be made through the vehicle's settings menu or physical buttons. During adjustment, observe the HUD's projection position, ensuring it is parallel to the driver's line of sight and located in the lower-middle part of the windshield. To more accurately determine the HUD's actual projection position, test patterns or images can be used. These patterns or images may contain easily identifiable markers or lines to define the boundaries of the projection area on the windshield.

[0050] By observing the projection positions of these marked points or lines on the windshield, the actual projection position of the HUD can be accurately determined. After obtaining the actual projection position of the HUD, the eye area needs to be identified. This is typically done by capturing an image of the driver's face with a camera and using image processing algorithms to identify the position and shape of the eye area.

[0051] The eye region can be defined as a rectangular area (or a vertical strip corresponding to a one-dimensional shutter) surrounding a given eye, with the distance between the eyes in the horizontal direction. Based on the position and shape of the eye region, a gaze dot matrix can be constructed. The gaze dot matrix consists of a series of points representing the gaze direction, and these points can be arranged and adjusted according to the position and shape of the eye region.

[0052] The position and size of the shutter pupil can be determined based on binocular parallax 3D display methods. A line is drawn from the right edge of any image display pixel to the left edge of the eye region, and another line is drawn from the left edge of the same image display pixel to the right edge of the same eye region. The intersection of these two lines defines an imaging pinhole plane. Then, based on the lateral distance between the intersection points on the imaging pinhole plane, the shutter screen position is finely adjusted longitudinally to determine the position and size of the shutter pupil.

[0053] Furthermore, establishing the gaze dot matrix may require consideration of multiple factors, such as individual driver differences, seat position, and windshield tilt angle. Therefore, in practical applications, dynamic adjustment and calibration of the gaze dot matrix may be necessary. After establishing the gaze dot matrix, verification and optimization are required. This can be done by having the driver view a test pattern or image on the HUD and observing whether the gaze dot matrix accurately reflects the driver's gaze direction. If deviations or errors are found in the gaze dot matrix, optimization can be achieved by adjusting the camera position, image processing algorithm, or the arrangement of the gaze dot matrix.

[0054] Step S230: Determine the user's pupil position information based on the gaze dot matrix, and obtain the mapping relationship between the pupil position information and the actual projection position.

[0055] For example, gaze tracking technology can utilize the pupil-corneal reflection method, which relies on the location of the Pulcim spot (generated by the reflection of light entering the pupil on the corneal surface) on the outer surface of the cornea and the center of the pupil. By locating the positions of the pupil and Pulcim spot on an eye image in real time, the corneal reflection vector can be calculated, and then a geometric model can be used to estimate the user's gaze direction. In an established gaze dot matrix, each point represents a possible gaze direction. When a user gazes at a location on the HUD, the user's current gaze direction can be determined by detecting the positions of the pupil and Pulcim spot, and their changes relative to the gaze dot matrix.

[0056] Furthermore, a corresponding point can be found in the gaze dot matrix based on the direction of gaze. This point represents the user's current pupil position information (relative to the HUD projection position). In order to establish the mapping relationship between the pupil position information and the actual projection position, a calibration process is usually required. During the calibration process, the user is asked to look at specific points on the HUD (these points are usually pre-set projection points with known coordinates). At the same time, the system records the pupil position information (relative to the gaze dot matrix) when the user looks at these points.

[0057] In step S240, if a change in pupil position information is detected, the target projection position of the head-up display system is determined based on the changed pupil position information and the mapping relationship, so as to control the reflection angle of the reflector of the head-up display system based on the target projection position.

[0058] For example, a head-up display (HUD) system first monitors the user's pupil position. This is typically achieved through a built-in camera or sensor that captures the user's eye position, particularly the pupil position, in real time. The pupil position changes as the user moves their head, eyes, or body. This change may be subtle but sufficient to affect the HUD's projection. Once a change in pupil position is detected, the HUD uses this information, combined with a previously established mapping (the correspondence between pupil position and projection position), to determine a new target projection position. This mapping may be established through a pre-calibration process to ensure the projected content is accurately positioned at the user's line of sight. After determining the target projection position, the HUD adjusts the reflection angle of its internal mirrors. The mirrors are a key component of the HUD system, responsible for reflecting the projection light source into the user's line of sight. By adjusting the angle of the reflector, the direction of the projected light can be changed, thereby ensuring that the projected content appears accurately at the target projection position. The whole process is dynamic, meaning that the head-up display system continuously monitors the pupil position information and adjusts the projection position and the angle of the reflector in real time as needed. In this way, no matter how the user moves, the head-up display system can ensure that the projected content always stays in the user's line of sight, providing a clear and comfortable viewing experience.

[0059] In some embodiments of this application, by capturing and analyzing images of the user's eye region, particularly pupil position information, the projection position of the head-up display system can be precisely adjusted. This makes the information display more in line with the user's viewing habits and comfort, providing a personalized driving or information display experience. Real-time monitoring of pupil position information ensures that the displayed content of the head-up display system is always within the user's optimal field of vision, reducing distraction or information omissions caused by gaze deviation, thereby improving driving or operational safety. On the other hand, since the user's eye position may change due to head movement, posture changes, or physiological differences among different users, dynamically adjusting the projection position can continuously maintain the effectiveness and accuracy of the information display, adapting to various usage scenarios and user states, enhancing the user experience, and improving driving safety.

[0060] Based on the above embodiments, please refer to Figure 3 In one exemplary embodiment provided in this application, the specific implementation process of establishing the gaze dot matrix corresponding to the eye region image based on the actual projection position may further include steps S310 to S330, which are described in detail below:

[0061] Step S310: Establish a first dot matrix coordinate system in the eye region image based on the actual projection position.

[0062] For example, a first dot matrix coordinate system can be established in the eye region image using the actual projection position of the head-up display system at the current moment, as can be seen in [reference needed]. Figure 4 The first dot matrix coordinate system is used to obtain the relative spatial position of the user's eyes with respect to the head-up display system. In other words, the position information corresponding to the user's eye area can be represented by coordinates in the first dot matrix coordinate system.

[0063] Optionally, in a head-up display (HUD) system, the actual projection position refers to the specific location of the image or information formed by the projected light in front of the user's eyes. This position can be monitored by sensors or cameras in the HUD system and compared with the user's pupil position to ensure that the projected content is at the user's focal point. The eye area image is typically captured by a camera built into the HUD system. This image includes the user's eyes and the surrounding area, and forms the basis for establishing a bit-matrix coordinate system. After determining the actual projection position and acquiring the eye area image, the next step is to establish a bit-matrix coordinate system on this image, i.e., the first bit-matrix coordinate system. This coordinate system is used to describe and locate various points on the image, especially those related to the actual projection position.

[0064] A bitmap coordinate system typically consists of a set of mutually perpendicular lines (i.e., coordinate axes), with points on these lines (i.e., coordinate points) representing specific locations on the image. The first bitmap coordinate system may be established based on specific features of the eye region (such as the pupil center, corner of the eye, etc.) to ensure its accuracy and stability. Once established, this first bitmap coordinate system can be used to precisely describe and locate the position of the projected content on the image within the eye region. This helps the HUD system more accurately adjust the projection position and the angle of the reflector to ensure that the projected content remains in the user's focal point.

[0065] Furthermore, in some feasible embodiments, the first bit coordinate system can also be used for other image processing tasks, such as image enhancement, feature extraction, and image recognition. These tasks help the HUD system provide a clearer and more comfortable viewing experience, and enhance the overall performance and reliability of the system.

[0066] Step S320: Extract the target peri-eye region from the eye region image based on the first dot matrix coordinate system.

[0067] Step S330: Establish a second dot matrix coordinate system in the target peri-eye region based on the actual projection position, and adjust the second dot matrix coordinate system based on the target dot matrix density to obtain the corresponding line-of-sight dot matrix.

[0068] like Figure 4 As shown, multiple coordinate points near the user's eyes can be determined using the first dot matrix coordinate system. Then, the user's target peri-eye region can be extracted based on these coordinate points. For example, by introducing a "dot matrix" onto the captured image, a horizontal and vertical coordinate system roughly equivalent to the height of the human eye (approximately one centimeter in width) is established for the first-level coarse matching of eye tracking, outputting the approximate location of the eyes. The matching method will incorporate the Euclidean distance algorithm, a commonly used method for measuring the direct distance between two points in space. This algorithm boasts advantages such as high computational efficiency and strong stability, enabling the rapid calculation of the point in the dot matrix closest to the user's eye with relatively low performance overhead. Finally, the user's target peri-eye region is extracted based on this closest point.

[0069] For example, these points form a grid or frame for locating and extracting specific image regions. Here, the "eye region image" is an image containing the eye and its surrounding area. By applying a first bitmap coordinate system, the part of interest, the "target peri-eye region," can be precisely located and extracted. This region typically includes the eye and its surrounding skin, eyelashes, etc. Next, based on the actual projection position (which could be the center of the eyeball, the position of the pupil, or other reference points), a new "second bitmap coordinate system" is established within the extracted target peri-eye region.

[0070] like Figure 5 As shown, this new second lattice coordinate system is used to further refine the analysis of the periocular region, especially to capture the direction of gaze. Then, this second lattice coordinate system is adjusted according to the target lattice density. The lattice density in the target lattice density refers to the number of points per unit area. By adjusting this density, the features of the periocular region can be captured and analyzed more finely or coarsely. Finally, the adjusted second lattice coordinate system is used to generate a "gaze lattice," which is a lattice representation that reflects the direction of gaze.

[0071] Optionally, a predefined first lattice coordinate system is used to accurately extract the target peri-eye region from the eye region image. This region contains key features of the eye and its surroundings. Then, within this extracted target peri-eye region, a new second lattice coordinate system is established based on the actual projection position (e.g., the center of the pupil). This new coordinate system is used for further analysis of the peri-eye region, particularly to capture the direction of gaze. To obtain more accurate gaze information, the second lattice coordinate system is adjusted according to the target lattice density. The adjusted lattice coordinate system more accurately reflects the features of the peri-eye region, thereby generating a gaze lattice that represents the direction of gaze.

[0072] In some embodiments of this application, the target peri-eye region can be extracted more effectively from the eye region image based on the first dot matrix coordinate system. This extraction method not only improves the recognition rate of the peri-eye region but also reduces unnecessary image processing time, optimizes peri-eye region extraction, and flexibly adjusts the gaze dot matrix. Furthermore, it can more accurately respond to changes in the user's gaze, providing HUD display content that better meets the user's needs. This not only improves the user's driving safety but also enhances the user's overall satisfaction and comfort.

[0073] Based on the above embodiments, please refer to Figure 6 In one exemplary embodiment provided in this application, the specific implementation process of extracting the target peri-eye region from the eye region image based on the first dot matrix coordinate system may further include steps S610 to S630, which are described in detail below:

[0074] Step S610: Obtain the target human eye model, and determine the candidate peri-eye region from the eye region image based on the target human eye model;

[0075] Step S620: Extract multiple peri-ocular coordinate data from the candidate peri-ocular region based on the first dot matrix coordinate system;

[0076] Step S630: Determine the target periorbital region based on multiple periorbital coordinate data.

[0077] For example, localization can be achieved by detecting the shape features of the eyes. For instance, potential eye regions can be extracted from an eye region image using multiple standard eye models, and the geometric features of these eye regions can be calculated. Then, the extracted geometric features are matched with a predefined target human eye model to obtain candidate eye peri-regions. Multiple eye peri-region coordinate data are then extracted from the candidate eye peri-regions based on a first dot matrix coordinate system. Finally, the target eye peri-region for the user is determined based on the extracted multiple eye peri-region coordinate data, and the eye region image of the target eye peri-region is extracted.

[0078] Optional, a target eye model. This model can be a general model built based on average facial features or a model customized for a specific individual. It contains structural and shape information about the eye and its surrounding area. With the target eye model, it can be compared or matched with an image of the eye region. Through this process, the region in the image that may contain the eye, i.e., the "candidate peri-eye region," can be roughly determined. This region may contain some additional background or noise, but it is close enough to the real peri-eye region. Next, a predefined "first-level coordinate system" is applied to the candidate peri-eye region. This coordinate system consists of a series of uniformly distributed points that cover the entire candidate peri-eye region. By sampling these points, multiple peri-eye coordinate data can be extracted. These data represent pixel values ​​or feature information at different locations within the candidate peri-eye region. With this peri-eye coordinate data, various image processing and analysis techniques (such as edge detection, morphological operations, machine learning algorithms, etc.) can be used to further process and filter this data. Finally, a more accurate "target peri-eye region" that is closer to the real peri-eye region can be determined from this data. This area typically includes the eye itself and the surrounding skin area adjacent to the eye.

[0079] In some embodiments of this application, candidate peri-ocular regions can be quickly selected from eye region images based on the target human eye model. This selection method reduces unnecessary image processing steps and improves overall processing efficiency. By using the first dot matrix coordinate system to extract multiple peri-ocular coordinate data from the candidate peri-ocular regions, the accuracy of the coordinate data can be ensured.

[0080] Based on the above embodiments, please refer to Figure 7 In one exemplary embodiment provided in this application, the specific implementation process of the above-mentioned head-up display system control method may further include steps S710 to S730, which are described in detail below:

[0081] Step S710: Extract pupil features from the target periorbital region;

[0082] Step S720: Obtain multiple pupil perimeter coordinate data corresponding to the pupil features in the second dot matrix coordinate system;

[0083] Step S730: Determine the pupil position information in the second lattice coordinate system based on multiple pupil circumference coordinate data.

[0084] For example, the second dot matrix coordinate system is used to capture the user's pupil, that is, the change of the user's gaze. Therefore, after obtaining the target eye area from the first dot matrix coordinate system, a coordinate system can be established in the target eye area and set as a more refined gaze dot matrix coordinate system. The position of the pupil is the direct determining factor of the gaze direction. In this frame, the positional details of the pupil will be magnified, and the subtle movements of the pupil can be captured with higher precision. The pupil position is matched with the dot matrix coordinates by the Euclidean distance algorithm, thereby outputting higher precision gaze position information.

[0085] Optionally, from the target peri-eye region, the pupil's features must first be extracted. These features may include the pupil's outline, color, brightness, etc., depending on the image processing techniques and algorithms used. Within the established second-level coordinate system, multiple peri-pupil coordinate data corresponding to the pupil features need to be found. These coordinate data describe the locations of key points or boundaries around the pupil. Based on the multiple peri-pupil coordinate data obtained in the previous step, the specific location information of the pupil in the second-level coordinate system can be further calculated or inferred. This may involve techniques such as statistical analysis of the coordinate data, geometric calculations, or pattern recognition.

[0086] Within the target peri-eye region, a more refined gaze-tracking coordinate system (i.e., the second coordinate system) is established. This coordinate system is designed to improve the accuracy of pupil position detection by capturing subtle pupil movements through denser coordinate points. In this refined gaze-tracking coordinate system, the pupil's orientation becomes the direct determinant of the gaze direction. To capture these subtle pupil movements, the pupil's positional details are magnified, which may involve techniques such as image scaling and interpolation. Using mathematical tools such as the Euclidean distance algorithm, the pupil's position in the gaze-tracking coordinate system is matched with preset coordinates. This step aims to accurately determine the pupil's current position, providing foundational data for subsequent gaze tracking.

[0087] In some embodiments of this application, by extracting pupil features in the periorbital region, the pupil position can be located more accurately, which can improve the accuracy of pupil recognition, reduce the possibility of false recognition, and improve the effectiveness of tracking the user's gaze.

[0088] Based on the above embodiments, please refer to Figure 8 In one exemplary embodiment provided in this application, the specific implementation process of the above-mentioned head-up display system control method may further include steps S810 to S830, which are described in detail below:

[0089] Step S810: Filter the acquired coordinate data to obtain filtered coordinate data;

[0090] Step S820: Calculate the difference between the multiple coordinate data and the filtered coordinate data;

[0091] Step S830: Replace the coordinate data in the multiple coordinate data whose difference with the filtered coordinate data reaches or exceeds a preset difference threshold with the filtered coordinate data.

[0092] For example, in the process of determining the shortest distance point in the above embodiments, it is inevitable to encounter situations where the actual measured distances to multiple coordinate points are close, and even slight changes can cause changes in the position of the acquisition point, ultimately causing the output HUD display position to fluctuate frequently within a small range. To avoid this situation, the acquired coordinate data can be filtered to obtain filtered data. Then, the difference between the multiple coordinate data and the filtered coordinate data can be calculated separately, and the coordinate data whose difference with the filtered coordinate data reaches a preset difference threshold can be replaced with the filtered coordinate data to achieve a smooth transition among the multiple coordinate data and avoid the occurrence of display position fluctuations.

[0093] Optionally, a suitable filtering algorithm is selected to process the coordinate data. The choice of filtering algorithm depends on the characteristics of the data and the desired result. Common filtering methods include moving average filtering, Gaussian filtering, and Kalman filtering. The selected filtering algorithm is applied to obtain filtered coordinate data. This filtered data is usually smoother than the original data, reducing noise and abrupt changes. For each original coordinate data point, the difference between it and the corresponding filtered coordinate data point is calculated. This typically involves calculating the difference between each coordinate component (e.g., x and y). These differences are summarized so that they can be used in subsequent steps to determine which original data points need to be replaced. A preset difference threshold is set. This threshold determines which original data points are considered significantly different from the filtered data points and therefore need to be replaced. All original coordinate data points are iterated through, and it is checked whether their differences with the filtered data points exceed the preset threshold. For those original data points whose differences exceed the threshold, they are replaced with the corresponding filtered data points. This ensures that significant noise or abrupt changes in the data are smoothed out while preserving most of the original features of the data.

[0094] Optionally, in some implementable embodiments, such as Figure 9As shown, a moving average filtering algorithm can be used to filter multiple coordinate data collected to obtain filtered coordinate data. The earliest acquired data can be removed from the multiple coordinate data based on the filtered coordinate data, and then the filtered coordinate data can be added to the multiple coordinate data. Alternatively, the average value of all data in the multiple coordinate data can be used as the actual calculated value and then used in the calculation of the shortest distance point in the above embodiment. This smoothing process can minimize the impact of jitter on the final output value.

[0095] In this embodiment, filtering removes noise and outliers from the original coordinate data, resulting in smoother and more accurate coordinate data. This helps improve the accuracy and reliability of subsequent processing. By calculating the difference between the original coordinate data and the filtered coordinate data and replacing coordinate data with differences exceeding a preset threshold, fluctuations and instabilities in the data can be further reduced, which helps ensure data stability and consistency.

[0096] Based on the above embodiments, please refer to Figure 10 In one exemplary embodiment provided in this application, before adjusting the second dot matrix coordinate system based on the target dot matrix density to obtain the corresponding line-of-sight dot matrix, the specific implementation process of the head-up display system control method may further include steps S1010 and S1020, which are described in detail below:

[0097] Step S1010: Obtain vehicle operating condition information and driver information. The driver information includes driver identity information and personalized information corresponding to the driver identity information.

[0098] Step S1020: Determine the target lattice density based on working condition information and personalized information, so as to determine the second lattice coordinate system through the target lattice density.

[0099] For example, driver identity information includes basic information such as the driver's name, gender, and age, while personalized information may involve the driver's driving habits (such as preferred speed, acceleration, and braking habits), eye level, and information reading speed. This information helps the system better understand the driver's needs, thereby providing more personalized services. A driver's personalized needs may change with increased driving time. For instance, as driver fatigue increases, the driver may require a simpler and clearer information display, thus requiring less precision in eye-tracking. In this case, the target dot density can be set lower. For example, when driving at low speeds in congested urban areas, or when the driver primarily relies on other sensors and displays for information, the HUD system can set a lower target dot density. A lower dot density means the information display may be simpler, reducing detail, but still meeting basic driving needs. For drivers with less driving experience, they may rely more on the HUD system for assistance, thus requiring higher precision information display. In this case, setting a higher target dot density is reasonable, as it provides more detailed and intuitive information, helping novice drivers adapt to and understand vehicle status more quickly. In other words, the HUD system should be able to dynamically adjust according to the driver's real-time status and needs, providing more personalized services.

[0100] In some embodiments of this application, by acquiring the driver's identity information and its corresponding personalized information (such as driving habits, preference settings, etc.), the actual needs of the driver can be matched more accurately. This personalized matching helps improve the driving experience and makes the driving assistance system more in line with the driver's usage habits. By acquiring vehicle operating condition information (such as vehicle speed, engine status, steering angle, etc.), the vehicle's driving status can be monitored in real time, and the parameters of the driving assistance system can be adjusted according to this information. This real-time adjustment helps ensure that the driving assistance system can provide the best effect under different operating conditions, thereby enhancing driving safety.

[0101] Based on the above embodiments, please refer to Figure 11 In one exemplary embodiment provided in this application, the specific implementation process of the above-mentioned head-up display system control method may further include steps S1110 to S1140, which are described in detail below:

[0102] Step S1110: Obtain a calibration sample set, which includes multiple pupil position information and the target projection position corresponding to each of the multiple pupil position information;

[0103] Step S1120: Establish a mapping relationship between pupil position information and target projection position based on multiple pupil position information and multiple target projection positions;

[0104] Step S1130: Obtain user operation information, including user operation information on the head-up display system;

[0105] Step S1140: Based on the operation information, correct the mapping relationship to obtain the mapping relationship between the pupil position information and the actual projection position.

[0106] For example, using a camera or other sensor in a HUD system, pupil position information of the user in different head poses is captured. This typically includes the X and Y coordinates of the pupil in the screen or image coordinate system. For each pupil position, a desired projection position, i.e., the target projection position, is manually or automatically set. This can be a specific point or region on the HUD image. Using data from a calibrated sample set, a machine learning model (such as a regression model, neural network, etc.) is trained to learn the mapping relationship between pupil position information and the target projection position. The accuracy of the model can be evaluated using cross-validation or other validation methods to ensure that the mapping relationship performs well even on unseen data.

[0107] The HUD system's sensors and interface continuously monitor user actions, such as head movements and eye gaze. User action information is recorded, including action type, timestamp, and related pupil position information. Based on this information, user satisfaction and feedback regarding the HUD projection position are analyzed. If user feedback indicates inaccurate mapping, incremental or online learning from a machine learning model is used to update the mapping. This can be achieved by introducing new training data (i.e., pupil position information from user actions and the actual projection position). The corrected mapping is then validated: after correction, validation methods are used again to ensure accuracy and consistency.

[0108] Optionally, in some feasible embodiments, the position of the human eye may vary due to differences in driver height, driving habits, and driving posture. The initial position of the human eye has a relatively small impact on the perceived quality of the HUD imaging effect. The "human eye dot matrix" can be divided into several points with lower precision for calibration, for example, calibration every 5 centimeters. The calibration process requires the calibration tester to actually sit in the vehicle and change postures to position their eyes differently, simulating various driving postures a driver might adopt. Simultaneously, a professional calibration tool is used to calibrate all adjustable hardware parameters of the HUD as variable parameters of the tool, allowing for real-time changes to the various angle parameters of the HUD.

[0109] Furthermore, calibration testers need to simulate various driver postures, positioning the driver's eyes sequentially at each calibration point in the "dot matrix." Then, calibration tools are used to optimize the HUD's display, recording the calibrated projection angle information. The calibration of the "line-of-sight dot matrix" requires full coverage, and even increases in coordinate density if accuracy is insufficient. For example, since the HUD changes its display position by altering the projection angle, during calibration, the HUD image is adjusted to the most suitable viewing position using these two dot matrix combinations, recording the projection angle information as (-30, 120). These two dot matrix combinations are then linked to the projection angle. In actual operation, when the driver's line-of-sight position matches this, the HUD projection will adjust to the (-30, 120) angle position, achieving the same optimal display effect as during calibration.

[0110] In some feasible embodiments of this application, a calibration sample set is obtained and a mapping relationship between pupil position information and target projection position is established. By correcting the mapping relationship, this embodiment can be personalized according to user preferences and driving habits, thereby improving the user experience.

[0111] Figure 12 This is a simplified flowchart illustrating the control of a head-up display system in an exemplary application scenario. In the application scenario shown in Figure 12, image information including the user's eyes is acquired to obtain an eye region image; the actual projection position of the head-up display system is acquired to establish a gaze dot matrix corresponding to the eye region image based on the actual projection position; the user's pupil position information is determined based on the gaze dot matrix, and the mapping relationship between the pupil position information and the actual projection position is obtained; a first dot matrix coordinate system is established in the eye region image based on the actual projection position; the target peri-eye region is extracted from the eye region image based on the first dot matrix coordinate system; a second dot matrix coordinate system is established in the target peri-eye region based on the actual projection position, and the second dot matrix coordinate system is adjusted based on the target dot matrix density to obtain the corresponding gaze dot matrix. If a change in pupil position information is detected, the target projection position of the head-up display system is determined based on the changed pupil position information and the mapping relationship, and the reflection angle of the head-up display system's mirror is controlled based on the target projection position. For detailed implementation processes, please refer to the descriptions in the aforementioned embodiments; they will not be repeated here.

[0112] Figure 13 This is a block diagram illustrating a heads-up display system control device according to an exemplary embodiment of this application. This device can be applied to… Figure 1 The implementation environment shown is specifically configured in the smart terminal 210. This device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0113] like Figure 13 As shown, the exemplary head-up display system control device includes: an acquisition module 1310, which acquires image information including the user's eyes to obtain an eye region image based on the image information; an establishment module 1320, which acquires the actual projection position of the head-up display system to establish a gaze dot matrix corresponding to the eye region image based on the actual projection position; a determination module 1330, which determines the user's pupil position information based on the gaze dot matrix and obtains the mapping relationship between the pupil position information and the actual projection position; and a control module 1340, which, if a change in pupil position information is detected, determines the target projection position of the head-up display system based on the changed pupil position information and the mapping relationship, and controls the reflection angle of the reflector of the head-up display system based on the target projection position.

[0114] According to one aspect of the embodiments of this application, the above-mentioned establishment module 1320 further includes: establishing a first dot matrix coordinate system based on the actual projection position in the eye region image; extracting the target peri-eye region from the eye region image based on the first dot matrix coordinate system; establishing a second dot matrix coordinate system based on the actual projection position in the target peri-eye region, and adjusting the second dot matrix coordinate system based on the target dot matrix density to obtain the corresponding gaze dot matrix.

[0115] According to one aspect of the embodiments of this application, the above-mentioned establishment module 1320 further includes: acquiring a target human eye model, and determining candidate peri-eye regions from the eye region image based on the target human eye model; extracting multiple peri-eye coordinate data from the candidate peri-eye regions based on a first dot matrix coordinate system; and determining the target peri-eye region based on the multiple peri-eye coordinate data.

[0116] According to one aspect of the embodiments of this application, the above-mentioned establishment module 1320 further includes: extracting pupil features in the target peri-eye region; obtaining multiple peri-pupil coordinate data corresponding to the pupil features in the second dot matrix coordinate system; and determining the pupil position information in the second dot matrix coordinate system based on the multiple peri-pupil coordinate data.

[0117] According to one aspect of the embodiments of this application, the head-up display system control device further includes: a filtering module, which performs filtering processing on the acquired multiple coordinate data to obtain filtered coordinate data; calculates the difference between the multiple coordinate data and the filtered coordinate data; and replaces the coordinate data in the multiple coordinate data whose difference with the filtered coordinate data reaches or exceeds a preset difference threshold with the filtered coordinate data.

[0118] According to one aspect of the embodiments of this application, the above-mentioned establishment module 1320 further includes: acquiring vehicle operating condition information and driver information, the driver information including driver identity information and personalized information corresponding to the driver identity information; determining the target dot matrix density based on the operating condition information and personalized information, so as to determine the second dot matrix coordinate system through the target dot matrix density.

[0119] According to one aspect of the embodiments of this application, the head-up display system control device further includes: a calibration module, which acquires a calibration sample set, the calibration sample set including multiple pupil position information and target projection positions corresponding to each of the multiple pupil position information; establishes a mapping relationship between the pupil position information and the target projection positions based on the multiple pupil position information and the multiple target projection positions; acquires user operation information, the operation information including user operation information on the head-up display system; and corrects the mapping relationship based on the operation information to obtain a mapping relationship between the pupil position information and the actual projection positions.

[0120] The head-up display system control device and the head-up display system control method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the head-up display system control device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0121] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the head-up display system control method provided in the above embodiments.

[0122] Figure 14 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Figure 14 The computer system 1400 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0123] like Figure 14As shown, the computer system 1400 includes a Central Processing Unit (CPU) 1401, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 1402 or programs loaded from Storage Unit 1408 into Random Access Memory (RAM) 1403. The RAM 1403 also stores various programs and data required for system operation. The CPU 1401, ROM 1402, and RAM 1403 are interconnected via a bus 1404. An Input / Output (I / O) interface 1405 is also connected to the bus 1404.

[0124] The following components are connected to I / O interface 1405: an input section 1406 including a keyboard, mouse, etc.; an output section 1407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1408 including a hard disk, etc.; and a communication section 1409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to I / O interface 1405 as needed. Removable media 1411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1410 as needed so that computer programs read from them can be installed into storage section 1408 as needed.

[0125] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1409, and / or installed from removable medium 1411. When the computer program is executed by central processing unit (CPU) 1401, it performs various functions defined in the system of this application.

[0126] The computer-readable medium shown in the embodiments of this application may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0129] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the head-up display system control method described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0130] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the head-up display system control method provided in the various embodiments described above.

[0131] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A control method for a head-up display system, characterized in that, include: Acquire image information including the user's eyes, and obtain an image of the eye region based on the image information; Obtain the actual projection position of the head-up display system, and establish a gaze dot matrix corresponding to the eye region image based on the actual projection position; Based on the gaze dot matrix, determine the user's pupil position information, and obtain the mapping relationship between the pupil position information and the actual projection position; If a change in the pupil position information is detected, the target projection position of the head-up display system is determined based on the changed pupil position information and the mapping relationship, so as to control the reflection angle of the reflector of the head-up display system based on the target projection position; The step of establishing the gaze dot matrix corresponding to the eye region image based on the actual projection position includes: A first dot matrix coordinate system is established in the eye region image based on the actual projection position; The target peri-eye region is extracted from the eye region image based on the first dot matrix coordinate system; A second dot matrix coordinate system is established in the target peri-eye region based on the actual projection position, and the second dot matrix coordinate system is adjusted based on the target dot matrix density to obtain the corresponding line-of-sight dot matrix.

2. The method as described in claim 1, characterized in that, Extracting the target peri-eye region from the eye region image based on the first dot matrix coordinate system includes: Obtain a target human eye model, and determine candidate peri-eye regions from the eye region image based on the target human eye model; Multiple peri-ocular coordinate data are extracted from the candidate peri-ocular region based on the first dot matrix coordinate system; The target periorbital region is determined based on the aforementioned multiple periorbital coordinate data.

3. The method as described in claim 1, characterized in that, The method further includes: Extract pupil features from the target periorbital region; Obtain multiple pupillary coordinate data corresponding to the pupil feature in the second dot matrix coordinate system; The pupil position information in the second dot matrix coordinate system is determined based on the multiple pupil perimeter coordinate data.

4. The method as described in any one of claims 2 and 3, characterized in that, The method further includes: The acquired coordinate data is filtered to obtain the filtered coordinate data. Calculate the difference between the plurality of coordinate data and the filtered coordinate data; Replace the coordinate data in the plurality of coordinate data whose difference with the filtered coordinate data reaches or exceeds a preset difference threshold with the filtered coordinate data.

5. The method as described in claim 1, characterized in that, Before adjusting the second lattice coordinate system based on the target lattice density to obtain the corresponding line-of-sight lattice, the method further includes: Obtain vehicle operating condition information and driver information, wherein the driver information includes driver identity information and personalized information corresponding to the driver identity information; The target dot matrix density is determined based on the operating condition information and the personalized information, and the second dot matrix coordinate system is determined by the target dot matrix density.

6. The method as described in claim 1, characterized in that, The method further includes: Obtain a calibration sample set, which includes multiple pupil position information and the target projection position corresponding to each of the multiple pupil position information; A mapping relationship between the pupil position information and the target projection position is established based on the multiple pupil position information and the multiple target projection positions; Obtain the user's operation information, which includes the user's operation information regarding the head-up display system; Based on the operation information, the mapping relationship is corrected to obtain the mapping relationship between the pupil position information and the actual projection position.

7. A head-up display system control device, characterized in that, The device includes: The acquisition module acquires image information including the user's eyes, and acquires an image of the eye region based on the image information; A module is established to obtain the actual projection position of the head-up display system, and to establish a gaze dot matrix corresponding to the eye region image based on the actual projection position; and to establish a first dot matrix coordinate system in the eye region image based on the actual projection position; to extract the target peri-eye region from the eye region image based on the first dot matrix coordinate system; to establish a second dot matrix coordinate system in the target peri-eye region based on the actual projection position, and to adjust the second dot matrix coordinate system based on the target dot matrix density to obtain the corresponding gaze dot matrix; The determination module determines the user's pupil position information based on the gaze dot matrix and obtains the mapping relationship between the pupil position information and the actual projection position; If the control module detects a change in the pupil position information, it determines the target projection position of the head-up display system based on the changed pupil position information and the mapping relationship, and controls the reflection angle of the reflector of the head-up display system based on the target projection position.

8. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the head-up display system control method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores computer-readable instructions, which, when executed by the computer's processor, cause the computer to perform the head-up display system control method according to any one of claims 1 to 6.

Images