Head-up display system, carrier and control method of head-up display system

By using the combination technology of the pupil tracking module, image source and curved mirror in the head-up display system, multiple visual areas and rotation adjustments are spliced, the problem of large field of view angle design under the space limitations in the vehicle is solved, and a larger Longitudinal field of view and a better viewing experience.

CN118962983AActive Publication Date: 2024-11-15HANGZHOU FERVCLOUD TECHNOLOGY CO LTD

Patent Information

Application Number
CN202411255348.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-15
Estimated Expiration
2044-09-09

AI Technical Summary

Technical Problem

In limited space in the vehicle, it is difficult to design a head-up display system with a large field of view angle. The prior art requires a larger screen and curved mirror, but this violates the limitations of the car space.

Method used

The eye movement data is obtained through the pupil tracking module, and the head-up display device is controlled to splice multiple visual areas in the vertical direction to form a larger longitudinal field of view angle, reduce the volume of the display device, and adjust the rotation of the image source and the curved mirror , achieve a larger field of view angle.

Benefits of technology

It realizes the formation of a larger longitudinal field of view angle in a small display device, avoids cumbersome windshield imaging position adjustment, reduces the sense of mutation in the image seen by the eyes, and improves the viewing experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118962983A_ABST
    Figure CN118962983A_ABST
Patent Text Reader

Abstract

The invention provides a head-up display system, a carrier and a control method of the head-up display system, and relates to the technical field of head-up display. The head-up display system includes: a head-up display device which projects to form an image of one of at least two visible areas, the at least two visible areas including a first visible area and a second visible area; the pupil tracking module is used for tracking eye movement information and acquiring eye movement data; and the controller is electrically connected with the head-up display device and the pupil tracking module, and controls the head-up display device to at least project an image of the first visual area or the second visual area according to the eye movement data. According to the embodiment of the invention, the size of the head-up display device is reduced, and a larger longitudinal field angle is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of head-up display technology, and in particular to a head-up display system, a vehicle, and a control method for the head-up display system. Background Art

[0002] Head up display (HUD) can accurately project the dashboard information and navigation information elements obtained by analyzing the vehicle and road conditions through intelligent driving sensors (such as cameras and radars) onto the vehicle's windshield or imaging components such as displays, so that users can observe the information displayed by the HUD.

[0003] With the improvement of material and cultural level, people have more and more demands for the amount of information presented by HUD virtual images, and the application of super-large virtual image size is becoming more and more widespread. In order to meet the demand for expanding the field of view, the existing technology usually chooses a larger screen or a larger curved mirror to expand the field of view.

[0004] The limited space in the car does not allow for the placement of larger screens and curved mirrors, so it is very difficult to design a large field of view imaging system within the limited space. Summary of the invention

[0005] Embodiments of the present invention provide a head-up display system, a vehicle, and a control method of the head-up display system, so as to reduce the volume of the head-up display device and achieve a larger longitudinal field of view.

[0006] In a first aspect, an embodiment of the present invention provides a head-up display system, including:

[0007] A head-up display device, projecting an image of one of at least two visible areas, the at least two visible areas comprising a first visible area and a second visible area;

[0008] Pupil tracking module, tracking eye movement information and obtaining eye movement data;

[0009] The controller is electrically connected to the head-up display device and the pupil tracking module, and controls the head-up display device to at least project an image of the first visible area or the second visible area according to the eye movement data.

[0010] Optionally, the first visible area and the second visible area are arranged in a set direction.

[0011] Optionally, the first visible area overlaps with the second visible area to form a first overlapping area.

[0012] Optionally, a height of the first overlapping area along the set direction is greater than a first preset height.

[0013] Optionally, at least two of the visible areas further include a third visible area, and along the set direction, the first visible area is located between the second visible area and the third visible area.

[0014] Optionally, the first visible area overlaps with the third visible area to form a second overlapping area.

[0015] Optionally, a height of the second overlapping area along the set direction is greater than a second preset height.

[0016] Optionally, the eye movement data includes pupil states, and the pupil states include pupil upward, pupil mid-position, and pupil downward;

[0017] The third visible area is located below the first visible area;

[0018] When the pupil state is upward, the head-up display device projects to form the second visible area; when the pupil state is in the middle position, the head-up display device projects to form the first visible area; when the pupil state is downward, the head-up display device projects to form the third visible area.

[0019] Optionally, a height of the first visible area along the set direction is greater than a height of the second visible area along the set direction;

[0020] And / or, a height of the first visible area along the vertical direction is greater than a height of the third visible area along the vertical direction.

[0021] Optionally, the head-up display device includes an image source and a curved mirror;

[0022] The image source is used to generate an image light beam and emit the image light beam to the curved mirror;

[0023] The curved mirror is used to rotate according to the eye movement data to a preset angle, so that the head-up display device at least projects an image of the first visible area or the second visible area.

[0024] Optionally, at least two of the visible areas further include a third visible area, the first visible area and the second visible area are arranged in a set direction, and along the set direction, the first visible area is located between the second visible area and the third visible area;

[0025] The angle of the curved mirror when the head-up display device is projected to form the first visible area is a first angle, the angle of the curved mirror when the head-up display device is projected to form the second visible area is a second angle, and the angle of the curved mirror when the head-up display device is projected to form the third visible area is a third angle, and the first angle is between the second angle and the third angle.

[0026] Optionally, the image source includes a display area;

[0027] When projecting to form an image of the first visible area, the first area in the display area generates a first image light beam; when projecting to form an image of the second visible area, the second area in the display area generates a second image light beam;

[0028] The overlap degree between the first region and the second region is greater than a preset overlap degree, wherein the overlap degree is a ratio of an overlapping area between the first region and the second region to a sum of an area of ​​the first region and an area of ​​the second region.

[0029] Optionally, the pupil tracking module includes an optical flow module.

[0030] In a second aspect, an embodiment of the present invention provides a vehicle, comprising the head-up display system described in the first aspect, and a windshield;

[0031] The image light beam emitted by the head-up display device is projected onto the windshield to form an image of at least one of the two visible areas.

[0032] Optionally, the pupil tracking module is arranged on the windshield.

[0033] In a third aspect, an embodiment of the present invention provides a control method of a head-up display system, which is applied to the head-up display device system described in the first aspect, comprising:

[0034] Get eye movement data from the pupil tracking module;

[0035] According to the eye movement data, the head-up display device is controlled to at least project an image of the first visible area or the second visible area.

[0036] In an embodiment of the present invention, the controller controls the head-up display device to at least project an image of the first visible area or the second visible area based on the eye movement data. The first visible area and the second visible area are spliced ​​in the vertical direction to form a larger visible area. In this way, on the one hand, the volume of the head-up display device can be reduced, that is, at least the image of the first visible area or the second visible area can be generated without using a large head-up display device. On the other hand, the first visible area and the second visible area are spliced ​​in the vertical direction to form a larger field of view. Thus, a complete and larger image can be formed in the vertical direction by splicing. A larger longitudinal field of view is thereby achieved. On the other hand, the embodiment of the present invention does not require the cumbersome step of adjusting the imaging position on the windshield, thereby avoiding accidents caused by distraction during driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of a head-up display system provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of a visible area provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of another visual area provided by an embodiment of the present invention;

[0040] Figure 4 A schematic diagram of another visual area provided by an embodiment of the present invention;

[0041] Figure 5 A schematic diagram of another visual area provided by an embodiment of the present invention;

[0042] Figure 6 A schematic diagram of another visual area provided by an embodiment of the present invention;

[0043] Figure 7 A schematic diagram of another head-up display system provided by an embodiment of the present invention;

[0044] Figure 8 A schematic diagram of another visual area provided by an embodiment of the present invention;

[0045] Fig. 9 A schematic diagram of an image source provided by an embodiment of the present invention;

[0046] Fig.10 A schematic diagram of a carrier provided by an embodiment of the present invention;

[0047] Fig.11 A flow chart of a control method for a head-up display system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0049] In order to facilitate the explanation of the solution of the embodiment of the present invention, the embodiment of the present invention provides a possible application scenario using the head-up display system applied to a car as an example. Those skilled in the art should understand that the head-up display system of the embodiment of the present invention can also be applied to, for example, sanitation vehicles, fire trucks, military vehicles, and of course, can also be applied to the fields of ships, aviation, etc. For example, it can be applied to aircraft such as fighter jets, so that the driver can track and aim at objects with the assistance of the head-up display system.

[0050] The head-up display system usually transmits the image light to a projection medium, such as a specially made screen in front of the driver, or directly reflects it to a suitable position of the windshield of the car, and the image light after being reflected by the projection medium enters the eye box range, thereby forming a target virtual image that can be observed within the eye box range in front of the driver. In the exemplary implementation of the embodiment of the present invention, the windshield is used as an example of the projection medium that finally reflects the image light to the human eye.

[0051] The area where the target virtual image generated by the head-up display system is located is the visible area. In related technologies, a larger screen (i.e., image source) and a curved mirror are usually set up to achieve a larger target virtual image, a larger visible area, and a larger longitudinal field of view. However, the limitation of the vehicle space does not allow the placement of a larger screen and a curved mirror, so it is very difficult to design a large field of view imaging system in a limited space.

[0052] It should be further explained that there is a rotating reflective element in the related art to change the position of the light projected into the pupil of the eye, so as to adapt to the movement of the eye position and / or the movement of the pupil position. No matter where the target virtual image is observed, the complete image needs to be observed. Therefore, the target virtual image generated by the head-up display system is a complete image. In order to achieve this complete image and achieve a larger field of view, a larger screen (i.e., image source) and a curved mirror need to be set. Or conversely, in order to achieve this complete image without setting a larger screen (i.e., image source) and a curved mirror, the field of view needs to be reduced.

[0053] Figure 1 A schematic diagram of a head-up display system provided by an embodiment of the present invention, Figure 2 A schematic diagram of a visual area provided by an embodiment of the present invention, referring to Figure 1 and Figure 2The head-up display system includes a head-up display device 100, a pupil tracking module 300 and a controller ( Figure 1 The head-up display device 100 projects an image of at least one of the two visible areas 200. The at least two visible areas 200 include a first visible area 210 and a second visible area 220. The pupil tracking module 300 tracks eye movement information and obtains eye movement data. The controller is electrically connected to the head-up display device 100 and the pupil tracking module 300. The controller controls the head-up display device 100 to project an image of the first visible area 210 or the second visible area 220 according to the eye movement data.

[0054] It should be noted that in the embodiment of the present invention, the at least two visible areas 200 including the first visible area 210 and the second visible area 220 are used as an example for explanation, and the number of visible areas is not limited. When the at least two visible areas 200 include the first visible area 210 and the second visible area 220, the head-up display device 100 projects an image of the first visible area 210 or the second visible area 220. When the head-up display device 100 projects to form N visible areas, the head-up display device 100 projects to form an image of one of the N visible areas. Wherein, N is a positive integer greater than 2. When the head-up display device 100 projects to form N visible areas, the head-up display device 100 at least projects to form an image of the first visible area 210 or the second visible area 220. Images of other visible areas can also be projected. In short, the head-up display device 100 projects to form an image of one of the at least two visible areas 200.

[0055] In an embodiment of the present invention, the controller controls the head-up display device 100 to project an image of the first visible area 210 or the second visible area 220 according to the eye movement data. The first visible area 210 and the second visible area 220 are spliced ​​in the vertical direction to form a larger visible area. In this way, on the one hand, the volume of the head-up display device 100 can be reduced, that is, the image of the first visible area 210 or the second visible area 220 can be generated without using a large head-up display device 100. On the other hand, the first visible area 210 and the second visible area 220 are spliced ​​in the vertical direction to form a larger field of view. Thus, a complete and larger image can be formed in the vertical direction by splicing. And then a larger longitudinal field of view is achieved. On the other hand, the embodiment of the present invention does not need to perform the cumbersome step of adjusting the imaging position on the windshield 500, so as to avoid accidents caused by distraction during driving.

[0056] Optionally, the first visible area 210 and the second visible area 220 are arranged in a set direction. Figure 1 and Figure 2As shown, the embodiment of the present invention is explained by taking the setting direction including the vertical direction as an example. However, it is not limited to this. In other embodiments, the setting direction may also include the horizontal direction or a certain inclined direction. For simplicity, the various embodiments of the present invention are explained by taking the setting direction including the vertical direction as an example, and the first visible area 210 and the second visible area 220 are arranged in the vertical direction.

[0057] The vertical direction is the vertical direction commonly understood, that is, the direction of gravity. Usually, the vertical direction is perpendicular to the direction of the line connecting the two eyes.

[0058] Optionally, refer to Figure 1 and Figure 2 , the first visible area 210 and the second visible area 220 overlap to form a first overlapping area 212. Therefore, when switching between the images displayed in the first visible area 210 and the second visible area 220, the images displayed in the first visible area 210 and the second visible area 220 both include the image displayed in the first overlapping area 212. This reduces the sense of sudden change in the image seen by the eyes and improves the viewing experience.

[0059] For example, reference Figure 2 , the first visible area 210 includes a first sub-visible area 211 and a first overlapping area 212. The second visible area 220 includes a second sub-visible area 221 and a first overlapping area 212. The first overlapping area 212 is located between the first sub-visible area 211 and the second sub-visible area 221. The second sub-visible area 221, the first overlapping area 212 and the first sub-visible area 211 are arranged in the vertical direction. The visible area formed by the first visible area 210 and the second visible area 220 spliced ​​in the vertical direction includes the second sub-visible area 221, the first overlapping area 212 and the first sub-visible area 211. The visible area formed by the first visible area 210 and the second visible area 220 spliced ​​in the vertical direction is larger than the first visible area 210, and larger than the second visible area 220. Thereby achieving a larger longitudinal field of view angle.

[0060] Optionally, refer to Figure 2 , the height of the first overlapping area 212 in the vertical direction is H12, and H12 is greater than the first preset height. If the height H12 of the first overlapping area 212 is relatively small, when switching between the images displayed by the first visible area 210 and the second visible area 220, the image seen by the eyes will have a relatively large sense of sudden change. In this embodiment of the present invention, H12 is controlled to be greater than the first preset height to control the height H12 of the first overlapping area 212 not to be too small. Thus, when switching between the images displayed by the first visible area 210 and the second visible area 220, the sense of sudden change of the image seen by the eyes is reduced.

[0061] Figure 3 A schematic diagram of another visual area provided by an embodiment of the present invention, referring to Figure 3 , the edge of the first visible area 210 is aligned with the edge of the second visible area 220. The first visible area 210 and the second visible area 220 are spliced ​​in the vertical direction. The first visible area 210 and the second visible area 220 do not have an overlapping area.

[0062] Figure 4 A schematic diagram of another visual area provided by an embodiment of the present invention, referring to Figure 4 , at least two visible areas include a first visible area 210, a second visible area 220 and a third visible area 230. In the vertical direction, the first visible area 210 is located between the second visible area 220 and the third visible area 230. The controller is electrically connected to the head-up display device 100 and the pupil tracking module 300. The controller controls the head-up display device 100 to project an image of the first visible area 210, the second visible area 220 or the third visible area 230 according to the eye movement data. In the embodiment of the present invention, the results of the pupil gaze point obtained by analysis are divided into three categories, corresponding to three different visible areas.

[0063] It is understandable that the more the number of divided visible areas is, the more computing resources are required when switching. On the other hand, the more the number of divided visible areas is, the smaller the volume of the head-up display device 100 can be set. Alternatively, a larger longitudinal field of view angle is achieved while keeping the volume of the head-up display device 100 unchanged. Alternatively, the volume of the head-up display device 100 is reduced while achieving a larger longitudinal field of view angle.

[0064] In one embodiment, the number of divided visible areas is sufficient to achieve stepless switching of the visible areas.

[0065] Optionally, refer to Figure 4 , the first visible area 210 overlaps with the third visible area 230 to form a second overlapping area 213. The first visible area 210 includes a first overlapping area 212, a first sub-visible area 211, and a second overlapping area 213. The third visible area 230 includes a third sub-visible area 231 and a second overlapping area 213. The second overlapping area 213 is located between the first sub-visible area 211 and the third sub-visible area 231. The second sub-visible area 221, the first overlapping area 212, the first sub-visible area 211, the second overlapping area 213, and the third sub-visible area 231 are arranged in a vertical direction. The visible area formed by the first visible area 210, the second visible area 220, and the third visible area 230 in the vertical direction is larger than the first visible area 210, larger than the second visible area 220, and larger than the third visible area 230. Thus, a larger longitudinal field of view angle is achieved.

[0066] Optionally, refer to Figure 4 , the height of the second overlapping area 213 in the vertical direction is H13, and H13 is greater than the second preset height. If the height H13 of the second overlapping area 213 is relatively small, when switching between the images displayed by the first visible area 210 and the third visible area 230, the image seen by the eyes will have a relatively large sense of sudden change. In this embodiment of the present invention, H13 is controlled to be greater than the second preset height to control the height H13 of the second overlapping area 213 not to be too small. Thus, when switching between the images displayed by the first visible area 210 and the third visible area 230, the sense of sudden change of the image seen by the eyes is reduced.

[0067] Exemplarily, the first preset height is equal to the second preset height. In other embodiments, the first preset height may be greater than or less than the second preset height.

[0068] Figure 5 A schematic diagram of another visual area provided by an embodiment of the present invention, referring to Figure 5 , the edge of the first visible area 210 is aligned with the edge of the second visible area 220. The edge of the first visible area 210 is aligned with the edge of the third visible area 230. The first visible area 210, the second visible area 220 and the third visible area 230 do not have an overlapping area.

[0069] Figure 6 A schematic diagram of another visual area provided by an embodiment of the present invention, referring to Figure 6 , the first visible area 210 overlaps with the second visible area 220 to form a first overlapping area 212. The edge of the first visible area 210 is aligned with the edge of the third visible area 230. There is no overlapping area between the first visible area 210, the second visible area 220 and the third visible area 230. In other embodiments, the first visible area 210 and the second visible area 220 may not overlap, and the first visible area 210 and the third visible area 230 overlap to form an overlapping area.

[0070] Figure 7 A schematic diagram of another head-up display system provided by an embodiment of the present invention, with reference to Figure 1 , Figure 4 and Figure 7 The eye movement data includes pupil states, and the pupil states include pupil upward, pupil middle, and pupil downward. In one embodiment, when the eye 410 changes the gaze point in the eye socket in the vertical direction, the pupil 420 moves in the vertical direction, so that the pupil 420 has three positions, and the states represented by these three positions are pupil upward, pupil middle, and pupil downward.

[0071] The third visible area 230 is located below the first visible area 210, and the second visible area 220 is located above the first visible area 210. When the pupil state is that the pupil is facing upward, the head-up display device 100 projects to form the second visible area 220, and the eye 410 can see the image in the second visible area 220 at an upper position. When the pupil state is the middle position of the pupil, the head-up display device 100 projects to form the first visible area 210, and the eye 410 can see the image in the first visible area 210 at a middle position. When the pupil state is that the pupil is facing downward, the head-up display device 100 projects to form the third visible area 230, and the eye 410 can see the image in the third visible area 230 at a lower position.

[0072] For example, reference Figure 7 , the visible area corresponding to the pupil state when the pupil is facing upward overlaps with the visible area corresponding to the pupil state when the pupil is facing downward. That is, the second visible area 220 overlaps with the third visible area 230. In other embodiments, the second visible area 220 and the third visible area 230 may not overlap.

[0073] Figure 8 A schematic diagram of another visual area provided by an embodiment of the present invention, referring to Figure 8 When the pupil state is upward, the visible areas formed by the head-up display device 100 are area 1 and area 2. When the pupil state is in the middle position, the visible areas formed by the head-up display device 100 are area 2, area 3 and area 4. When the pupil state is downward, the visible areas formed by the head-up display device 100 are area 4 and area 5.

[0074] Combined with reference Figure 4 and Figure 7 , the height of the first visible area 210 along the vertical direction is H21, the height of the second visible area 220 along the vertical direction is H22, and the height of the third visible area 230 along the vertical direction is H23. H21 is greater than H22, and / or H21 is greater than H23. Since the eyes 410 look straight ahead most of the time, the first visible area 210 at the middle position can be made larger, and the second visible area 220 at the upper position and the third visible area 230 at the lower position can be made smaller.

[0075] Combined with reference Figure 4 and Figure 7The head-up display device 100 includes an image source 101 and a curved mirror 102. The image source 101 is used to generate an image beam and emit the image beam to the curved mirror 102. The curved mirror 102 is used to rotate according to the eye movement data to a preset angle, so that the head-up display device 100 projects an image of at least one of the two visible areas. For example, the image in the first visible area 210, the second visible area 220, or the third visible area 230 is projected.

[0076] Exemplarily, the head-up display device 100 may further include a curved mirror servo motor, which is connected to the curved mirror 102. The curved mirror servo motor is also electrically connected to the controller, and is used to drive the curved mirror 102 to rotate under the control of the controller.

[0077] Combined with reference Figure 4 and Figure 7 , the angle of the curved mirror 102 when the head-up display device 100 projects to form the first visible area 210 is the first angle. The angle of the curved mirror 102 when the head-up display device 100 projects to form the second visible area 220 is the second angle. The angle of the curved mirror 102 when the head-up display device 100 projects to form the third visible area 230 is the third angle. The first angle is between the second angle and the third angle. Corresponding to the pupil states of the pupil upward, the pupil mid-position, and the pupil downward, the second angle, the first angle, and the third angle gradually increase or decrease, and there will be no angle jump.

[0078] Fig. 9 A schematic diagram of an image source provided by an embodiment of the present invention, referring to Figure 1 , Figure 7 and Fig. 9, the image source 101 includes a display area 110. When projecting an image of the first visible area 210, the first area 111 in the display area 110 generates a first image beam. The image formed by the first image beam in the eye 410 corresponds to the first visible area 210. When projecting an image of the second visible area 220, the second area 112 in the display area 110 generates a second image beam. The image formed by the second image beam in the eye 410 corresponds to the second visible area 220. The overlap between the first area 111 and the second area 112 is greater than a preset overlap, wherein the overlap between the first area 111 and the second area 112 is the ratio of the overlapping area of ​​the first area 111 and the second area 112 to the sum of the areas of the first area 111 and the second area 112. In the embodiment of the present invention, when the head-up display device 100 projects to form the first visible area 210 or the second visible area 220, the overlap between the first area 111 and the second area 112 of the image source 101 is greater than the preset overlap, so as to control the majority of pixels in the image source 101 for display, that is, control most of the area of ​​the display area 110 in the image source 101 for display, thereby improving display effects such as resolution and brightness.

[0079] In one embodiment, the first area 111 overlaps the second area 112. When the head-up display device 100 projects to form the first visible area 210 or the second visible area 220, all pixels in the image source 101 are displayed, that is, the entire area of ​​the display area 110 in the image source 101 is controlled to display, further improving display effects such as resolution and brightness.

[0080] For example, reference Figure 1 , Figure 7 and Fig. 9 When projecting an image of the third visible area 230, the third area 113 in the display area 110 generates a third image beam. The image formed by the third image beam in the eye 410 corresponds to the third visible area 230. The overlap between the first area 111 and the third area 113 is greater than a preset overlap.

[0081] In general, optical flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between the observer (eye or camera) and the scene. An optical flow module is a vision sensor that is capable of measuring optical flow or visual motion and outputs measurements based on optical flow. Figure 1 or Figure 7 , the pupil tracking module 300 includes an optical flow module ( Figure 1 and Figure 7 (not shown) to increase the operating speed of the pupil tracking module 300.

[0082] Fig.10 A schematic diagram of a carrier provided by an embodiment of the present invention, referring to Fig.10, the vehicle includes the head-up display system in the above embodiment and a windshield 500. The image beam emitted by the head-up display device 100 is projected onto the windshield 500 to form an image of at least one of the two visible areas. For example, in one imaging, the image beam emitted by the head-up display device 100 is projected onto the windshield 500 to form an image in the first visible area 210. In another imaging, the image beam emitted by the head-up display device 100 is projected onto the windshield 500 to form an image in the first visible area 210.

[0083] Optionally, refer to Figure 1 , Figure 7 and Fig.10 , the pupil tracking module 300 is disposed on the windshield 500. Compared with disposing the pupil tracking module 300 on the housing of the head-up display device 100, the accuracy of pupil tracking can be improved.

[0084] For example, reference Figure 1 , Figure 7 and Fig.10 The pupil tracking module 300 is disposed at the upper end of the windshield 500, where the field of vision is not blocked. It is understood that the image beam emitted by the head-up display device 100 is projected onto the windshield 500, and the area on the windshield 500 corresponding to the image beam is the projection area. The pupil tracking module 300 can be disposed at the upper end of the projection area.

[0085] Based on the same technical concept, an embodiment of the present invention further provides a control method of a head-up display system. The control method is applied to the head-up display device system in the above embodiment and can be executed by the controller in the above embodiment. Fig.11 A flow chart of a control method of a head-up display system provided by an embodiment of the present invention, referring to Figure 1-Figure 11 , the control method comprises:

[0086] S101. Acquire eye movement data from a pupil tracking module.

[0087] S102: Control the head-up display device to at least project an image of the first visible area or the second visible area according to the eye movement data.

[0088] In an embodiment of the present invention, eye movement data is obtained from a pupil tracking module. Based on the eye movement data, the head-up display device is controlled to at least project an image of the first visible area or the second visible area. On the one hand, the volume of the head-up display device 100 can be reduced, that is, the image of the first visible area 210 or the second visible area 220 can be generated without using a large head-up display device 100. On the other hand, the first visible area 210 and the second visible area 220 are spliced ​​in the vertical direction to form a larger field of view. Thus, a complete and larger image can be formed in the vertical direction by splicing. Thereby achieving a larger longitudinal field of view.

[0089] In one embodiment, the eye movement data includes pupil states, and the pupil states include pupil upward, pupil middle, and pupil downward. According to the eye movement data, the curved mirror is controlled to rotate to a preset angle. When the pupil state is pupil upward, the head-up display device 100 projects to form a second visual area 220, and the eye 410 can see the image in the second visual area 220 in an upper position. When the pupil state is pupil middle, the head-up display device 100 projects to form a first visual area 210, and the eye 410 can see the image in the first visual area 210 in the middle position. When the pupil state is pupil downward, the head-up display device 100 projects to form a third visual area 230, and the eye 410 can see the image in the third visual area 230 in a lower position.

[0090] There is a 5-degree area in the center of the human eye's (i.e., eye) field of view, which is the visual center of the human eye. Due to the existence of the fovea, this area is the highest-definition part of the human eye's field of view. We name the sunken part of the human eye's macula as the fovea, which contains the highest concentration of cone photoreceptors and is the highest focus area of ​​the human eye. The high concentration of cones makes the focus of this part of the eye clearer, and the fovea is also the main part of human color vision. Eye tracking technology (EyeTracking Object Detection, ETOD) is an eye tracking target detection method based on foveal visual area mapping. Its function is to accurately find the object information of the user's real focus area, that is, according to the movement state of the human eye, detect the category of the object being looked at, and filter out other irrelevant information. It is an end-to-end eye tracking target detection method. The model is based on the YOLOv4 network, and the CSPANet network is designed to achieve feature fusion. At the same time, the multi-task loss function is used to train the network, so as to achieve targeted target detection in the sight area. YOLOv is the abbreviation of You Only Look Once. YOLOv4 is its fourth generation version.

[0091] In one embodiment of the present invention, an end-to-end eye tracking target detection method is designed to track two types of human eye movements: eye movement in the central area of ​​the pupil and smooth trailing tracking, and instantaneously transmit eye movement signals to a controller. The controller sends an adjustment signal based on the eye movement signal to a signal receiver on the curved mirror servo motor to read and process the adjustment signal, thereby controlling the curved mirror to make corresponding rotations and project a virtual image on the corresponding area on the windshield.

[0092] Among them, saccades and smooth pursuit movement are two different human eye movements. Saccades refer to the sudden change of the gaze point or gaze direction of the human eye (usually occurring when a person is scanning), and smooth pursuit movement refers to the gaze point following the movement of an object (usually occurring when a person is looking at a moving object).

[0093] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, combinations and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A head-up display system, characterized in that: include: A head-up display device, projecting an image of one of at least two visible areas, the at least two visible areas comprising a first visible area and a second visible area; Pupil tracking module, tracking eye movement information and obtaining eye movement data; The controller is electrically connected to the head-up display device and the pupil tracking module, and controls the head-up display device to at least project an image of the first visible area or the second visible area according to the eye movement data.

2. The head-up display system according to claim 1, characterized in that: The first visible area and the second visible area are arranged in a set direction.

3. The head-up display system according to claim 2, characterized in that: The first visible area overlaps with the second visible area to form a first overlapping area.

4. The head-up display system according to claim 3, characterized in that: A height of the first overlapping area along the set direction is greater than a first preset height.

5. The head-up display system according to claim 2, characterized in that: At least two of the visible areas further include a third visible area, and along the set direction, the first visible area is located between the second visible area and the third visible area.

6. The head-up display system according to claim 5, characterized in that: The first visible area overlaps with the third visible area to form a second overlapping area.

7. The head-up display system according to claim 6, characterized in that: A height of the second overlapping area along the set direction is greater than a second preset height.

8. The head-up display system according to claim 7, characterized in that: The eye movement data includes pupil states, and the pupil states include pupil upward, pupil middle, and pupil downward; The third visible area is located below the first visible area; When the pupil state is upward, the head-up display device projects to form the second visible area; when the pupil state is in the middle position, the head-up display device projects to form the first visible area; when the pupil state is downward, the head-up display device projects to form the third visible area.

9. The head-up display system according to claim 7, characterized in that: The height of the first visible area along the set direction is greater than the height of the second visible area along the set direction; And / or, a height of the first visible area along the set direction is greater than a height of the third visible area along the set direction.

10. The head-up display system according to claim 1, characterized in that: The head-up display device includes an image source and a curved mirror; The image source is used to generate an image light beam and emit the image light beam to the curved mirror; The curved mirror is used to rotate according to the eye movement data to a preset angle, so that the head-up display device at least projects an image of the first visible area or the second visible area.

11. The head-up display system according to claim 10, characterized in that: At least two of the visible areas further include a third visible area, the first visible area and the second visible area are arranged in a set direction, and along the set direction, the first visible area is located between the second visible area and the third visible area; The angle of the curved mirror when the head-up display device is projected to form the first visible area is a first angle, the angle of the curved mirror when the head-up display device is projected to form the second visible area is a second angle, and the angle of the curved mirror when the head-up display device is projected to form the third visible area is a third angle, and the first angle is between the second angle and the third angle.

12. The head-up display system according to claim 10, characterized in that: The image source includes a display area; When projecting to form an image of the first visible area, the first area in the display area generates a first image light beam; when projecting to form an image of the second visible area, the second area in the display area generates a second image light beam; The overlap degree between the first region and the second region is greater than a preset overlap degree, wherein the overlap degree is a ratio of an overlapping area between the first region and the second region to a sum of an area of ​​the first region and an area of ​​the second region.

13. The head-up display system according to claim 1, characterized in that: The pupil tracking module includes an optical flow module.

14. A carrier, characterized in that: A head-up display system comprising any one of claims 1 to 13, and a windshield; The image light beam emitted by the head-up display device is projected onto the windshield to form an image of at least one of the two visible areas.

15. The carrier according to claim 14, characterized in that: The pupil tracking module is arranged on the windshield.

16. A method for controlling a head-up display system, applied to the head-up display system according to any one of claims 1 to 13, characterized in that: include: Get eye movement data from the pupil tracking module; According to the eye movement data, the head-up display device is controlled to at least project an image of the first visible area or the second visible area.

Citation Information

Patent Citations

  • Projection position determination method and device, vehicle-mounted terminal and vehicle

    CN117761904A

  • Head-up display device, method, and computer program

    WO2021065698A1

  • Display control device, head-up display device, and display control method

    WO2023003045A1

Cited By

  • Head-up display system, vehicle, and control method for head-up display system

    WO2026051314A1