Head-up display system, vehicle, and control method for head-up display system

Abstract

A head-up display system, a vehicle, and a control method for the head-up display system. The head-up display system comprises: a head-up display apparatus (100), for projecting to form an image of one of at least two visible areas (200), the at least two visible areas (200) including a first visible area (210) and a second visible area (220); a pupil tracking module (300), for tracking eye movement information and acquiring eye movement data; and a controller, which is electrically connected to the head-up display apparatus (100) and the pupil tracking module (300), and used for controlling, on the basis of the eye movement data, the head-up display apparatus (100) to at least project to form an image of the first visible area (210) or the second visible area (220).

Description

Head-up display system, vehicle and control method of head-up display system

[0001] This application claims priority to the Chinese patent application No. 202411255348.5, filed on September 9, 2024, with the Chinese Patent Office, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of head-up display, for example, to a head-up display system, a vehicle and a control method of the head-up display system. BACKGROUND

[0003] A head-up display (HUD) can project instrument panel information, navigation information elements, etc. obtained by analyzing a vehicle and a road condition through sensors (such as a camera and a radar) of intelligent driving of the vehicle onto a windshield glass or an imaging component of a display, so that a user can observe information displayed by the HUD.

[0004] With the improvement of material and cultural level, people have more and more demand for the amount of information presented by a HUD virtual image, and super-large virtual image size is applied more and more widely. In order to meet the demand of expanding a field of view, a related technology usually selects a larger screen or a larger curved mirror to realize expansion of the field of view.

[0005] The limitation of a car machine space does not allow a larger screen and a curved mirror to be placed, so it is very difficult to design a large field of view imaging system in a limited space. SUMMARY

[0006] Embodiments of the present application provide a head-up display system, a vehicle and a control method of the head-up display system, to realize reduction of a volume of a head-up display device and realization of a larger longitudinal field of view.

[0007] In a first aspect, embodiments of the present application provide a head-up display system, comprising:

[0008] A head-up display device projects an image of at least one of at least two visual regions, the at least two visual regions comprising a first visual region and a second visual region;

[0009] A pupil tracking module tracks eye movement information and obtains eye movement data;

[0010] A controller is electrically connected to the head-up display device and the pupil tracking module, and controls the head-up display device to project an image of at least the first visual region or the second visual region according to the eye movement data.

[0011] Optionally, the first visual region and the second visual region are arranged in a set direction.

[0012] Optionally, the first visual area and the second visual area overlap to form a first overlap area.

[0013] Optionally, a height of the first overlap area along the set direction is greater than a first preset height.

[0014] Optionally, the at least two visual areas further comprise a third visual area, and the first visual area is located between the second visual area and the third visual area along the set direction.

[0015] Optionally, the first visual area and the third visual area overlap to form a second overlap area.

[0016] Optionally, a height of the second overlap area along the set direction is greater than a second preset height.

[0017] Optionally, the eye movement data comprises an eye pupil state, and the eye pupil state comprises an upward eye pupil, a middle eye pupil and a downward eye pupil.

[0018] The third visual area is located below the first visual area.

[0019] When the eye pupil state is the upward eye pupil, the head-up display device projects to form the second visual area; when the eye pupil state is the middle eye pupil, the head-up display device projects to form the first visual area; and when the eye pupil state is the downward eye pupil, the head-up display device projects to form the third visual area.

[0020] Optionally, a height of the first visual area along the set direction is greater than a height of the second visual area along the set direction.

[0021] And / or, a height of the first visual area along the vertical direction is greater than a height of the third visual area along the vertical direction.

[0022] Optionally, the head-up display device comprises an image source and a curved mirror.

[0023] The image source is configured to generate an image light beam and emit the image light beam to the curved mirror.

[0024] The curved mirror is configured to rotate according to the eye movement data to a preset angle, so that the head-up display device at least projects an image of the first visual area or the second visual area.

[0025] Optionally, the at least two visual areas further comprise a third visual area, the first visual area and the second visual area are arranged in a set direction, along the set direction, the first visual area is located between the second visual area and the third visual area.

[0026] The angle of the curved mirror when the head-up display device projects to form the first visual area is a first angle, the angle of the curved mirror when the head-up display device projects to form the second visual area is a second angle, and the angle of the curved mirror when the head-up display device projects to form the third visual area is a third angle, the first angle is between the second angle and the third angle.

[0027] Optionally, the image source comprises a display area.

[0028] In the image projected to form the first visual area, a first area in the display area generates a first image light beam; in the image projected to form the second visual area, a second area in the display area generates a second image light beam.

[0029] The coincidence degree of the first area and the second area is greater than a preset coincidence degree, wherein the coincidence degree is a ratio of an overlapping area of the first area and the second area to a sum of areas of the first area and the second area.

[0030] Optionally, the pupil tracking module comprises a light flow module.

[0031] In a second aspect, an embodiment of the present application provides a vehicle, comprising the head-up display system in the first aspect and a windshield.

[0032] The image light beam emitted by the head-up display device is projected to the windshield to form an image of one of the at least two visual areas.

[0033] Optionally, the pupil tracking module is arranged on the windshield.

[0034] In a third aspect, an embodiment of the present application provides a control method of a head-up display system, applied to the head-up display device system in the first aspect, comprising:

[0035] Obtaining eye movement data from the pupil tracking module;

[0036] According to the eye movement data, controlling the head-up display device to project at least an image of the first visual area or the second visual area.

[0037] In the embodiments of the present application, the controller controls the head-up display device to project at least an image of the first visual area or the second visual area according to the eye movement data. The first visual area and the second visual area are spliced in the vertical direction to form a larger visual area. In this way, on the one hand, the volume of the head-up display device can be reduced, that is, the head-up display device with a large volume is not needed to at least generate an image of the first visual area or the second visual area. On the other hand, the first visual area and the second visual area are spliced in the vertical direction to form a larger visual angle. Thus, a complete larger image can be formed in the vertical direction by splicing. A larger longitudinal visual angle is further achieved. On the other hand, the embodiments of the present application do not need to perform the cumbersome step of adjusting the imaging position on the windshield, and the attention of the driver is not distracted during driving, so that accidents can be avoided. BRIEF DESCRIPTION OF DRAWINGS

[0038] FIG. 1 is a schematic diagram of a head-up display system provided by an embodiment of the present application;

[0039] FIG. 2 is a schematic diagram of a visual area provided by an embodiment of the present application;

[0040] FIG. 3 is a schematic diagram of another visual area provided by an embodiment of the present application;

[0041] FIG. 4 is a schematic diagram of another visual area provided by an embodiment of the present application;

[0042] FIG. 5 is a schematic diagram of another visual area provided by an embodiment of the present application;

[0043] FIG. 6 is a schematic diagram of another visual area provided by an embodiment of the present application;

[0044] FIG. 7 is a schematic diagram of another head-up display system provided by an embodiment of the present application;

[0045] FIG. 8 is a schematic diagram of another visual area provided by an embodiment of the present application;

[0046] FIG. 9 is a schematic diagram of an image source provided by an embodiment of the present application;

[0047] FIG. 10 is a schematic diagram of a carrier provided by an embodiment of the present application;

[0048] FIG. 11 is a flowchart of a control method of a head-up display system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0049] The present application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the structures.

[0050] For the convenience of explaining the scheme of the embodiments of the present application, a possible application scenario provided by the embodiments of the present application is that the head-up display system is applied to a car. It should be understood by those skilled in the art that the head-up display system of the embodiments of the present application can also be applied to, for example, a sanitation vehicle, a fire vehicle, a military vehicle, and of course, can also be applied to the field of ships, aviation, and the like. For example, the head-up display system can be applied to a fighter aircraft and the like, so that the pilot can track and aim at an object based on the assistance of the head-up display system.

[0051] The head-up display system usually transmits image light to a projection medium, for example, a specially designed screen in front of the driver, or directly reflects to a proper position of the windshield of the car. The light reflected after the image light is reflected by the projection medium enters the eyebox range, so as to form a target virtual image in front of the driver which can be observed in the eyebox range. In the exemplary embodiments of the present application, the windshield is taken as an example to illustrate the projection medium which finally reflects the image light to the human eye.

[0052] The area where the target virtual image generated by the head-up display system is located is the visible area. In the related art, a larger screen (i.e., image source) and a curved mirror are usually arranged to realize a larger target virtual image, a larger visible area, and a larger longitudinal field of view angle. However, the limitation of the car space does not allow a larger screen and a curved mirror to be placed, so it is very difficult to design a large field of view imaging system in a limited space.

[0053] It should be further explained that there is a rotating reflecting element in the related art, which changes the position of the light projected into the pupil of the eye, so as to adapt to the movement of the position of the eye and / or the movement of the position of the pupil. No matter where the target virtual image is observed, the complete image needs to be observed. Therefore, the target virtual image generated by the head-up display system is a complete image. In order to realize the complete image and realize a larger field of view angle, a larger screen (i.e., image source) and a curved mirror need to be arranged. Alternatively, in order to realize the complete image and not to arrange a larger screen (i.e., image source) and a curved mirror, the field of view angle needs to be reduced.

[0054] FIG. 1 is a schematic diagram of a head-up display system according to an embodiment of the present application, and FIG. 2 is a schematic diagram of a visual area according to an embodiment of the present application. Referring to FIGS. 1 and 2, the head-up display system includes a head-up display device 100, a pupil tracking module 300, and a controller (not shown in FIG. 1). The head-up display device 100 projects an image of one of at least two visual areas 200. The at least two visual areas 200 include a first visual area 210 and a second visual area 220. The pupil tracking module 300 tracks eye movement information and obtains eye movement data. The controller is electrically connected to the head-up display device 100 and the pupil tracking module 300. The controller controls the head-up display device 100 to project an image of the first visual area 210 or the second visual area 220 according to the eye movement data.

[0055] It should be noted that the present application is described by taking the at least two visual areas 200 including the first visual area 210 and the second visual area 220 as an example, but is not limited to the number of visual areas. When the at least two visual areas 200 include the first visual area 210 and the second visual area 220, the head-up display device 100 projects an image of the first visual area 210 or the second visual area 220. When the head-up display device 100 projects N visual areas, the head-up display device 100 projects an image of one of the N visual areas. N is a positive integer greater than 2. When the head-up display device 100 projects N visual areas, the head-up display device 100 projects an image of at least one of the first visual area 210 or the second visual area 220. The head-up display device 100 can also project an image of other visual areas. In summary, the head-up display device 100 projects an image of one of the at least two visual areas 200.

[0056] In the present application, the controller controls the head-up display device 100 to project an image of the first visual area 210 or the second visual area 220 according to the eye movement data. The first visual area 210 and the second visual area 220 are spliced in a vertical direction to form a larger visual area. In this way, on the one hand, the size of the head-up display device 100 can be reduced, i.e., the head-up display device 100 does not need to have a large size to project an image of the first visual area 210 or the second visual area 220. On the other hand, the first visual area 210 and the second visual area 220 are spliced in a vertical direction to form a larger visual angle. Thus, a complete larger image can be formed in the vertical direction by splicing. A larger vertical visual angle is achieved. On the other hand, the present application does not need to adjust the imaging position on the windshield 500, which avoids distraction during driving.

[0057] Optionally, the first visual area 210 and the second visual area 220 are arranged in a set direction. As shown in FIG. 1 and FIG. 2, the embodiments of the present application are explained and illustrated by taking the set direction as the vertical direction. However, the set direction is not limited to the vertical direction, and can also include the horizontal direction or a certain oblique direction. For the sake of simplicity, the embodiments of the present application are explained and illustrated by taking the set direction as the vertical direction, and the first visual area 210 and the second visual area 220 are arranged in the vertical direction.

[0058] The vertical direction is the vertical direction in the conventional sense, i.e., the direction of the earth's gravity. Generally, the vertical direction is perpendicular to the direction of the line connecting the two eyes.

[0059] Optionally, as shown in FIG. 1 and FIG. 2, the first visual area 210 and the second visual area 220 overlap to form a first overlap area 212. Thus, when the images displayed by the first visual area 210 and the second visual area 220 are switched, the images displayed by the first overlap area 212 are included in the images displayed by the first visual area 210 and the second visual area 220. The abruptness of the images seen by the eyes is reduced, and the viewing experience is improved.

[0060] Exemplarily, as shown in FIG. 2, the first visual area 210 includes a first sub-visual area 211 and the first overlap area 212. The second visual area 220 includes a second sub-visual area 221 and the first overlap area 212. The first overlap area 212 is located between the first sub-visual area 211 and the second sub-visual area 221. The second sub-visual area 221, the first overlap area 212 and the first sub-visual area 211 are arranged in the vertical direction. The visual area formed by splicing the first visual area 210 and the second visual area 220 in the vertical direction includes the second sub-visual area 221, the first overlap area 212 and the first sub-visual area 211. The visual area formed by splicing the first visual area 210 and the second visual area 220 in the vertical direction is larger than the first visual area 210 and larger than the second visual area 220. A larger longitudinal field of view is achieved.

[0061] Optionally, as shown in FIG. 2, the height of the first overlap area 212 in the vertical direction is H12, and H12 is greater than the first preset height. If the height H12 of the first overlap area 212 is relatively small, the abruptness of the images seen by the eyes is relatively large when the images displayed by the first visual area 210 and the second visual area 220 are switched. In the embodiments of the present application, H12 is controlled to be greater than the first preset height, so that the height H12 of the first overlap area 212 is not too small. Thus, the abruptness of the images seen by the eyes is reduced when the images displayed by the first visual area 210 and the second visual area 220 are switched.

[0062] FIG. 3 is a schematic diagram of another visual area according to an embodiment of the present application. Referring to FIG. 3, the edge of the first visual area 210 is aligned with the edge of the second visual area 220. The first visual area 210 and the second visual area 220 are spliced in the vertical direction. There is no overlapping area between the first visual area 210 and the second visual area 220.

[0063] FIG. 4 is a schematic diagram of another visual area according to an embodiment of the present application. Referring to FIG. 4, the at least two visual areas include the first visual area 210, the second visual area 220, and the third visual area 230. In the vertical direction, the first visual area 210 is located between the second visual area 220 and the third visual area 230. The controller is electrically connected with the head-up display device 100 and the pupil tracking module 300. The controller controls the head-up display device 100 to project an image forming the first visual area 210, the second visual area 220, or the third visual area 230 according to the eye movement data. The embodiments of the present application divide the results of the analyzed pupil fixation points into three categories to obtain three different visual areas.

[0064] It can be understood that the more the number of divided visual areas, the more the computing resources required for switching. On the other hand, the more the number of divided visual areas, the smaller the volume of the head-up display device 100 can be set. Or, in the case of maintaining the volume of the head-up display device 100, a larger longitudinal field of view angle is achieved. Or, both the volume of the head-up display device 100 is reduced and a larger longitudinal field of view angle is achieved.

[0065] In one embodiment, the number of divided visual areas is sufficient to achieve infinite switching of the visual areas.

[0066] Optionally, referring to FIG. 4, the first visual area 210 and the third visual area 230 overlap to form a second overlapping area 213. The first visual area 210 includes a first overlapping area 212, a first sub-visual area 211, and a second overlapping area 213. The third visual area 230 includes a third sub-visual area 231 and a second overlapping area 213. The second overlapping area 213 is located between the first sub-visual area 211 and the third sub-visual area 231. The second sub-visual area 221, the first overlapping area 212, the first sub-visual area 211, the second overlapping area 213, and the third sub-visual area 231 are arranged in the vertical direction. The visual area formed by splicing the first visual area 210, the second visual area 220, and the third visual area 230 in the vertical direction is greater than the first visual area 210, greater than the second visual area 220, and greater than the third visual area 230. Further, a larger longitudinal field of view angle is achieved.

[0067] Optionally, referring to FIG. 4, the height of the second overlap region 213 in the vertical direction is H13, and H13 is greater than the second preset height. If the height H13 of the second overlap region 213 is relatively small, the image seen by the eye when switching between the image displayed by the first viewable region 210 and the image displayed by the third viewable region 230 has a relatively large sense of abrupt change. In the embodiment of the present application, H13 is controlled to be greater than the second preset height, so that the height H13 of the second overlap region 213 is not too small. Thus, the sense of abrupt change of the image seen by the eye when switching between the image displayed by the first viewable region 210 and the image displayed by the third viewable region 230 is reduced.

[0068] For example, the first preset height is equal to the second preset height. In other embodiments, the first preset height can be greater than or less than the second preset height.

[0069] FIG. 5 is a schematic diagram of another viewable region provided by an embodiment of the present application. Referring to FIG. 5, the edge of the first viewable region 210 is aligned with the edge of the second viewable region 220. The edge of the first viewable region 210 is aligned with the edge of the third viewable region 230. The first viewable region 210, the second viewable region 220, and the third viewable region 230 do not have overlap regions.

[0070] FIG. 6 is a schematic diagram of another viewable region provided by an embodiment of the present application. Referring to FIG. 6, the first viewable region 210 and the second viewable region 220 overlap to form a first overlap region 212. The edge of the first viewable region 210 is aligned with the edge of the third viewable region 230. The first viewable region 210, the second viewable region 220, and the third viewable region 230 do not have overlap regions. In other embodiments, the first viewable region 210 and the second viewable region 220 can not have overlap regions, and the first viewable region 210 and the third viewable region 230 overlap to form an overlap region.

[0071] FIG. 7 is a schematic diagram of another head-up display system provided by an embodiment of the present application. Referring to FIGS. 1, 4, and 7, the eye movement data includes an eye pupil state, and the eye pupil state includes an eye pupil up, an eye pupil center, and an eye pupil down. In one embodiment, when the eye 410 changes the gaze point in the eye socket in the vertical direction, the pupil 420 moves in the vertical direction, and thus the pupil 420 has three positions, which represent the states of the eye pupil up, the eye pupil center, and the eye pupil down.

[0072] The third visual area 230 is located below the first visual area 210, and the second visual area 220 is located above the first visual area 210. When the eye pupil state is the eye pupil upward, the head-up display device 100 projects to form the second visual area 220, and the eye 410 can see the image in the second visual area 220 in the upper position. When the eye pupil state is the eye pupil median, the head-up display device 100 projects to form the first visual area 210, and the eye 410 can see the image in the first visual area 210 in the median position. When the eye pupil state is the eye pupil downward, the head-up display device 100 projects to form the third visual area 230, and the eye 410 can see the image in the third visual area 230 in the lower position.

[0073] Exemplarily, referring to FIG. 7, the corresponding visual area when the eye pupil state is the eye pupil upward overlaps the corresponding visual area when the eye pupil state is the eye pupil downward. That is, the second visual area 220 overlaps the third visual area 230. In other embodiments, the second visual area 220 can not overlap the third visual area 230.

[0074] FIG. 8 is a schematic diagram of another visual area provided by an embodiment of the present application. Referring to FIG. 8, when the eye pupil state is the eye pupil upward, the head-up display device 100 projects to form the visual area of area 1 and area 2. When the eye pupil state is the eye pupil median, the head-up display device 100 projects to form the visual area of area 2, area 3 and area 4. When the eye pupil state is the eye pupil downward, the head-up display device 100 projects to form the visual area of area 4 and area 5.

[0075] With reference to FIG. 4 and FIG. 7, the height of the first visual area 210 along the vertical direction is H21, the height of the second visual area 220 along the vertical direction is H22, and the height of the third visual area 230 along the vertical direction is H23. H21 is greater than H22, and / or H21 is greater than H23. Since the eye 410 mostly looks straight ahead, the first visual area 210 in the median position can be made larger, and the second visual area 220 in the upper position and the third visual area 230 in the lower position can be made smaller.

[0076] With reference to FIG. 4 and FIG. 7, the head-up display device 100 includes an image source 101 and a curved mirror 102. The image source 101 is configured to generate an image light beam and emit the image light beam to the curved mirror 102. The curved mirror 102 is configured to rotate according to the eye movement data to a preset angle, so that the head-up display device 100 projects to form an image in one of the at least two visual areas. For example, to project to form an image in the first visual area 210, the second visual area 220 or the third visual area 230.

[0077] Exemplarily, the head-up display device 100 further comprises a curved mirror servo motor connected with the curved mirror 102. The curved mirror servo motor is further electrically connected with the controller, and is configured to drive the curved mirror 102 to rotate under the control of the controller.

[0078] With reference to FIG. 4 and FIG. 7, the angle of the curved mirror 102 when the head-up display device 100 projects to form the first visual area 210 is a first angle. The angle of the curved mirror 102 when the head-up display device 100 projects to form the second visual area 220 is a second angle. The angle of the curved mirror 102 when the head-up display device 100 projects to form the third visual area 230 is a third angle. The first angle is between the second angle and the third angle. Corresponding to the eye pupil states of the upward eye pupil, the middle eye pupil and the downward eye pupil, the second angle, the first angle and the third angle are gradually increased or gradually decreased, and there is no jump of the angle.

[0079] FIG. 9 is a schematic diagram of an image source provided in an embodiment of the present application. With reference to FIG. 1, FIG. 7 and FIG. 9, the image source 101 comprises the display area 110. When projecting to form the image of the first visual area 210, the first region 111 in the display area 110 generates a first image light beam. The image formed by the first image light beam in the eye 410 corresponds to the first visual area 210. When projecting to form the image of the second visual area 220, the second region 112 in the display area 110 generates a second image light beam. The image formed by the second image light beam in the eye 410 corresponds to the second visual area 220. The coincidence degree of the first region 111 and the second region 112 is greater than a preset coincidence degree, wherein the coincidence degree of the first region 111 and the second region 112 is a ratio of the overlapping area of the first region 111 and the second region 112 to the sum of the areas of the first region 111 and the second region 112. In the embodiment of the present application, when the head-up display device 100 projects to form the first visual area 210 or the second visual area 220, the coincidence degree of the first region 111 and the second region 112 of the image source 101 is greater than the preset coincidence degree, so as to control most of the pixels in the image source 101 to display, that is, to control most of the display area 110 in the image source 101 to display, thereby improving the display effects such as resolution and brightness.

[0080] In one embodiment, the first region 111 and the second region 112 overlap. When the head-up display device 100 projects to form the first visual area 210 or the second visual area 220, all the pixels in the image source 101 display, that is, all the display area 110 in the image source 101 displays. The display effects such as resolution and brightness are further improved.

[0081] Exemplarily, referring to FIG. 1, FIG. 7 and FIG. 9, the third region 113 in the display area 110 generates a third image beam when the image of the third visual region 230 is projected. The image formed in the eye 410 corresponds to the third visual region 230. The coincidence degree of the first region 111 and the third region 113 is greater than the preset coincidence degree.

[0082] Generally, the optical flow is the apparent motion pattern of objects, surfaces and edges in a visual scene caused by the relative motion between the observer (eye or camera) and the scene. The optical flow module is a visual sensor capable of measuring the optical flow or visual motion and outputting the measured visual motion based on the optical flow. Referring to FIG. 1 or FIG. 7, the pupil tracking module 300 includes an optical flow module (not shown in FIG. 1 and FIG. 7), which improves the running speed of the pupil tracking module 300.

[0083] FIG. 10 is a schematic diagram of a vehicle provided by an embodiment of the present application. Referring to FIG. 10, the vehicle includes the head-up display system in the above embodiments and a windshield 500. The image beam emitted by the head-up display device 100 is projected to the windshield 500 to form an image of one of the at least two visual regions. For example, in one imaging, the image beam emitted by the head-up display device 100 is projected to the windshield 500 to form an image in the first visual region 210. In another imaging, the image beam emitted by the head-up display device 100 is projected to the windshield 500 to form an image in the first visual region 210.

[0084] Optionally, referring to FIG. 1, FIG. 7 and FIG. 10, the pupil tracking module 300 is arranged on the windshield 500. Compared with arranging the pupil tracking module 300 on the housing of the head-up display device 100, the accuracy of pupil tracking can be improved.

[0085] Exemplarily, referring to FIG. 1, FIG. 7 and FIG. 10, the pupil tracking module 300 is arranged at the upper end of the windshield 500 in a position that does not block the field of view. It can be understood that the image beam emitted by the head-up display device 100 is projected to the windshield 500, and the region of the windshield 500 corresponding to the image beam is a projection region. The pupil tracking module 300 can be arranged at the upper end of the projection region.

[0086] Based on the same technical concept, an embodiment of the present application further provides a control method of a head-up display system. The control method is applied to the head-up display device system in the above embodiments and can be executed by the controller in the above embodiments. FIG. 11 is a flowchart of a control method of a head-up display system provided by an embodiment of the present application. Referring to FIG. 1-FIG. 11, the control method includes:

[0087] S101, obtaining eye movement data from the pupil tracking module.

[0088] S102, according to the eye movement data, control the head-up display device to project at least an image forming the first visual area or the second visual area.

[0089] In the embodiments of the present application, the eye movement data is obtained from the pupil tracking module. According to the eye movement data, the head-up display device is controlled to project at least an image forming the first visual area or the second visual area. On the one hand, the volume of the head-up display device 100 can be reduced, that is, the image of the first visual area 210 or the second visual area 220 can be generated without using a large volume of head-up display device 100. On the other hand, the first visual area 210 and the second visual area 220 are spliced in the vertical direction to form a larger field of view angle. Thus, a complete larger image can be formed in the vertical direction by splicing. A larger longitudinal field of view angle is further achieved.

[0090] In one embodiment, the eye movement data includes eye pupil states, and the eye pupil states include eye pupil up, eye pupil center and eye pupil down. According to the eye movement data, the curved mirror is rotated to a preset angle. When the eye pupil state is eye pupil up, the head-up display device 100 projects to form the second visual area 220, and the eye 410 can see the image in the second visual area 220 in the upper position. When the eye pupil state is eye pupil center, the head-up display device 100 projects to form the first visual area 210, and the eye 410 can see the image in the first visual area 210 in the middle position. When the eye pupil state is eye pupil down, the head-up display device 100 projects to form the third visual area 230, and the eye 410 can see the image in the third visual area 230 in the lower position.

[0091] There is a 5-degree area in the center of the human eye (i.e. the eye) visual field range, i.e. the human eye visual center. Due to the existence of the fovea, this area is the highest resolution part in the human eye visual field range. We name the fovea as the fovea of the human eye macular region, which contains the highest concentration of cone photoreceptors, and is the highest area of the human eye focus. High concentration of cone cells makes the focus of this part of the eye more clear, and the fovea is also the main part of human color vision. Eye tracking technology (Eye Tracking Object Detection, ETOD) is an eye tracking target detection method based on foveal visual area mapping. Its function is to accurately find the object information of the user's real attention area, i.e. to detect the category of the object of attention according to the movement state of the human eye, and to filter out other irrelevant information. It is an end-to-end eye tracking target detection method, and the model takes YOLOv4 network as the basis, designs CSPANet network to realize feature fusion, and uses multi-task loss function to train the network, so as to realize the targeted detection of the visual field area. YOLOv is the abbreviation of You Only Look Once. YOLOv4 is the fourth generation version.

[0092] In one embodiment of the present application, a method of end-to-end eye tracking target detection is designed to track both saccades and pursuit movements of the central region of the pupil of the human eye, to instantaneously transmit the eye movement signals to the controller, and to have the controller send adjustment signals to the signal receiver on the servo motor of the curved mirror to read and process the adjustment signals, so as to control the corresponding rotation of the curved mirror and to project the virtual image on the corresponding region of the windshield glass.

[0093] In the method, saccades and pursuit movements are two different human eye movements, saccades refer to the sudden change of the fixation point or the fixation direction of the human eye (usually occurring when the human eye scans), and pursuit movements refer to the pursuit of the fixation point to the movement of the object (usually occurring when the human eye fixates on the moving object).

Claims

1. A head-up display system, comprising: a head-up display device, configured to project an image forming one of at least two visual regions, the at least two visual regions comprising a first visual region and a second visual region; a pupil tracking module, configured to track eye movement information and obtain eye movement data; a controller, electrically connected with the head-up display device and the pupil tracking module, configured to control the head-up display device to project an image forming at least one of the first visual region or the second visual region according to the eye movement data.

2. The head-up display system of claim 1, wherein, The first visual region and the second visual region are arranged in a set direction.

3. The head-up display system of claim 2, wherein, The first visual region and the second visual region overlap to form a first overlap region.

4. The heads-up display system of claim 3, wherein, A height of the first overlap region along the set direction is greater than a first preset height.

5. The heads-up display system of claim 2, wherein, The at least two visual regions further comprise a third visual region, and the first visual region is located between the second visual region and the third visual region along the set direction.

6. The heads-up display system of claim 5, wherein, The first visual region and the third visual region overlap to form a second overlap region.

7. The head-up display system of claim 6, wherein, A height of the second overlap region along the set direction is greater than a second preset height.

8. The head-up display system of claim 7, wherein, The eye movement data comprises an eye pupil state, and the eye pupil state comprises an upward eye pupil, a middle eye pupil and a downward eye pupil. The third visual region is located below the first visual region. When the eye pupil state is the upward eye pupil, the head-up display device projects the second visual region; when the eye pupil state is the middle eye pupil, the head-up display device projects the first visual region; and when the eye pupil state is the downward eye pupil, the head-up display device projects the third visual region.

9. The head-up display system of claim 7, wherein, The height of the first visual region along the set direction is greater than the height of the second visual region along the set direction. And / or, the height of the first visual region along the set direction is greater than the height of the third visual region along the set direction.

10. The heads-up display system of claim 1, wherein, The head-up display device comprises an image source and a curved mirror. The image source is configured to generate an image light beam and emit the image light beam to the curved mirror. The curved mirror is configured to rotate according to the eye movement data to a preset angle, so that the head-up display device projects an image forming at least one of the first visual region or the second visual region.

11. The head-up display system of claim 10, wherein, The at least two visual regions further comprise a third visual region, and the first visual region and the second visual region are arranged in a set direction, and the first visual region is located between the second visual region and the third visual region along the set direction. An angle of the curved mirror when the head-up display device projects the first visual region is a first angle, an angle of the curved mirror when the head-up display device projects the second visual region is a second angle, and an angle of the curved mirror when the head-up display device projects the third visual region is a third angle, and the first angle is between the second angle and the third angle.

12. The head-up display system of claim 10, wherein, The image source comprises a display area. The first region in the display area generates a first image light beam when projecting an image of the first visual area; and the second region in the display area generates a second image light beam when projecting an image of the second visual area. The overlap degree of the first region and the second region is greater than a preset overlap degree, wherein the overlap degree is a ratio of an overlapping area of the first region and the second region to a sum of areas of the first region and the second region.

13. The heads-up display system of claim 1, wherein, The pupil tracking module comprises a light flow module.

14. A vehicle comprising the head-up display system of any one of claims 1-13 and a windshield. The image light beam emitted by the head-up display device is projected onto the windshield to form an image of one of the at least two visual areas.

15. The vehicle of claim 14, wherein, The pupil tracking module is arranged on the windshield.

16. A control method of a head-up display system, applied to the head-up display device system of any one of claims 1-13, comprising: obtaining eye movement data from the pupil tracking module; controlling the head-up display device to project at least an image of the first visual area or the second visual area according to the eye movement data.

Images