Head-up display system, vehicle, and control method for a head-up display system

Abstract

The application provides a head-up display system, a carrier and a control method of the head-up display system, and relates to the technical field of head-up display. The head-up display system comprises: a head-up display device, which projects an image of one of at least two visual regions, and the at least two visual regions comprise a first visual region and a second visual region; a pupil tracking module, which tracks eye movement information and acquires eye movement data; and a controller, which is electrically connected with the head-up display device and the pupil tracking module, and controls the head-up display device to project an image of at least the first visual region or the second visual region according to the eye movement data. The embodiment of the application can reduce the volume of the head-up display device and realize a larger longitudinal field of view.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of head-up display, in particular to a head-up display system, a vehicle and a control method of the head-up display system. BACKGROUND

[0002] The head-up display (HUD) can project the instrument panel information, navigation information elements obtained by analyzing the vehicle and road conditions through sensors (such as cameras and radars) of the intelligent driving vehicle onto the windshield glass or the imaging component of the display, so that the user can observe the information displayed by the HUD.

[0003] With the improvement of material and cultural level, people demand more information presented by the HUD virtual image, and the application of large virtual image size is more and more widely. In order to meet the demand of expanding the field of view, the prior art usually selects a larger screen or a larger curved mirror to realize the expansion of the field of view.

[0004] The limitation of the car machine space does not allow the placement of larger screens and curved mirrors, so it is very difficult to design a large field of view imaging system in a limited space. SUMMARY

[0005] The embodiments of the present application provide a head-up display system, a vehicle and a control method of the head-up display system, so as to realize the reduction of the volume of the head-up display device and the realization of a larger longitudinal field of view.

[0006] In a first aspect, the embodiments of the present application provide a head-up display system, comprising:

[0007] A head-up display device projects an image of at least one of at least two visible regions, the at least two visible regions comprising a first visible region and a second visible region;

[0008] A pupil tracking module tracks eye movement information and obtains eye movement data;

[0009] A controller is electrically connected with the head-up display device and the pupil tracking module, and controls the head-up display device to project an image of at least the first visible region or the second visible region according to the eye movement data.

[0010] Optionally, the first visible region and the second visible region are arranged in a set direction.

[0011] Optionally, the first visible region and the second visible region overlap to form a first overlapping region.

[0012] Optionally, the height of the first overlapping region along the set direction is greater than a first preset height.

[0013] Optionally, the at least two visual areas further comprise a third visual area, the first visual area is located between the second visual area and the third visual area along the set direction.

[0014] Optionally, the first visual area and the third visual area overlap to form a second overlapping area.

[0015] Optionally, a height of the second overlapping area along the set direction is greater than a second preset height.

[0016] Optionally, the eye movement data comprises an eye pupil state, the eye pupil state comprises an upward eye pupil, a middle eye pupil and a downward eye pupil.

[0017] The third visual area is located below the first visual area.

[0018] When the eye pupil state is the upward eye pupil, the head-up display device projects to form the second visual area; when the eye pupil state is the middle eye pupil, the head-up display device projects to form the first visual area; when the eye pupil state is the downward eye pupil, the head-up display device projects to form the third visual area.

[0019] Optionally, a height of the first visual area along the set direction is greater than a height of the second visual area along the set direction.

[0020] And / or, a height of the first visual area along the vertical direction is greater than a height of the third visual area along the vertical direction.

[0021] Optionally, the head-up display device comprises an image source and a curved mirror.

[0022] The image source is configured to generate an image light beam and emit the image light beam to the curved mirror.

[0023] The curved mirror is configured to rotate according to the eye movement data to a preset angle, so that the head-up display device projects at least an image of the first visual area or the second visual area.

[0024] Optionally, the at least two visual areas further comprise a third visual area, the first visual area and the second visual area are arranged in a set direction, the first visual area is located between the second visual area and the third visual area along the set direction.

[0025] The angle of the curved mirror when the head-up display device projects to form the first visible area is a first angle, the angle of the curved mirror when the head-up display device projects to form the second visible area is a second angle, and the angle of the curved mirror when the head-up display device projects to form the third visible area is a third angle, and the first angle is between the second angle and the third angle.

[0026] Optionally, the image source comprises a display area.

[0027] In the image projected to form the first visible area, a first area in the display area generates a first image light beam; and in the image projected to form the second visible area, a second area in the display area generates a second image light beam.

[0028] The coincidence degree of the first area and the second area is greater than a preset coincidence degree, wherein the coincidence degree is a ratio of an overlapping area of the first area and the second area to a sum of areas of the first area and the second area.

[0029] Optionally, the pupil tracking module comprises a light flow module.

[0030] In a second aspect, an embodiment of the present application provides a vehicle comprising the head-up display system of the first aspect and a windshield.

[0031] The image light beam emitted by the head-up display device is projected to the windshield to form an image of one of the at least two visible areas.

[0032] Optionally, the pupil tracking module is arranged on the windshield.

[0033] In a third aspect, an embodiment of the present application provides a control method of a head-up display system, applied to the head-up display device system of the first aspect, comprising:

[0034] Obtaining eye movement data from the pupil tracking module;

[0035] According to the eye movement data, controlling the head-up display device to project at least an image of the first visible area or the second visible area.

[0036] In the embodiment of the present application, the controller controls the head-up display device to project at least an image of the first visual area or the second visual area according to the eye movement data. The first visual area and the second visual area are spliced in the vertical direction to form a larger visual area. In this way, on the one hand, the volume of the head-up display device can be reduced, that is, the head-up display device with a large volume is not needed to at least generate an image of the first visual area or the second visual area. On the other hand, the first visual area and the second visual area are spliced in the vertical direction to form a larger visual angle. Thus, a complete larger image can be formed in the vertical direction by splicing. A larger longitudinal visual angle is further achieved. On the other hand, the embodiment of the present application does not need to perform the cumbersome step of adjusting the imaging position on the windshield, and avoids the distraction of attention during driving. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A schematic diagram of a head-up display system provided by the embodiment of the present application is shown in FIG. 1.

[0038] Figure 2 A schematic diagram of a visual area provided by the embodiment of the present application is shown in FIG. 2.

[0039] Figure 3 A schematic diagram of another visual area provided by the embodiment of the present application is shown in FIG. 3.

[0040] Figure 4 A schematic diagram of another visual area provided by the embodiment of the present application is shown in FIG. 4.

[0041] Figure 5 A schematic diagram of another visual area provided by the embodiment of the present application is shown in FIG. 5.

[0042] Figure 6 A schematic diagram of another visual area provided by the embodiment of the present application is shown in FIG. 6.

[0043] Figure 7 A schematic diagram of another head-up display system provided by the embodiment of the present application is shown in FIG. 7.

[0044] Figure 8 A schematic diagram of another visual area provided by the embodiment of the present application is shown in FIG. 8.

[0045] Figure 9 A schematic diagram of an image source provided by the embodiment of the present application is shown in FIG. 9.

[0046] Figure 10 A schematic diagram of a vehicle provided by the embodiment of the present application is shown in FIG. 10.

[0047] Figure 11 A flowchart of a control method of a head-up display system provided by the embodiment of the present application is shown in FIG. 11. DETAILED DESCRIPTION

[0048] The application will be further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended for the purpose of interpretation of the application and are not limiting of the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for the purpose of description.

[0049] For the purpose of illustrating the solutions of the embodiments of the application, a possible application scenario provided by the embodiments of the application is taken as an example in which the head-up display system is applied to a vehicle. It should be understood by those skilled in the art that the head-up display system of the embodiments of the application can also be applied to, for example, a sanitation vehicle, a fire vehicle, a military vehicle, and of course, can also be applied to the field of ships, aviation, etc. For example, the head-up display system can be applied to a fighter aircraft, so that the pilot can track and aim at an object based on the assistance of the head-up display system.

[0050] The head-up display system usually transmits image light to a projection medium, for example, a specially designed screen in front of the driver, or directly reflects to a proper position of the windshield of the vehicle. The light reflected after the image light is transmitted to the projection medium enters the eyebox range, so that a target virtual image that can be observed in the eyebox range is formed in front of the driver. In the exemplary embodiments of the embodiments of the application, the windshield is taken as an example to be described as the projection medium for finally reflecting the image light to the human eye.

[0051] The area where the target virtual image generated by the head-up display system is located is the visible area. In the related art, a larger screen (i.e., image source) and a curved mirror are usually provided to achieve a larger target virtual image, a larger visible area, and a larger longitudinal field of view angle. However, the limitation of the vehicle space does not allow a larger screen and a curved mirror to be placed, so it is very difficult to design a large field of view imaging system in a limited space.

[0052] It should be further noted that there is a rotating reflecting element in the related art, which changes the position of the light projected into the pupil of the eye, so as to adapt to the movement of the position of the eye and / or the movement of the position of the pupil. No matter where the target virtual image is observed, the complete image needs to be observed. Therefore, the target virtual image generated by the head-up display system is a complete image. In order to achieve the complete image and achieve a larger field of view angle, a larger screen (i.e., image source) and a curved mirror need to be provided. Or conversely, in order to achieve the complete image and not to provide a larger screen (i.e., image source) and a curved mirror, the field of view angle needs to be reduced.

[0053] Figure 1 A schematic diagram of a head-up display system provided by the embodiments of the application, Figure 2 A schematic diagram of a visible area provided by the embodiments of the application, with reference to Figure 1 and Figure 2The head-up display system includes a head-up display device 100, a pupil tracking module 300, and a controller. Figure 1 (Not shown in the image). The head-up display device 100 projects an image of one of at least two visual areas 200. The at least two visual areas 200 include a first visual area 210 and a second visual area 220. The pupil tracking module 300 tracks eye movement information and acquires eye movement data. The controller is electrically connected to the head-up display device 100 and the pupil tracking module 300. Based on the eye movement data, the controller controls the head-up display device 100 to project an image of either the first visual area 210 or the second visual area 220.

[0054] It should be noted that the embodiments of the present invention use at least two visible areas 200, including a first visible area 210 and a second visible area 220, as an example for explanation and illustration, and are not a limitation on the number of visible areas. When at least two visible areas 200 include a first visible area 210 and a second visible area 220, the head-up display device 100 projects an image of either the first visible area 210 or the second visible area 220. When the head-up display device 100 projects N visible areas, it projects an image of one of the N visible areas. Here, N is a positive integer greater than 2. When the head-up display device 100 projects N visible areas, it projects at least an image of either the first visible area 210 or the second visible area 220. It may also project images of other visible areas. In summary, the head-up display device 100 projects an image of at least one of the at least two visible areas 200.

[0055] In this embodiment of the invention, the controller controls the head-up display device 100 to project an image of a first visible area 210 or a second visible area 220 based on eye-tracking data. The first visible area 210 and the second visible area 220 are vertically joined to form a larger visible area. This reduces the size of the head-up display device 100, eliminating the need for a large head-up display device 100 to generate an image of the first visible area 210 or the second visible area 220. Furthermore, the vertical joining of the first visible area 210 and the second visible area 220 creates a larger field of view. This allows for the formation of a complete, larger image in the vertical direction, resulting in a larger longitudinal field of view. Moreover, this embodiment eliminates the need for cumbersome adjustments to the image position on the windshield 500, preventing distractions and potential accidents during driving.

[0056] Optionally, the first visible area 210 and the second visible area 220 are arranged in a set direction. For example... Figure 1 and Figure 2As shown, embodiments of the present application are explained by taking the vertical direction as an example of the set direction. However, the set direction can also include the horizontal direction or a certain oblique direction in other embodiments. For the sake of simplicity, embodiments of the present application are explained by taking the vertical direction as an example of the set direction, and the first visible area 210 and the second visible area 220 are arranged in the vertical direction.

[0057] The vertical direction is the vertical direction in the conventional sense, i.e. the direction of the gravitational force. Generally, the vertical direction is perpendicular to the direction of the line connecting the two eyes.

[0058] Optionally, referring to Figure 1 and Figure 2 , the first visible area 210 and the second visible area 220 overlap to form a first overlap area 212. Thus, when the images displayed by the first visible area 210 and the second visible area 220 are switched, the images displayed by the first overlap area 212 are included in the images displayed by the first visible area 210 and the second visible area 220. The abruptness of the images seen by the eyes is reduced, and the viewing experience is improved.

[0059] Exemplarily, referring to Figure 2 , the first visible area 210 includes a first sub-visible area 211 and the first overlap area 212. The second visible area 220 includes a second sub-visible area 221 and the first overlap area 212. The first overlap area 212 is located between the first sub-visible area 211 and the second sub-visible area 221. The second sub-visible area 221, the first overlap area 212 and the first sub-visible area 211 are arranged in the vertical direction. The visible area formed by splicing the first visible area 210 and the second visible area 220 in the vertical direction includes the second sub-visible area 221, the first overlap area 212 and the first sub-visible area 211. The visible area formed by splicing the first visible area 210 and the second visible area 220 in the vertical direction is larger than the first visible area 210 and larger than the second visible area 220. A larger longitudinal field of view is achieved.

[0060] Optionally, referring to Figure 3 , the height of the first overlap area 212 in the vertical direction is H12, and H12 is greater than the first preset height. If the height H12 of the first overlap area 212 is relatively small, the abruptness of the images seen by the eyes is relatively large when the images displayed by the first visible area 210 and the second visible area 220 are switched. Embodiments of the present application control H12 to be greater than the first preset height, so as to control the height H12 of the first overlap area 212 not to be too small. Thus, the abruptness of the images seen by the eyes is reduced when the images displayed by the first visible area 210 and the second visible area 220 are switched.

[0061] Figure 3 Another schematic diagram of the visual area provided by an embodiment of the present application is shown in FIG. 2B. Referring to FIG. 2B, the edge of the first visual area 210 is aligned with the edge of the second visual area 220. The first visual area 210 and the second visual area 220 are spliced in the vertical direction. There is no overlapping area between the first visual area 210 and the second visual area 220. Figure 4

[0062] Figure 4 Another schematic diagram of the visual area provided by an embodiment of the present application is shown in FIG. 2C. Referring to FIG. 2C, the at least two visual areas include the first visual area 210, the second visual area 220, and the third visual area 230. In the vertical direction, the first visual area 210 is located between the second visual area 220 and the third visual area 230. The controller is electrically connected with the head-up display device 100 and the pupil tracking module 300. The controller controls the head-up display device 100 to project an image forming the first visual area 210, the second visual area 220, or the third visual area 230 according to the eye movement data. The embodiment of the present application divides the result of the analyzed pupil fixation point into three categories, and correspondingly obtains three different visual areas. Figure 4

[0063] It can be understood that the more the number of divided visual areas, the more the computing resources required for switching. On the other hand, the more the number of divided visual areas, the smaller the volume of the head-up display device 100 can be set. Or, in the case of keeping the volume of the head-up display device 100 unchanged, a larger longitudinal field of view angle is achieved. Or, both the volume of the head-up display device 100 is reduced and a larger longitudinal field of view angle is achieved.

[0064] In one embodiment, the number of divided visual areas is sufficient to achieve infinite switching of the visual area.

[0065] Optionally, referring to FIG. 2D, the first visual area 210 and the third visual area 230 overlap to form a second overlapping area 213. The first visual area 210 includes a first overlapping area 212, a first sub-visual area 211, and the second overlapping area 213. The third visual area 230 includes a third sub-visual area 231 and the second overlapping area 213. The second overlapping area 213 is located between the first sub-visual area 211 and the third sub-visual area 231. The second sub-visual area 221, the first overlapping area 212, the first sub-visual area 211, the second overlapping area 213, and the third sub-visual area 231 are arranged in the vertical direction. The visual area formed by splicing the first visual area 210, the second visual area 220, and the third visual area 230 in the vertical direction is greater than the first visual area 210, greater than the second visual area 220, and greater than the third visual area 230. Further, a larger longitudinal field of view angle is achieved. Figure 4 ​​​

[0066] Optionally, referring to Figure 5 , the height of the second overlap region 213 in the vertical direction is H13, and H13 is greater than the second preset height. If the height H13 of the second overlap region 213 is relatively small, the image seen by the eye when switching between the image displayed by the first viewable region 210 and the image displayed by the third viewable region 230 produces a relatively large sense of mutation. In the embodiment of the present application, H13 is controlled to be greater than the second preset height, so as to control the height H13 of the second overlap region 213 not to be too small. Thus, when switching between the image displayed by the first viewable region 210 and the image displayed by the third viewable region 230, the sense of mutation produced by the image seen by the eye is reduced.

[0067] Exemplarily, the first preset height is equal to the second preset height. In other embodiments, the first preset height can be greater than or less than the second preset height.

[0068] Figure 5 Another schematic diagram of a viewable region provided by an embodiment of the present application is shown in Figure 6 , the edge of the first viewable region 210 is aligned with the edge of the second viewable region 220. The edge of the first viewable region 210 is aligned with the edge of the third viewable region 230. There is no overlap region between the first viewable region 210, the second viewable region 220 and the third viewable region 230.

[0069] Figure 6 Another schematic diagram of a viewable region provided by an embodiment of the present application is shown in Figure 7 , the first viewable region 210 and the second viewable region 220 overlap to form a first overlap region 212. The edge of the first viewable region 210 is aligned with the edge of the third viewable region 230. There is no overlap region between the first viewable region 210, the second viewable region 220 and the third viewable region 230. In other embodiments, the first viewable region 210 and the second viewable region 220 can not overlap, and the first viewable region 210 and the third viewable region 230 overlap to form an overlap region.

[0070] Figure 1 Another schematic diagram of a head-up display system provided by an embodiment of the present application is shown in combination with Figure 4 , Figure 7 and Figure 7 , the eye movement data includes an eye pupil state, and the eye pupil state includes an eye pupil upward, an eye pupil center and an eye pupil downward. In one embodiment, when the eye 410 changes the gaze point in the eye socket in the vertical direction, the pupil 420 moves in position in the vertical direction, and thus the pupil 420 has three positions, which represent the states of the eye pupil upward, the eye pupil center and the eye pupil downward.

[0071] The third visual area 230 is located below the first visual area 210, and the second visual area 220 is located above the first visual area 210. When the eye pupil state is the eye pupil upward, the head-up display device 100 projects to form the second visual area 220, and the eye 410 can see the image in the second visual area 220 in the upper position. When the eye pupil state is the eye pupil median, the head-up display device 100 projects to form the first visual area 210, and the eye 410 can see the image in the first visual area 210 in the middle position. When the eye pupil state is the eye pupil downward, the head-up display device 100 projects to form the third visual area 230, and the eye 410 can see the image in the third visual area 230 in the lower position.

[0072] Exemplarily, referring to Figure 8 The corresponding visual area when the eye pupil state is the eye pupil upward overlaps with the corresponding visual area when the eye pupil state is the eye pupil downward. That is, the second visual area 220 overlaps with the third visual area 230. In other embodiments, the second visual area 220 and the third visual area 230 can not overlap.

[0073] Figure 8 Another schematic diagram of the visual area provided by the embodiment of the present application is provided, referring to Figure 4 When the eye pupil state is the eye pupil upward, the visual area projected by the head-up display device 100 is area 1 and area 2. When the eye pupil state is the eye pupil median, the visual area projected by the head-up display device 100 is area 2, area 3 and area 4. When the eye pupil state is the eye pupil downward, the visual area projected by the head-up display device 100 is area 4 and area 5.

[0074] Combined with reference to Figure 7 and Figure 4 The height of the first visual area 210 along the vertical direction is H21, the height of the second visual area 220 along the vertical direction is H22, and the height of the third visual area 230 along the vertical direction is H23. H21 is greater than H22, and / or H21 is greater than H23. Since the eye 410 mostly looks straight ahead, the first visual area 210 in the middle position can be made larger, and the second visual area 220 in the upper position and the third visual area 230 in the lower position can be made smaller.

[0075] Combined with reference to Figure 7 and Figure 4The head-up display device 100 comprises an image source 101 and a curved mirror 102. The image source 101 is configured to generate an image light beam and emit the image light beam to the curved mirror 102. The curved mirror 102 is configured to rotate according to the eye movement data to a preset angle, so that the head-up display device 100 projects an image in at least one of the two visible regions. For example, the image is projected in the first visible region 210, the second visible region 220 or the third visible region 230.

[0076] Exemplarily, the head-up display device 100 further comprises a curved mirror servo motor connected with the curved mirror 102. The curved mirror servo motor is further electrically connected with the controller, and is configured to drive the curved mirror 102 to rotate under the control of the controller.

[0077] With reference to Figure 7 and Figure 9 , the angle of the curved mirror 102 when the head-up display device 100 projects the first visible region 210 is a first angle. The angle of the curved mirror 102 when the head-up display device 100 projects the second visible region 220 is a second angle. The angle of the curved mirror 102 when the head-up display device 100 projects the third visible region 230 is a third angle. The first angle is between the second angle and the third angle. Corresponding to the eye pupil states of the upward eye pupil, the middle eye pupil and the downward eye pupil, the second angle, the first angle and the third angle are gradually increased or gradually decreased, and there is no jump in the angle.

[0078] Figure 1 A schematic diagram of an image source provided by an embodiment of the present application is shown in Figure 7 , Figure 9 and Figure 1The image source 101 includes a display area 110. When the image of the first visual area 210 is projected, the first area 111 in the display area 110 generates a first image beam. The image formed by the first image beam in the eye 410 corresponds to the first visual area 210. When the image of the second visual area 220 is projected, the second area 112 in the display area 110 generates a second image beam. The image formed by the second image beam in the eye 410 corresponds to the second visual area 220. The coincidence degree of the first area 111 and the second area 112 is greater than a preset coincidence degree. The coincidence degree of the first area 111 and the second area 112 is the ratio of the overlapping area of the first area 111 and the second area 112 to the sum of the areas of the first area 111 and the second area 112. In the embodiment of the application, when the head-up display device 100 projects the image of the first visual area 210 or the second visual area 220, the coincidence degree of the first area 111 and the second area 112 of the image source 101 is greater than the preset coincidence degree, so that most of the pixels in the image source 101 are controlled to display, that is, most of the display area 110 in the image source 101 is controlled to display, thereby improving the display effect such as resolution and brightness.

[0079] In one embodiment, the first area 111 and the second area 112 overlap. When the head-up display device 100 projects the image of the first visual area 210 or the second visual area 220, all the pixels in the image source 101 display, that is, all the display area 110 in the image source 101 is controlled to display. The display effect such as resolution and brightness is further improved.

[0080] For example, referring to Figure 7 , Figure 9 and Figure 1 , when the image of the third visual area 230 is projected, the third area 113 in the display area 110 generates a third image beam. The image formed by the third image beam in the eye 410 corresponds to the third visual area 230. The coincidence degree of the first area 111 and the third area 113 is greater than the preset coincidence degree.

[0081] Generally, the optical flow is the apparent motion pattern of objects, surfaces and edges in a visual scene caused by the relative motion between the observer (eye or camera) and the scene. The optical flow module is a visual sensor capable of measuring the optical flow or visual motion and outputting the measured visual motion based on the optical flow. Referring to Figure 7 or Figure 1 , the pupil tracking module 300 includes an optical flow module (not shown in Figure 7 and Figure 10 ), which improves the running speed of the pupil tracking module 300.

[0082] Figure 10 A schematic diagram of a vehicle provided by an embodiment of the application is shown in Figure 1The vehicle includes the head-up display system in the above embodiments and the windshield 500. The image light beams emitted by the head-up display device 100 are projected to the windshield 500 to form an image in one of the at least two visual areas. For example, in one imaging, the image light beams emitted by the head-up display device 100 are projected to the windshield 500 to form an image in the first visual area 210. In another imaging, the image light beams emitted by the head-up display device 100 are projected to the windshield 500 to form an image in the first visual area 210.

[0083] Optionally, referring to Figure 7 , Figure 10 and Figure 1 , the pupil tracking module 300 is arranged on the windshield 500. Compared with arranging the pupil tracking module 300 on the housing of the head-up display device 100, the accuracy of pupil tracking can be improved.

[0084] Illustratively, referring to Figure 7 , Figure 10 and Figure 11 , the pupil tracking module 300 is arranged at the upper end of the windshield 500, at a position that does not block the field of view. It can be understood that the image light beams emitted by the head-up display device 100 are projected to the windshield 500, and the area on the windshield 500 corresponding to the image light beams is a projection area. The pupil tracking module 300 can be arranged at the upper end of the projection area.

[0085] Based on the same technical concept, the present application also provides a control method of a head-up display system. The control method is applied to the head-up display device system in the above embodiments and can be executed by the controller in the above embodiments. Figures 1-11 A flowchart of a control method of a head-up display system according to an embodiment of the present application is shown in FIG. 10. The control method includes the following steps. ​ S101, obtaining eye movement data from the pupil tracking module.

[0086] S102, controlling the head-up display device to project at least an image forming the first visual area or the second visual area according to the eye movement data.

[0087] S102, controlling the head-up display device to project at least an image forming the first visual area or the second visual area according to the eye movement data.

[0088] In the embodiment of the present application, the eye movement data is obtained from the pupil tracking module. According to the eye movement data, the head-up display device is controlled to project at least an image forming the first visual area or the second visual area. On the one hand, the volume of the head-up display device 100 can be reduced, that is, the image of the first visual area 210 or the second visual area 220 can be generated without using a large volume of head-up display device 100. On the other hand, the first visual area 210 and the second visual area 220 are spliced in the vertical direction to form a larger field of view angle. Thus, a complete larger image can be formed in the vertical direction by splicing. A larger longitudinal field of view angle is further achieved.

[0089] In one embodiment, the eye movement data includes an eye pupil state, and the eye pupil state includes an upward eye pupil, a median eye pupil, and a downward eye pupil. According to the eye movement data, the curved mirror is controlled to rotate to a preset angle. When the eye pupil state is the upward eye pupil, the head-up display device 100 projects to form the second visual area 220, and the eye 410 can see the image in the second visual area 220 in the upper position. When the eye pupil state is the median eye pupil, the head-up display device 100 projects to form the first visual area 210, and the eye 410 can see the image in the first visual area 210 in the middle position. When the eye pupil state is the downward eye pupil, the head-up display device 100 projects to form the third visual area 230, and the eye 410 can see the image in the third visual area 230 in the lower position.

[0090] There is a 5-degree area in the center of the human eye (i.e., the eye) visual field range, i.e., the human eye visual center. Due to the existence of the fovea, this area is the highest clear part in the human eye visual field range. We name the fovea of the macular region of the human eye as the fovea, which contains the highest concentration of cone photoreceptors, and is the highest area of the human eye focus. The high concentration of cone cells makes the focus of this part of the eye more clear, and the fovea is also the main part of human color vision. Eye tracking technology (Eye Tracking Object Detection, ETOD) is an eye tracking target detection method based on foveal visual area mapping. Its function is to accurately find the object information of the user's real attention area, i.e., to detect the category of the object of attention according to the motion state of the human eye, and to filter out other irrelevant information. It is an end-to-end eye tracking target detection method, and the model takes the YOLOv4 network as the basis, designs the CSPANet network to realize feature fusion, and uses a multi-task loss function to train the network, so as to realize the targeted detection of the visual field area. YOLOv is the abbreviation of You Only Look Once. YOLOv4 is the fourth generation version.

[0091] In one embodiment of the present application, an end-to-end eye movement tracking target detection method is designed to track two kinds of human eye movements, i.e. saccades and pursuit movements, in the central region of the human eye pupil, and to instantaneously transmit the eye movement signals to a controller. The controller sends adjustment signals to a signal receiver on a servo motor of the curved mirror according to the eye movement signals, reads and processes the adjustment signals, and controls the curved mirror to rotate accordingly to project the virtual image on the corresponding region on the windshield.

[0092] wherein saccades and pursuit movements are two different kinds of human eye movements, saccades refer to sudden changes in the fixation point or direction of the human eye (usually occurring when the human is scanning), and pursuit movements refer to the tracking of the fixation point following the movement of an object (usually occurring when the human is looking at a moving object).

[0093] It is noted that the above only describes the preferred embodiments of the present application and the principles of the applied technology. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, re-adjustments, mutual combinations and substitutions can be made without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the appended claims.

Claims

1. A heads-up display system, characterized by, The head-up display device comprises: a head-up display device, which projects an image of at least one of at least two visual areas, the at least two visual areas comprising a first visual area and a second visual area; a pupil tracking module, which tracks eye movement information and obtains eye movement data; a controller, which is electrically connected to the head-up display device and the pupil tracking module, and controls the head-up display device to project an image of at least one of the first visual area or the second visual area according to the eye movement data; the first visual area and the second visual area are arranged in a set direction, and the set direction comprises a vertical direction; the first visual area and the second visual area overlap to form a first overlapping area; a height of the first overlapping area along the set direction is greater than a first preset height; the at least two visual areas further comprise a third visual area, and the first visual area is located between the second visual area and the third visual area along the set direction; the eye movement data comprises an eye pupil state, and the eye pupil state comprises an upward eye pupil, a middle eye pupil, and a downward eye pupil; the third visual area is located below the first visual area; when the eye pupil state is the upward eye pupil, the head-up display device projects the second visual area; when the eye pupil state is the middle eye pupil, the head-up display device projects the first visual area; and when the eye pupil state is the downward eye pupil, the head-up display device projects the third visual area; and the first visual area, the second visual area, and the third visual area are spliced in the vertical direction to form a larger field of view angle.

2. The head-up display system of claim 1, wherein, the first visual area and the third visual area overlap to form a second overlapping area; 3. The head-up display system of claim 2, wherein, a height of the second overlapping area along the set direction is greater than a second preset height; 4. The head-up display system of claim 3, wherein, a height of the first visual area along the set direction is greater than a height of the second visual area along the set direction; and / or, a height of the first visual area along the set direction is greater than a height of the third visual area along the set direction.

5. The head-up display system of claim 1, wherein, the head-up display device comprises an image source and a curved mirror; the image source is configured to generate an image light beam and emit the image light beam to the curved mirror; the curved mirror is configured to rotate to a preset angle according to the eye movement data, so that the head-up display device projects an image of at least one of the first visual area or the second visual area.

6. The head-up display system of claim 5, wherein, the at least two visual areas further comprise a third visual area, and the first visual area and the second visual area are arranged in a set direction, and the first visual area is located between the second visual area and the third visual area along the set direction; an angle of the curved mirror when the head-up display device projects the first visual area is a first angle, an angle of the curved mirror when the head-up display device projects the second visual area is a second angle, and an angle of the curved mirror when the head-up display device projects the third visual area is a third angle, and the first angle is located between the second angle and the third angle.

7. The head-up display system of claim 5, wherein, the image source comprises a display area; The first region in the display area generates a first image light beam when projecting an image of the first visual area; and the second region in the display area generates a second image light beam when projecting an image of the second visual area; The coincidence degree of the first region and the second region is greater than a preset coincidence degree, wherein the coincidence degree is a ratio of an overlapping area of the first region and the second region to a sum of areas of the first region and the second region.

8. The head-up display system of claim 1, wherein, The pupil tracking module comprises a light flow module.

9. A carrier, characterized by The head-up display system comprises the head-up display device and a windshield glass. The image light beam emitted by the head-up display device projects onto the windshield glass to form an image of one of the at least two visual areas.

10. The vehicle of claim 9, wherein, The pupil tracking module is arranged on the windshield glass.

11. A control method of a head-up display system, applied to the head-up display system of any one of claims 1-8, characterized in that, The head-up display device comprises: The eye movement data is obtained from the pupil tracking module. The head-up display device is controlled to project at least an image of the first visual area or the second visual area according to the eye movement data.

Images