Head-mounted display and near-eye display method

Abstract

The application provides a head-mounted display, comprising: a display module configured to display a compensation image; an imaging module configured to modulate the compensation image into a near-eye display image and project the near-eye display image into a user's eye; a driving module connected to the imaging module and configured to adjust a position of the imaging module; an eye movement tracking module configured to obtain a contour image of the eye; and a processor electrically connected to the eye movement tracking module and the driving module, configured to calculate a gaze position of the eye on the display module according to the contour image, and configured to transmit a driving control signal to the driving module, so that the driving module adjusts the position of the imaging module according to the driving control signal, so that an optical axis of the imaging module coincides with the gaze position; the processor is also electrically connected to the display module, and configured to make the display module generate the compensation image with the gaze position as the center. The application also provides a near-eye display method.

Description

Technical Field

[0001] This application relates to the field of displays, and more particularly to a head-mounted display and a near-eye display method. Background Technology

[0002] Existing virtual reality (VR) displays typically project the image displayed on the display module onto the viewer's eye through a lens. Since the lens itself distorts the image, the image to be displayed is usually pre-distorted on the display module; that is, the display module displays a compensated image. This compensated image is then projected onto the viewer's eye through the lens, allowing the observer to see a corrected near-eye display image. However, the greater the degree of distortion compensated for in the compensated image, the less clear the near-eye display image becomes after passing through the lens. Because the optical axis of the lens is usually fixed and coincides with the center of the display module, the image clarity at the edges of the display module is poor. When the viewer focuses on the edges of the display module, the degree of image distortion is high, affecting the user experience. Summary of the Invention

[0003] This application provides a head-mounted display, comprising:

[0004] Display module, used to display a compensated image;

[0005] An imaging module is used to modulate the compensated image into a near-eye display image and project the near-eye display image into the user's eyes;

[0006] A driving module, connected to the imaging module, is used to adjust the position of the imaging module;

[0007] An eye-tracking module for acquiring the contour image of the eyes; and

[0008] The processor, electrically connected to the eye-tracking module and the driving module, is used to calculate the gaze position of the eye on the display module based on the contour image, and to transmit a driving control signal to the driving module so that the driving module adjusts the position of the imaging module according to the driving control signal so that the optical axis of the imaging module coincides with the gaze position; the processor is also electrically connected to the display module so that the display module generates the compensation image centered on the gaze position.

[0009] The head-mounted display provided in this application embodiment, by setting an eye-tracking module and a processor, facilitates the calculation of the eye's gaze position. Then, by setting a driving module to adjust the position of the imaging module, the optical axis of the imaging module can move with the eye's gaze center. By generating a compensation image centered on the gaze center, the eye's gaze position always has high clarity and low distortion, which is beneficial to improving the visual experience.

[0010] In one embodiment, the processor is further configured to set the position of each pixel on the compensation image based on the distance between each pixel on the display module and the gaze position.

[0011] In one embodiment, the processor is further configured to calculate the angle by which the imaging module needs to move based on the gaze position, in order to generate the drive control signal.

[0012] In one embodiment, the driving module includes a first driving module and a second driving module, wherein the first driving module is used to control the optical axis of the imaging module to move along a first direction, and the second driving module is used to control the optical axis of the imaging module to move along a second direction.

[0013] This application also provides a near-eye display method for projecting a near-eye display image onto a user's eye via an imaging module, including:

[0014] Obtain an image of the eye's outline and calculate the eye's gaze position;

[0015] A compensation image is generated, wherein the compensation center of the compensation image coincides with the gaze position;

[0016] Adjust the imaging module so that its optical axis coincides with the gaze position.

[0017] The near-eye display method provided in this application calculates the eye's gaze position and adjusts the position of the imaging module and the center position of the compensated image according to the gaze position. This ensures that the eye's gaze position always has high clarity and low distortion, which helps improve the clarity of the image and enhances the user experience.

[0018] In one embodiment, the steps of generating the compensation image and adjusting the imaging module are performed simultaneously.

[0019] In one embodiment, generating the compensation image specifically includes: setting the position of each pixel on the compensation image according to the distance between each pixel on the compensation image and the gaze position.

[0020] In one embodiment, before adjusting the imaging module to make its optical axis coincide with the gaze position, the method further includes: calculating the angle that needs to be adjusted to make the optical axis of the imaging module coincide with the gaze position.

[0021] In one embodiment, the step of adjusting the imaging module to make the optical axis of the imaging module coincide with the gaze position includes: controlling the optical axis of the imaging module to move along a first direction and a second direction respectively.

[0022] In one embodiment, the steps of acquiring an eye contour image, calculating the eye's gaze position, and adjusting the imaging module are performed continuously. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the structure of a head-mounted display in one embodiment of this application.

[0024] Figure 2 for Figure 1 A schematic diagram showing the relative positions of the imaging module and the display module.

[0025] Figure 3 This is a schematic diagram of the optical path structure of a head-mounted display in one embodiment of this application.

[0026] Figure 4 for Figure 3 A schematic diagram showing the relative positions of the imaging module and the display module.

[0027] Figure 5 This is a flowchart illustrating a near-eye display method in one embodiment of this application.

[0028] Explanation of main component symbols

[0029] Head-mounted display 100

[0030] Display Module 10

[0031] Imaging module 30

[0032] Optical axis 31

[0033] Drive Module 50

[0034] First drive module 51

[0035] Second drive module 53

[0036] Eye-tracking module 70

[0037] Connecting cable 80

[0038] Processor 90

[0039] Eyes E

[0040] gaze positions O, O'

[0041] First direction X

[0042] Second direction Y

[0043] Distances r0, r1, r2, r3

[0044] Steps S1, S2, S3

[0045] The following detailed description, in conjunction with the accompanying drawings, will further illustrate this application. Detailed Implementation

[0046] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0047] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0048] To further illustrate the technical means and effects adopted by this application in achieving its intended purpose, the following detailed description of this application is provided in conjunction with the accompanying drawings and preferred embodiments.

[0049] Please see Figure 1 The head-mounted display 100 of this application embodiment includes: a display module 10, an imaging module 30, a driving module 50, an eye-tracking module 70, and a processor 90. The display module 10 displays a compensation image, the imaging module 30 modulates the compensation image into a near-eye display image, and projects the near-eye display image onto the user's eye E. The driving module 50 is connected to the imaging module 30 and is used to adjust the position of the imaging module 30. The eye-tracking module 70 acquires a contour image of the eye E. The processor 90 is electrically connected to the display module 10, the driving module 50, and the eye-tracking module 70, and is used to calculate the gaze position O of the eye E based on the contour image, and transmit a driving control signal to the driving module 50, causing the driving module 50 to adjust the position of the imaging module 30 according to the driving control signal, so that the optical axis 31 of the imaging module 30 coincides with the gaze position O, and to cause the display module 10 to generate a compensation image centered on the gaze position O.

[0050] The display module 10 can be a liquid crystal display module or a self-emissive display module such as a micro light-emitting diode display module, and this application does not limit it.

[0051] The imaging module 30 can be a single lens or a combination of multiple lenses. For example, the imaging module 30 can be a single Fresnel lens, or a folded optical path formed by a combination of multiple lenses and optical thin films; this application does not impose any limitations on this.

[0052] Please refer to the following: Figure 1 and Figure 2 The driving module 50 includes a first driving module 51 and a second driving module 53. The first driving module 51 is used to control the optical axis 31 of the imaging module 30 to move in a first direction X by adjusting the position of the imaging module 30. The second driving module 53 is used to control the optical axis 31 of the imaging module 30 to move in a second direction Y by adjusting the position of the imaging module 30. Specifically, the first driving module 51 and the second driving module 53 are used to adjust the position of the imaging module 30 in different directions, so that the imaging module 30 can rotate relative to the eye E. The adjustment range of the position of the imaging module 30 is greater than the rotation range of the eye E, so that when the eye E looks at any position, the optical axis 31 of the imaging module 30 can coincide with the gaze direction of the eye E.

[0053] The first drive module 51 and the second drive module 53 can be voice coil motors or other drive devices, and this application does not limit them.

[0054] An eye-tracking module 70 is used to acquire a contour image of the eye E. Specifically, the eye-tracking module 70 may include an infrared light source and a camera (not shown). The infrared light source emits infrared light towards the eye E, and the camera receives the infrared light reflected back from the eye E, thereby obtaining a contour image of the eye E. The contour image may include information such as an image of the cornea or an image of the pupil; this application is not limited to this, as long as the gaze position of the eye E can be calculated from the contour image, it is within the scope of this application.

[0055] The processor 90 can be a central processing unit or an integrated circuit composed of multiple chips; this application does not limit it in this regard.

[0056] The processor 90 is electrically connected to the eye-tracking module 70 via a connecting cable 80. It receives the contour image of the eye E and calculates the gaze position O of the eye E based on the contour image. Specifically, since the contour image records information such as the cornea or pupil of the eye E, the gaze direction of the eye E can be obtained by calculating the information recorded in the contour image. Because the distance between the eye E and the display module 10 is relatively fixed, the position of the display module 10 that the eye E is gazing at can be calculated using the calculated gaze direction of the eye E.

[0057] For example, please refer to the following: Figure 3 and Figure 4When the gaze position of eye E shifts from the center position of display module 10, the contour image of eye E acquired by eye tracking module 70 will change. Based on the change in the contour image, processor 90 can calculate the angle of eye E deflection, thereby calculating the gaze direction of eye E. Given the distance between eye E and display module 10, the gaze position O' corresponding to the gaze direction of eye E on display module 10 can be calculated.

[0058] The processor 90 is electrically connected to the drive module 50 via a connecting cable 80. It calculates the angle that the imaging module 30 needs to move based on the gaze position O' of the eye E, generates a drive control signal, and sends the drive control signal to the drive module 50. Specifically, after calculating the gaze position O' of the eye E, the processor 90 also calculates the angle that the imaging module 30 needs to move from its current position to make the optical axis 31 completely coincide with the gaze position O', including the angle to move in the first direction X and the angle to move in the second direction Y. After calculating the angles to move in the first direction X and the second direction Y, the processor 90 generates the drive control signal and transmits it to the drive module 50, thereby driving the imaging module 30 to move.

[0059] Please refer to the following: Figures 1 to 4 The processor 90 is electrically connected to the display module 10 via a connecting cable 80. It generates a compensation image centered on the gaze position O of the eye E. The processor 90 sets the position of each pixel on the compensation image according to the distance between each pixel on the display module 10 and the gaze position O. Specifically, the compensation image is used to adjust the distortion level of the imaging module 30. Near the optical axis 31, the distortion level of the portion of the image incident from the display module 10 to the eye E is relatively small; far from the optical axis 31, the distortion level of the portion of the image incident from the display module 10 to the eye E is relatively large. Different compensation coefficients need to be set to compensate for the distortion based on the distance between each pixel on the display module 10 and the gaze position O.

[0060] For example, when eye E is focused on the center position of display module 10 (e.g.) Figure 2 As shown), the distances between pixels on display module 10 and the gaze position O can be divided into r0, r1, r2, and r3 from near to far. r0 corresponds to the region with the least distortion, while r3 corresponds to the region with the most distortion. Different compensation coefficients can be set for different distances r0, r1, r2, and r3 to generate a compensated image. When eye E gazes at the lower left corner of display module 10 (e.g., ...), ... Figure 4As shown, the gaze position O' is located at the lower left corner of the display module 10. At this time, the distance between the pixel on the display module 10 and the gaze position O' can also be divided into r0, r1, r2, and r3 from near to far.

[0061] In this embodiment, the positions and numbers of distances r0, r1, r2, and r3 can be adjusted according to the distortion degree of the imaging module 30. Specifically, the distortion degree of the imaging module 30 is usually smaller near the optical axis 31, so the distance r0 can be set to a larger range. Further away from the optical axis 31, the distortion degree of the imaging module 30 is larger, and the rate of distortion change is also faster; therefore, the arrangement of distances r1, r2, and r3 can be set to gradually become denser. In other embodiments, more compensation distances can also be set at the position of the imaging module 30 to improve the compensation effect.

[0062] The head-mounted display 100 provided in this application embodiment uses an eye-tracking module 70 to acquire the contour image of the eye E in real time. A processor 90 calculates the gaze position O of the eye E, and based on this, calculates the angle at which the imaging module 30 needs to move, generates a compensation image centered on the gaze position O, sends a drive control signal to the drive module 50, and transmits the compensation image to the display module 10. Thus, while driving the imaging module 30, the corresponding compensation image is displayed. This ensures that the eye E can see the image with the least distortion and the highest resolution at different positions on the display module 10, thereby achieving the best visual effect and improving the user experience.

[0063] This application also provides a near-eye display method for projecting a near-eye display image onto the user's eye using an imaging module. Please refer to [link to relevant documentation]. Figure 5 It includes:

[0064] Step S1: Obtain the outline image of the eye and calculate the gaze position of the eye;

[0065] Step S2: Generate a compensation image, wherein the compensation center of the compensation image coincides with the gaze position;

[0066] Step S3: Adjust the imaging module so that the optical axis of the imaging module coincides with the gaze position.

[0067] The above-described near-eye display method can be used in conjunction with the head-mounted display 100 in the embodiments of this application. The near-eye display method will be further described below in conjunction with the head-mounted display 100. The above-described near-eye display method can also be applied to other head-mounted displays that can implement the corresponding steps, and this application does not impose any limitations on this.

[0068] Step S1 specifically includes: acquiring a contour image of the eye E through the eye-tracking module 70. The contour image is used to calculate the gaze direction of the eye E and may include information such as the cornea, pupil, and reflective area. After calculating the gaze direction of the eye E, the step further includes calculating the gaze position O of the eye E on the display module 10 based on the corresponding position between the eye E and the display module 10.

[0069] Steps S2 and S3 are performed simultaneously. Specifically, step S2 is used to adjust the position of the imaging module 30, and step S3 is used to display the corresponding compensation image on the display module 10. The compensation image is used to compensate for the distortion of the imaging module 30. Therefore, the desired near-eye display image can only be presented in the eye E when the center positions of the imaging module 30 and the compensation image coincide with the gaze position of the eye E.

[0070] Step S2 specifically includes: calculating the position of each pixel in the compensated image based on the distance between each pixel in the near-eye display image to be displayed and the gaze position O. Specifically, the closer the distance between each pixel in the near-eye display image to be displayed and the gaze position O, the smaller the distortion after passing through the imaging module 30; the farther the distance, the greater the distortion. Therefore, it is necessary to use different compensation coefficients to compensate for each pixel based on its distance from the gaze position O, thereby obtaining the compensated image.

[0071] Before adjusting the imaging module 30 so that its optical axis 31 coincides with the gaze position O, the process also includes calculating the angle that needs to be adjusted for the optical axis 31 of the imaging module 30 to coincide with the gaze position O. Specifically, the position of the imaging module 30 changes with the gaze position O. Therefore, each time the position that the imaging module 30 needs to be adjusted to is recalculated, the angle that needs to be adjusted needs to be recalculated based on the current position of the imaging module 30, so that the imaging module 30 can be moved to the preset position.

[0072] Step S3 further includes: controlling the imaging module 30 to move along the first direction X and the second direction Y respectively. Specifically, the imaging module 30 is controlled by the first driving module 51 and the second driving module 53 respectively, so when adjusting the angle, it needs to move in the first direction X and the second direction Y respectively.

[0073] Steps S1, S2, and S3 are performed continuously. Specifically, the gaze position O of eye E will constantly change, so it is necessary to acquire the contour image of eye E in real time to ensure that the gaze position of eye E always maintains the best visual effect.

[0074] The near-eye display method provided in this application acquires the contour image of the eye E, calculates the gaze position O of the eye E, and adjusts the optical axis of the imaging module 30 to coincide with the gaze position O, and generates a corresponding compensation image. This helps to ensure that the eye E always gazes at the position with the least distortion and the highest resolution, which helps to maintain the best viewing effect and improve the user experience.

[0075] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A head-mounted display, characterized in that, include: Display module, used to display a compensated image; An imaging module is used to modulate the compensated image into a near-eye display image and project the near-eye display image into the user's eyes; A driving module, connected to the imaging module, is used to adjust the position of the imaging module; An eye-tracking module is used to acquire the outline image of the eyes; as well as The processor, electrically connected to the eye-tracking module and the driving module, is used to calculate the gaze position of the eye on the display module based on the contour image, and to transmit a driving control signal to the driving module so that the driving module adjusts the position of the imaging module according to the driving control signal so that the optical axis of the imaging module coincides with the gaze position. The processor is also electrically connected to the display module, and is used to enable the display module to generate the compensation image centered on the gaze position; The processor is further configured to divide the near-eye display image into multiple regions r0, r1, r2, and r3 from near to far based on the distance between each pixel and the gaze position, and to set the distance of region r0 to be greater than the spacing between regions r1, r2, and r3 according to the distortion degree of the imaging module, so that the arrangement of regions r0, r1, r2, and r3 gradually becomes denser as they move away from the gaze position. The processor is also configured to set the position of each pixel on the compensation image according to different compensation coefficients corresponding to different regions to generate the compensation image.

2. The head-mounted display as described in claim 1, characterized in that, The processor is also configured to calculate the angle that the imaging module needs to move based on the gaze position, so as to generate the drive control signal.

3. The head-mounted display as described in claim 1, characterized in that, The driving module includes a first driving module and a second driving module. The first driving module is used to control the optical axis of the imaging module to move along a first direction, and the second driving module is used to control the optical axis of the imaging module to move along a second direction.

4. A near-eye display method for projecting a near-eye display image onto a user's eye via an imaging module, characterized in that, include: Obtain an image of the eye's outline and calculate the eye's gaze position; A compensation image is generated, wherein the compensation center of the compensation image coincides with the gaze position; Adjust the imaging module so that its optical axis coincides with the gaze position; The step of generating the compensation image includes: dividing the near-eye display image into multiple regions r0, r1, r2, and r3 from near to far based on the distance between each pixel and the gaze position; setting the distance of region r0 to be greater than the spacing between regions r1, r2, and r3 according to the distortion degree of the imaging module; and making the arrangement of regions r0, r1, r2, and r3 gradually denser as they move away from the gaze position; thereby setting the position of each pixel on the compensation image according to the different compensation coefficients corresponding to different regions to generate the compensation image.

5. The near-eye display method as described in claim 4, characterized in that, The steps of generating the compensated image and adjusting the imaging module are performed simultaneously.

6. The near-eye display method as described in claim 4, characterized in that, Before adjusting the imaging module to align its optical axis with the gaze position, the method further includes: Calculate the angle that needs to be adjusted to make the optical axis of the imaging module coincide with the gaze position.

7. The near-eye display method as described in claim 6, characterized in that, The step of adjusting the imaging module to make the optical axis of the imaging module coincide with the gaze position includes: controlling the optical axis of the imaging module to move along a first direction and a second direction respectively.

8. The near-eye display method as described in claim 4, characterized in that, The steps of acquiring the contour image of the eye, calculating the gaze position of the eye, generating the compensation image, and adjusting the imaging module are performed continuously.

Images