System and method for enhancing vision

Abstract

A head-mounted display system includes a target object detection module configured to receive image pixels and corresponding depths of a first portion and a second portion of a target object; a first light emitter configured to emit first eye light signals to display a first eye virtual image of the first portion and the second portion of the target object to a viewer; a first light direction adjuster configured to adjust a light direction of each of the first eye light signals from the first light emitter; a first collimator; and a first light combining element configured to redirect and converge the first eye light signals to a first eye of the viewer. The first portion of the first eye virtual image of the target object in a first field of view has a higher number of first eye light signals per degree than a second portion of the first eye virtual image of the target object in a second field of view.

Description

Background of the Invention

[0002] Related applications

[0003] This application claims priority to U.S. Provisional Patent Application No. 63 / 147,214, filed February 8, 2021, entitled "AR / VR SYSTEM AND METHOD FORIMPROVING VISUAL CAPABILITY", and U.S. Provisional Patent Application No. 63 / 217,297, filed July 1, 2021, entitled "SYSTEM AND METHODFOR IMPROVING VISUAL ACUITY IN AR / VR ENVIRONMENT".

[0004] Furthermore, the entire contents of PCT international patent application No. PCT / US21 / 52750, filed on September 30, 2021, entitled "Virtual Image Display System for Virtual Reality and Augmented Reality Devices," and PCT international patent application No. PCT / US2020 / 059317, filed on November 6, 2020, entitled "System and Method for Displaying an Object with Depth," are incorporated herein by reference. Technical Field

[0005] This invention relates to systems and methods for enhancing vision; more specifically, to systems and methods for generating machine-assisted vision with a visual acuity exceeding 1.0. Background Technology

[0006] Visual ability is often limited by the anatomy of the human eye. In particular, parameters such as the refractive power of the lens, the axial length of the eyeball, and the condition of the cornea and retina significantly affect visual ability. Currently, there are no machine vision systems on the market that outperform normal vision. However, in recent years, machine vision has been developed and applied in many fields to improve the vision of people with visual impairments. Specifically, machine vision has been used in the medical field to help patients with blindness or visual impairments, such as glaucoma and myopic macular degeneration. For example, pixel art enhancement can help restore vision by increasing the brightness or contrast of the object the patient is viewing. However, this can only partially restore the patient's vision.

[0007] Therefore, it is necessary to provide a head-mounted display device / system that is easy to use in daily life and can partially or substantially replace / enhance human natural vision, providing users with vision superior to normal human vision. Summary of the Invention

[0008] The present invention envisions using a light-emitting device based on light signal scanning in a head-mounted display system / device (i.e., AR glasses or VR goggles) or retinal scanning device to help people with visual impairments or visual disturbances resume normal life. Specifically, the invention can capture real-time image information of a target object or the viewer's surrounding environment for the viewer to see, and reproduce a stereoscopic digital (or pixelated) image with depth perception to the viewer. Furthermore, the invention can replace traditional prescription glasses for vision correction in people with myopia or presbyopia. The invention can also improve the vision of normal individuals, making it superior to normal visual ability.

[0009] The head-mounted display system includes a target object detection module, a first light emitter, a first light direction adjuster, a first collimator, and a first light combining element. The target object detection module receives several image pixels from a first portion and a second portion of a target object. The first light emitter emits several first-eye light signals related to the target object. For example, these first-eye signals can directly reproduce the image pixels of the target object, allowing the viewer to see a first-eye virtual image of the target object through the head-mounted display system. The first light emitter can generate light pulses to create pixelated images. In some examples, the light emitter may include a red laser diode, a green laser diode, and a blue laser diode. The first light direction adjuster receives the several first-eye light signals emitted by the first light emitter and changes the light direction of the several first light signals from the first light emitter. The direction of the light beam can change in several spatial dimensions over time, thereby creating an image through the periodic scanning motion of the first light beam direction adjuster, producing an image frame over a period of time. The light beam direction adjuster mentioned in this invention can be a mechanical or optical element capable of dynamically changing the direction of the light beam emitted by the light emitter over time. The first collimator can be disposed between the first light emitter and the first light beam direction adjuster to make several first eye light signals from the first light emitter parallel. In another embodiment, the first collimator can be disposed between the first light beam direction adjuster and the first light combining element. Furthermore, the first collimator can change the optical path of the first eye light signal.

[0010] The first light-combining element is used to redirect and converge the plurality of first-eye light signals to the viewer's first eye. In some embodiments, the first light-combining element receives the plurality of first-eye light signals from the first light direction adjuster, converges the plurality of first-eye light signals, and directs them to the viewer's first eye. The first light direction adjuster can rotate within a certain range or move linearly within a certain range. The light direction of the plurality of first-eye light signals can also be changed within a specific range; when the first light-combining element receives the plurality of first-eye light signals from the first light direction adjuster 100, the first light-combining element directs the first-eye light signals with different incident angles to the viewer's first eye. The plurality of first-eye light signals enter the viewer's first eye within a preset incident angle range, the incident angle range being equivalent to the maximum field of view generated by the head-mounted display system. In some embodiments, the size of the light spot of the first-eye light signal projected onto the retina of the first eye can be manipulated by changing the distance between the light direction adjuster and the first collimator.

[0011] In one embodiment, the spot size can be adjusted by changing the projection time of a single pixel. Thus, the spot size can be adjusted in real time to meet the Rayleigh criterion for different visual acuity settings. In some variations of this embodiment, the projection time can be effectively increased by repeatedly projecting the same pixel or image pixel in different rows or columns.

[0012] According to one embodiment of the present invention, the distance between two adjacent projected light signals can be changed by adjusting the rate of change of the direction of the light direction adjuster. In one embodiment of the present invention, the spot size of the plurality of first eye light signals can be reduced so that there is no gap between any two adjacent first eye light signals; therefore, it is not necessary to change the oscillation frequency of the light direction adjuster or the emission frequency of the light emitter.

[0013] In another embodiment of the invention, only a portion of the virtual image frames are projected with a higher amount of light signal per unit angle (i.e. per degree) of field of view.

[0014] In some embodiments, the present invention can be applied to a head-mounted device for vision correction or vision training. The present invention can be used to correct or improve eye conditions, such as, but not limited to, myopia, hyperopia, strabismus, amblyopia, and convergence disorder.

[0015] This invention can capture real-time image pixels of a target object or its surrounding environment and reproduce a stereoscopic digital image with improved image quality for a viewer in an AR / VR system. The viewer can adjust the image quality to achieve better than normal vision (i.e., above 20 / 20 visual acuity or VA 1.0). Furthermore, this invention can assist visually impaired individuals or replace traditional prescription glasses for vision correction in individuals with myopia or hyperopia. This invention can be used by healthcare workers, military personnel, precision manufacturing industries, pilots, law enforcement officers, emergency medical personnel, and athletes, among others. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the Rayleigh Criterion.

[0017] Figure 2 This is a schematic diagram illustrating the change in the size of the light spot on different projection planes.

[0018] Figure 3 This is a schematic diagram illustrating the display system of the present invention.

[0019] Figure 4A This is a schematic diagram illustrating the effect of changing the position of the collimator.

[0020] Figure 4BThis is another schematic diagram illustrating the effect of changing the collimator position.

[0021] Figure 5A This is a schematic diagram illustrating the effect of changing the emission time on the size of the light spot.

[0022] Figure 5B This is another schematic diagram illustrating the effect of changing the emission time on the size of the light spot.

[0023] Figure 6 This is a schematic diagram illustrating the effect of changing the spatial spacing between adjacent optical signals on the field of view (FOV).

[0024] Figure 7 This is a schematic diagram illustrating an embodiment of changing the spatial spacing between adjacent optical signals.

[0025] Figure 8 This is a schematic diagram illustrating an embodiment of changing the oscillation frequency of the light direction adjuster to alter vision.

[0026] Figure 9 This is a schematic diagram illustrating another embodiment of the display system of the present invention.

[0027] Figure 10 This is another schematic diagram illustrating another embodiment of the display system of the present invention.

[0028] Figure 11 This is another schematic diagram illustrating another embodiment of the display system of the present invention.

[0029] Figure 12 This is another schematic diagram illustrating another embodiment of the display system of the present invention.

[0030] Figure 13 This is a schematic diagram illustrating an embodiment of the optical element in this invention.

[0031] Figure 14 The following is another schematic diagram illustrating an embodiment of the optical element in this invention.

[0032] Figure 15 This is a schematic diagram illustrating one application of the present invention. Detailed Implementation

[0033] The terminology used herein is for the purpose of describing details in specific embodiments of the invention, and all terms should be interpreted in the broadest sense. Certain terms will be specifically highlighted below; any limiting terms will be defined by the specific embodiments.

[0034] The objectives of this invention are, on the one hand, to improve vision and surpass normal visual abilities for healthy individuals. On the other hand, to provide improved vision for visually impaired individuals, such as those with myopia or astigmatism, or to assist in training impaired eyes to improve vision. Furthermore, this invention can substantially replace the vision of blind or other severely visually impaired individuals. This invention can serve as an assistive device to enhance the viewer's vision; in some cases, it can partially or almost completely replace the eye function of visually impaired individuals. Moreover, for visually impaired viewers, the system and method of this invention can bypass damaged or destroyed eye tissue and provide a clear image to the healthy portion of the viewer's retina. This invention takes into account the axial length of the eyeball and the condition of the viewer's cornea and retina, incorporating these factors into its design; furthermore, this invention can provide multiple or continuous depth perception, enabling the viewer to perceive the highest degree of realism. This invention can receive several image pixels of a near or distant target object and reproduce a virtual image of the target object for the viewer with higher visual acuity.

[0035] The term "visual acuity" relates to the number of critical gaps that a viewer can distinguish within one arcminute of their field of vision. The general definition of visual acuity (VA) is as follows:

[0036] Visual acuity = 1 / gap size (in arc minutes); and

[0037] 1 arc minute = 1 / 60 degree

[0038] For visual acuity of 1.0 (or 20 / 20), the viewer's eye must be able to distinguish one contrast pattern (i.e., a black and white pattern) within one arcminute of the visual field. In other words, for a person with visual acuity of 1.0, the viewer's eye must be able to distinguish 60 contrast patterns within one degree of the visual field. For example, if a person can only distinguish one contrast image within two arcminutes of the visual field, their visual acuity is 0.5; and if a person can distinguish one contrast image within 0.5 arcminutes of the visual field, their visual acuity is 2. The visual acuity described in this invention relates to the number of pixels that the retina can distinguish / separate within one arcminute (i.e., one-sixtieth of a degree of the visual field). In other words, to achieve a visual acuity superior to general (i.e., VA 1.0) using a pixelated display system, the goal is to project an image of at least a portion of a target object to the viewer, which increases the number of resolvable pixels per degree of the visual field for the retina; or the goal is to project an image of the target object with more than sixty resolvable light signals / pixels onto the retina per degree of the visual field. This invention is designed to generate a virtual image of at least a portion of a target object, the virtual image having a visual acuity greater than 1.0 for the viewer. Therefore, in some embodiments of this invention, the objective is to project a virtual image of the target object having more than 120 (VA2.0) or 180 (VA3.0) resolvable light signals / pixels per degree of visual field onto the retina, the virtual image comprising at least a portion of the virtual image. That is, to achieve optimal visual acuity, a display system must project the highest possible number of resolvable / separable pixels onto the retina of the viewer's eye per degree of visual field. The spatial spacing between two adjacent light signals / pixels on the retina (or other parts of the viewer's eye) can be the spatial distance between the center points of any two adjacent light signals / pixels on the cross-section of the optical path. This spatial distance needs to satisfy a criterion that the two adjacent light signals are separated. Therefore, in order to project more discrete light signals (resolvable pixels) onto the retina within one arc minute of the visual field, it is necessary to control the size of the light signal / pixel on the retina, usually by reducing it, so that more discrete light signals can be contained within one arc minute of the visual field.

[0039] Reference Figure 1Regardless of the distance and size of the target object to be reproduced in a virtual image for the viewer, the virtual image of the target object must be resolvable on the viewer's retina. More specifically, the light signals (light pulse signals or beam signals) that generate the virtual image must satisfy the Rayleigh criterion, which is a specification of the minimum interval between two resolvable / separable light signals. In a pixelated display system, light signals correspond to pixels that generate the virtual image. For example, each light signal may contain one pixel of the virtual image of the target object. When each light signal is projected onto a cross-section of the light path (i.e., the retina of the viewer's eye), each light signal has a near-circular or elliptical cross-sectional area (also known as the spot size). According to the Rayleigh criterion, the spatial distance / interval between the center points of two adjacent light signals must be greater than half the maximum diameter of the adjacent light signals (approximately half the spot size) for the two light signals to be resolvable. For the reasons stated above, while increasing the number of light signals per unit field of view (e.g., per degree) to improve visual acuity is feasible, the spacing between two adjacent light signals must satisfy the Rayleigh criterion for them to be resolvable / separable. The number of light signals per degree of field of view (similar to the concept of resolution) does not necessarily need to be increased for the viewer. A system must also maintain the spot size of the light signals and the spatial distance / spacing between the center points of two adjacent light signals to ensure that these light signals are resolvable / separable.

[0040] The spot size of the optical signal is related to the dispersion angle and the distance between the light emitter and the cross-section of the optical path (projection plane). (Refer to...) Figure 2 With the dispersion angle α remaining constant throughout the optical path, the size of the light spot projected onto the projection plane increases with the distance between the light emitter and the projection plane (the size of the light spot on plane 1 is smaller than the size of the light spot on plane 2). Similarly, with the distance between the light emitter and the projection plane remaining constant, the size of the light spot increases with the dispersion angle. In this invention, the final dispersion angle of the light signal entering the eye, and the optical path between the light emitter and the viewer's retina, can be adjusted in several ways. These will be described in more detail below.

[0041] To improve vision, pixel density needs to be increased and any two adjacent light signals need to be resolvable (distinguishable / separable). This pixel density is the number of light signals (such as pixels) projected onto a unit area and is proportional to the unit angle of the field of view (such as per degree).

[0042] This invention provides a system and method for enhancing visual acuity by increasing the number of resolvable and separable light signals projected per unit angle of field of view (in degrees), thereby creating a virtual image frame on the retina with maximum field of view and light signal density. In one embodiment, the system and method for enhancing visual acuity can be applied to a head-mounted display system, such as AR / VR glasses, goggles, helmets, or other similar commercial or medical devices. In other examples, the invention can be applied to other fixed or non-head-mounted display devices. The embodiments of the invention are explained below using a head-mounted display system; furthermore, this head-mounted display system can provide pixelated virtual images to the viewer; therefore, light signals projected by a light emitter correspond to pixels in the virtual image. However, the invention is not limited to this embodiment.

[0043] Reference Figure 3 The head-mounted display system includes a target object detection module 300, a first light emitter 10, a first light direction adjuster 100, a first collimator 1000, and a first light combining element 20. The target object detection module 300 receives several image pixels of a first portion and a second portion of a target object. The target object can be the viewer's surrounding environment, a part of the surrounding environment, or a specific object in the surrounding environment. The target object detection module 300 may further include a distance detection unit 301 for determining the depth of the target object or at least a portion of the target object. To present a stereoscopic virtual image of the target object, the target object detection module 300 measures several points of the target object and establishes a stereoscopic outline of the target object so that a stereoscopic virtual image of the target object can be subsequently created.

[0044] The target object detection module 300 functions to receive several image pixels from different viewpoints. Each target object detection module can be mounted on a receiving position of the frame of the head-mounted display device via a connection structure, and this receiving position is adjustable. In one embodiment, the target object detection module can be moved via the connection structure to adjust the receiving position or a viewpoint. The connection structure may include a hinge to adjust the viewpoint of the target object detection module. The target object detection module is electrically connected to the frame via the connection structure to receive power or transmit data. In some embodiments, the target object detection module can be a camera incorporating a wide-angle lens, zoom lens, fisheye lens, or multi-purpose lens for various applications. Furthermore, the camera is not limited to an optical camera, but also includes an infrared camera for measuring temperature, a range imaging sensor (e.g., a time-of-flight rangefinder camera) for measuring depth, and other physical parameter measurement sensing modules.

[0045] In some embodiments, the target object detection module is rotatable. The target object detection module may include a first target object detection module and a second image module. In this embodiment, the target object detection module captures user or environmental images and processes them to identify the images.

[0046] The first light emitter 10 emits a first-eye light signal related to the target object. For example, through a head-mounted display system, the plurality of first-eye light signals can generate a first-eye virtual image of a first part and a second part of the target object for the viewer. The first light emitter 10 can generate light pulses to create pixelated virtual images. For example, the first light emitter 10 can be a laser emitter capable of emitting one light signal or pixel at a time. For example, the light emitter may include a red laser diode, a green laser diode, and a blue laser diode. The first light direction adjuster 100 receives the plurality of first-eye light signals emitted by the first light emitter 10 and changes the light direction of the plurality of first-eye light signals. The light direction can change over time in several spatial dimensions, thereby creating an image through the periodic scanning action of the first light direction adjuster 100 to generate an image frame over a period of time. The light direction adjuster of the present invention can be a mechanical or optical element capable of dynamically changing the direction of light emitted by a light emitter over time. One example could be, but is not limited to, a one-dimensional, two-dimensional, or three-dimensional microelectromechanical system (MEMS) mirror. The first collimator 1000 can be disposed between the first light emitter 10 and the first light direction adjuster 100 to make the plurality of first optical signals from the first light emitter 10 parallel. In another embodiment, the first collimator 100 can be disposed between the first light direction adjuster 100 and the first light combining element 20. Furthermore, the first collimator 1000 can separate the plurality of first optical signals from each other at the optical waist position of each of the plurality of first optical signals.

[0047] The first light-combining element 20 is used to redirect and converge the plurality of first-eye light signals to the viewer's first eye. The first eye can be the viewer's right or left eye. In some embodiments, the first light-combining element 20 receives the plurality of first-eye light signals from the first light direction adjuster 100, converges and redirects the plurality of light signals, and directs them to the viewer's first eye 50. More specifically, as an example, the first light direction adjuster 100 can rotate or linearly displace within a range. Thus, the light direction of the plurality of first-eye light signals can also change within a specific range; when the first light-combining element 20 receives the plurality of first-eye light signals (each light signal has a different incident angle) from the first light direction adjuster 100, the first light-combining element 20 redirects each first-eye light signal with a different incident angle to the viewer's first eye 50. Since the rotation or linear displacement of the first light direction adjuster 100 is preset, the incident angle range of the several first-eye light signals redirected to the viewer's first eye 50 is also preset, which is equivalent to the maximum field of view generated by the head-mounted display system.

[0048] In some embodiments of the invention, the head-mounted display system may employ a dual-axis design for the light-combining element to adjust its pitch and roll (rotation angles in the horizontal and vertical directions). Furthermore, the X, Y, and Z axis positions of the light-combining element can also be adjusted to accommodate the interpupillary distance of each viewer. In some other embodiments, the X, Y, and Z axis positions of the light-combining element may also be individually adjustable for each viewer.

[0049] Reference Figures 4A to 4B The dispersion angle of each first-eye light signal from the first combining element 20 to the first eye 50 determines the size of the light spot projected onto the viewer's retina. The spot size increases with increasing dispersion angle, and vice versa. According to one embodiment, the spot size of the first-eye light signal projected onto the retina can be adjusted by changing the distance between the first light direction adjuster 100 and the first collimator 1000. (Refer to...) Figures 4A to 4BThis figure illustrates how changing the distance between the first light direction adjuster 100 and the first collimator 1000 affects the spot size. The beam in the figure represents the optical path of a single first-eye light signal emitted by the first light emitter 10. Along the entire light path (optical path) from the first light emitter 10 to the viewer's first eye 50, the beam of the first-eye light signal undergoes several diverging / converging cycles. The cross-sectional area of ​​this beam (equivalent to the spot size) varies along different locations along the light path. In other words, the spot size will also differ at different locations along the light path. By changing the total light path between the light emitter and the viewer's eye, the cross-sectional area projected onto the viewer's retina will also change, as will the spot size. The spot size perceived by the viewer's eye will also depend on the specifications of each viewer's eye, such as the total refractive power, axial length, and retinal condition. These factors need to be considered during initial calibration for different users / viewers. Figure 4A and 4B In the process, the initial light signal emerges from the first beam combining element 20 and gradually converges, forming a beam waist (the point of minimum beam cross-sectional area) at different locations. Figure 4A In the process, before the light signal is incident on the first beam combining element, it forms a beam waist and diverges. Before reaching the viewer's first eye 50, the light signal is reflected by the first beam combining element 20 and then converges again. Figure 4B In the two figures, the beam waist is formed between the first beam combining element 20 and the viewer's first eye 50. Therefore, the spot size of the first-eye light signal provided to the viewer's first eye 50 is different. In this embodiment, the position of the beam waist can be adjusted by changing the position of the first collimator 1000, thereby adjusting the spot size of several first-eye light signals projected onto the viewer's first eye 50 (i.e., the retina), making the several first-eye light signals separable and resolvable according to the Rayleigh criterion. In other cases, the position of the beam waist can be adjusted by changing the distance between the first beam direction adjuster 100 and the first collimator 1000. Obviously, by manipulating the distance between the first beam direction adjuster 100 and the first collimator 1000, the most suitable spot size and beam spacing can be evaluated and determined for viewers with different eye conditions. Generally, the curvature of the collimator and the beam combining element can be customized for different users, as these parameters also affect the spot size. Furthermore, since the spot sizes in the two images are different, the spatial interval between the center points of two adjacent first-eye light signals will also change to see if it conforms to the Rayleigh criterion.

[0050] The following describes several methods to alter the viewer's perceived visual acuity by changing the number of primary light signals per degree within the viewer's field of vision. These methods involve modifying the spot size of the light signals and the spatial spacing between adjacent light signals.

[0051] Reference Figure 5A and 5B In one embodiment, the spot size can be adjusted by changing the projection time of a single pixel (e.g., via a control program). A light emitter projects the light signal / pixel of the image one at a time to form an image frame; the light emitter (fixed) changes its projection position by moving the light direction adjuster (not shown) to form another pixel of the image at a new position. Therefore, if the projection time of the light signal is shortened, the width of the pixel in the rotation direction of the light direction adjuster will also decrease; if the projection time of the light signal is extended, the width of the pixel in the rotation direction of the light direction adjuster will also increase. Thus, the spot size can be adjusted in real time under different visual acuity settings to conform to the Rayleigh criterion. In some variations of this embodiment, the projection time can be effectively increased by repeatedly projecting the same pixel or image pixel in pixels in different rows or columns. Figure 5A In this case, the light emission pulse is compared to Figure 5B There is a longer projection time. Figure 5A and 5B With the same oscillation frequency of the light direction adjuster, a longer projection time allows the light signal to sweep over a wider area, thus increasing the size of the light spot.

[0052] Considering that changing the spot size also affects the spacing between each optical signal, it is necessary to handle increases or decreases in the distance between each optical signal. Several embodiments for changing the spacing between each optical signal are described below.

[0053] According to an embodiment of the present invention, the distance between each projected light signal can be changed by adjusting the rate at which the light direction adjuster changes direction (i.e., the oscillation frequency). As previously described, the light direction adjuster of the present invention can change the light direction on one or two independent axes (e.g., a two-dimensional microelectromechanical system mirror). For example, a two-dimensional microelectromechanical system mirror can deflect light signals at high speed on two axes, so that its optical scanning angle reaches close to 30 degrees. However, the maximum scanning angle on one axis (the primary scanning axis) will be larger than the scanning angle on the other axis (the secondary scanning axis). The oscillation frequency and oscillation amplitude of the microelectromechanical system mirror can be controlled by applying driving voltages / electromagnetic fields of different frequencies and different intensities to the microelectromechanical system mirror, respectively. These are all prior art in the art. According to this embodiment, the first light direction adjuster 100 changes the first coordinate component or the second coordinate component of several first eye light signals at a non-constant oscillation frequency (or oscillation rate, i.e., the rotation or movement speed of the light direction adjuster relative to a reference point). The first or second coordinate component can be the X or Y coordinate component in a Cartesian coordinate system, or the θ coordinate component in a polar coordinate system. Coordinate components. When the present invention displays an image with higher visual acuity, the spot size of the light signal is reduced. The oscillation rate of the first light direction adjuster 100 can be reduced, thereby allowing each adjacent light signal to be projected with a smaller angular displacement, bringing the light signals projected onto the viewer's retina closer together (see reference). Figure 6 Thus, the amount of light signal projected per unit angle (e.g., per degree) within the field of view increases, increasing the viewer's perceived visual acuity. Since the swing angle of the light direction adjuster is constant, a virtual image frame with the same size but a greater light signal density is produced.

[0054] like Figure 6 As shown, to cope with the reduced oscillation frequency while maintaining a constant image projection frame rate, the scanning area of ​​the light direction adjuster can be reduced. This, in turn, leads to a smaller spatial interval between each projected light signal, thus reducing the field of view. However, the reduction in the field of view can actually increase visual acuity.

[0055] like Figure 7 As shown, in one embodiment of the present invention, the spot size of several first-eye light signals can be adjusted so that there is little or no gap between any two adjacent first-eye light signals. When the light signal is projected onto the cross-section of the light path, a spot area is generated. Figure 7 As shown, the spot regions of two adjacent light signals on the cross-section of the optical path are closely connected. Therefore, it is not necessary to change the oscillation frequency of the light direction adjuster or the emission frequency of the light emitter. However, in this embodiment, because the minimum spot size is limited, the highest visual acuity is also limited.

[0056] To maintain a sufficient frame rate, in another embodiment of the invention, only a portion of the virtual image frames are projected with a higher amount of light signal per unit angle (e.g., per degree) of field of view (i.e., high visual acuity). The concept of this embodiment is that when a human eye views a target object, the visual axis of the eye is oriented towards the target object, causing the image of the target object to be concentrated in the macula of the eye (the most sensitive part of the retina); therefore, the target object appears in the central visual field. Other parts of the image may become less clear relative to the target object in the image because these parts are projected onto other, less light-sensitive areas of the retina. Based on the aforementioned properties of human vision, this invention provides a central field of view (or first field of view), representing the first part (i.e., the central portion) of the virtual image frame of the target object, and a peripheral field of view (or second field of view), representing the second part (i.e., the peripheral portion) of the virtual image frame of the target object. The first part has a greater number of light signals per degree than the second part, so the user of this invention can see that the central field of view of the target object has a higher pixel density (higher visual acuity); while the image in the peripheral field of view of the target object does not need to be as clear as the central field of view, because the human eye cannot perceive higher quality images in the peripheral field of view. In other words, the number of first-eye light signals per degree of the first part of the target object in the first field of view is greater than the number of first-eye light signals per degree of the second part of the target object in the second field of view. In one embodiment, when the first light emitter 10 projects light signals for the central field of view, the number of projected light signals (or pixels) in the central field of view can be controlled by changing the oscillation frequency of the light direction adjuster. As mentioned above, in addition to changing the oscillation frequency, the number of light signals (or pixels) projected into the central field of view can also be further changed by changing the projection frequency or projection time of the light signals. These methods allow the time required for the light emitter to generate a frame (frame rate) to remain at a high rate. Virtual images generated using this method will have inconsistent pixel density.

[0057] For example, under normal conditions (when the oscillation frequency is a preset value), the first light emitter 10 and the first light direction adjuster 100 can form an image frame composed of a preset resolution (e.g., 1280*720 pixels) within a first spatial range (e.g., 40 degrees horizontally or 22.5 degrees vertically). The spatial range mentioned in this invention refers to the range of the first coordinate component and the second coordinate component, which can be represented by two coordinates. In this embodiment, the first spatial range corresponds to the field of vision of the first eye 50. The first image frame can be divided into two fields of vision: the peripheral field of vision and the central field of vision. The central field of vision of the image frame has a higher number of light signals per unit angle (e.g., per degree) compared to the peripheral field of vision (higher light signal / pixel density projected onto the retina). For example, the two coordinate components of the central field of vision can be set to 10 degrees of the total field of vision in the two coordinate components; the peripheral fields of vision on both sides can be set to 15 degrees of the total field of vision. When the viewer selects a higher visual acuity setting, the spot size of the light signal in the central field of vision decreases; simultaneously, the oscillation rate of the light direction adjuster decreases to compensate for the increased gap between each light signal. The light signals in the peripheral area can be generated with a preset spot size and a preset oscillation frequency of the light direction adjuster. Therefore, the viewer can perceive a higher visual acuity in the central field of vision than in the peripheral field of vision in image frame F1 with inconsistent pixel density. (See reference...) Figure 8 The following example demonstrates how to change pixel density along a coordinate component (e.g., horizontal or vertical). In this example, the total field of view in one direction is 40 degrees; the central field of view is set to 10 degrees, and the preset resolution per frame is 1280*720. If the target's visual acuity is 2.0, the total number of pixels required within the central field of view is:

[0058] 60 * 2 * 10 = 1200 pixels (where 60 is the number of pixels required to achieve a visual acuity of 1.0 per degree of field of view).

[0059] For the remaining field of view (peripheral field of view), the pixel density remains the same. The total number of pixels in the peripheral field of view in this direction is:

[0060] 1280*30 / 40=960 pixels

[0061] VA = 960 / 30 / 60 = 0.53

[0062] During the formation of the first image frame, the first light direction adjuster 100 continuously rotates and changes the projection direction of the first light emitter 10 to generate the first image frame row by row or column by column. Specifically, the first light emitter 10 generates the first image frame by projecting one pixel of the image at a time; the first light direction adjuster 100 then changes the direction of the first eye light signal 1000 to generate another pixel of the image at a new position. This new position is typically adjacent to the previous image pixel in a horizontal or vertical direction. Therefore, after a period of time, the first light emitter 10 generates one row or column of image pixels (e.g., 1280*1 or 1*720). The first light direction adjuster 100 then changes the direction of the first eye light signal 1000 to the next row or column, continuing to generate a second row or column of image pixels. This process is repeated until a complete image frame (e.g., a complete 1280*720 pixel image) is formed.

[0063] In some embodiments, while keeping the projection frequency of the light emitter constant, the oscillation frequency of the light direction adjuster can be reduced to increase the pixel density of the light direction adjuster in the secondary scanning direction (by reducing the spatial spacing between adjacent pixels). To increase the pixel density in the primary scanning direction, the projection frequency of the light emitter can be increased. For example, the oscillation frequency of the light direction adjuster can be reduced by half to increase the number of pixels in the secondary scanning direction from 720p to 1440p. The projection frequency of the light emitter can be doubled to increase the number of pixels in the primary scanning direction from 720p to 1440p.

[0064] In other cases, it is beneficial to have two one-dimensional light direction adjusters because the oscillation frequency of the light direction adjuster in the horizontal and vertical directions can vary depending on different areas of the field of view. As mentioned earlier, the oscillation frequency of the light direction adjuster in the horizontal and vertical directions varies according to the horizontal and vertical positions. When projecting an image in the central field of view, the projection rate can be increased and / or the oscillation frequency of the light direction adjuster in the horizontal and vertical directions can be decreased to project a higher density of pixels or image pixels in the central field of view (thus increasing the visual acuity of the central field of view). The projection rate and / or the oscillation frequency of the light direction adjuster in both directions can return to normal in the peripheral field of view.

[0065] In a variation of the above embodiments, the image in the central field of vision can have higher visual acuity, while the light emitter does not emit any light signals related to the peripheral field of vision. Specifically, in an AR head-mounted display system, only a portion of the target object (e.g., a distant target) is selected by the viewer to be displayed with higher visual acuity; the other portions of the target object are viewed by the viewer's natural vision. Therefore, only a small portion of the total field of vision needs to be displayed with a higher light signal per degree, while the remaining field of vision is displayed with zero light signal per degree. Because the field of vision that needs to be scanned by the light direction adjuster is small, the frame rate can be maintained at a relatively high level.

[0066] As previously mentioned, to surpass the visual acuity of ordinary people, it is necessary to increase the number of resolvable light signals projected onto the viewer's retina per unit field of view. To achieve this, the present invention takes into account the following factors: the size of the light spot projected onto the viewer's retina; the size of the field of view perceived by the viewer; and the spatial spacing between each light signal. In one embodiment of the present invention, the light spot size is related to the pixel size projected onto the retina. If the light spot size is too large, or the spatial spacing between adjacent pixels is too small (i.e., the spatial spacing between the center points of adjacent pixels), the pixels will overlap, making the pixels or image indistinguishable. On the other hand, if the light spot size is too small, or the spacing between pixels is too large, the number of pixels that can be placed in a unit angle of view (a region of the retina) will decrease. In both cases, visual acuity will decrease. For example, the following table illustrates the relationship between visual acuity and the above key factors. In this example, a laser projector is used as a light emitter. The resolution of the laser projector of the present invention is typically 1280*720 or 1920*1080 pixels. However, the resolution of the light emitter is not limited to these values. The data shown in the table below are derived from experimental results.

[0067]

[0068] Table 1

[0069] According to the table above, to achieve a visual acuity of 1.0, the number of resolvable pixels (satisfying the Rayleigh criterion) required in a 1-degree field of view is 60. Using a laser projector capable of producing 1280*720p resolution images, to achieve a visual acuity of 2.0, 120 resolvable pixels are required in a 10.67-degree field of view. To achieve a visual acuity of 3.0, 180 resolvable pixels are needed in a 7.1-degree field of view. To achieve a visual acuity of 4.0, 240 resolvable pixels are needed in a 5.33-degree field of view. Using a laser projector capable of producing 1920*1080p resolution images, to achieve a visual acuity of 2.0, 120 resolvable pixels are needed in a 16-degree field of view. To achieve a visual acuity of 4.0, 240 resolvable pixels are needed in an 8-degree field of view.

[0070] Furthermore, in order to achieve visual acuity exceeding that of ordinary people, it is necessary to select an appropriate curvature (focal length) of the light combining element for different users. Once selected, the distance between the light direction adjuster and the collimator can be adjusted to project a light signal with an appropriate spot size so that the viewer can experience the desired visual acuity setting.

[0071] The following describes an exemplary embodiment of the present invention. A head-mounted display system is used by a viewer as a visual aid to obtain vision superior to that of an average person (e.g., visual acuity greater than 1.0). A target object detection module 300 captures several image pixels of the target object. These image pixels may relate to a portion or the entire target object, depending on the physical size of the target object. Since each part of the target object may have a different position or depth relative to the viewer, the target object detection module 300 determines the distance or relative depth of at least a portion of the target object. To better understand the invention, suppose there are two objects of the same physical size around the viewer. When the viewer looks at both objects simultaneously, the one farther away (deeper) occupies a smaller portion of the total field of view relative to the one closer (shallower). In order for the viewer to see the object farther away as having the same amount of detail as the object closer, the object farther away needs to be displayed with a higher visual acuity (e.g., greater than 1.0). Therefore, as mentioned earlier, in order to present virtual images of different parts of a target object with different depths, it is necessary to adjust the dispersion angle of each of several light signals according to the depth changes, as well as the spatial interval between the center points of any two adjacent light signals presenting the object's image. Furthermore, the target object mentioned here can be the viewer's surrounding environment or an individual object within that environment. If the target object occupies a relatively large portion of the viewer's field of vision, the virtual image of the target object can be divided into several fields of vision. The number of first-eye light signals per degree in the first field of vision containing the first part of the target object may exceed the number of first-eye light signals per degree in the second field of vision containing the second part of the target object, depending on the depth of the different parts of the target object. In this embodiment, the first field of vision and the second field of vision represent different parts of the target object, respectively.

[0072] In a variation of the aforementioned embodiment, when the target object moves relative to the viewer, the target object detection module 300 captures image pixels of the moving target object. The distance detection unit 301 of the target object detection module 300 dynamically determines the distance or depth of the moving target object in the surrounding area. The field of view of the virtual image of the moving target object changes depending on the position of the moving target object. The dispersion angle of each of the plurality of light signals, and the spatial interval between the center points of any two adjacent light signals presenting the image of the moving target object, need to be adjusted according to the depth of the object to compensate for the change in field of view. Therefore, the number of first-eye light signals per degree in the first field of view of the target object (e.g., when the target object is far away) may exceed the number of first-eye light signals per degree in the second field of view of the target object (e.g., when the target object is close). In this embodiment, the first field of view and the second field of view represent different virtual images of the moving target object at different times.

[0073] Another aspect of the invention is the ability to generate images of a target object or a portion thereof with high visual acuity and depth perception.

[0074] Reference Figure 9In some embodiments, the present invention includes a first light projector 1, comprising: a first light projector 1, a first light direction adjuster 100, a first collimator 1000, and a first light combining element 20. Furthermore, the present invention further includes a second light projector 3, comprising: a second light projector 3 for emitting a plurality of second eye light signals corresponding to a plurality of first eye light signals to display a second eye virtual image of the target object; a second collimator for adjusting the position of the light waist of each of the plurality of second eye light signals, thereby separating the plurality of second eye light signals from each other; and a second light direction adjuster 300 for changing the light direction of the plurality of second eye light signals from the second light projector 3. The present invention further includes a second light combining element 40 for converging the plurality of second eye light signals and focusing them onto the viewer's second eye 60. The functions of the second light projector 3, the second collimator, the second light direction adjuster 300, and the second light combining element are similar to their counterparts. For example, the plurality of second-eye light signals are perceived by the viewer's left eye, while the plurality of first-eye light signals are perceived by the viewer's right eye (or vice versa). Each of the plurality of second-eye light signals has a corresponding first-eye light signal; that is, a first-eye light signal and its corresponding second-eye light signal are fused together to form a virtual binocular pixel of the binocular virtual image. Each of the first-eye light signal and each of the second-eye light signals has its own angle of incidence into the first eye 50 and the second eye 60, respectively. Furthermore, the second light projector 3, the second collimator 3000, the second light direction adjuster 300, and the second light combining element 40 can change the light spot size by changing the light waist position of the plurality of second-eye light signals and the spatial interval between adjacent second-eye light signals, similar to their corresponding groups. The first eye 50 and the second eye 60 perceive a first-eye light signal and its corresponding second-eye light signal to generate binocular vision, and the first-eye light signal and its corresponding second-eye light signal are fused together to form a binocular pixel for the viewer. In particular, in this embodiment, refer to Figure 10 and Figure 11The first eye light signal is projected by the first light projector 1 and redirected by the first light combining element 20 before entering the viewer's first eye 50. A corresponding second eye light signal is projected by the second light projector 3 and redirected by the second light combining element 40 before entering the viewer's second eye 60. The first and second light signals are perceived by the viewer to form a virtual binocular vision 72 of an object 70 having a first depth (d1), which is related to a first angle (θ1) between the light path extensions of the redirected first and second eye light signals. More specifically, the light path extensions of the first and second eye light signals are on the other side of the first and second light combining elements 40 and virtually converge at a position P1. When the first angle (θ1) between the light paths of the first eye light signal and the second eye light signal increases, the first depth d1 perceived by the viewer decreases; conversely, when the first angle (θ1) decreases, the first depth d1 perceived by the viewer increases. The first depth d1 of the first virtual binocular pixels 72 can be approximately calculated using the following formula:

[0075]

[0076] Reference Figures 9 to 12 When forming an image frame using the above method, the pixels of the first image frame F1 and the corresponding pixels of the second image frame F2 form a virtual binocular pixel at a first depth, which is related to a first angle between the first and second eye light signals projected onto the viewer's eye. After receiving several light signals, the viewer will perceive several right pixels of the object in the first image frame F1 in region A, which is defined by the extension of the redirected second eye light signal from the combining element. Region A refers to the field of view (FOV) of the second eye 50. Similarly, several first eye light signals of the second image frame F2 are redirected by the first combining element 20, pass through the center of the left pupil 62, and are finally received by the left retina 64. After receiving the redirected first eye light signal, the viewer will perceive several left pixels of the object in region B, which is defined by the extension of the redirected first eye light signal. Region B refers to the field of view (FOV) of the first eye. When several right pixels from the first image frame F1 and a left pixel from the second image frame F2 are displayed in region C, which is the overlapping portion of regions A and B, at least one second eye-light signal displaying a right pixel and one first eye-light signal displaying a left pixel are merged and displayed in region C as a virtual binocular pixel with a specific depth. This depth is related to an angle between the redirected second eye-light signal and the redirected first eye-light signal. This angle is also called the convergence angle.

[0077] Further reference Figures 9 to 12As described above, the plurality of second-eye light signals are generated by the second light projector 3, redirected by the second light combining element 40, and then scanned by the right retina to form a right retinal image on the right retina. Similarly, the plurality of first-eye light signals are generated by the first light projector 1, redirected by the first light combining element 20, and then scanned by the left retina to form a left retinal image on the left retina. In one embodiment, a right retinal image from the first image frame F1 comprises 36 right pixels (a 6*6 matrix), and a left retinal image from the second image frame F2 comprises 36 left pixels (a 6*6 matrix). Figure 11 In another embodiment, a right retinal image from the first image frame F1 contains 921,600 right pixels (1280*720 pixels), and a left retinal image from the second image frame F2 contains 921,600 left pixels (1280*720 pixels). The object display system generates several second eye light signals and corresponding several first eye light signals, which respectively form a right retinal image on the right retina and a left retinal image on the left retina. Thus, the viewer perceives a virtual binocular object with a specific depth in region C due to image fusion. The first second eye light signal 16 from the second light projector is received and reflected by the second light combining element 40. The first redirected second eye light signal 16' passes through the right pupil 52 to reach the viewer's right retina to form the right pixel R34. The corresponding first eye light signal 36 from the first light projector 1 is received and reflected by the first light combining element 20. The first redirected first-eye light signal 36' passes through the left pupil 62 to reach the viewer's left retina to form the left pixel L33. As a result of image fusion, a viewer perceives the virtual binocular object with several depths, where the depth is determined by the angle between several redirected second-eye light signals of the same object and the corresponding several redirected first-eye light signals. The angle between a redirected second-eye light signal and a corresponding redirected first-eye light signal is determined by the horizontal distance between the right pixel and the left pixel. In other words, the deeper a virtual binocular pixel is perceived by the viewer, the smaller the relative horizontal distance on the X-axis between the right and left pixels forming the virtual binocular pixel. For example, the second virtual binocular pixel 74 perceived by the viewer is deeper than the first virtual binocular pixel 72 (i.e., farther from the viewer). Therefore, on the retinal image, the horizontal distance between the second right pixel and the second left pixel is smaller than the distance between the first right pixel and the first left pixel.

[0078] In a variation of the above embodiments, the head-mounted display system of the present invention has a single light-combining element covering the user's eyes. The curvature of the light-combining element is designed to receive and converge the plurality of first eye light signals and the plurality of second eye light signals respectively. When each of the plurality of second eye light signals travels from the first light-combining element 20 to the viewer's second eye 60, each of the plurality of second eye light signals has a dispersion angle.

[0079] Using the above method, in an AR environment, a virtual image of a portion of the target object (e.g., an object within the target object) composed of several first and second eye signals can be presented at several depths, allowing the viewer to see a virtual image with optimal realism (depth perception and 3D effect). Furthermore, based on several depths at different points in the surrounding environment, the virtual image can be superimposed on a portion of the real image in the surrounding environment as an aid, allowing the viewer to see a portion of the surrounding environment with higher visual acuity. In this embodiment, the image information captured by the target object detection module 300 can be superimposed on the image seen by the viewer (as mentioned in U.S. Provisional Application 63 / 074444, the entire contents of which are incorporated herein by reference). In another embodiment, the invention is applied to a VR system where the viewer's vision relies entirely on the image information provided by the VR system.

[0080] Through the above embodiments, the eye-tracking device 302 can determine the gaze positions of the viewer's first eye 50 and second eye 60 respectively. The eye-tracking device 302 is used to track the positions of at least two pupils of a viewer. Furthermore, the eye-tracking module can provide more information about the viewer's eyes, including but not limited to eye movement, pupil size, gaze angle, and convergence angle for each eye. This eye information can be used not only to determine the direction and position of the light signal projected by the virtual object, but also to determine the viewer's gaze position and gaze depth. The eye-tracking device may include a first camera to track the first eye 50 and a second camera to track the second eye 60. In addition to a general eye-tracking camera, the first and second cameras can be constructed using a miniature microelectromechanical system (MEMS). The first and second cameras can utilize infrared light emission and sensors to detect and obtain various eye information. The eye-tracking device 302 may further include an integrated inertial measurement unit (IMU), an electronic device that can use a combination of accelerometers, gyroscopes, and sometimes magnetometers to measure and report body specific force, angular velocity, and sometimes body orientation. An exemplary embodiment of the invention is described below, in which the viewer's first and second eyes each have a light emitter, a collimator, a light direction adjuster, and a light combining element to obtain visual abilities superior to normal human vision (e.g., visual acuity greater than 1.0). The eye-tracking module 302 determines the viewer's gaze position to determine the portion of the target object being viewed. The target object detection module 300 captures several image pixels of the target object. These image pixels are associated with a portion or the entire target object. Since each portion of the target object has a different position and depth relative to the viewer, the target object detection module 300 needs to determine the distance or relative depth of at least a portion of the target object being gazed at by the viewer. As previously stated, in order to present virtual images of different parts of the target object with different depths, the dispersion angle of each of the plurality of light signals and the spatial interval between any two adjacent light signal center points presenting the image of the object need to be adjusted according to the depth variation. Furthermore, the target object mentioned here refers to the viewer's surrounding environment or a single object within that environment. If the target object or a portion of the target object occupies a relatively large portion of the viewer's field of vision, the virtual image of the target object can be divided into several fields of vision.Depending on the gaze position of the target object, the number of first-eye light signals per degree in a first field of view containing the first part of the target object (i.e., the number of first-eye light signals per degree) may exceed the number of first-eye light signals per degree in a second field of view containing the second part of the target object; simultaneously, the number of second-eye light signals per degree in a third field of view containing the first part of the target object (i.e., the number of second-eye light signals per degree) may exceed the number of first-eye light signals per degree in a fourth field of view containing the second part of the target object. The first field of view of the first eye corresponds to the third field of view of the second eye, both presenting the first part of the target object. The second field of view of the first eye corresponds to the fourth field of view of the second eye, both presenting the second part of the target object. In this embodiment, the spot size and spatial spacing of the plurality of first-eye light signals and second-eye light signals can be dynamically adjusted according to the viewer's gaze position.

[0081] As mentioned above, the dispersion angle of each of the plurality of light signals and the spatial interval between the center points of any two adjacent light signals presenting the image of the target object can be adjusted according to the above method; that is, by adjusting the projection time of the light emitter, the distance between the beam combining element and the collimator, the projection frequency of the light emitter, and the swing frequency of the light direction adjuster. In fact, the viewer's eyes can continuously change the gaze position to view different parts of the target object or different objects with different three-dimensional positions (including depth). Or, in some cases, the viewer's eyes may be focused on a moving object, so the viewer needs to continuously change the gaze position. Therefore, the projection time of the light emitter, the distance between the beam combining element and the collimator, the projection frequency of the light emitter, and the swing frequency of the light direction adjuster need to be dynamically adjusted according to the gaze position (i.e., the depth of the object being gazed at by the viewer).

[0082] Reference Figures 13 to 14 In another variation of this embodiment, an optical element can be installed between the light emitter and collimator described above to change the preset cross-sectional area of ​​the optical paths of several optical signals. Specifically, the optical element includes a lens group that can change the optical paths of several first-eye optical signals 1000 from the first light emitter 10 to the first light combining element 20, thereby changing the projection area or cross-sectional area (i.e., spot size) of each of the several first-eye optical signals. For example, referring to... Figure 13The optical element comprises Lens_1 and Lens_2. Both Lens_1 and Lens_2 are convex lenses. The light emitter is initially placed at the focal length of Lens_1. When Lens_1 moves to the new position shown by Lens_1', the distance between the light emitter and Lens_1 increases; thus, the dispersion angle of the light from the LBS also increases. Consequently, the cross-sectional area or projection area of ​​each of the plurality of first-eye light signals 1000 also increases, thereby increasing the spot size of the light signal. (Refer to...) Figure 14 In another embodiment, the optical elements include Lens_1, Lens_2, and LBS. Lens_1 and Lens_2 are both convex lenses. The light emitter is positioned at the focal length of Lens_1, so that the light signal emitted by the light emitter can be converted into a parallel light signal after passing through Lens_1. In this mode, the focal length of Lens_2 can be used to reduce the field of view. For example, changing the focal length of Lens_2 from 50 mm to 100 mm (and placing the pupil 100 mm away from Lens_2, which is the focal point of Lens_2) can reduce the field of view by half. When the invention is implemented in the form of a head-mounted device or AR / VR glasses, Lens_2 can be a light combining element. However, Lens_2 can also be other optical elements in other embodiments.

[0083] In some embodiments, to further improve the field of vision, Figure 13 and Figure 14 The methods shown can be used simultaneously. When the invention is implemented in the form of a head-mounted device or AR / VR glasses, Lens_2 can be a light-combining element. However, Lens_2 can also be other optical elements in other embodiments.

[0084] This invention can assist viewers with corneal and retinal damage (e.g., age-related macular degeneration). In the case of a normal eye, the optimal area for light perception is the macula; however, for individuals with macular degeneration or other eye conditions, other areas of the eye may be more suitable for light perception and receiving image pixels. Therefore, this invention can be used to project light signals onto a healthy portion of the retina to generate an image of the surrounding environment for the viewer. Prior to implementing this invention, corneal topography and retinal perimeter are used to find the optimal optical path and position for projecting light signals onto the viewer's eye. Simultaneously, the optimal angle of incidence for the least defective corneal region needs to be determined to project light onto the least defective portion of the retina. To achieve this, the light-combining element needs to be designed with an elliptical concave surface and / or have a customized reflection angle.

[0085] In this invention, the light combining elements 210 and 410 receive, redirect, and converge several optical signals generated by the light emitters 10 and 30. In one embodiment, the light combining elements 210 and 410 reflect the several optical signals, so the redirected optical signal and the incident optical signal are on the same side of the light combining elements 210 and 410. In another embodiment, the light combining elements 210 and 410 refract the several optical signals, so the redirected optical signal and the incident optical signal are on different sides of the light combining elements 210 and 410. When the light combining elements 210 and 410 also function as refractive mirrors, their reflectivity can be very wide, for example, 20% to 80%, depending in part on the intensity of the light signal generator. Those skilled in the art know how to determine an appropriate reflectivity based on the characteristics of the light emitters 10 and 30 and the light combining elements 210 and 410. Furthermore, in one embodiment, the light combining elements 210 and 410 are optically transparent to ambient light from the other side of the incident optical signal. The range of transparency is very wide, depending on the application. In AR / MR applications, transparency is preferably greater than 50%, for example, approximately 75% in one embodiment. In addition to redirecting the light signal, the light combining elements 210, 410 can converge several light signals forming the image of the light combining element, allowing them to pass through the pupil and reach the retina of the viewer's eyes. The light combining elements 210, 410 can be made of glass or plastic materials (e.g., lenses) coated with a specific material that makes them partially transparent or partially reflective, such as metal. One advantage of using reflective light combining elements instead of prior art waveguides when directing the light signal to the user's eyes is the elimination of adverse effects caused by diffraction, such as ghosting, color shift, etc. The light combining elements 210, 410 can be holographic light combining elements, but this is not the most preferred option because diffraction effects can cause ghosting and RGB shift. In some embodiments, we would like to avoid using holographic light combining elements. To avoid interference from ambient light, the present invention can utilize a shutter to significantly reduce ambient light entering the viewer's eyes. In one embodiment, the shutter can be a mechanical element to block ambient light. In another embodiment, the shutter can operate by reducing the transparency of the first light combining element and the second light combining element 210, 410.

[0086] In some embodiments, the present invention can be used as a head-mounted device for vision correction or vision training. The present invention can be used to correct or improve eye conditions, such as, but not limited to, myopia, hyperopia, strabismus, amblyopia, and convergence disorder. The principle of correcting or improving these conditions is to provide more visual stimulation to the viewer's eyes. In other words, providing appropriate stimulation to the eyes can promote improvements in visual acuity and muscle movement. For example, referring to… Figure 15This invention can be used to correct vision in nearsighted or farsighted individuals. In this embodiment, a head-mounted device with an AR / VR system can use the target object detection module 300 to capture a real-time image of a target object or surrounding environment for a viewer and project a pixelated image with depth perception onto the viewer. The pixelated image of this system can be focused at an ideal position using the methods mentioned above. For example, for nearsighted individuals, the pixelated image can be focused directly in front of the retina; for farsighted individuals, it can be focused directly behind the retina to stimulate the ciliary muscle to adjust the lens of the eye so that the image is correctly focused on the retina. By using this method, the eye muscles can be trained.

[0087] This invention can capture real-time image pixels of a target object or its surrounding environment and reproduce a stereoscopic digital image with improved image quality for the viewer of the AR / VR system. Viewers can adjust the image quality to exceed the visual acuity of ordinary people (e.g., higher than 20 / 20 or VA 1.0). Furthermore, this invention can assist visually impaired individuals or replace conventional prescription glasses, providing vision correction for those with myopia or hyperopia. This invention can be used by healthcare workers, military personnel, precision manufacturing industries, pilots, law enforcement officers, emergency medical personnel, and athletes, among others.

[0088] Although specific embodiments of the invention have been described in detail for illustrative purposes, various modifications and improvements may be made without departing from the spirit and scope of the invention as disclosed herein.

Claims

1. A head-mounted display system, the system comprising: a target object detection module configured to receive a plurality of image pixels of a first portion and a second portion of a target object and corresponding depths of the first portion and the second portion; a first light emitter configured to emit a plurality of first-eye light signals to display a first-eye virtual image of the first portion and the second portion of the target object to a viewer; a first light direction adjuster configured to change a light direction of each of the plurality of first-eye light signals from the first light emitter; a first collimating element disposed between the first light emitter and the first light direction adjuster and configured to adjust a beam waist position of each of the plurality of first-eye light signals such that the plurality of first-eye light signals are separated from each other; a first light combining element configured to redirect and converge the plurality of first-eye light signals to a first eye of the viewer; and wherein the first-eye virtual image of the first portion of the target object in a first field of view has a higher number of first-eye light signals per degree than the first-eye virtual image of the second portion of the target object in a second field of view; wherein the number of the plurality of first-eye light signals per degree in the first-eye virtual image of the target object is adjusted according to the corresponding depths of the plurality of image pixels of the target object, and a spot size and a spatial separation of the plurality of first-eye light signals are dynamically adjusted according to a gaze position of the viewer.

2. The head-mounted display system of claim 1, wherein the plurality of first-eye light signals are separated from each other if a spatial separation of center points of any two adjacent first-eye light signals on a cross-section of an optical path is greater than half of a maximum diameter of the two adjacent first-eye light signals.

3. The head-mounted display system of claim 1, wherein the number of the plurality of first-eye light signals per degree in the first-eye virtual image of the first portion of the target object in the first field of view exceeds 120.

4. The head-mounted display system of claim 1, wherein a projection time of each of the plurality of first-eye light signals by the first light emitter is adjustable to adjust the spot size of the plurality of first-eye light signals of the first-eye virtual image of the first portion of the target object.

5. The head-mounted display system of claim 1, wherein a projection frequency of the first light emitter is adjustable to adjust the number of the plurality of first-eye light signals per degree in the first-eye virtual image of the first portion of the target object.

6. The head-mounted display system of claim 1, wherein a wobbling frequency of the first light direction adjuster is adjustable to adjust the number of the plurality of first-eye light signals per degree in the first-eye virtual image of the first portion of the target object.

7. The head-mounted display system of claim 1, wherein a spot area of each of two adjacent first-eye light signals on a cross-section of an optical path is adjacent to each other.

8. The head-mounted display system of claim 1, wherein the first light emitter comprises a red laser diode, a green laser diode, and a blue laser diode. ​ 9. The head-mounted display system of claim 6, wherein the light directions of the first plurality of light signals vary with time at the dithering frequency to display the first eye virtual image of the target object, and the dithering frequency is non-constant.

10. The head-mounted display system of claim 9, wherein the light direction includes a first coordinate component and a second coordinate component; the first coordinate component and the second coordinate component of the light direction of the first plurality of light signals vary with time at a first dithering frequency and a second dithering frequency, respectively, to display the first eye virtual image of the target object; and the first dithering frequency or the second dithering frequency is non-constant.

11. The head-mounted display system of claim 9, wherein each of the first plurality of light signals forms a pixel in the first eye virtual image of the target object, and a pixel density of the first eye virtual image is non-constant.

12. The head-mounted display system of claim 11, wherein the first portion of the target object is a central portion, the second portion of the target object is a peripheral portion, and the first eye virtual image of the first portion of the target object has a higher pixel density than the first eye virtual image of the second portion of the target object.

13. The head-mounted display system of claim 1, wherein a first distance between the first light direction adjuster and the first collimating element is adjustable to adjust a spot size of the first plurality of light signals in the first eye virtual image of the first portion of the target object.

14. The head-mounted display system of claim 1, further comprising: a second light emitter for emitting a second plurality of light signals corresponding to the first plurality of light signals to display a second eye virtual image of the first and second portions of the target object to the viewer; a second light direction adjuster for varying a light direction of each of the second plurality of light signals from the second light emitter; a second collimating element disposed between the second light emitter and the second light direction adjuster to adjust a waist position of the second plurality of light signals to separate the second plurality of light signals from each other; wherein the first light combining element receives and converges the second plurality of light signals to the viewer; wherein the second eye virtual image of the first portion of the target object in a third field of view has a higher number of the second plurality of light signals per degree than the second eye virtual image of the second portion of the target object in a fourth field of view; and wherein both the first light signals and the corresponding second light signals are perceived by the viewer to display a virtual binocular pixel of a binocular virtual image of the target object having a depth related to an angle between the first light signals and the corresponding second light signals projected to the viewer's eyes.

15. The head-mounted display system of claim 1, further comprising: ​ a second light emitter for emitting a plurality of second-eye light signals corresponding to the plurality of first-eye light signals to display a second-eye virtual image of the first portion and the second portion of the target object to the viewer; a second light direction adjuster for changing a light direction of each of the plurality of second-eye light signals from the second light emitter; a second collimating element disposed between the second light emitter and the second light direction adjuster for adjusting a beam waist position of the plurality of second-eye light signals to separate the plurality of second-eye light signals from each other; wherein the second-eye virtual image of the first portion of the target object in a third field of view has a larger number of the plurality of second-eye light signals per degree than the second-eye virtual image of the second portion of the target object in a fourth field of view; and wherein the first-eye light signals and the corresponding second-eye light signals are perceived by the viewer to display a virtual binocular pixel of a binocular virtual image of the target object having a depth related to an angle between the first-eye light signals and the corresponding second-eye light signals projected to the viewer's eyes.

16. The head-mounted display system of claim 15, further comprising an eye movement tracking device for determining a visual axis of the viewer's first eye and second eye, respectively, to determine the gaze position.

17. The head-mounted display system of claim 15, wherein the target object detection module determines a three-dimensional coordinate for each of the plurality of image pixels of the first portion and the second portion of the target object.

18. The head-mounted display system of claim 15, wherein the binocular virtual image of the target object is displayed and superimposed on the target object.

19. The head-mounted display system of claim 15, further comprising a shutter for substantially reducing ambient light entering the viewer's eyes.

20. The head-mounted display system of claim 19, wherein the shutter operates by reducing transparency of the first light combining element.

21. The head-mounted display system of claim 16, wherein a projection frequency of the first light emitter and the second light emitter varies according to the gaze position to adjust a number of the plurality of first-eye light signals per degree in the first-eye virtual image of the first portion of the target object and a number of the plurality of second-eye light signals per degree in the second-eye virtual image of the first portion of the target object, respectively.

22. The head-mounted display system of claim 16, wherein a wobble frequency of the first light direction adjuster and the second light direction adjuster varies according to the gaze position to adjust a number of the plurality of first-eye light signals per degree in the first-eye virtual image of the first portion of the target object and a number of the plurality of second-eye light signals per degree in the second-eye virtual image of the second portion of the target object, respectively. ​ 23. The head-mounted display system of claim 16, wherein a projection frequency of the first light emitter and the second light emitter varies according to the gaze position to adjust a number of first eye light signals per degree in the first eye virtual image of the first portion of the target object and a number of second eye light signals per degree in the second eye virtual image of the second portion of the target object, respectively.

24. The head-mounted display system of claim 16, wherein a projection time of each of a plurality of first eye light signals by the first light emitter and a projection time of each of a plurality of second eye light signals by the second light emitter varies according to the gaze position to adjust a spot size of the plurality of first eye light signals in the first eye virtual image of the first portion of the target object and a spot size of the plurality of second eye light signals in the second eye virtual image of the second portion of the target object, respectively.

25. The head-mounted display system of claim 16, wherein a first distance between the first light direction adjuster and the first collimating element and a second distance between the second light direction adjuster and the second collimating element varies according to the gaze position to adjust a spot size of the plurality of first eye light signals in the first eye virtual image of the first portion of the target object and a spot size of the plurality of second eye light signals in the second eye virtual image of the second portion of the target object, respectively.

Images