Image display system for expanding a visual space and method therefor

Abstract

The present invention discloses a system and method for expanding the eyebox of a viewer, including but not limited to the application of retinal projection technology from a head mounted device (e.g. smart glasses) on a near-eye display. The present invention includes two embodiments, the first of which applies the principle of "light splitting", containing an optical replicator that creates multiple incident light signals in a manner that expands the eyebox of the viewer. The second embodiment applies the principle of "time splitting", containing an optical reflector that moves at different incident angles to redirect multiple light signals in a manner that expands the eyebox of the viewer.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 041,740, filed June 19, 2020, entitled “Methods and Systems for Eyebox Expansion” and U.S. Provisional Patent Application No. 63 / 085,172, filed September 30, 2020, entitled “Systems and Methods for Projecting Virtual Images with Multiple Depths”.

[0003] In addition, the entire contents of PCT International Patent Application No. PCT / US20 / 59317, filed on November 6, 2020, entitled “System and Method for Displaying an Object with Depth,” and U.S. International Patent Application No. 17 / 179,423, filed on February 19, 2021, entitled “Head Wearable Device with Adjustable Image Sensing Modules and Its System,” are incorporated herein by reference. Technical Field

[0004] This invention generally relates to an image display system for expanding the range of eye movement and a method of manufacturing such a system and method, particularly the application of the principles of "spectral splitting" or "temporal segmentation" to expand the range of eye movement for an observer. Background Technology

[0005] One of the main challenges in designing head-mounted AR / VR devices is reducing the physical size of the device while maintaining sufficient image quality, field of view, and viewpoint. The range of viewpoints visible to the viewer from the image provided by the device is called the "eye range." The size and position of the eye range significantly impact the user experience. For example, if the eye range is too small, the viewer may not see the image from the head-mounted AR / VR device if their gaze deviates slightly from the direction of the incoming image. Increasing the eye range (in other words, increasing the number and range of viewpoints provided by a head-mounted AR / VR device) can be achieved optically. However, increasing the eye range often adds bulky optical components to the head-mounted AR / VR device. Therefore, there is a need to design a system and method to increase the eye range without sacrificing user experience or affecting the physical size of the head-mounted AR / VR device. Summary of the Invention

[0006] The purpose of this invention is to provide an image display system and method that expands the field of vision for viewers. This system and method includes (but is not limited to) applying retinal projection technology from a head-mounted device (e.g., smart glasses) to a near-eye display. This invention includes two embodiments.

[0007] The image display system in this first embodiment includes a first image projector, a first optical replicator, and a first beam combiner. The first image projector generates several optical signals of a first image. The first optical replicator receives an optical signal generated by the first image projector, replicates the optical signal into N non-parallel beams, and redirects each of the N beams of the optical signal to a first beam combiner, where N is an integer greater than one. The first beam combiner is located between the first optical replicator and one of the observer's eyes. The beam combiner receives the N beams of the optical signal and converges them to N viewpoints within the eye's range of motion of the observer's eye. The image viewing system may further include a second image projector, a second optical replicator, and a second beam combiner to similarly expand the eye's range of motion of the observer's other eye. Therefore, the image display system can simultaneously expand the eye's range of motion of the observer's left and right eyes.

[0008] The second embodiment applies the principle of "time division." This embodiment includes an optical reflector that moves at different incident angles to redirect multiple light signals, thereby expanding the viewer's eye movement range. The image display system of this second embodiment includes a first image projector, a first optical reflector, and a first light combining element. The first image projector generates multiple light signals of a first image. The first optical reflector receives the multiple light signals generated by the first image projector and moves to redirect the multiple light signals to the first light combining element. The movement of the first optical reflector results in different incident angles when the multiple light signals reach the first light combining element. The first light combining element is located between the first optical reflector and one of the viewer's eyes, and is used to receive and converge the multiple light signals to a first visible area of ​​the viewer's eye to expand the viewer's eye movement range. Furthermore, the movement frequency of the first optical reflector is adjusted according to the projection frequency of the first image projector so that the multiple light signals of the first image are projected into the visible area of ​​the viewer's eye during the visual persistence time. The image display system may further include a second image projector, a second optical reflector, and a second light combining element to expand the eye movement range of the observer's other eye in the same manner. Therefore, the image display system can simultaneously expand the eye movement range of the observer's left and right eyes.

[0009] In the first and second embodiments, the image display system for the viewer's eyes is used to display an object with depth. The light signal redirected from the second light combining element is a first redirected right light signal, and the relative light signal redirected from the first light combining element is a first redirected left light signal. A first virtual binocular pixel for displaying an object, perceived by the viewer using the first redirected right light signal and the first redirected left light signal, has a depth related to a first angle between the first redirected right light signal and the relative first redirected left light signal. Generally, this first depth is determined by the horizontal distance between the first redirected right light signal and the relative first redirected left light signal.

[0010] In AR and MR applications, an image display system may further include a support structure that can be worn on the observer's head. The first image projector, the second image projector, the first optical replicator and the second optical replicator (and the first optical reflector and the second optical reflector of the second embodiment), the first light combining element, and the second light combining element are all supported by the support structure. In one embodiment, the system is a head-mounted device, particularly a pair of glasses, such as smart glasses. In this case, the support structure may be a frame that may have lenses, which may be prescription lenses for correcting myopia or hyperopia.

[0011] Other features and advantages of the invention will be described below, and some may be learned from the description or examples of the invention. The objectives and other advantages of the invention will be realized by the structures and methods particularly pointed out in the written description, claims, and drawings. It is to be understood that the foregoing general description and the following detailed description are exemplary and illustrative, intended to provide further explanation of the claimed invention. Attached Figure Description

[0012] Figure 1A This is a schematic diagram illustrating an embodiment of an image display system of the present invention, the image display system having a first optical replicator.

[0013] Figure 1B This is a schematic diagram illustrating the image display system of the present invention, which is carried by a pair of glasses.

[0014] Figure 2 This is a schematic diagram illustrating an embodiment of the image display system with a beam splitter according to the present invention.

[0015] Figure 3 This is a schematic diagram illustrating an embodiment of an image display system of the present invention, the system having a polarizer in which N beams of an optical signal converge onto a first beam combining element.

[0016] Figure 4 This is a schematic diagram illustrating an embodiment of an image display system of the present invention, wherein the extensions of N non-parallel beam paths of the optical signal converge to a virtual converging plane located behind the first beam combining element.

[0017] Figure 5A This is a schematic diagram illustrating an image display system for a viewer's binoculars to perceive the depth of an object, according to the present invention.

[0018] Figure 5B This is a schematic diagram illustrating an image display system for a viewer's eyes, used to perceive two virtual binocular pixels of an object with depth.

[0019] Figure 6 This is a flowchart illustrating the process of an embodiment of the present invention, which expands the eye movement range for a viewer's eye through an image display with a first optical replicator.

[0020] Figure 7A This is a schematic diagram illustrating an embodiment of an image display system of the present invention, the system having a first optical reflector.

[0021] Figure 7B This is a schematic diagram illustrating the image display system of the present invention, which is mounted on a pair of glasses.

[0022] Figure 8 This is a schematic diagram illustrating an embodiment of an image display system of the present invention, wherein a first image projector is a digital light processing projector.

[0023] Figure 9A -D are all schematic diagrams illustrating an embodiment of an image display system of the present invention with a continuously moving optical reflector, the system displaying image pixels.

[0024] Figure 10 This is a schematic diagram illustrating an embodiment of an image display system of the present invention, which generates several viewpoints in a visible area.

[0025] Figure 11A This is a schematic diagram illustrating an embodiment of an image display system of the present invention, the system having a pentagonal column reflector.

[0026] Figure 11B This is a schematic diagram illustrating an embodiment of an image display system of the present invention with a pentagonal column reflector, the system generating a first viewing area.

[0027] Figure 12 This is a flowchart illustrating the process of an embodiment of the present invention, which uses an image display system with a pentagonal prism reflector to expand the eye movement range of an observer's eye. Detailed Implementation

[0028] The terminology used herein is for the purpose of describing details in specific embodiments of the invention, and all terms should be interpreted in the broadest sense. Certain terms will be specifically highlighted below; any limiting terms will be defined by the specific embodiments.

[0029] This invention relates to one or more methods, systems, and apparatuses for expanding the eye-tracking range of an image display, including (but not limited to) near-eye displays employing retinal projection technology from head-mounted devices (e.g., smart glasses). The invention comprises two embodiments. Description of the first embodiment may be used in connection with the second embodiment, and vice versa. The first embodiment utilizes the principle of "beam splitting" to expand the eye-tracking range for a viewer, and includes an optical replicator to generate a plurality of beams of an incident light signal. An image display apparatus of the first embodiment includes a first image projector, a first optical replicator, and a first beam combiner. The first image projector generates a plurality of light signals of a first image. The first optical replicator receives a light signal generated by the first image projector, replicates the light signal into N non-parallel beams, and redirects each of the N beams of the light signal to a first beam combiner, where N is an integer greater than one. The first beam combiner is located between the first optical replicator and one of the observer's eyes. This beam combiner receives N beams of the light signal and converges them onto N viewpoints within the observer's eye's range of motion. The image viewing system may further include a second image projector, a second optical replicator, and a second beam combiner to similarly expand the range of motion of the observer's other eye. Therefore, the image display system can simultaneously expand the range of motion of both the observer's left and right eyes.

[0030] The second embodiment applies the principle of "time division." This embodiment includes an optical reflector that moves at different incident angles to redirect multiple light signals, thereby expanding the viewer's eye movement range. The image display system of this second embodiment includes a first image projector, a first optical reflector, and a first light combining element. The first image projector generates multiple light signals of a first image. The first optical reflector receives the multiple light signals generated by the first image projector and moves to redirect the multiple light signals to the first light combining element. The movement of the first optical reflector results in different incident angles when the multiple light signals reach the first light combining element. The first light combining element is located between the first optical reflector and one of the viewer's eyes, and is used to receive and converge the multiple light signals to a first visible area of ​​the viewer's eye to expand the viewer's eye movement range. Furthermore, the movement frequency of the first optical reflector is adjusted according to the projection frequency of the first image projector so that the multiple light signals of the first image are projected into the visible area of ​​the viewer's eye during the visual persistence time. The image display system may further include a second image projector, a second optical reflector, and a second light combining element to expand the eye movement range of the observer's other eye in the same manner. Therefore, the image display system can simultaneously expand the eye movement range of the observer's left and right eyes.

[0031] In the first and second embodiments, the image display system for the viewer's eyes is used to display an object with depth. The light signal redirected from the second light-combining element is a first redirected right light signal, and the relative light signal redirected from the first light-combining element is a first redirected left light signal. A first virtual binocular pixel for displaying an object, perceived by the viewer using the first redirected right light signal and the first redirected left light signal, has a depth related to a first angle between the first redirected right light signal and the relative first redirected left light signal. Generally, this first depth is determined by the horizontal distance between the first redirected right light signal and the relative first redirected left light signal.

[0032] First Embodiment

[0033] like Figure 1A As shown, in a first embodiment, an image display system 100 includes a first image projector 110, a first optical replicator 120, and a first beam combining element 130. Using the principle of "beam splitting," this first embodiment uses the first optical replicator 120 to receive an optical signal of a first image and generate several beams of the optical signal. These beams converge to several viewpoints (i.e., 151, 152, 153) to expand the eye movement range of a viewer's eye. Traditionally, an eye movement range contains only one viewpoint. With this invention, the eye movement range can be expanded to include several viewpoints. A viewpoint can be separate from, adjacent to, or overlap with adjacent viewpoints. This eye movement range is the area where an observer's eye 140 can see the complete image. In other words, as long as the viewer's eye moves within this eye movement range, the viewer can see the complete image. The image display system 100 can expand the eye movement range for a viewer's eye.

[0034] The image display system 100 can be carried by a head-wearable device (HWD), which in one embodiment can be as follows: Figure 1BA pair of smart glasses 180 is shown. The glasses have a frame 185 and a pair of lenses 190. The frame 185 includes a first image projector 110 and a first image replicator 120. The positions of the first image projector 110 and the first image replicator 120 are adjusted according to the optical path design. The lenses 190 include a first light combining element 130. In one embodiment, the first light combining element 130 and the lenses 190 are integrated into a single component. In this case, the image display system 110 can expand the eye movement range for the wearer of the head-mounted device. A viewer can see the complete image from different viewpoints within the eye movement range (i.e., 151, 152, 153). Furthermore, because the smart glasses 180 can be customized for the viewer, the interpupillary distance can be adjusted for each viewer. Those skilled in the art will recognize that in other embodiments, the image display system 110 can be used to expand the eye movement range for several viewers simultaneously.

[0035] The light source of the first image projector 100 can be a laser, a light-emitting diode (LED), including mini or micro LEDs, organic light-emitting diodes (OLEDs), superluminescent diodes (SLDs), liquid crystal on silicon (LCoS), or a liquid crystal display (LCD), or a combination thereof. In one embodiment, the first image projector 110 is a laser scanning projector (LBS projector), which consists of a light source (including a red laser, a green laser, and a blue laser), a light color modifier (such as a dual-color combining element and a polarizing combining element), and a two-dimensional adjustable reflector (such as a microelectromechanical system mirror). The LBS projector generates and scans light signals sequentially one after another at a preset resolution (e.g., 1280x720 pixels per frame). Then, the light signal of one pixel is generated and projected onto the first optical replicator 120 one at a time. In order for a viewer to see the two-dimensional image with one eye, the LBS projector must sequentially generate the light signal (e.g., 1280x720 light signals) for each pixel of the first image within the visual persistence time (e.g., 1 / 18 of a second). Therefore, the duration of each light signal is approximately 60.28 nanoseconds.

[0036] In another embodiment, the first image projector 110 may be a digital light processing (DLP) projector capable of generating a two-dimensional color image at a time. Texas Instruments' DLP technology is one such technology that can be applied to the manufacture of DLP projectors. Each complete two-dimensional color image frame, for example, may include 1280x720 pixels, is simultaneously projected onto the first optical replicator 120. Therefore, after receiving N non-parallel beams of an incident light signal, the first optical replicator 120 can simultaneously redirect N non-parallel beams of several light signals (e.g., 1280x720 light signals) of a frame to the first light combining element 130, where N is an integer greater than 1.

[0037] When an LBS projector is used as the first image projector 110, the first optical replicator 120 is used to simultaneously receive several light signals generated by the first image projector. The replicator is located and faces the optical path between the first image projector 110 and the first light combining element 130. For each received light signal, the first optical replicator 120 replicates the light signal into N non-parallel beams, and redirects each of the N beams of the light signal to the first light combining element 130. The first light combining element 130 is located and faces the first optical replicator 120 and one eye of the observer 140, and is used to redirect each of the N non-parallel beams of the light signal to N viewpoints (such as 151, 152, 153...) within the eye movement range of the observer's eye. Again, a viewpoint can be separate from, adjacent to, or overlap with adjacent viewpoints. Those skilled in the art should know how to determine the number of viewpoints, the range of viewpoints, and the distance between two adjacent viewpoints based on pupil size, image resolution, the scanning rate of the first image projector 110, and the interference effect between different beams of the light signal. The average adult pupil is 2-4 cm in diameter in bright light and 4-8 cm in dark light. In one embodiment, the distance between two adjacent central viewpoints is approximately 2.6-3 cm.

[0038] N non-parallel beams of light from the first optical replicator 120 can converge at a single point on the first light combining element 130. In another embodiment, the N non-parallel beams of light from the first optical replicator 120 are reflected at different points on the first light combining element 130, and the extensions of the paths of the N non-parallel beams of the reflected light signal converge onto a virtual converging plane 135, which is located at a distance d behind the first light combining element 130, far from the viewer's eye. In both embodiments, after reflection by the first light combining element 130, the N non-parallel beams of light from the same image pixel (such as the first beam, the second beam, and the third beam) are redirected to relative viewpoints (such as the first viewpoint, the second viewpoint, and the third viewpoint) within the eye movement range 150. From the viewer's perspective, because N non-parallel beams of light signals from the same image pixel physically converge at a point on the first light-combining element 130, or because the extensions of these light paths converge at a point on a virtual converging plane 135, the image pixel is considered to be at the same location when the viewer's eye sees it from a first viewpoint, a second viewpoint, or a third viewpoint. In other words, the first, second, and third beams of light signals seen by the viewer's eye all represent the same image pixel because they all originate from the same point on the first light-combining element 130 or the converging plane 135. Therefore, the two-dimensional image from the image display system 100 will be at the same location, regardless of which viewpoint the viewer's eye sees the two-dimensional image. Furthermore, after reflection from the first light-combining element 130, the relative beams of light signals from different image pixels reflected from the first light-combining element 130 (e.g., the first beam, the second beam, and the third beam) will converge to relative viewpoints (e.g., the first viewpoint, the second viewpoint, and the third viewpoint) within the eye movement range 150.

[0039] like Figure 2 As shown, the image display system 100 may further include a first collimator 160, located between the first image projector 110 and the first optical replicator 120, to make the direction of motion of the light signals more consistent (parallel) in a specific direction. In other words, light signals from different pixels of the first image projector 110 become approximately parallel after passing through the first collimator 160. Therefore, the first collimator 160 ensures that each light signal has approximately the same angle of incidence on the first optical replicator 120. The first collimator 160 can be a curved lens or a convex lens.

[0040] The first optical replicator 120 is used to replicate an incident light signal into N non-parallel beams. In other words, after receiving a light signal, the first optical replicator 120 generates N beams of the light signal and redirects them to the first beam combining element 130, where N is an integer greater than 1 (e.g., N equals 3, 4, 5). Due to the "splitting," the light intensity of the N non-parallel beams of the incident light signal is reduced. The first optical replicator 120 can be a beam splitter, polarizer, semi-silvered mirror, partial reflector, dichroic prism, dichroic optical coating, or dielectric optical coating. The first optical replicator 120 can include at least two optical elements to replicate the incident light into at least two beams. Each of the optical elements can be a lens, reflector, partial reflector, prism, mirror, or a combination thereof.

[0041] like Figure 2 As shown, in one embodiment, when N equals 3, the first optical replicator 120 is a beam splitter. The first optical replicator 120 includes two partial reflectors and one total reflector, which utilizes the principle of partial reflection to split an incident light signal into three beams. For example... Figure 3 As shown, in another embodiment, the first optical replicator 120 is a polarizing beam splitter that uses the principle of partial reflection to split an incident light signal into three beams.

[0042] The first optical replicator 120 can adjust its position, including orientation and distance, to allow N non-parallel beams of an optical signal to converge. Figure 2 and Figure 3 The first light beam (S11), the second light beam (S12), and the third light beam (S13) of the first light signal (L1) converge at point C1 of the first light combining element 130. Similarly, the first light beam (S31), the second light beam (S32), and the third light beam (S33) of the third light signal (L3) converge at point C3 of the first light combining element 130. When the first light signal and the third light signal are the leftmost and rightmost image pixels of the image, respectively, the distance between points C1 and C3 is called the field of view (FOV). In this embodiment, the field of view seen by the viewer from one viewpoint can almost cover the entire area of ​​the first light combining element 130. Alternatively, the field of view seen by the viewer from one viewpoint can cover more than 80% of the area of ​​the first light combining element. In conventional cases, when generating parallel light beams for a light signal, the area of ​​a light combining element is divided by multiple viewpoints, so the field of view seen by the viewer from one viewpoint is much smaller than the field of view of this invention.

[0043] like Figure 4In another embodiment of the image processor 100 shown, the first beam (S11), the second beam (S12), and the third beam (S13) of the first optical signal (L1) are reflected at points C11, C12, and C13 on the first light combining element 130, respectively. However, the light paths of the first reflected beam (RS11), the second reflected beam (RS12), and the third reflected beam (RS13) of the first optical signal (L1) converge at a point D1 on the virtual convergence plane 135, which is located d behind the first light combining element 130 and further away from the viewer's eye. In this embodiment, because all the beam paths of the optical signal of each image pixel converge at a point on the virtual convergence plane 135, regardless of the viewer's perspective from which they see the image, the viewer perceives each image pixel (and the entire image) as being located at the same position on the virtual convergence plane 135. This embodiment can be applied to augmented reality-assisted surgery (ARAS), where images generated by the imaging display system 100, such as images initially acquired from a computed tomography scan, can be superimposed on a corresponding portion of the patient within the clinic. In some cases, the distance D behind the first light-combining element 130 is approximately 30-40 centimeters.

[0044] The first light-combining element 130 reflects several beams of light from the first optical replicator 120 and converges the relative beams of each light signal to a corresponding viewpoint within the viewer's eye movement range. In one embodiment, the first light-combining element 130 is sufficiently transparent to allow ambient light to penetrate to the viewer's eye. Figure 2 , Figure 3 , Figure 4As shown, each first beam (solid lines S11, S21, S31) of the three incident light signals (L1, L2, L3) from the first optical replicator 120 is reflected by the first light combining element 130 and converged at the first viewpoint P1; each second beam (dashed lines S12, S22, S32) of the three incident light signals (L1, L2, L3) from the first optical replicator 120 is reflected by the first light combining element 130 and converged at the second viewpoint P2; each third beam (dotted lines S13, S23, S33) of the three incident light signals (L1, L2, L3) from the first optical replicator 120 is reflected by the first light combining element 130 and converged at the third viewpoint P3. Regardless of whether the first beam, the second beam, or the third beam of each optical signal from the first replicator 120 is present, they all converge at the first combining element 130. After reflection, each first reflected beam (RS11, RS21, RS31) of the three optical signals (L1, L2, L3) from the first optical element 130 converges at the first viewpoint P1. For ease of explanation, the first reflected beams (RS11, RS21, RS31) of the three optical signals (L1, L2, L3) are depicted as converging at the first viewpoint P1 (151). In fact, since the three optical signals represent three different pixels, they are projected onto three slightly separated points on the retina. The same applies to the second and third beams of each optical signal.

[0045] The first beam combining element 130 can be made of glass or plastic as a lens and coated with a specific material, such as metal, to make it partially transparent and partially reflective. The first beam combining element 130 can be a holographic beam splitter, but this is not the optimal choice because diffraction effects can cause several shadows and RGB shifts. In some embodiments, the use of a holographic beam splitter is avoided.

[0046] As described above, the image display system 100, including the first image projector 110, the first optical replicator 120, and the first light combining element 130, can expand the eye movement range for one eye of the viewer. In one embodiment, the image display system 100 may further include a second image projector 115, a second optical replicator 125, and a second light combining element 135, which function in the same manner as the first image projector 110, the first optical replicator 120, and the first light combining element 130, expanding the eye movement range for another point of the viewer. Similarly, the second image projector generates several light signals of a second image. The second optical replicator receives a light signal generated by the second image projector, replicates the light signal into M non-parallel beams, and redirects each of the M beams of the light signal to a second light combining element, where M is an integer greater than one. The second beam combining element is located between the first optical replicator and one of the observer's eyes. This element receives M beams of the light signal and converges them onto M viewpoints within the eye movement range of the observer's other eye. Furthermore, the second image projector has a similar structure to the first image projector; the second optical replicator has a similar structure to the first optical replicator; and the second beam combining element has a similar structure to the first beam combining element. Therefore, the image display system 100 can simultaneously expand the eye movement range of the observer's left and right eyes.

[0047] The image projector system 100 may include a support structure that can be worn on a viewer's head to support the first image projector 110, the second image projector 115, the first optical replicator 120, the second optical replicator 125, the first light combining element 130, and the second light combining element 135. The first light combining element 130 and the second light combining element 135 are located within the viewer's field of vision. Therefore, in this embodiment, the image display system 100 is a head-mounted device (HWD). Specifically, by Figure 1B As shown, the image display system is supported by a pair of glasses, referred to as smart glasses. In this case, the support structure can be a frame that may have lenses, which can be prescription lenses for correcting myopia or hyperopia. The first image projector 110 and the first optical replicator 120 are supported by the right temple, and the second image projector and the second optical replicator are supported by the left temple. The first light-combining element can be supported by the right lens, and the second light-combining element can be supported by the left lens. The support can be achieved in various ways, with the light-combining element movably or fixedly connected to or integrated into the lens. The light-combining element can be combined with the lens (including prescription lenses). When the support structure does not contain lenses, the right and left light-combining elements can be directly supported by the frame or edge.

[0048] All components and variations in the embodiments of the image display system 100 can be applied to a head-mounted device. Therefore, the head-mounted device including smart glasses can further accommodate other components of the image display system, such as a control unit, a first collimator 160, and a second collimator 165. The first collimator 160 is located between the first image projector and the first optical replicator, and the second collimator 165 is located between the second image projector and the optical replicator. When the image display system 100 is applied to smart glasses, the lenses of the smart glasses can simultaneously possess refractive properties for correcting the viewer's vision and the function of a light-combining element. The smart glasses can provide prescription lenses to meet the vision correction needs of people with myopia or hyperopia. In this case, each lens of the smart glasses can include a refractive unit and a light-combining element. The refractive unit and the light-combining element can be manufactured together using the same or different types of materials, or they can be manufactured separately and then assembled together. These two components can be temporarily connected to each other (e.g., using built-in magnetic materials) or permanently connected together. In both cases, the light-combining element is positioned on the side of the lens closest to the viewer's eye. If the lens is a single piece, the light-combining element forms the inner surface of the lens. If the lens has two parts, the light-combining element is the inner part of the lens. This combining element allows ambient light to pass through while reflecting the light signal generated by the image projector to the viewer's eye to form a virtual image of the real environment. The light-combining element has an appropriate curvature to reflect and converge all light signals from the optical replicator into the pupil and finally onto the retina.

[0049] like Figure 5A In one embodiment shown, the image display system 100 for a viewer's eyes is used to display an object with depth. Because the depth of the object is the same as the position being viewed by the viewer's eyes, convergence-accommodation conflict (VAC) and focus competition can be avoided. The light signal redirected from the second beam combiner 135 is a first redirected right light signal (e.g., RRL21), and the relative light signal redirected from the first beam combiner 130 is a first redirected left light signal (e.g., RLL21). The first redirected right light signal (e.g., RRL21) and the first redirected left light signal (e.g., RLL21) are perceived by the viewer to display a first virtual binocular pixel 72 of an object 70 having a first depth (d1), which is related to a first angle (θ1) between the optical path extensions of the first redirected right light signal (e.g., RRL21) and the first redirected left light signal (e.g., RLL21). Generally, the first depth is determined by the relative horizontal distance between the first redirected right light signal and the first redirected left light signal.

[0050] Figure 5AThe image display system 100 shown includes a first image projector 110, a first optical replicator 120, a first light combining element 130, a second image projector 115, a second optical replicator 125, and a second light combining element 135. The first image projector 110 generates a left light signal (LL2) to the first optical replicator 120, then replicates the left light signal into three beams (LL21, LL22, LL23) and redirects them to the first light combining element 130. The first light combining element 130 reflects the three beams of the left light signal at points C21(L), C22(L), and C23(L), ​​respectively. The three redirected beams (LL21, LL22, LL23) of the left light signal are projected onto three left viewing points P1(L), P2(L), and P3(L), ​​respectively, and then onto the viewer's retina. The optical paths of the three redirected beams of the left light signal converge at a point D2(L) on the left virtual converging plane, which is located d1 behind the first light combining element 130 and further away from the viewer's eye.

[0051] Similarly, the second image projector 115 generates a right light signal (RL2) to the second optical replicator 125, then replicates the left signal into three beams (RL21, RL22, RL23) and redirects them to the second beam combiner 135. The second beam combiner 135 reflects the three beams of the right signal at points C21(R), C22(R), and C23(R), respectively. The three redirected beams (RL21, RL22, RL23) of the right light signal are projected onto three right viewpoints P1(R), P2(R), and P3(R), respectively, and then onto the viewer's retina. The optical paths of the three redirected beams of the right light signal converge at a point D2(R) on a right virtual converging plane, which is located d1 behind the second beam combiner 135 and further away from the viewer's eye. The image display system 100 can make the position D2(L) the same as the position D2(R), which is the stereoscopic position of the virtual binocular pixels 72 of the object perceived by the viewer.

[0052] In this embodiment, as the eye movement range expands, the viewer's eyes can receive light signals from three pairs of viewpoints: the first right viewpoint P1(R) and the opposite first left viewpoint P1(L), the second right viewpoint P2(R) and the opposite second left viewpoint P2(L), and the third right viewpoint P3(R) and the opposite third left viewpoint P3(L). The viewer's right eye 50 includes a right pupil 52 and a right retina 54; the viewer's left eye 60 includes a left pupil 62 and a left retina 64. Therefore, from the first pair of viewpoints (i.e., the first right viewpoint P1(R) and the opposite first left viewpoint P1(L)), the viewer's eyes can receive the first beam of the redirected right light signal RRL21 and the first beam of the opposite redirected left light signal RLL21 through the pupil and project them onto the retina. As a result, the viewer perceives a first virtual binocular pixel 72 of an object, which displays a first depth (d1) related to a first angle (θ1) between the optical path extensions of the first beam of the redirected right light signal RRL21 and the opposite first beam of the redirected left light signal RLL21. Similarly, from the second pair of viewpoints (i.e., the second right viewpoint P2(R) and the opposite first left viewpoint P2(L)), the viewer's eye receives the second beam of the redirected right light signal RRL22 and the opposite second beam of the redirected left light signal RLL22 through the pupil and projects it onto the retina. As a result, the viewer perceives the same first virtual binocular pixel 72 of the object, which displays a first depth (d1) related to the first angle (θ1) between the optical path extensions of the second beam of the redirected right light signal RRL22 and the opposite second beam of the redirected left light signal RLL22. The above description can also be applied to the third pair of viewpoints. The distance between each pair of viewpoints is roughly the same because a viewer's interpupillary distance (IPD) does not change as the viewer moves.

[0053] like Figure 5B In one embodiment shown, an object, such as the dinosaur 70, is perceived to have several depths. In addition to the object's first virtual binocular pixels 72, when a second redirected right light signal 18' and a corresponding second redirected left light signal 38' are perceived by the viewer and a second virtual binocular pixel 74 of the object is displayed, this pixel displays a second depth d2, which is related to a second angle (θ2) between the optical path extensions of the second redirected right light signal 18' and the corresponding second redirected left light signal 38'. Figure 5B In order to simplify the illustration, only the first beam of each left and right optical signal from the first optical replicator 120 and the second optical replicator 125 is shown. Figure 5A It has been explained that the first optical replicator and the second optical replicator respectively generated three beams of light for the left light signal and the right light signal.

[0054] exist Figure 5B In the image of the dinosaur object 70, a first virtual binocular pixel 72 is displayed at a first depth d1 and a second binocular pixel 74 is displayed at a second depth d2. The first angle between the first redirected right light signal 16' and the opposite first redirected left light signal 36' is θ1. The first depth d1 is related to the first angle θ1. Specifically, the first depth of the first virtual binocular pixel of the object can be determined by the first angle θ1 between the optical path extensions of the first redirected right light signal and the opposite first redirected left light signal. The first depth d1 of the first virtual binocular pixel 72 can be approximately calculated by the following formula:

[0055]

[0056] The distance between the right pupil 52 and the left pupil 62 is the interpupillary distance (IPD). Similarly, the second angle between the optical path extensions of the second redirected right light signal 18' and the opposite second redirected left light signal 38' is θ2. The second depth d2 is related to the second angle θ2. In particular, the second depth d2 of the object's second virtual binocular pixel 74 can be approximately determined by the same formula using the second angle θ2 between the optical path extensions of the second redirected right light signal and the opposite second redirected left light signal. Because the second virtual binocular pixel 74 is perceived at a greater distance from the viewer (i.e., with greater depth) than the first virtual binocular pixel 72, the second angle θ2 is smaller than the first angle θ1.

[0057] Furthermore, the first redirected right light signal 16' and the corresponding first redirected left light signal 36' together display a first virtual binocular pixel 72 at the first depth d1. In one embodiment, the first redirected right light signal 16' is not the parallax of the corresponding first redirected left light signal 36'. Because the right eye and the left eye see the same object from different angles, the parallax between the image received by the right eye and the image received by the left eye is used by a viewer to perceive a stereoscopic image with depth. Therefore, the first redirected right light signal 16' and the corresponding first redirected left light signal 36' have the same viewing angle. However, in another embodiment, the right light signal and the corresponding left light signal of the virtual binocular pixel can display images from different viewing angles (with parallax). Furthermore, one or both of the right light signal and the left light signal can be modified to present certain three-dimensional effects, such as shadows.

[0058] As described above, the plurality of right light signals are generated by the second image projector, copied by the second optical replicator, redirected by the second light combining element, and scanned by the right retina to form a right retinal image. Similarly, the plurality of left light signals are generated by the first image projector, copied by the first optical replicator, redirected by the first light combining element, and scanned by the left retina to form a left retinal image. Figure 5B In one embodiment, a right retinal image 80 comprises 36 right pixels (6x6 matrix) and a left retinal image 90 also comprises 36 left pixels (6x6 matrix). In another embodiment, a right retinal image 80 comprises 921,600 right pixels (1280x720 matrix) and a left retinal image also comprises 921,600 left pixels (1280x720 matrix). The image display system 100 can be used to generate several right light signals and several corresponding left light signals, which form the right retinal image and the left retinal image respectively on the right and left retina. Therefore, due to image fusion, the viewer perceives a virtual binocular object with a specific depth.

[0059] Reference Figure 5B The first right light signal 16 from the second image projector 115 is replicated by the second optical replicator 125 and then reflected by the second light combining element 135. The first redirected right light signal (first beam) 16' reaches the viewer's right retina 54 through the right pupil 52 to display the right pixel R34. Conversely, the left light signal 36 from the first image projector 110 is reflected by the first optical replicator 120 and then reflected by the first light combining element 130. The first redirected left light signal 36' reaches the viewer's retina 64 through the left pupil 62 to display the left retinal pixel L33. In this embodiment, the first redirected right light signal and the corresponding first redirected left light signal are directed to approximately the same height on the retinas of the viewer's eyes. Due to image fusion, a viewer perceives several depths in the virtual binocular object, which can be determined by the angles between the several redirected right light signals and the corresponding several redirected left light signals of the same object. The angle between a redirected right light signal and a corresponding redirected left light signal is determined by the horizontal distance between the right and left pixels. Therefore, the depth of a virtual binocular pixel is negatively correlated with the distance between the right and left pixels forming the virtual binocular pixel. In other words, the deeper a virtual binocular pixel is perceived by the viewer, the smaller the relative horizontal distance on the X-axis between the right and left pixels forming the virtual binocular pixel. For example, as... Figure 5BAs shown, the second virtual binocular pixel 74, perceived by the viewer, is deeper (i.e., farther away) than the first virtual binocular pixel 72. Therefore, in the retinal image, the horizontal distance between the second right pixel and the second left pixel is smaller than the horizontal distance between the first right pixel and the first left pixel. Specifically, the distance between the second right pixel R41 and the second left pixel R51, which form the second virtual binocular pixel, is four pixels. However, the distance between the first right pixel R43 and the first left pixel L33, which form the first virtual binocular pixel, is six pixels.

[0060] As described above, this embodiment can be applied to augmented reality-assisted surgery (ARAS), where the image generated by the imaging display system 100, such as an image initially acquired from a computed tomography scan, is superimposed on the corresponding portion of the clinical patient. In some cases, the distance d1 behind the first beam combining element 130 is approximately 30-40 centimeters. In this application, the depth of the stereoscopic image can be fixed or limited to a relatively short distance.

[0061] When the first image projector 110 is a digital light processing (DLP) projector, the projector generates the entire image at once, for example, 1270x720 pixels per frame, and projects it simultaneously onto the first optical replicator 120. The above description generally applies to situations using digital light processing projectors.

[0062] Figure 6 This describes a method for expanding a viewer's eye movement range. In step 610, the first image projector 110 generates a light signal to a first optical replicator. In one embodiment, the image projector can be a laser scanning projector (LBS projector) that sequentially generates the light signals of the image pixels one by one. In another embodiment, the image projector 110 can be a digital light processing projector that simultaneously generates all the light signals of the image (e.g., a frame of 1280x720 pixels). In either embodiment, when the image projector 110 generates the light signal at high speed, the viewer can smoothly view the image due to visual persistence.

[0063] In step 620, the first optical replicator 120 receives the optical signal and replicates it into N non-parallel beams of the optical signal, where N is an integer greater than one. The first optical replicator 120 also redirects the N non-parallel beams to a first beam combiner 130. In step 630, the first beam combiner 130 redirects and converges each beam of the optical signal to a relative viewpoint within a viewer's eye movement range. The first beam combiner 130 is located between the first optical replicator and one of the viewer's eyes. The first optical replicator 120 and the first beam combiner 130 are used to converge the N non-parallel beams of each optical signal. For example, the first non-parallel beam of each optical signal converges to the first viewpoint and the second non-parallel beam of each optical signal converges to the second viewpoint. The first optical replicator 120 and the first beam combiner 130 are used to implement one of the following two embodiments. In one embodiment, the N non-parallel beams of an optical signal are physically converged at a single point on the first beam combiner 130. In another embodiment, the N non-parallel beams of the optical signal from the first optical replicator 120 are reflected at different points on the first light combining element 130. After reflection by the first light combining element 130, the optical path extensions of the N non-parallel beams of each optical signal virtually converge at a position D1, which is located d behind the first light combining element 130 and far from the viewer's eye.

[0064] In addition to the three steps described above, in one embodiment, after step 610 and before step 620, the method further includes a step 615. In step 615, a first collimator 160 ensures that the light signals from the plurality of image pixels generated by the first image projector 110 have approximately the same angle of incidence on the first optical replicator 120. The first collimator can be placed in the optical path between the first image projector 110 and the first optical replicator 120 to achieve this function.

[0065] In summary, a key feature of the various image display systems described in the first embodiment is that, regardless of the viewpoint from which the viewer sees the image, the viewer's eye can perceive the image / object (whether two-dimensional or three-dimensional) generated by the image display system as if the image were located at the same position on the first light-combining element 130 or the converging plane 135. In other words, when the viewer's eye moves from one viewpoint to another within the eye-tracking range, the viewer can see the complete image / object at exactly the same position. In the prior art, because N beams of each light signal are parallelly redirected to the viewpoint after reflection by the light-combining element, the observer perceives the object moving when their eye moves from one viewpoint to another within the eye-tracking range.

[0066] Another feature is that when the N non-parallel beams representing each light signal of a pixel from the first optical replicator 120 converge to a point on the first beam combiner 130, almost the entire area of ​​the first beam combiner 130 can be used as the field of view (FOV). In the prior art, the N beams representing each light signal of a pixel are directed to different areas of a beam combiner, and therefore, after being reflected by the beam combiner, the N beams of each light signal are redirected in parallel from different points on the beam combiner to the viewpoint. Therefore, only a small area of ​​the beam combiner (approximately the beam combiner divided by N) can be used as the field of view.

[0067] Second Embodiment

[0068] If the content provided in the first embodiment is consistent with the content provided in the second embodiment, it will be included in the second embodiment. In the second embodiment, such as... Figure 7A As shown, an image display system 200 includes a first image projector 210, a first optical reflector 220, and a first light combining element 230. This image processor system 200 can expand the eye movement range for each viewer's eye. Using the principle of "time division," this second embodiment utilizes the rapid movement of the first optical reflector 220 to receive light signals from an image, and through the movement of the first optical reflector 220, the light signals are rapidly redirected to the first light combining element 230 at different incident angles. The first light combining element 230 is located between the first optical reflector 220 and one of the viewer's eyes, and is used to receive the multiple light signals and converge them into a first visible area in the viewer's eye, thereby expanding the viewer's eye movement range 250. The movement frequency of the first optical reflector 220 is adjusted according to the projection frequency of the first image projector 210, so the multiple light signals of the first image are projected into the visible area in the viewer's eye during the persistence of vision.

[0069] The eye-tracking range 250 is the visible area within which a viewer's eye 240 can see a complete image. In other words, as long as the viewer's eye moves within the eye-tracking range, the viewer can see a complete image. This eye-tracking range (visual area) may comprise a continuous region or several viewpoints, where a viewpoint may be separate from, adjacent to, or overlap with adjacent viewpoints. The average adult pupil is 2-4 cm in diameter in bright light and 4-8 cm in dark light. In one embodiment, the distance between two adjacent central viewpoints is approximately 2.6-3 cm. Those skilled in the art will understand how the number of viewpoints, the range of viewpoints, and the distance between two adjacent central viewpoints can be determined by pupil size, image resolution, the scanning speed of the first image projector 210, and the interference effects between different beams of light. When the first optical reflector 220 moves continuously, the eye-tracking range is a continuous visual area, rather than several separate viewpoints. Therefore, as a viewer’s eyes move within the visible area (eye movement range), including from one viewpoint to the next, the viewer’s eyes can continuously see the complete image without interruption.

[0070] The first optical reflector 220 can be a one-dimensional microelectromechanical system (MEMS) mirror, a two-dimensional MEMS mirror, a polygonal reflector / mirror, a cylindrical reflector / mirror, etc. The first optical reflector 220 can move in two modes. In the first mode, the optical reflector 220 moves between N positions, each corresponding to a viewpoint within the first visual field (eye-tracking range), where N is an integer greater than one. Depending on the size of the viewpoint and the diameter of the pupil, there can be several viewpoints within an eye-tracking range, and the viewer can see the entire image from each viewpoint. In the second mode, the first optical reflector 220 moves continuously in one pattern, repeatedly redirecting the light signal and converging it into the first visual field of the viewer's eye.

[0071] The image display system 200 can be carried by a head-mounted device, such as Figure 7BAs shown, in one embodiment, this can be a pair of smart glasses 280. The glasses have a frame 285 and a pair of lenses 290. The frame 285 carries the first image projector 210 and the first optical reflector 220. The positions of the first image projector 210 and the first optical reflector 220 can be adjusted by designing the optical path. The lens 290 has a first light-combining element 230. In one embodiment, the first light-combining element 230 can be integrated with the glasses 290 as a single element. In this case, the image display system 200 can expand the eye movement range for the wearer of the head-mounted device. A viewer can see a complete image from any position, including, in some cases, different viewpoints (e.g., 251, 252, 254) within the first visual area (eye movement range). Furthermore, because the smart glasses can be customized for the viewer, the interpupillary distance (IPD) can be adjusted for each viewer. Those skilled in the art will recognize that in other embodiments, the image display system 200 can be used to expand the eye movement range for several viewers simultaneously.

[0072] The light source of the first optical projector 210 can be a laser, a light-emitting diode (LED), including miniature or micro LEDs, organic light-emitting diodes (OLEDs), superluminescent diodes (SLDs), liquid crystal on silicon (LCoS), or a liquid crystal display (LCD), or a combination thereof. In one embodiment, the first image projector 210 is a laser scanning projector (LBS projector), which consists of a light source (including a red laser, a green laser, and a blue laser), a light color modifier (such as a dual-color combining element and a polarizing combining element), and a two-dimensional adjustable reflector (such as a microelectromechanical system mirror). The LBS projector generates and scans light signals sequentially one after another at a preset resolution (e.g., 1280x720 pixels per frame). Then, the light signal of one pixel is generated and projected onto the first optical replicator 220 one at a time. In order for a viewer to see the two-dimensional image with one eye, the LBS projector must sequentially generate the light signal (e.g., 1280x720 light signals) for each pixel of the first image within the visual persistence time (e.g., 1 / 18 of a second). Therefore, the duration of each light signal is approximately 60.28 nanoseconds.

[0073] In another embodiment, the first image projector 210 may be a digital light processing (DLP) projector capable of generating a two-dimensional color image at a time. Texas Instruments' DLP technology is one such technology that can be applied to the manufacture of DLP projectors. The complete two-dimensional color image frame, for example, may include 1280x720 pixels, and is simultaneously projected onto the first optical replicator 220. Therefore, the first optical reflector 220 can simultaneously redirect several light signals (e.g., 1280x720 light signals) of a frame to the first light combining element 230.

[0074] The first optical reflector 220 is located and faces the optical path between the first image projector 210 and the second light combining element 230, and is used to simultaneously receive one or more light signals from the first image projector 210. The first light combining element 230 is located and faces between the first optical reflector 220 and a viewer's eye 240, and is used to redirect one or more light signals from the first optical reflector 220 and converge several light signals to the first visible area of ​​the viewer's eye, thereby expanding the viewer's eye movement range.

[0075] In the first mode, the first optical reflector 220 moves between N positions and reflects light signals to different portions of the first light combining element 230, where N is an integer greater than one. For example, such as Figure 10As shown, when N equals 5, the first optical reflector 220 moves very rapidly between five positions (X1, X2, X3, X4, X5). In one embodiment, the first optical reflector 220 is a one-dimensional (1D) microelectromechanical system (MEMS) mirror, repeatedly moving from X1 to X5 and back to X1 in the pattern X1→X2→X3→X4→X5→X4→X3→X2→X1. When the first optical reflector 220 is located at X1, the first combined light signal 230 reflects the light signal and then converges to the viewpoint P1. Specifically, when the laser scanning projector scans the first complete image frame (F1), the one-dimensional MEMS mirror is still at position X1 and then moves to position X2. Similarly, when the first optical reflector 220 is located at X2, the first combined light signal 230 reflects the light signal and then converges to the viewpoint P2. Specifically, when the laser scanning projector scans the second complete image frame (F2), the one-dimensional microelectromechanical system (MEMS) mirror remains at position X2. Then, the one-dimensional MEMS mirror moves to position X3, where it scans, reflects, and converges the third complete image frame (F3) to the third viewpoint P3. The first optical reflector 220 moves to position X4, where it scans, reflects, and converges the fourth complete image frame (F4) to the fourth viewpoint P4. The first optical reflector 220 moves to position X5, where it scans, reflects, and converges the fifth complete image frame (F5) to the fifth viewpoint P5. The first optical reflector 220 moves to position X4, where it scans, reflects, and converges the sixth complete image frame (F6) to the fourth viewpoint P4. The first optical reflector 220 moves to position X3, where it scans, reflects, and converges the seventh complete image frame (F7) to the third viewpoint P3. The first optical reflector 220 moves to position X2, where it scans, reflects, and converges the eighth complete image frame (F8) to the second viewpoint P2. When the first optical reflector 220, such as the one-dimensional microelectromechanical system mirror, returns to position X1, the second cycle begins. For a smooth viewing of the moving image, a viewer must see at least one complete image frame within the persistence of vision (e.g., 1 / 18 of a second).

[0076] When the first image projector 210 is a laser scanning projector, the light signal of each pixel is received and reflected sequentially to the relative position of the first optical reflector 220. In one embodiment, the first optical reflector 220 can sequentially reflect the light signal of each pixel of a first image frame (e.g., 1280x720 pixels) at position X1. Similarly, the first optical reflector 220 can sequentially reflect the light signal of each pixel of a second image frame at position X2. In this case, the first optical reflector 220 needs to remain in the same location for at least a period of time so that the laser scanning projector can scan the complete image frame.

[0077] like Figure 8 As shown, when the first image projector 210 is a digital light processing projector, the light signals of all pixels are received and simultaneously reflected at the relative position of the first optical reflector 220. The first optical reflector 220 can simultaneously reflect the light signals of all pixels of a first image frame (e.g., 1280x720 pixels) at position X1, and these signals are redirected and converged to the first viewpoint P1 by the first light combining element 230. This also applies to other positions and viewpoints.

[0078] In the second mode, the first optical reflector 220 moves continuously to reflect light signals to different positions of the first light combining element 230. In one embodiment, the optical reflector 220 is a one-dimensional microelectromechanical system mirror, which moves back and forth at both ends (e.g., X1→X5→X1). When the first image projector 210 is a laser scanning projector, as the first image projector moves continuously, the light signal of each pixel is received and reflected one after another.

[0079] Figures 9A-9D The imaging process in the second mode is further explained. As described above, during the imaging process of an image frame, when the first image projector 210 (e.g., a laser scanning projector) scans row by row or column by column to form the image frame, the first optical reflector 220 (e.g., a one-dimensional microelectromechanical system mirror) continuously moves (repeatedly rotates in one dimension) and changes position. (See reference...) Figure 9AWhen the one-dimensional microelectromechanical system (MEMS) mirror remains stationary, the image frame generated by the laser scanning projector may be rectangular. For example, line 910 represents the first row of image pixels; line 920 represents the second row; and line 930 represents the third row. However, in the second mode, due to the movement of the one-dimensional MEMS mirror, the image frame may be distorted into a parallelogram. This is because the laser projector generates an image frame by projecting one image pixel at a time; the laser projector then changes the projection position and / or angle to scan another image at a new position, typically adjacent to the previous pixel in the horizontal or vertical direction. Therefore, after a period of time, the laser scanning projector generates a row or column of image pixels (e.g., 1280x1 or 1x720). The laser scanning projector then changes the projection position and / or angle to the next row (row-by-row scanning) or the next column (column-by-column scanning) and continues to generate a second row or column of image pixels. This process continues until a complete image frame (e.g., a complete 1280x720 image pixel frame) is generated. However, in the second mode of the invention, not only does the laser scanning projector change its projection position and / or angle, but the movement of the microelectromechanical system (MEMS) mirror also affects the final shape of the image frame. Specifically, due to the movement / rotation of the one-dimensional MEMS mirror, the projection starting point of each row or column of image pixels in an image frame is translated. As a result, the shape of the image frame, such as... Figure 9B As shown, it may resemble a parallelogram because the movement of the mirror causes a change in the angle of incidence of the light signal traveling toward the one-dimensional microelectromechanical system mirror.

[0080] refer to Figure 9CIn some embodiments, the time (TEP, time between end points, 1 / 2f) required for the one-dimensional microelectromechanical system (MEMS) mirror to move from one end point to another (e.g., X1→X5, half a cycle) is set to be the same as the time (TF, time of a frame) required for the laser scanning projector to completely scan an image frame. In other words, the moving frequency of the first optical reflector (e.g., the one-dimensional MEMS mirror) must be adjusted according to the projection frequency of the first image projector (e.g., the laser scanning projector) so that several light signals of the first image can be projected onto the visible area in the viewer's eye during the persistence of vision. During the period when the one-dimensional MEMS mirror moves back from X5 to X1, the laser scanning projector completes a second image frame 902. In one embodiment of the invention, the first image frame 901 and the second image frame 902 may contain substantially the same image information (pixels). In other words, the content of the first image frame 901 and the second image frame 902 is substantially the same. The amount of difference between the content of the first image frame 901 and the second image frame 902 is determined by the frame rate of the laser scanning projector. A higher frame rate results in a smaller difference between the content of the first image frame 901 and the second image frame 902, and vice versa. In another embodiment, due to a lower frame rate, the image information of the first image frame 901 and the second image frame 902 may contain slight differences.

[0081] In addition, refer to Figure 9C In some embodiments, a portion of the image frame may extend beyond the observer's field of vision, creating a blind spot 91 within the field of vision, such as... Figure 9C The first image frame 901 is shown in region A. However, because the first image frame 901 and the second image frame 902 contain substantially the same image information, some of the image information (pixels) contained in region A can be seen at point 92 in region A', which is displayed as the second image frame 902. Therefore, the viewer can still see the complete image frame. In order for a viewer to see a complete image frame, the first image frame 901 and the second image frame 902 must be projected completely within the persistence of vision. Furthermore, the second image frame 902 is a refresh of the first image frame 901, wherein the image refresh rate is 1 / TF. However, in other embodiments, the first and second image frames 901 and 902 may contain different image information depending on the frame rate.

[0082] refer to Figure 9DIn another embodiment of the second mode, the time (TEP) required for the one-dimensional microelectromechanical system (MEMS) mirror to move from one endpoint to another (e.g., X1→X5) is set to be several times the time (TF) required for the laser scanning projector to completely scan an image frame. Therefore, N*TF = TEP, where N is a positive integer and TF is the time required for the laser scanning projector to scan an image frame. In this embodiment, several (N) image frames can be generated within the time it takes for the one-dimensional MEMS mirror to move from one endpoint to another (e.g., X1→X5). Because the first optical reflector 220 moves continuously to change the angle of incidence, thereby changing the convergence position of the light signal from the first light combining element 230, making it no longer a viewpoint, the eye-tracking range is expanded into a continuous visual area 950. Figure 9D An exemplary embodiment is described, wherein the first, second, and third image frames are formed consecutively as the one-dimensional microelectromechanical system (MEMS) mirror moves from X1 to X5; and the fourth, fifth, and sixth image frames are formed consecutively as the one-dimensional MEMS mirror moves from X5 back to X1. In some embodiments, due to the high frame rate, all six image frames contain substantially the same image information (pixels). To make these six image frames appear smooth, the first to sixth image frames must be fully scanned during the persistence of vision. However, in another embodiment, the six image frames do not need to contain the same image information. For example, the first, second, and third image frames may contain substantially the same image information, and the fourth, fifth, and sixth image frames may contain substantially the same image information. As described above, some image frames may contain a blind spot 91. However, since the remaining image frames may contain the same image information, the image information (pixels) of the blind spot 91 can be filled by a portion of the other image frames, so the viewer can still see the complete image frame.

[0083] For a viewer to see a complete image, they must see all the different parts of that image within a visual persistence time (e.g., 1 / 18 of a second). A complete image frame can be automatically stitched together from the different parts seen by the eye of a viewer located in a first viewing area. However, these different parts may come from different image frames. Because high frame rates result in very similar content between different image frames, it is difficult for a viewer to distinguish between different parts from different image frames. Furthermore, for a viewer to smoothly view a moving image, they must see at least one complete image frame at the same location within the first viewing area within the visual persistence time (e.g., 1 / 18 of a second). Additionally, to achieve better image quality for a viewer, interference effects need to be reduced and phase shift compensation provided. One method to reduce interference effects is to synchronize the frequency of the laser scanning projector with the round-trip frequency (X1→X5→X1) of the one-dimensional microelectromechanical system mirror. For example, if the image projector 210 generates a first light signal for an image frame while the optical reflector 220 begins to move from the starting position X1 so that the first light signal can be seen at the first viewpoint P1, this better synchronization can improve image quality.

[0084] When the first image projector 210 is a digital light processing projector, the light signals of all pixels are simultaneously received and reflected to the relative position of the first optical reflector 220. Therefore, at any moment when the first optical reflector 220 is continuously moving, the light signals of all pixels of an image frame (e.g., 1280x720 pixels) can be simultaneously reflected by the first optical reflector 220 and then redirected and converged by the first light combining element 230 to the viewer's visual field. When the first optical reflector 220 is a one-dimensional MSM mirror and is constantly moving between two endpoints (e.g., X1 and X5), the light signals of the image frame are converged to the first visual field.

[0085] In another embodiment of the second mode, the first optical reflector 220 is a polygonal prism reflector that continuously rotates clockwise or counterclockwise to reflect light signals to the first light combining element 230. The light combining element 230 redirects and converges the light signals to the first visible area 1100 of a viewer's eye to expand the viewer's eye movement range. However, for ease of explanation, the continuous first visible area is divided into five viewpoints. When the first image projector 210 is a laser scanning projector and the first optical reflector 220 is a pentagonal prism reflector, as the first optical reflector 220 moves continuously, the light signal of each pixel is received and reflected one by one. The pentagonal prism reflector has five faces; therefore, during a first time period, as the first optical reflector 220 continuously moves from the starting point X10 of the first face of the pentagonal prism reflector to the ending point X15 of the same face, the light signal of the first portion (e.g., the first 1 / 5) of the first image frame is reflected and redirected to the spatial range of the first viewpoint P1. During the second time period, the first optical reflector 220 continuously moves towards the endpoint X15 of the first surface, and the light signal of the second portion (e.g., the second 1 / 5) of the first image frame is reflected and redirected to the spatial range of the second viewpoint P1. Similarly, during the fifth time period, the first optical reflector 220 continuously moves towards the endpoint X15 of the first surface, and the light signal of the fifth portion (e.g., the fifth 1 / 5) of the first image frame is reflected and redirected to the spatial range of the fifth viewpoint P5. In fact, due to the continuous rotation of the first optical reflector (such as a pentagonal prism reflector), the first image is continuously projected and focused on the first viewing area 1100. Conceptually, this is divided into five viewpoints that overlap to some extent to form the first viewing area 1100. The pentagonal prism reflector then continues to rotate to the starting point X20 of the second surface of the pentagonal prism reflector. Simultaneously, the light signal of the second image frame has scanned the front end of the second portion (e.g., the second 1 / 5), indicating that during the sixth time period, the first optical reflector 220 continuously moves from the starting point X20 of the second surface to the ending point X25 of the second surface, and the light signal of the second portion (e.g., the second 1 / 5) of the second image frame is reflected and redirected to the spatial range of the first viewpoint P1. Similarly, during the seventh time period, the first optical reflector 220 continuously moves towards the ending point X25 of the second surface, and the light signal of the third portion (e.g., the third 1 / 5) of the second image frame is reflected and redirected to the spatial range of the first viewpoint P2. Finally, in order for a viewer to see a complete image, the viewer needs to see different portions of a complete image (e.g., the first 1 / 5, the second 1 / 5, the third 1 / 5, the fourth 1 / 5, and the fifth 1 / 5) within the persistence of vision (e.g., 1 / 18 of a second). However, these different portions may originate from different image frames.Because these different image frames are very close in time, and the pixels of two adjacent image frames are roughly similar, it is difficult for a viewer to detect that different parts come from different image frames. Complete image frames can be formed by automatically stitching together different parts of different image frames seen by the viewer in the first viewing area 1100 during the persistence of vision. Furthermore, in order for a viewer to smoothly watch moving images, the viewer must see multiple complete image frames in the first viewing area 1100 within the persistence of vision (e.g., 1 / 18 of a second).

[0086] As described above, in the second mode, where the first image projector 210 uses a laser scanning projector, to ensure a viewer sees better image quality, it is necessary to reduce interference effects and provide phase shift compensation. One way to reduce interference effects is to synchronize the frequency of the laser scanning projector, the number of faces of the pentagonal prism reflector, and the rotation frequency. For example, if the first optical reflector 220 begins to move from the starting position X1 of each face of the pentagonal prism reflector, and simultaneously the first image projector 210 begins to generate a light signal for an appropriate portion of an image frame, as described above, the complete image frame can be seen at every point in the first viewing area 1100. This better synchronization improves image quality. For example, during the period when the first optical reflector 220 continuously moves from the starting point X10 of the first face of the pentagonal prism reflector to the ending point X15 of the same face, the first image projector 210 projects one or more complete image frames.

[0087] like Figure 11A As shown, when the first image projector 210 is a digital light processing projector and the first optical reflector 220 is a pentagonal prism reflector, the light signals of all pixels are received and simultaneously reflected to the corresponding positions of the first optical reflector 220. As described above, due to the continuous movement of the pentagonal prism reflector, in Figure 11BThe first visible area 1100 is a continuous region. However, for ease of explanation, this continuous first visible area 1100 is conceptually divided into five viewpoints. The pentagonal prism reflector has five faces. When the starting point X10 of the first face of the pentagonal prism reflector receives light signals from all pixels of the first image projector 210, the first light combining element 230 redirects and converges these light signals to the front end of the spatial range of the first viewpoint P1. As the pentagonal prism reflector continues to move towards the ending point X15 of the first face of the pentagonal prism reflector, the first light combining element 230 redirects and converges the light signals of that pixel to the rear end of the spatial range of the last viewpoint P5. Then, as the pentagonal prism reflector continues to rotate and the starting point X20 of the second face of the pentagonal prism reflector receives light signals from all pixels of the first image projector 210, the first light combining element 230 redirects and converges these light signals back to the front end of the spatial range of the first viewpoint P1. As the pentagonal prism reflector continues to move toward the tail end X25 of the second face of the pentagonal mirror reflector, the first light combining element 230 also redirects and converges the light signal of the pixel to the tail end of the spatial range of the last viewpoint P5. The same steps are repeated as the pentagonal prism reflector continues to rotate to the third, fourth, and fifth faces. Depending on the frame rate of the first image projector 210 and the rotation speed, the viewer can see one or more image frames within the time period during which the light signal is received on the same face of the pentagonal prism reflector. In fact, due to the continuous rotation of the first optical reflector 220 (such as the pentagonal prism reflector), the first image is continuously projected and focused on the first viewing area 1100. Conceptually, five viewpoints overlap to some extent to form the first viewing area 1100. Similarly, in order for the viewer to see the complete image frame, the viewer needs to see all the different portions of the entire image frame within the period of visual persistence, for example, 1 / 18 of a second. In any case, these different portions can come from different image frames. Because these different image frames are very close in time, and the pixels of two adjacent image frames are roughly similar, it is difficult for a viewer to detect that different parts come from different image frames. A complete image frame can be formed by automatically stitching together different parts of different image frames seen by the viewer in the first viewing area 1100 during the persistence of vision. Furthermore, in order for a viewer to smoothly watch moving images, the viewer must see multiple complete image frames from the same viewpoint within the persistence of vision (e.g., 1 / 18 of a second).

[0088] Those with ordinary knowledge in the field will know that, in particular when the optical reflector 220 is a polygonal cylindrical reflector, several image display systems can be implemented simultaneously to expand the eye movement range for several viewers.

[0089] The first beam combining element 230 can be made of glass or plastic as a lens and coated with a specific material, such as metal, to make it partially transparent and partially reflective. The first beam combining element 230 can be a holographic beam splitter, but this is not the optimal choice because diffraction effects can cause several shadows and RGB shifts. In some embodiments, the use of a holographic beam splitter is avoided.

[0090] like Figure 8 or Figure 11A As shown, the image display system 200 may further include a first collimator 260, located between the first image projector 210 and the first optical reflector 220, to make the direction of motion of the light signals more consistent (parallel) in a specific direction. In other words, light signals from different pixels of the first image projector 210 become approximately parallel after passing through the first collimator 260. Therefore, the first collimator 260 ensures that each light signal has approximately the same angle of incidence on the first optical reflector 220. The first collimator 260 can be a curved lens or a convex lens.

[0091] As described above, the image display system 200, with its first image projector 210, first optical reflector 220, and first light combiner 230, can expand the eye movement range of a viewer's eye. In one embodiment, the image display system 200 may further include a second image projector 215, a second optical reflector 225, and a second light combiner 235, which function in the same manner as the first image projector 210, first optical reflector 225, and first light combiner 230, expanding the eye movement range of another point for the viewer. Similarly, the second image projector generates several light signals for a second image. The second optical reflector receives the several light signals generated by the second image projector and, through movement of the second optical reflector, redirects the several light signals to a second light combiner at different incident angles. The second light-combining element is located between the second optical reflector and one of the observer's eyes. This element receives and converges the multiple light signals into a second visual field of the observer's other eye, thereby expanding the range of eye movement for that eye. Furthermore, the movement frequency of the second optical reflector is adjusted according to the projection frequency of the second image projector, so that the multiple light signals of the second image can be projected onto a second visual field at another point in the observer's field of vision during the persistence of vision.

[0092] Furthermore, the second image projector has a similar structure to the first image projector; the second optical reflector has a similar structure to the first optical reflector; and the second light combining element has a similar structure to the first light combining element. Therefore, the image display system 100 can simultaneously expand the eye movement range of the viewer's left and right eyes.

[0093] The image projector system 200 may include a support structure that can be worn on the viewer's head to support the first image projector 210, the second image projector 215, the first optical reflector 220, the second optical reflector 225, the first light combining element 230, and the second light combining element 235. The first light combining element 230 and the second light combining element 235 are located within the viewer's field of vision. Therefore, in this embodiment, the image display system 200 is a head-mounted device (HWD). Specifically, by Figure 7B As shown, the image display system is supported by a pair of glasses, which is referred to as smart glasses. In this case, the support structure can be a frame that may have lenses, which can be prescription lenses for correcting myopia or hyperopia. The first image projector 210 and the first optical reflector 220 are supported by the right temple, and the second image projector 215 and the second optical reflector 225 are supported by the left temple. The first light-combining element 230 can be supported by the right lens, and the second light-combining element 235 can be supported by the left lens. The support can be achieved in various ways, with the light-combining element being movably or fixedly connected to or integrated into the lens. The light-combining element can be combined with the lens (including prescription lenses). When the support structure does not contain lenses, the right and left light-combining elements can be directly supported by the frame or edge.

[0094] Similar to the first embodiment, the image display system 100 for the viewer's eyes is used to display an object with depth. Because the depth of the object is the same as the position where the viewer's eyes are focused, visual convergence-accommodation conflict (VAC) and focus competition can be avoided. In this embodiment, a light signal converged from the second light combining element 235 is a first redirected right light signal, and a corresponding light signal converged from the first light combining element is a first redirected left light signal. The first redirected right light signal and the first redirected left light signal are perceived by the viewer to display a first virtual binocular pixel of an object having a first depth, which is related to a first angle between the first redirected right light signal and the corresponding first redirected left light signal. Generally, the first depth is determined by the relative horizontal distance between the first redirected right light signal and the corresponding first redirected left light signal.

[0095] Figure 12This describes a method for expanding a viewer's eye movement range using the "time segmentation" principle in the second embodiment. In step 1210, the first image projector 210 generates multiple light signals to a first optical reflector 220. In one embodiment, the image projector 210 can be a laser scanning projector (LBS projector), generating the light signals of the image pixels sequentially. In another embodiment, the image projector 220 can be a digital light processing projector, simultaneously generating all the light signals of the image (e.g., a frame of 1280x720 pixels). In either embodiment, when the image projector 210 generates the light signals at a high speed (e.g., 60 frames per second), the viewer can smoothly view the image due to visual persistence.

[0096] In step 1220, when the first optical reflector 220 moves, it receives the light signal and redirects it to different parts of the first light combining element 230. The first optical reflector 220 can be a one-dimensional microelectromechanical system (MEMS) mirror, a two-dimensional MEMS mirror, a polygonal cylindrical reflector / mirror, a cylindrical reflector / mirror, etc. The first optical reflector 220 can move in two modes. In the first mode, the first optical reflector 220 moves between N positions, each corresponding to a viewpoint, where N is an integer greater than one. In the second mode, the first optical reflector 220 moves continuously in one mode, allowing the first light combining element 230 to repeatedly redirect and converge the light signal to a first visible area in the viewer's eye, thereby expanding the viewer's eye movement range.

[0097] In step 1230, when the first optical reflector 220 moves, the first light combining element 230 reflects and converges the plurality of light signals to a first visible area of ​​the viewer, thereby expanding the range of eye movement of the viewer's eyes. The first light combining element 230 is located between the first optical reflector 220 and one of the viewer's eyes.

[0098] Furthermore, the movement frequency of the first optical reflector is adjusted according to the projection frequency of the first image projector, so that several light signals of the first image can be projected onto the first visible area of ​​the viewer's eye within the visual persistence time.

[0099] In addition to the three steps described above, in one embodiment, after step 1210 and before step 1220, the method further includes step 1215, which causes the light signals of several image pixels to have approximately the same angle of incidence on the first optical reflector 220. A first collimator can be positioned in the optical path between the first optical projector 210 and the first optical reflector 220 to achieve this function.

[0100] In summary, a key feature of this method is that almost the entire area of ​​the first light-combining element 230 can be considered as the field of view (FOV). The first optical reflector 220 redirects the light signal of a complete image to almost the entire area of ​​the light-combining element 230, which converges the light signal to a viewer's first visible area. As the first optical reflector 220 moves, the light signal of a complete image is redirected to slightly different portions of the first light-combining element 230. Therefore, considering the movement of the first optical reflector 220, a certain area of ​​the light-combining element 230 needs to be preserved. Besides this reserved area, the remaining area of ​​the first light-combining element 230 can be considered as the field of view (FOV).

[0101] The description of the embodiments provided above is intended to enable those skilled in the art to make and use the invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the basic principles established herein can be applied to other embodiments without inventive effort. Therefore, the subject matter claimed herein is not limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. Other embodiments are contemplated to be within the spirit and scope of the invention disclosed herein. Therefore, the invention is intended to cover modifications and variations that fall within the scope of the appended claims and their equivalents.

Claims

1. An image display system for expanding the range of eye movement, the system comprising: A first image projector for generating several optical signals of a first image; A first optical replicator includes at least one optical element for receiving a first light signal generated by a first image projector, the first light signal being pixels of the first image; the first optical replicator replicates the first light signal into N non-parallel light beams, and each of the N light beams of the first light signal is redirected to a first light combining element, where N is an integer greater than one; and A second image projector is used to generate several light signals for a second image: A second optical copier includes at least one optical element for receiving a second light signal generated by the second image projector, the second light signal being a pixel of the second image, the second optical copier replicating the second light signal into M non-parallel beams, and redirecting each of the M beams of the second light signal to a second combining element, wherein M is an integer greater than one. The first light-combining element is located between the first optical replicator and one eye of a viewer. The first light-combining element is configured to receive and redirect each of the N non-parallel beams of the first light signal and converge them to N viewpoints within the eye movement range of the viewer's eye. The second light-combining element is located between the second optical replicator and the other eye of the viewer. The second light-combining element is configured to receive and redirect each of the M non-parallel beams of the second light signal and converge them to M viewpoints within the eye movement range of the other eye of the viewer. The light signal redirected by the second light combining element is a first redirected right light signal, and the relative light signal redirected by the first light combining element is a first redirected left light signal. The viewer perceives the first redirected right light signal and the first redirected left light signal to display a first virtual binocular pixel of an object. N non-parallel light beam paths and M non-parallel light beam paths converge at the same point on the virtual convergence plane. The first depth of the object is related to the first angle between the first redirected right light signal and the relative first redirected left light signal.

2. The image display system of claim 1, wherein N non-parallel beams of the first optical signal from the first optical replicator physically converge on the first light combining element, or the extensions of the N non-parallel beam paths of the first optical signal redirected by the first light combining element virtually converge to a converging plane located behind the first light combining element, the converging plane being far from the viewer's eye.

3. The image display system of claim 1, wherein the viewer's field of view covers more than 80% of the first light combining element.

4. The image display system as claimed in claim 1, wherein the first optical replicator comprises one or more beam splitters, polarizers, semi-silvered mirrors, semi-reflectors, dichroic mirror prisms, dichroic optical coatings, dielectric optical coatings, or combinations thereof.

5. The image display system as described in claim 1, wherein N equals 3 and the first optical replicator is a beam splitter comprising two partial reflectors and a total reflector, splitting the first optical signal into three beams.

6. The image display system of claim 1, wherein the first optical replicator is a polarizer.

7. The image display system of claim 1, wherein a light source of the first image projector is a laser, a light-emitting diode (LED), an organic light-emitting diode (OLED), a superluminescent diode (SLD), a liquid crystal on silicon (LCoS), or a liquid crystal display (LCD), or a combination thereof.

8. The image display system as claimed in claim 1, wherein the first image projector is a laser scanning (LBS) projector or a digital light processing (DLP) projector.

9. The image display system as described in claim 1, wherein the first beam combining element is not a holographic beam splitter.

10. The image display system of claim 1, further comprising a first collimator disposed between the first image projector and the first optical replicator, so that the direction of motion of the first optical signal is more aligned with a specific direction.

Images