An augmented reality display system based on retinal projection

Abstract

The application provides a kind of based on retina projection's augmented reality display system, including collimating light source, image display, 4f optical system, phase grating, focusing eyepiece and half mirror.The collimating light source is used to provide collimating illumination light;The image display is used to load the virtual image that needs to be presented in augmented reality display;The 4f optical system is composed of two optical lenses, the distance between two optical lenses is equal to the sum of the focal length of two optical lenses, its main function is used to converge and collimate light;The phase grating is at the focal point position in the middle of 4f optical system, can form multiple parallel light by diffraction with 4f optical system combining a parallel light;The focusing eyepiece is used to converge multiple parallel light formed by 4f optical system and phase grating into multiple view points at the pupil of human eye;The half mirror is used for virtual-real fusion, forms augmented reality display effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of augmented reality display technology, in particular to an augmented reality display system based on retinal projection. BACKGROUND

[0002] Augmented reality display technology forms an augmented reality display effect by superimposing virtual image information into the real world. Augmented reality display can not only present two-dimensional image information in a plane, but also present three-dimensional image information in a stereoscopic manner using binocular parallax, giving a more realistic experience, and has great application value in the fields of medical treatment, aviation, military, education display, etc., and is regarded as the target of the next generation of mobile display terminals.

[0003] Currently, mainstream augmented reality display technology mainly uses binocular parallax to achieve 3D display effect, i.e. making the left and right eyes of a person view different parallax images to produce 3D stereoscopic visual effect through the brain. Most of the mainstream technology uses lens imaging, which easily causes vergence-accommodation conflict problem in 3D visual effect, which is the main factor causing visual fatigue. The current technologies for solving the vergence-accommodation conflict problem in 3D display mainly include holographic technology and light field display technology. Although the holographic technology can provide good visual effect, it is still difficult to achieve high-quality dynamic images due to the limitation of the space-bandwidth product. The light field display technology is difficult to be widely applied in near-eye display due to the requirement of a large number of viewing angles. Retinal projection display based on Maxwellian view can directly converge images on the pupil of the human eye, making the virtual image independent of the accommodation of the human eye. Regardless of the focusing depth of the human eye, it can provide images that are always in focus on the retina, thus solving the vergence-accommodation conflict problem in 3D display. However, the current retinal projection display technology has a very small exit pupil, which is difficult to meet the eyebox requirement of augmented reality display, so it is necessary to use pupil expansion to increase the eyebox size. Among them, view point replication is a mainstream pupil expansion method, which can replicate the view point in the near-eye position to form a larger eyebox size. However, the current view point replication technology mainly uses display panel segmentation, which easily reduces the resolution of the display image, especially as the number of view points increases, the image resolution decreases proportionally, which greatly hinders the application of retinal projection technology in augmented reality display. SUMMARY

[0004] To this end, the application aims to provide a retinal projection-based augmented reality display system, which can transform a single beam of light into multiple beams of light by combining a 4f optical system and a phase grating, and then combine a focusing eyepiece to form multiple convergent view points at the pupil, thereby realizing the expansion of the eyebox size without reducing the image resolution. The optical system can be applied to augmented reality display to realize the augmented reality display effect of full-resolution and high-quality image information.

[0005] The application provides a retinal projection-based augmented reality display system, which comprises a collimating light source, an image display, a 4f optical system, a phase grating, a focusing eyepiece and a half-transmission half-reflection mirror; the collimating light source is used for providing collimating illumination light; the image display is used for loading a virtual image to be presented in augmented reality display; the 4f optical system is used for converging and collimating light; the phase grating is used for transforming a single beam of parallel light into multiple beams of parallel light in combination with the 4f optical system; the focusing eyepiece is used for converging multiple beams of parallel light into multiple view points at the pupil of the human eye; and the half-transmission half-reflection mirror is used for virtual-real fusion to form an augmented reality display effect; the application can realize the expansion of the pupil without reducing the image resolution, increase the eyebox size, and be applied to augmented reality display, which can not only solve the problem of vergence-accommodation conflict, but also obtain the augmented reality display effect of full-resolution image information.

[0006] Further, the collimating light source can be directly generated by a light source or generated by other light sources through collimation.

[0007] Further, the image display is used for loading a virtual image, and the loaded virtual image can be taken by a real object or virtually drawn. The image display can be a transmissive image display or a reflective image display, such as an LCD, an LCOS, a DMD and the like, but is not limited to these displays.

[0008] Further, the 4f optical system is composed of two optical lenses, the distance between the two optical lenses is equal to the sum of the focal lengths of the two optical lenses, and the focal lengths of the two optical lenses can be the same or different.

[0009] Further, the phase grating is a transmission grating, which is composed of periodic stripes and has multiple diffraction effects on light. The phase grating is located at a focal point between two optical lenses in the 4f optical system. Parallel light carrying image information is converged by a front optical lens in the 4f optical system to the phase grating, is diffracted by the phase grating, forms multiple beams of diffracted light at different diffraction orders, and finally passes through a rear optical lens in the 4f optical system to form multiple beams of parallel light, each of which carries virtual image information to be presented in augmented reality display. Therefore, the phase grating combined with the 4f optical system can realize replication of single-beam image information into multiple-beam image information, and increase the channel of image information.

[0010] Further, the focusing eyepiece mainly forms converging view points of the multiple beams of parallel light at a human eye pupil, and the human eye pupil is located on a rear focal plane of the focusing eyepiece, so as to realize retinal projection display based on Maxwell view angle. The focusing eyepiece can be made of glass or other materials, and can be composed of a single lens or multiple lenses.

[0011] Further, the half-mirror can reflect and transmit light, wherein the reflectivity is used to reflect converging light to the human eye pupil to form near-eye display effect, and the transmissivity is used to transmit light of a real object into the human eye, so that the augmented reality display effect of superimposing virtual image on real object can be realized in the human eye.

[0012] Further, the application combines the 4f optical system and the phase grating, uses the converging and collimating effects of the 4f optical system and the multiple diffraction effects of the phase grating, can realize transformation of single-beam parallel light into multiple-beam parallel light, and further realize replication of view points, so as to meet the requirement of eyebox range in retinal projection display.

[0013] The application has the following advantages:

[0014] First, the retinal projection technology based on Maxwell view angle is applied to augmented reality display, can solve the problem of vergence-accommodation conflict in near-eye display, and can provide augmented reality display effect without visual fatigue.

[0015] Second, the application uses the 4f optical system combined with the phase grating to replicate view points, can not only increase the size of the eyebox, but also can ensure that the resolution of the image is not reduced, is applied to augmented reality display, can realize full-resolution image display effect, greatly increases the application prospect of augmented reality, and can be widely applied to fields such as military, medical treatment and education. BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and / or additional aspects and advantages of the embodiments of the present application will become apparent and be readily appreciated from the following description, including the references to the figures, in which:

[0017] Figure 1 is the schematic diagram of the phase grating structure modulating light under the XY axis coordinate.

[0018] Figure 2 is the schematic diagram of the 4f optical system and the phase grating jointly modulating light under the embodiment of the present application.

[0019] Figure 3 is the schematic diagram of another 4f optical system and the phase grating jointly modulating light under the embodiment of the present application.

[0020] Figure 4 is the working principle diagram of an augmented reality display system based on retinal projection under the embodiment of the present application.

[0021] Figure 5 is the working principle diagram of another augmented reality display system based on retinal projection under the embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0023] As described in the background, the retinal projection technology based on Maxwell view angle can solve the vergence-accommodation conflict problem in near-eye display and obtain better visual experience effect. At present, the contradiction between image resolution and eyebox size is a key factor that has been puzzling its popularization and application. The present application combines 4f optical system with phase grating, which can expand the convergence view point number of retinal projection display without reducing the image resolution, increase the eyebox size, and be applied to augmented reality display, which can reproduce full-resolution and high-quality virtual images, and can be favored by those skilled in the art, and become one of the most likely large-scale applications in augmented reality display systems in the future.

[0024] Please refer to Figure 1 , Figure 1is a schematic diagram of the principle of the phase grating 102 structure modulating light under XY axis coordinates. The phase grating 102 is composed of periodic or non-periodic interval fringes 103, and the fringes are medium structures. According to the diffraction theory of light, the converging spherical light wave 101 converging on the surface of the phase grating 102 is diffracted by the phase grating 102 to form multi-order diffracted diverging spherical light waves (for example, 0-order diffracted diverging spherical light wave, ±1-order diffracted diverging spherical light wave, ±2-order diffracted diverging spherical light wave, etc.), and the diffraction angle θ (the angle with the X axis) corresponding to the order satisfies the grating equation in the diffraction optics. The light wave intensity of each order of the phase grating 102 is determined by the fringe duty cycle and height, so the fringe duty cycle and height of the phase grating 102 can be designed to ensure that the brightness of each order of the diffraction region is approximately consistent, thereby ensuring that the image brightness formed by each order of diffracted light is approximately consistent.

[0025] The technical scheme of the present application is that the virtual image loaded by the image display is converged on the surface of the phase grating by the front optical lens in the 4f optical system, diffracted by the phase grating to form multiple beams of diffracted light with virtual image information, collimated by the rear optical lens in the 4f optical system, and finally formed into multiple converging eye points at the pupil of the human eye through the focusing ocular lens and the half-transmission half-reflection mirror, thereby expanding the eyebox size of the retinal projection display and forming an augmented reality display effect. The phase grating is located at the focal point position between the two optical lenses of the 4f optical system, and the diffraction angles of each order thereof satisfy the grating equation. Since the present application combines the 4f optical system with the phase grating, the number of converging eye points of the retinal projection display can be expanded without reducing the image resolution, the eyebox size is increased, and the present application is applied to augmented reality display, which can reproduce a virtual image with full resolution and high quality, effectively solves the contradiction between the image resolution and the eyebox size in the retinal projection display, and has a wider practical application scenario.

[0026] In the following, the technical scheme of the present application will be described in detail in the specific implementation manner.

[0027] Please refer to Figure 2 , Figure 2is a schematic diagram of a 4f optical system and a phase grating jointly modulating light according to an embodiment of the present application. The 4f optical system is composed of optical lens 202a and optical lens 202b with focal lengths f1 and f2 respectively. The phase grating 204 is located at the focal point between the optical lens 202a and the optical lens 202b, i.e. the distance from the phase grating 204 to the optical lens 202a is f1 and the distance from the phase grating 204 to the optical lens 202b is f2. The collimated parallel light 201 forms convergent light 203 on the surface of the phase grating 204 after passing through the optical lens 202a. After diffraction by the phase grating 204, three orders of diffracted light 205a, 205b and 205c (i.e. -1 order, 0 order and +1 order respectively) are formed. The three orders of diffracted light 205a, 205b and 205c are collimated by the optical lens 202b and finally form three beams of parallel light 206a, 206b and 206c. The phase grating 204 can be designed according to the diffraction theory to produce three orders of diffracted light, and the diffraction angles of the three orders of diffracted light can be calculated according to the grating equation.

[0028] Please refer to Figure 3 , Figure 3 is another schematic diagram of a 4f optical system and a phase grating jointly modulating light according to an embodiment of the present application. The 4f optical system is composed of optical lens 302a and optical lens 302b with focal lengths f1 and f2 respectively. The phase grating 304 is located at the focal point between the optical lens 302a and the optical lens 302b, i.e. the distance from the phase grating 304 to the optical lens 302a is f1 and the distance from the phase grating 304 to the optical lens 302b is f2. The collimated parallel light 301 forms convergent light 303 on the surface of the phase grating 304 after passing through the optical lens 302a. After diffraction by the phase grating 304, five orders of diffracted light 305a, 305b, 305c, 305d and 305e (i.e. -2 order, -1 order, 0 order, +1 order and +2 order respectively) are formed. The five orders of diffracted light 305a, 305b, 305c, 305d and 305e are collimated by the optical lens 302b and finally form five beams of parallel light 306a, 306b, 306c, 306d and 306e. The phase grating 304 can be designed according to the diffraction theory to produce five orders of diffracted light, and the diffraction angles of the five orders of diffracted light can be calculated according to the grating equation.

[0029] Please refer to Figure 4 , Figure 4is a working principle diagram of a kind of augmented reality display system based on retina projection under the embodiment of the application. The augmented reality display system includes collimating light source 401, image display 402, 4f optical system, phase grating 404, focusing eyepiece 405 and half-mirror 406. Image display 402 is transmissive, 4f optical system is composed of optical lens 403a and 403b, focal length is f1 and f2 respectively, phase grating 404 is located at the focal point position between optical lens 403a and 403b, that is, the distance of phase grating 404 to optical lens 403a is f1, and the distance to optical lens 403b is f2. Collimating light source 401 is transmitted through image display 402 to form a virtual image information, converges on the surface of phase grating 404 through optical lens 403a, forms multiple secondary diffraction lights through phase grating 404 diffraction, then collimates to form multiple parallel lights through optical lens 403b, finally, through the transmission of focusing eyepiece 405 and the reflection of half-mirror 406, multiple convergent view points can be formed at the pupil of human eye 407, wherein the effective distance of the pupil of human eye 407 to focusing eyepiece 405 is equal to the focal length of focusing eyepiece 405, for example, as shown in Figure 4 the pupil of human eye 407 forms three convergent view points (view point 1, view point 2 and view point 3 respectively), and the virtual image 408 can be viewed by human eye at any view point, and since half-mirror 406 is used, human eye 407 can not only view virtual image 408, but also view real object 409, to realize the effect of augmented reality display.

[0030] Please refer to Figure 5 , Figure 5 is another working principle diagram of an augmented reality display system based on retina projection under the embodiment of the application. The augmented reality display system includes collimating light source 501, image display 502, 4f optical system, phase grating 504, focusing eyepiece 505 and half-mirror 506. Image display 502 is reflective, 4f optical system is composed of optical lens 503a and 503b, focal length is f1 and f2 respectively, phase grating 504 is located at the focal point position between optical lens 503a and 503b, that is, the distance of phase grating 504 to optical lens 503a is f1, and the distance to optical lens 503b is f2. Collimating light source 501 is reflected through image display 502 to form a virtual image information, converges on the surface of phase grating 504 through optical lens 503a, forms multiple secondary diffraction lights through phase grating 504 diffraction, then collimates to form multiple parallel lights through optical lens 503b, finally, through the transmission of focusing eyepiece 505 and the reflection of half-mirror 506, multiple convergent view points can be formed at the pupil of human eye 507, wherein the effective distance of the pupil of human eye 507 to focusing eyepiece 505 is equal to the focal length of focusing eyepiece 505, for example, as shown in Figure 5As shown, three convergent view points (view point 1, view point 2 and view point 3) are formed at the pupil of the human eye 507, and the human eye can view the virtual image 508 at any one of the view points, and due to the adoption of the half-transmission half-reflection mirror 506, the human eye 507 can not only view the virtual image 508, but also view the real object 509, so as to realize the augmented reality display effect.

[0031] The phase grating described above can be manufactured by using photolithography and nano-imprinting. It should be noted that in the present application, the phase grating structure can be etched on the surface of the substrate by using the photolithography method, or a mask used for imprinting can be first manufactured by using the photolithography method, and then the phase grating described above can be imprinted on the substrate in large quantities by using the nano-imprinting technology. The number of view points formed by the present application is determined by the number of diffraction orders of the phase grating, so the phase grating parameters can be designed according to the number of view points, so as to ensure that the number of diffraction orders and the diffraction intensity of each order are approximately consistent.

[0032] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0033] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A retinal projection-based augmented reality display system, characterized by, The system comprises a collimating light source, an image display, a 4f optical system, a phase grating, a focusing eyepiece and a half-transmission half-reflection mirror; the collimating light source is used to provide collimating illumination light; the image display is used to load virtual images to be presented in the augmented reality display; the 4f optical system is used to converge and collimate light; the phase grating is used to transform a parallel light into multiple parallel lights in combination with the 4f optical system; the focusing eyepiece is used to converge the multiple parallel lights into multiple view points at a human eye pupil; the half-transmission half-reflection mirror is used for virtual-real fusion to form an augmented reality display effect; The collimating light source is directly generated by a light source or generated by collimation of other light sources; The image display is used to load virtual images, and the loaded virtual images are taken from real objects or virtually drawn images; The image display is a transmission type image display or a reflection type image display; The 4f optical system comprises two optical lenses, and the distance between the two optical lenses is equal to the sum of the focal lengths of the two optical lenses; The phase grating is a transmission type grating, which comprises periodic fringes and has multiple-order diffraction effects on light; The phase grating is located at the focal point between the two optical lenses in the 4f optical system, and parallel light carrying image information is converged on the phase grating through the front optical lens in the 4f optical system, diffracted by the phase grating to form multiple beams of diffracted light at different diffraction orders, and finally passed through the rear optical lens in the 4f optical system to form multiple parallel lights, each of which carries virtual image information to be presented in the augmented reality display; The focusing eyepiece is used to form convergent view points of the multiple parallel lights at a human eye pupil, and the human eye pupil is located on the rear focal plane of the focusing eyepiece, so as to realize retinal projection display based on Maxwell view angle; the focusing eyepiece is made of glass or other materials and comprises a single lens or multiple lenses; The half-transmission half-reflection mirror can reflect and transmit light, wherein the reflectivity is used to reflect convergent light rays to the human eye pupil to form a near-eye display effect, and the transmissivity is used to transmit real object light rays into the human eye to present the augmented reality display effect of superimposing virtual images on real objects; The augmented reality display system adopts the 4f optical system combined with the phase grating to realize view point replication, which can increase the eye box size.

Images