Augmented reality near-to-eye display system based on pixel grating array device

Abstract

The invention belongs to the technical field of display, and particularly provides an augmented reality near-to-eye display system based on a pixel grating array device, which comprises a transparent display screen and the pixel grating array device which are closely arranged, the transparent display screen is composed of a plurality of image pixels and used for displaying a target image and emitting light containing target image information to the pixel grating array device. The pixel grating array device is composed of a plurality of grating pixels with different grating vector parameters, and the grating pixels are aligned with image pixels on the transparent display screen one by one; light emitted by each image pixel on the transparent display screen is directionally diffracted to the crystalline lens optical center of the human eye by the grating pixels on the aligned pixel grating array device and finally reaches the retina, so that the retina projection near-to-eye display effect is formed, and a complex projection light path in an existing augmented reality near-to-eye display system is avoided. The device has the advantages of simple structure, small size and light weight.

Description

Technical Field

[0001] The present invention belongs to the field of display technology, and in particular relates to an augmented reality near-eye display system based on a pixel grating array device. Background Art

[0002] Augmented reality near-eye display is a new interactive window that integrates virtual digital information and real physical information. It is widely used in military training, industrial manufacturing, surgical navigation and other fields due to its characteristics of virtual-real fusion, three-dimensional immersion, and real-time interaction. It is expected to become the next generation of human-computer interactive display platform. The optical module is the core component of the augmented reality near-eye display. Its performance is a key factor that directly affects the imaging quality and interactive experience. The design of the optical module needs to take into account the visual characteristics of the human eye and the requirements of ergonomics, and comprehensively consider important indicators such as field of view, eye movement range, light efficiency, angular resolution, component transmittance, distortion, volume and quality. From the current status of technological development, augmented reality near-eye display technology mostly adopts binocular parallax stereoscopic perspective display mode, and uses different types of optical modules (free-form optical components / geometric optical waveguides / diffraction optical waveguides / holographic optical waveguides as the mainstream) to fuse real scenes with virtual images. However, when it comes to consumer-level applications, augmented reality near-eye display optical modules still generally have their own limitations. For example, the field of view and size are balanced when designing free-form surface structures, and optical waveguide structures can achieve thin and light systems but the field of view is limited. In addition, there are problems such as low display quality, small depth range, and visual fatigue caused by the contradiction between convergence and focusing. Therefore, it will become very important to overcome these limitations through innovative optical architectures, innovative model designs, and the use of new principle optical components. Metasurface is an array of sub-wavelength electromagnetic control units arranged in an artificial way on a two-dimensional plane. It can arbitrarily control the amplitude, phase, polarization, and frequency parameters of light waves. It has the advantages of ultra-thinness, planarization, low loss, and easy integration. It is generally considered to be the next generation of new optical components. With the powerful light field control ability and the special advantages of thinness and flatness of metasurfaces, it can provide new solutions to break through the bottleneck problems of existing near-eye display systems. Therefore, metasurface optical technology is expected to become a powerful platform for researching and improving augmented reality near-eye displays.

[0003] At present, researchers at home and abroad have conducted exploratory research and beneficial attempts to apply metasurfaces in the field of augmented reality near-eye display. For example, in 2017, C. Hong et al. from the University of Washington in the United States first proposed a design scheme for an augmented reality near-eye display system based on a microdisplay and a metasurface. Compared with the near-eye display system based on a free-form reflector, the system has the advantages of small size and large field of view. However, the system can only respond to red light and cannot achieve color display. Later, in 2021, E. Bayati et al. from the same group at the University of Washington in the United States improved the above scheme and proposed a design scheme for an achromatic augmented reality near-eye display system based on a composite metasurface, in which the metasurface close to the human eye realizes an achromatic phase distribution for reflecting light from the display; the other metasurface is used to avoid the distortion of the real light field information caused by the achromatic metasurface, and has the advantages of large field of view, small volume, and achromaticity. However, the internal structure of the composite metasurface is complex and the characteristic size is extremely small, which leads to its complex manufacturing process and expensive processing cost. At the same time, the two-sided metasurface needs to be precisely aligned during the manufacturing process, which further increases the processing difficulty of the composite metasurface. In addition, Chinese patent 201811187305.2 discloses an augmented reality near-eye display system based on an ultra-microstructure, wherein the ultra-microstructure has a semi-transparent and semi-reflective function, which is used to reflect and converge the light emitted by the microdisplay to the pupil of the human eye, thereby achieving the effect of augmented reality near-eye display. However, the above three schemes all adopt a reflective optical system design, resulting in a large space between the microdisplay and the metasurface, which hinders the further reduction of the volume of the near-eye display system. In addition, Chinese patent 202210019355.X discloses a metasurface micro-nano near-eye display based on retinal display, including a transparent micro-display image source, a metasurface micro-nano focusing lens group, and a metasurface micro-nano compensation lens group, which has the advantages of large field of view and small volume. The ultra-surface micro-nano focusing lens group focuses the light beam information emitted by the transparent micro-display image source on the optical center of the human eye lens, and finally reaches the retina for imaging; the ultra-surface micro-nano compensation lens group can offset the focusing effect of the ultra-surface micro-nano focusing lens group, so that the light emitted by objects in the real external environment can enter the human eye normally without lens effect after passing through the ultra-surface micro-nano compensation lens group and the ultra-surface micro-nano focusing lens group; the pixel of the transparent micro-display image source is composed of three sub-pixels of red, green and blue, and the nano-column array of a single pixel on the ultra-surface micro-nano focusing lens layer corresponds to a pixel of a transparent micro-display image source, thereby realizing achromatic color display. However, the structure of the ultra-surface micro-nano lens group is complex and delicate, and in order to accurately modulate the phase, the length, width and rotation angle of the nano-columns of different pixels must be different, which makes the preparation of the structure extremely difficult and the processing cost expensive. On this basis, an augmented reality near-eye display system based on a pixel grating array device is proposed. Summary of the invention

[0004] The present invention mainly provides an augmented reality near-eye display system based on a pixel grating array device, which overcomes the common problems of great difficulty and high cost in processing of metasurface optical devices used in augmented reality near-eye display systems at this stage, and aims to provide a low-cost and easy-to-process metasurface optical device, which cooperates with a transparent display screen to achieve a large field of view and small volume augmented reality near-eye display effect.

[0005] The technical solution adopted by the present invention to solve the above-mentioned technical problems is:

[0006] An augmented reality near-eye display system based on a pixel grating array device comprises a transparent display screen and a pixel grating array device placed closely together, wherein the pixel grating array device is close to a human eye side and the transparent display screen is close to a real object side;

[0007] The transparent display screen is composed of a plurality of image pixels, and is used to display a target image, and emits light containing target image information to be incident on a pixel grating array device;

[0008] The pixel grating array device is composed of a plurality of grating pixels with different grating vector parameters, and the grating pixels are aligned one by one with the image pixels on the transparent display screen;

[0009] The light emitted by each image pixel on the transparent display screen is directionally diffracted by the grating pixels on the aligned pixel grating array device to the optical center of the lens of the human eye and finally reaches the retina, thereby forming a retinal projection near-eye display effect.

[0010] A further improvement is that the size of the grating pixels is equal to the size of the image pixels on the transparent display screen, and the spacing between adjacent grating pixels is equal to the spacing between adjacent image pixels on the transparent display screen.

[0011] A further improvement is that the grating vector parameters of the grating pixels on the pixel grating array device include spatial frequency and grating line orientation angle, and the calculation method is as follows: establish a coordinate system of the display plane and the observation plane, record the distance between the two planes as d, record the center position of any grating pixel on the display plane as A(x, y, 0), and record the position of the optical center of the human eye lens on the observation plane as B(0, 0, d), then the spatial frequency and grating line orientation angle of the grating pixel are respectively:

[0012]

[0013]

[0014] Where λ is the wavelength of light, n is the refractive index of the grating material, and f sis the spatial frequency of the grating pixel, and α is the grating line orientation angle of the grating pixel. According to the above formula, the center position coordinates of all grating pixels on the pixel grating array device are traversed to calculate the spatial frequency and grating line orientation angle corresponding to all grating pixels.

[0015] A further improvement is that the transparent display screen and the pixel grating array device have the function of transmitting the light emitted by the real object, so that the light emitted by the real object can also enter the human eye and finally reach the retina, thereby realizing the superposition and fusion of the real object image and the image displayed on the transparent display screen on the retina, thus forming the effect of augmented reality near-eye display.

[0016] A further improvement is that it also includes a phase compensation device, which is placed closely on the side of the transparent display screen close to the real object, and is used to compensate for the phase modulation caused by the light emitted by the real object passing through the pixel grating array device, so that the light emitted by the real object passes through the phase compensation device and the pixel grating array device without phase modulation and enters the human eye normally, thereby avoiding the distortion of the real light field information caused by the pixel grating array device.

[0017] A further improvement is that the transparent display screen is any one of an organic light emitting diode display, a micron-scale organic light emitting diode display, a micron-scale inorganic light emitting diode display, and a nanometer-scale inorganic light emitting diode display.

[0018] A further improvement is that the material of the pixel grating array device is quartz glass or K9 glass.

[0019] A further improvement is that the spatial frequency of the grating pixels on the pixel grating array device is 0 to 3000 lines / mm.

[0020] A further improvement is that the grating line orientation angle of the grating pixels on the pixel grating array device is 0 to 180°.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. The augmented reality near-eye display system of the present invention is formed by closely attaching a transparent display screen and a pixel grating array device, avoiding the complex projection optical path in the existing augmented reality near-eye display system, and has a simple structure, small size and light weight;

[0023] 2. The augmented reality near-eye display system of the present invention has an appearance similar to that of ordinary eyeglass lenses, and the pixel grating array device is close to one side of the human eye, thereby effectively reducing the pupil distance and expanding the field of view of the near-eye display;

[0024] 3. Compared with the retinal projection display method based on laser scanning technology, the grating pixels on the pixel grating array device of the present invention are aligned one by one with the image pixels on the transparent display screen, so that the light emitted by each image pixel on the transparent display screen is diffracted to the optical center of the lens of the human eye and finally reaches the retina, thereby forming a retinal projection display effect, avoiding the safety problem of laser focusing on the optical center of the lens, and improving the safety of use;

[0025] 4. Compared with other metasurface optical devices used in augmented reality near-eye display systems, the pixel grating array device in the present invention has a larger characteristic size and does not require expensive electron beam lithography technology during processing, which helps to reduce processing difficulty and processing costs.

[0026] The present invention will be explained in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of an augmented reality near-eye display system based on a pixel grating array device proposed by the present invention;

[0028] Figure 2 A schematic diagram of alignment between grating pixels on the pixel grating array device of the present invention and image pixels on the transparent display screen;

[0029] Figure 3 A diagram showing the positional relationship between the display plane and the observation plane of the present invention;

[0030] Figure 4 is a spatial frequency distribution diagram of grating pixels on a pixel grating array device in specific embodiment 1 of the present invention;

[0031] Figure 5 is a distribution diagram of the orientation angles of grating pixel lines on the pixel grating array device in the specific embodiment 1 of the present invention;

[0032] Figure 6 This is a schematic diagram of an augmented reality near-eye display system based on a pixel grating array device and a phase compensation device proposed in Example 2 of the present invention.

[0033] Among them, 1-transparent display screen; 101-image pixel; 2-pixel grating array device; 201-grating pixel; 3-phase compensation device. DETAILED DESCRIPTION

[0034] In order to facilitate understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings, in which several embodiments of the present invention are given, but the present invention can be implemented in different forms and is not limited to the embodiments described in the text. On the contrary, these embodiments are provided to make the content disclosed by the present invention more thorough and comprehensive.

[0035] Example 1

[0036] Reference Figure 1-5 The present invention proposes an augmented reality near-eye display system based on a pixel grating array device. Figure 1 As shown, it includes a transparent display screen 1 and a pixel grating array device 2;

[0037] The transparent display screen 1 is composed of a plurality of image pixels 101 and is used for displaying a target image. The light beam containing the target image information is emitted and incident on the pixel grating array device 2 .

[0038] The pixel grating array device 2 is composed of a plurality of grating pixels 201 with different grating vector parameters. The grating pixels 201 are aligned one by one with the image pixels 101 on the transparent display screen 1. Figure 2 As shown, the size of the grating pixel 201 is equal to the size of the image pixel 101 on the transparent display screen 1 , and the spacing between adjacent grating pixels 201 is equal to the spacing between adjacent image pixels 101 on the transparent display screen 1 .

[0039] In the above solution, the pixel grating array device 2 is placed closely to the transparent display screen 1, wherein the pixel grating array device 2 is close to the human eye side and the transparent display screen 1 is away from the human eye side, that is, close to the real object side.

[0040] The light emitted by each image pixel 101 on the transparent display screen 1 is directionally diffracted by the aligned grating pixels 201 on the pixel grating array device 2 to the optical center of the lens of the human eye and finally reaches the retina, thereby forming a retinal projection near-eye display effect.

[0041] In the above scheme, the grating vector parameters of the grating pixel 201 on the pixel grating array device 2 include the spatial frequency and the grating line orientation angle. The specific calculation method is as follows: establish the coordinate system of the display plane and the observation plane, such as Figure 3 As shown, the distance between the two planes is recorded as d, the center position of any grating pixel on the display plane is recorded as A(x,y,0), and the position of the optical center of the human eye lens on the observation plane is recorded as B(0,0,d). From the geometric relationship, the diffraction angle (the angle between the diffracted light vector and the positive direction of the z-axis) and the diffraction azimuth (the angle between the projection component of the diffracted light vector on the xoy plane and the positive direction of the x-axis) of point A are obtained as follows:

[0042]

[0043]

[0044] According to the generalized grating equation, it can be deduced:

[0045]

[0046]

[0047] Where λ is the wavelength of light, n is the refractive index of the grating material, and f s is the spatial frequency of the grating pixel, and α is the grating line orientation angle of the grating pixel (the angle between the grating line and the positive direction of the y-axis).

[0048] Combining the above formulas, we can get that the spatial frequency f of any grating pixel is s and the grid line orientation angle α are:

[0049]

[0050]

[0051] According to the calculation formula of the spatial frequency and the grating line orientation angle, the center position coordinates of all grating pixels 201 on the pixel grating array device 2 are traversed to calculate the spatial frequency and the grating line orientation angle corresponding to all the grating pixels 201.

[0052] In the preferred technical solution, the transparent display screen 1 and the pixel grating array device 2 both have the function of transmitting light emitted by real objects, so that the light emitted by real objects can also enter the human eye and eventually reach the retina, thereby realizing the superposition and fusion of the real object image and the image displayed on the transparent display screen 1 on the retina, thereby forming an augmented reality near-eye display effect.

[0053] In the above technical solution, the transparent display screen 1 is an organic light emitting diode display (OLED), a micron-sized organic light emitting diode display (Micro-OLED), a micron-sized inorganic light emitting diode display (Micro-LED) or a nano-sized inorganic light emitting diode display (Nano-LED).

[0054] In the above technical solution, the material of the pixel grating array device 2 is quartz glass or K9 glass.

[0055] In the above technical solution, the spatial frequency of the grating pixels 201 on the pixel grating array device 2 is 0 to 3000 lines / mm.

[0056] In the above technical solution, the grating line orientation angle of the grating pixel 201 on the pixel grating array device 2 is 0-180°.

[0057] In the specific implementation process, the transparent display screen 1 in this embodiment uses an organic light-emitting diode display (OLED) device with a thickness of 1mm, which is composed of 1920×1080 image pixels 101, the size of each image pixel 101 is 18um×18um, and the spacing between adjacent image pixels 101 in the horizontal and vertical directions is 20um. Then the pixel grating array device 2 is composed of 1920×1080 grating pixels 201 with different grating vector parameters, and the size of each grating pixel 201 is also 18um×18um, and the spacing between adjacent grating pixels 201 in the horizontal and vertical directions is also 20um. The peak emission wavelength of the transparent display screen 1 is 532nm, and the material of the pixel grating array device 2 is K9 glass with a refractive index of 1.519. Assuming that the human eye is located on the central axis of the transparent display screen 1 and the observation distance from the transparent display screen 1 is 25 mm, the spatial frequency distribution and the grid line orientation angle distribution of all grating pixels 201 on the pixel grating array device 2 can be calculated according to the calculation formulas of the spatial frequency and the grid line orientation angle, respectively, as follows: Figure 5 and Figure 6 As shown, the spatial frequency of the grating pixels 201 on the pixel grating array device 2 varies between 0 and 1888 lines / mm, and the grating line orientation angle varies between 0 and 180°.

[0058] A preferred manufacturing process of the pixel grating array device 2 in this embodiment is as follows: first, the glass substrate is strictly cleaned, and then photoresist is coated on the substrate surface by spin coating to form a uniform photoresist layer, and then a pixel grating array pattern is made by photolithography technology to form a photoresist grating mask, and then the above-mentioned photoresist grating mask is pattern-transferred by ion beam etching technology. After the etching is completed, a pixel grating array pattern is formed on the surface of the glass substrate, that is, an etched grating, and finally the remaining photoresist on the top of the etched grating is removed and cleaned, thereby completing the manufacture of the pixel grating array device 2. The photolithography technology includes but is not limited to laser direct writing lithography, ultraviolet continuous variable space-frequency interference lithography, and two-photon polymerization lithography.

[0059] Afterwards, the manufactured pixel grating array device 2 and the transparent display screen 1 are integrated and adjusted under a microscope. When it is observed that the grating pixels 201 on the pixel grating array device 2 are aligned one by one with the image pixels 101 on the transparent display screen 1, the positions of the transparent display screen 1 and the pixel grating array device 2 are fixed, and then they are bonded and cured with glue, thereby forming an augmented reality near-eye display system based on the transparent display screen and the pixel grating array device.

[0060] The light emitted by each image pixel 101 on the transparent display screen 1 is directionally diffracted by the corresponding grating pixel 201 on the pixel grating array device 2 to the optical center of the lens of the human eye and finally reaches the retina, thereby achieving the effect of retinal projection near-eye display. At the same time, the light emitted by the real object enters the human eye through the transparent display screen 1 and the pixel grating array device 2 in turn, and finally reaches the retina to form a real object image. The real object image on the retina is superimposed and fused with the image displayed on the transparent display screen 1, and finally the effect of augmented reality near-eye display is achieved.

[0061] In this embodiment, the display size of the transparent display screen 1 is 38.4 mm×21.6 mm, the observation distance is 25 mm, and the horizontal, vertical and diagonal viewing angles are calculated to be 75.05°, 46.73° and 82.77°, respectively, which illustrates that the augmented reality near-eye display system based on the pixel grating array device proposed in the present invention has the advantage of a large field of view.

[0062] In summary, the present invention discloses an augmented reality near-eye display system based on a pixel grating array device. In the present invention, the augmented reality near-eye display system is composed of only two components, a transparent display screen and a pixel grating array device, which are closely attached to each other. The appearance is similar to that of ordinary eyeglass lenses, avoiding complex projection light paths, and having the advantages of small size, light weight, and a large field of view. In addition, compared with other metasurface optical devices used in augmented reality near-eye display systems, the pixel grating array device in the present invention is a low-cost, easy-to-process metasurface optical device, which does not require expensive electron beam lithography technology during processing, thereby effectively reducing the processing difficulty and processing cost.

[0063] Example 2

[0064] Focus on reference Figure 6 Based on Example 1, this embodiment also provides an augmented reality near-eye display system based on a pixel grating array device. Figure 6 As shown, it includes a transparent display screen 1, a pixel grating array device 2 and a phase compensation device 3. The phase compensation device 3 is placed closely on the side of the transparent display screen 1 close to the real object, and is used to compensate for the phase modulation caused by the light emitted by the real object passing through the pixel grating array device 2, so that the light emitted by the real object passes through the phase compensation device 3 and the pixel grating array device 2 without phase modulation and enters the human eye normally, thereby avoiding the distortion of the real light field information caused by the pixel grating array device 2. Although the embodiments of the present invention have been shown and described, it can be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An augmented reality near-eye display system based on a pixel grating array device, characterized in that: It includes a transparent display screen and a pixel grating array device placed closely together, wherein the pixel grating array device is close to a human eye side and the transparent display screen is close to a real object side; The transparent display screen is composed of a plurality of image pixels, and is used to display a target image, and emits light containing target image information to be incident on a pixel grating array device; The pixel grating array device is composed of a plurality of grating pixels with different grating vector parameters, and the grating pixels are aligned one by one with the image pixels on the transparent display screen; The light emitted by each image pixel on the transparent display screen is directionally diffracted by the grating pixels on the aligned pixel grating array device to the optical center of the lens of the human eye and finally reaches the retina, thereby forming a retinal projection near-eye display effect.

2. The augmented reality near-eye display system based on a pixel grating array device according to claim 1, characterized in that: The size of the grating pixel is equal to the size of the image pixel on the transparent display screen, and the spacing between adjacent grating pixels is equal to the spacing between adjacent image pixels on the transparent display screen.

3. The augmented reality near-eye display system based on a pixel grating array device according to claim 1, characterized in that: The grating vector parameters of the grating pixels on the pixel grating array device include spatial frequency and grating line orientation angle, and the calculation method is as follows: establish a coordinate system of the display plane and the observation plane, record the distance between the two planes as d, record the center position of any grating pixel on the display plane as A(x, y, 0), and record the position of the optical center of the human eye lens on the observation plane as B(0, 0, d), then the spatial frequency and grating line orientation angle of the grating pixel are respectively: Where λ is the wavelength of light, n is the refractive index of the grating material, and f s is the spatial frequency of the grating pixel, and α is the grating line orientation angle of the grating pixel; according to the above formula, the center position coordinates of all grating pixels on the pixel grating array device are traversed to calculate the spatial frequency and grating line orientation angle corresponding to all grating pixels.

4. The augmented reality near-eye display system based on a pixel grating array device according to claim 1, characterized in that: The transparent display screen and pixel grating array device have the function of transmitting light emitted by real objects, so that the light emitted by real objects can enter the human eye and eventually reach the retina, thereby realizing the superposition and fusion of the real object image and the image displayed on the transparent display screen on the retina, forming an augmented reality near-eye display effect.

5. The augmented reality near-eye display system based on a pixel grating array device according to claim 1, characterized in that: It also includes a phase compensation device, which is placed closely on the side of the transparent display screen close to the real object and is used to compensate for the phase modulation caused by the light emitted by the real object passing through the pixel grating array device.

6. The augmented reality near-eye display system based on a pixel grating array device according to claim 1, characterized in that: The transparent display screen is any one of an organic light emitting diode display, a micron-level organic light emitting diode display, a micron-level inorganic light emitting diode display, and a nanometer-level inorganic light emitting diode display.

7. The augmented reality near-eye display system based on a pixel grating array device according to claim 1, characterized in that: The material of the pixel grating array device is quartz glass or K9 glass.

8. The augmented reality near-eye display system based on a pixel grating array device according to claim 1, characterized in that: The spatial frequency of the grating pixels on the pixel grating array device is 0 to 3000 lines / mm.

9. The augmented reality near-eye display system based on a pixel grating array device according to claim 1, characterized in that: The grating line orientation angle of the grating pixels on the pixel grating array device is 0-180 degrees.

Images