Virtual image phased array based augmented reality display pupil expansion display apparatus and method

Abstract

The application relates to a device and method for pupil dilation and display of a virtual image phase array-based augmented reality display, which comprises a swept-frequency laser, a modulator, a collimator, a mirror, a MEMS and a virtual image phase array, the output end of the swept-frequency laser is connected with the input port of the modulator, the output port of the modulator is connected with the collimator, a video signal is used for modulating laser through the modulator, the output end of the modulator is connected with the collimator through an optical waveguide, the output of the collimator is reflected to the virtual image phase array through the mirror and then output, the mirror is controlled by the MEMS and scans along the X direction, the output spectrum of the swept-frequency laser is real-time changeable, and scanning along the Y direction is realized after the virtual image phase array.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of augmented reality near-eye display technology, and particularly relates to an apparatus and method for pupil expansion and display of an augmented reality display based on a virtual image phase array. BACKGROUND

[0002] With the development of 5G, a research wave of virtual reality and augmented reality has been triggered for the "next generation mobile intelligent computing platform". Virtual reality separates the real world from the user through a display device, enabling the user to immerse in a computer-generated virtual environment and interact with virtual world objects. It can only be used in an indoor closed environment. Augmented reality is a computer-generated virtual image superimposed on a real scene, and the user can interact with virtual and real objects at the same time. Therefore, augmented reality is more interactive than virtual reality and has a wider range of applications. Head-mounted display AR (HMD) devices belong to near-eye display devices, which couple the virtual image formed by a miniature display into the human eye through optical elements, and the external light is transmitted to the human eye through the optical device. Therefore, the user can experience a more realistic and natural virtual and real scene superimposition effect, achieving the purpose of augmented reality. Existing HMDs can be mainly divided into AR display devices based on traditional optical lenses and AR display devices based on optical waveguide optical elements.

[0003] The biggest advantage of optical waveguide in near-eye display devices is pupil expansion. In AR display systems based on traditional optical lenses, the pupil distance and the field of view angle are inversely proportional. An optical system with good imaging needs a large field of view angle, however, as the field of view angle increases, the pupil distance decreases, which will result in a reduced eye movement range of the observer, and the eye can only move in a small conical area to observe the virtual image. Therefore, in several schemes of AR display devices based on traditional lenses, the relationship between the field of view angle and the eye movement range cannot be balanced, and the development is limited. Pupil expansion is also divided into one-dimensional expansion and two-dimensional expansion, and two-dimensional expansion is realized on the basis of one-dimensional expansion. Diffractive gratings can also be used as waveguide coupling elements in waveguide imaging systems to realize the coupling and transmission of light waves and pupil expansion. Microsoft's Hololens2 uses a laser scanning method for pupil expansion, but the optical effect is general.

[0004] VIPA (Virtually imaged phased array) is an optical device with large angular dispersion and high resolution characteristics, which can be used to achieve wave division multiplexing, dispersion compensation and filter functions, but due to its high requirements for the incident light beam, multi-stage diffraction and large loss, the application is not very extensive. Until 2004, Shijun Xiao of Purdue University combined VIPA and diffraction grating, and VIPA and diffraction grating combined to map the broadband spectrum onto a two-dimensional plane, and the research based on VIPA gradually increased. The present application is a new method and device for expanding the pupil and displaying of an augmented reality display based on a virtually imaged phased array, which mainly utilizes the pupil expansion characteristics of the virtually imaged phased array. SUMMARY

[0005] The present application aims to provide a new device and method for expanding and displaying an augmented reality display based on a virtually imaged phased array, which is expected to change the current augmented reality light waveguide expansion and display technology.

[0006] In order to achieve the above-mentioned purpose, the present application provides a device for expanding and displaying an augmented reality display based on a virtually imaged phased array, comprising a frequency-sweeping laser, a modulator, an optical waveguide, a collimator, a mirror, a MEMS and a virtually imaged phased array, the output end of the frequency-sweeping laser is connected with the input port of the modulator, the output port of the modulator is connected with the collimator through the optical waveguide, a video signal modulates the laser through the modulator, the modulated laser beam is output after being reflected by the mirror and output to the virtually imaged phased array, the mirror is controlled by the MEMS and scans along the X direction, the output spectrum of the frequency-sweeping laser is real-time changeable, and the laser beam will scan along the Y direction after passing through the virtually imaged phased array.

[0007] The virtually imaged phased array is a double-sided coated glass flat plate, the incident surface of the glass flat plate is coated with a high reflectivity film with a reflectivity of more than 99% except for the anti-reflection film coated in the incident window area, and the exit surface is coated with a high reflectivity film with a reflectivity of 95% to 98%.

[0008] The frequency-sweeping laser comprises a red-green-blue or red-yellow-blue three-color laser, and the modulated laser output by the frequency-sweeping laser is combined and then enters the optical waveguide.

[0009] Preferably, the mirror is controlled by a MEMS or a PZT and scans along the x direction, stays at each point for a period of time tx, during which the modulated laser output by the frequency-sweeping laser enters the virtually imaged phased array, and the laser can be swept due to the change of frequency, and scans along the y direction point by point according to different frequencies.

[0010] The device for expanding and displaying of the virtual image phase array based augmented reality display comprises two laser scanning modes of transverse scanning and longitudinal scanning.

[0011] The transverse scanning refers to that the scanning speed of the MEMS along the x direction is much greater than the sweeping speed of the frequency sweeping laser, when the frequency sweeping laser is stabilized at a certain frequency, the output light beam is firstly scanned along the x direction under the action of the MEMS, after scanning a row, the frequency sweeping laser is adjusted to the next frequency, and the laser starts to scan the next row; the scanning of the whole image is completed by the combination of the MEMS movement and the frequency sweeping of the frequency sweeping laser.

[0012] The longitudinal scanning refers to that the scanning speed of the MEMS along the x direction is much less than the sweeping speed of the frequency sweeping laser, when the MEMS is stabilized at a certain position, the frequency sweeping laser is adjusted to the next frequency, and the output light beam is firstly scanned along the y direction under the action of the virtual image phase array, after scanning a column, the MEMS moves to the next position, and the output light beam is translated along the x direction to start to scan the next column; the scanning of the whole image is completed by the combination of the frequency sweeping of the frequency sweeping laser and the movement of the MEMS.

[0013] Preferably, a converging lens or a cylindrical mirror is arranged between the collimator and the virtual image phase array to focus the light beam, so that the light can smoothly enter the window of the virtual image phase array.

[0014] Preferably, the mirror is omitted, and the collimator is directly driven by the motor to perform the X direction scanning; or the mirror and the collimator are omitted, and the optical fiber is directly driven by the motor to perform the X direction scanning.

[0015] The application provides a method for expanding and displaying of the virtual image phase array based augmented reality display, comprising the following steps:

[0016] Firstly, the wavelength of the light output by the frequency sweeping laser is λ1, λ2, …, λ n λ n+1 λ n+2 … λ N However, the wavelength range covered by the frequency sweeping laser only accounts for a part of the whole bandwidth, and only a part of the natural light outside is blocked, and the light of other wavelengths can normally pass through the virtual image phase array, so that the person can see the light outside; and the wavelength of the laser output by the frequency sweeping laser is only within λ1, …, λ N , so that the virtual image phase array can be expanded, and then the light enters the human eye to generate the virtual image.

[0017] Second, the video signal modulates the laser through the modulator, loads the video signal to the laser, assuming that the frame rate of the video is f, and the pixel is M, then the modulation frequency of the modulator is Mf; in the horizontal scanning mode, the modulator and the MEMS maintain a synchronous relationship, and the scanning frequency of the MEMS is an integer multiple of the frequency of the frequency scanning laser; in the vertical scanning mode, the modulator and the frequency scanning laser maintain a synchronous relationship, and the frequency of the frequency scanning laser is an integer multiple of the frequency of the MEMS.

[0018] Third, through the optical path adjustment, the laser beam output by the collimator is reflected by the mirror and output to the window of the virtual image phase array; the pupil expansion in the X direction is realized by controlling the mirror to change the direction of the mirror along the X direction, so that the light beam scans along the X direction; the pupil expansion in the Y direction is realized by periodically changing the wavelength of the frequency scanning laser, and the angle of the output light beam of the virtual image phase array changes corresponding to the laser with different wavelengths.

[0019] Advantages

[0020] Compared with the prior art, the advantages of the present application are:

[0021] The virtual image phase array based augmented reality display pupil expansion and display device and method of the present application is a new type of pupil expansion and display method, and the image information is contained in an optical signal, so that the control signal can be transmitted to the system through one optical fiber, and the optical fiber has the advantages of low loss, light weight and anti-interference, so that the control signal and the lens can be completely separated, which greatly facilitates the design and manufacture of the AR glasses, and plays an important role in reducing the volume, weight and power consumption of the glasses. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings are included to provide a further understanding of the technical solutions of the present application, and constitute a part of the specification, and are used together with the specific embodiments of the present application to explain the technical solutions of the present application, and do not constitute a limitation on the technical solutions of the present application.

[0023] Figure 1 is a structural schematic diagram of the virtual image phase array based augmented reality display pupil expansion and display device. DETAILED DESCRIPTION

[0024] The present application is described in more detail below to facilitate the understanding of the present application.

[0025] As Figure 1As shown, the virtual image phase array based augmented reality display pupil expansion and display device of the present application comprises a swept frequency laser 1, a modulator 2, an optical waveguide 4, a collimator 5, a mirror 6, a MEMS 7 and a virtual image phase array 8, the output end of the swept frequency laser 1 is connected with the input port of the modulator 2, the output port of the modulator 2 is connected with the collimator 5 through the optical waveguide 4, the video signal 3 modulates the laser through the modulator 2, and the modulation can also be in the form of internal modulation (i.e. the video signal 3 directly modulates the swept frequency laser 1 without using a special modulator 2). The modulated laser beam is output after being reflected by the mirror 6 and output to the virtual image phase array 8, the mirror 6 is controlled by the MEMS 7 and scans along the X direction, and the output spectrum of the swept frequency laser 1 changes in real time, so that the scanning along the Y direction is realized after the virtual image phase array 8.

[0026] The optical waveguide 4 can be an optical fiber, an integrated optical waveguide or a planar optical waveguide.

[0027] The virtual image phase array 8 (VIPA) is a double-sided coated glass flat plate, the incident surface of the glass flat plate is coated with a high reflectivity film with a reflectivity of more than 99% except for the anti-reflection film coated in the incident window area, and the general exit surface is coated with a high reflectivity film with a reflectivity of 95% to 98%. The swept frequency laser 1 comprises a red-green-blue or red-yellow-blue three-color laser, the laser output by the swept frequency laser 1 is combined after being modulated, and then enters the optical waveguide 4, the collimator 5, the mirror 6, the MEMS 7 and the virtual image phase array 8, the mirror is controlled by a MEMS or a PZT and scans along the x direction, each point stays for a period of time tx, during which the laser output by the swept frequency laser enters the VIPA after being modulated, and the frequency change can realize frequency sweeping, and according to different frequencies, the points are scanned along the y direction.

[0028] The virtual image phase array based augmented reality display pupil expansion and display device comprises two laser scanning modes of transverse scanning and longitudinal scanning.

[0029] The transverse scanning refers to that the scanning speed of the MEMS along the x direction is much greater than the frequency sweeping speed of the swept frequency laser, when the swept frequency laser is stabilized at a certain frequency, the output beam is scanned along the x direction under the action of the MEMS, after scanning a row, the swept frequency laser is adjusted to the next frequency, and the laser starts to scan the next row; the combination of the MEMS movement and the swept frequency laser frequency sweeping completes the scanning of the entire image.

[0030] The longitudinal scanning refers to that the scanning speed of the MEMS along the x direction is much smaller than the sweeping speed of the sweeping laser, when the MEMS is stable at a position, the sweeping laser is tuned to the next frequency and first scans along the y direction under the action of the virtual image phase array (VIPA), after scanning a column, the MEMS moves to the next position, the output light beam is translated in the x direction, and the next column is scanned; the sweeping of the sweeping laser and the movement of the MEMS combine to complete the scanning of the whole image.

[0031] Preferably, between the collimator 5 and the virtual image phase array 8, in addition to passing through the mirror 6, converging lenses or cylindrical mirrors can be added as needed to focus the light beam, so that the light smoothly enters the window of the virtual image phase array 8.

[0032] Preferably, the mirror 6 can be omitted, and the collimator 5 is directly driven by a motor to perform X direction scanning. The mirror 6 and the collimator 5 can also be omitted, and the optical fiber is directly driven by a motor to perform X direction scanning.

[0033] The device and method for expanding pupil display of the virtual image phase array-based augmented reality display according to the present application comprises the following steps:

[0034] First, the light wavelength output by the sweeping laser 1 is λ1λ2……λ n λ n+1 λ n+2 ……λ N However, the wavelength range covered by the sweeping laser 1 only accounts for a very small part of the total bandwidth, and only a small part of the external natural light is blocked, and the light of other wavelengths can normally pass through the virtual image phase array 8, so that the person can see the external light; and the light wavelength of the laser output by the sweeping laser 1 is only within λ1……λ N , so that the virtual image can be expanded and entered into the human eye through the virtual image phase array 8;

[0035] Second, the video signal 3 is modulated by the modulator 2 to load the video signal onto the laser, assuming that the frame rate of the video is f and the pixel is M, then the modulation frequency of the modulator 2 is Mf; in the horizontal scanning mode, the modulator 2 and the MEMS 7 maintain a synchronous relationship, and the scanning frequency of the MEMS is an integer multiple of the sweeping frequency of the sweeping laser 1; in the longitudinal scanning mode, the modulator 2 and the sweeping laser 1 maintain a synchronous relationship, and the frequency of the sweeping laser is an integer multiple of the sweeping frequency of the MEMS 7;

[0036] Third, through the light path adjustment, the light beam outputted by the collimator 5 is reflected by the mirror 6 and outputted to the window of the virtual image phase array 8; the X-direction pupil expansion is realized by changing the direction of the mirror along the X-direction through the MEMS control mirror, so as to realize the purpose of the light beam scanning along the X-direction; the Y-direction pupil expansion is realized by periodically changing the laser wavelength through the frequency-sweeping laser, and the angle of the output light beam of the virtual image phase array will change corresponding to the laser with different wavelengths, so as to realize the purpose of the light beam scanning along the Y-direction.

[0037] As a new type of pupil expansion and display mode, the image information is contained in a light signal, so that the image information can be transmitted to the system through one optical fiber, and the optical fiber has the advantages of low loss, light weight and anti-interference, so that the control signal and the lens can be completely separated, which greatly facilitates the design and manufacture of the AR glasses, and plays an important role in reducing the volume, weight and power consumption of the glasses.

[0038] The above describes the preferred embodiments of the present application, but is not intended to limit the present application. Those skilled in the art can make improvements and changes to the embodiments disclosed herein without departing from the scope and spirit of the present application.

Claims

1. A device for pupil expansion and display of augmented reality display based on virtual image phase array, characterized in that, The aforementioned device for pupil expansion and display of augmented reality display based on virtual image phase array includes a swept laser, a modulator, an optical waveguide, a collimator, a mirror, a MEMS, and a virtual image phase array. The output end of the swept laser is connected to the input port of the modulator, and the output port of the modulator is connected to the collimator through the optical waveguide. The video signal modulates the laser through the modulator. The modulated laser beam is output through the collimator and then reflected by the mirror before being output to the virtual image phase array. The mirror is controlled by the MEMS and scans along the X direction. The output spectrum of the swept laser changes in real time and scans along the Y direction after passing through the virtual image phase array. The virtual image phase array is a double-sided coated glass plate. Except for the incident window area, which is coated with an anti-reflection film, the incident surface of the glass plate is coated with a high-reflection film with a reflectivity of more than 99%, and the exit surface is coated with a high-reflection film with a reflectivity of 95% to 98%. The sweeping laser includes red-green-blue or red-yellow-blue tri-color lasers. The laser output from the sweeping laser is modulated and then combined before entering the optical waveguide.

2. The apparatus for pupil expansion and display of augmented reality display based on virtual image phase array according to claim 1, characterized in that, The reflector is controlled by a MEMS and scans along the x-direction, staying at each point for a period of time tx. During this period, the laser emitted by the frequency sweep laser is modulated and enters the virtual image phase array. Due to the frequency change, frequency sweeping can be performed, and the laser is scanned point by point along the y-direction according to different frequencies.

3. The apparatus for pupil expansion and display of augmented reality display based on virtual image phase array according to claim 1, characterized in that, The device for pupil expansion and display of the augmented reality display based on virtual image phase array includes two laser scanning methods: horizontal scanning and vertical scanning.

4. The apparatus for pupil expansion and display of augmented reality display based on virtual image phase array according to claim 3, characterized in that, The lateral scanning refers to the fact that the scanning speed of the MEMS along the x-direction is greater than the scanning speed of the frequency sweep laser. When the frequency sweep laser is stabilized at a certain frequency, the output beam is first scanned along the x-direction under the action of the MEMS. After scanning one line, the frequency sweep laser is adjusted to the next frequency, and the laser starts scanning the next line. The combination of MEMS motion and frequency sweep laser scanning completes the scanning of the entire image.

5. The apparatus for pupil expansion and display of augmented reality display based on virtual image phase array according to claim 3 or 4, characterized in that, The longitudinal scanning refers to the fact that the scanning speed of the MEMS along the x-direction is less than the scanning speed of the sweep laser. When the MEMS is stable at a certain position, the sweep laser is tuned to the next frequency and scans along the y-direction under the action of the virtual image phase array. After completing the scanning of one column, the MEMS moves to the next position, allowing the output beam to translate in the x-direction and start scanning the next column. The combination of the sweep laser's sweep and the MEMS's movement completes the scanning of the entire image.

6. The apparatus for pupil expansion and display of augmented reality display based on virtual image phase array according to claim 1, characterized in that, Between the collimator and the virtual image phase array, a converging lens or cylindrical mirror is also provided to focus the light beam, so that the light can smoothly enter the window of the virtual image phase array.

7. The apparatus for pupil expansion and display of augmented reality display based on virtual image phase array according to claim 1, characterized in that, The reflector can be omitted, and the collimator can be driven directly by a motor to perform scanning in the X direction; or the reflector and collimator can be omitted, and the optical fiber can be driven directly by a motor to perform scanning in the X direction.

8. The display method of the device for pupil expansion and display of augmented reality display based on virtual image phase array according to any one of claims 1 to 7, characterized in that, Includes the following steps: First, the wavelength of light output by the frequency-sweeping laser is ... However, the wavelength range covered by the swept-frequency laser only occupies a portion of the total bandwidth. Only a portion of the external natural light is blocked, while other wavelengths can pass normally through the virtual phase array, allowing people to see the external light. The wavelength of the laser output by the swept-frequency laser is only within... Within this, the pupil can be expanded through the virtual image phase array, thus allowing the image to enter the human eye and generate a virtual image. Second, the video signal is modulated onto the laser by a modulator, and the video signal is loaded onto the laser. Assuming the frame rate of the video is f and the number of pixels is M, the modulation frequency of the modulator is Mf. In the horizontal scanning mode, the modulator and the MEMS maintain a synchronous relationship, and the scanning frequency of the MEMS is an integer multiple of the scanning frequency of the sweep laser. In the vertical scanning mode, the modulator and the sweep laser maintain a synchronous relationship, and the frequency of the sweep laser is an integer multiple of the sweep frequency of the MEMS. Third, by adjusting the optical path, the laser beam output from the collimator is reflected by the mirror and output to the window of the virtual image phase array. The pupil expansion in the X direction is controlled by MEMS to continuously change the direction of the mirror along the X direction, so as to achieve the purpose of scanning the beam along the X direction. The pupil expansion in the Y direction is achieved by periodically changing the laser wavelength of the frequency sweep laser. Corresponding to different wavelengths of laser, the angle of the output beam of the virtual image phase array will change, so as to achieve the purpose of scanning the beam along the Y direction.

Images