AR glasses based on integrated imaging three-dimensional display

Abstract

The invention provides AR glasses based on integrated imaging three-dimensional display. According to the AR glasses, the integrated imaging technology is adopted, a three-dimensional light field is reconstructed through the micro-lens array, and multi-view three-dimensional image display is achieved. The display module adopts a Micro-LED display screen and is combined with a micro-lens array to realize superposition of a virtual image and a real scene. The optical waveguide system uses a holographic waveguide or a free-form surface optical device to guide virtual image light to the visual field of a user, so that the equipment is light and the display definition is ensured. Meanwhile, the system is provided with a head tracking and parallax adjusting module, and an inertial measurement unit (IMU) and an external camera are used for tracking the head movement of a user in real time, dynamically adjusting three-dimensional image display and ensuring the consistency of a virtual image and a real scene. According to the design, through precise matching of the display screen and the micro lens array, high-quality reconstruction of a three-dimensional image is ensured, and the cruising ability of equipment is improved by optimizing power consumption management.

Description

Technical Field

[0001] The present invention belongs to the field of AR display technology, and in particular relates to a 3D display AR glasses based on body pixel modeling Background Art

[0002] Augmented Reality (AR) technology provides users with an enhanced visual experience by integrating virtual images and information with real scenes. In recent years, with the rapid development of display technology, sensor technology, and computing processing capabilities, AR glasses, as an important application form, have received extensive attention and research. However, existing AR glasses face many technical challenges, especially in displaying three-dimensional images and achieving seamless integration of virtual information and real scenes.

[0003] Most traditional AR glasses use perspective displays (such as optical waveguides, translucent lenses, etc.), projecting virtual images through flat displays and superimposing them in the user's field of view. However, this flat display method can only provide two-dimensional images or images with limited stereoscopic sense, and cannot truly and accurately restore the depth information and parallax changes in three-dimensional scenes. In addition, the existing technology is prone to image distortion when processing complex three-dimensional scenes, and the fusion effect of virtual images and real scenes is poor, which makes users prone to visual fatigue when using it for a long time.

[0004] In order to solve these problems, three-dimensional display technology based on integrated imaging has been proposed in recent years. Integrated imaging captures or reproduces light fields from different perspectives through a microlens array or a grating array, thereby realizing true three-dimensional image display. This technology can present multi-viewpoint images without relying on additional glasses or tracking systems, providing a more natural three-dimensional visual experience. In addition, integrated imaging also has the advantages of good parallax continuity and obvious stereoscopic effect, and is particularly suitable for three-dimensional information display in augmented reality.

[0005] Although integrated imaging technology has shown great potential in the field of 3D display, its application in AR glasses still faces many technical challenges. For example, how to combine integrated imaging modules with transparent display technology to ensure seamless integration of virtual 3D images and real scenes; how to achieve a stable visual experience through parallax adjustment and head tracking in dynamic head movement; and how to achieve lightweight and low-power design while ensuring high-quality 3D image display.

[0006] Therefore, the present invention aims to propose an AR glasses based on the principle of integrated imaging three-dimensional display, to solve the technical bottlenecks of existing AR glasses in three-dimensional image display and virtual-reality fusion, and to provide a better augmented reality visual experience. Summary of the invention

[0007] Purpose of the invention: The present invention aims to provide an augmented reality (AR) glasses based on the principle of integrated imaging three-dimensional display, so as to solve the technical problems of existing AR glasses in three-dimensional image display, virtual-real fusion and parallax adjustment.

[0008] The technical solution of the present invention is as follows:

[0009] A pair of AR glasses based on integrated imaging three-dimensional display, comprising a transparent display screen, a collimation system, an optical waveguide system, and a microlens array; the transparent display screen processes a real scene image into a sub-image array; the collimation system converts the sub-image array into a parallel light beam; the optical waveguide system guides the parallel sub-image array to the microlens array; the microlens array optically reconstructs the sub-image array to generate a multi-viewpoint three-dimensional image; the real scene image and the multi-viewpoint three-dimensional image are superimposed and enter the human eye.

[0010] Preferably, the transparent display screen is a Micro-LED display screen with a resolution of not less than 480p, so as to ensure that the number of viewpoints under a single microlens is sufficient, the details of the three-dimensional image are clearer, and the viewing angle is richer.

[0011] Preferably, the optical waveguide system adopts holographic waveguide material, which has high light transmittance and low dispersion characteristics.

[0012] Preferably, the focal length of the microlens array is 2 mm, the diameter is 0.5 mm, and the microlenses are arranged in a regular rectangular array and are 1 mm away from the transparent display screen.

[0013] Preferably, the microlens array is made of optical materials with high light transmittance (such as PMMA or optical glass) to ensure that light attenuation is minimized. In order to further improve the optical performance, an anti-reflection coating can be added to the lens surface to reduce light reflection loss and increase the brightness and clarity of the virtual image.

[0014] Preferably, the refractive index of the microlens material is consistent to reduce the dispersion of light of different wavelengths when passing through the lens, avoiding chromatic aberration and image edge blur. Using optical materials with uniform refractive index and low dispersion can improve the color accuracy and sharpness of the image.

[0015] Preferably, the optical axis of the microlens array is aligned with the sub-image array derived from the optical waveguide system, ensuring that the viewpoint information reconstructed by light passing through each microlens array corresponds one-to-one to the light field information of the transparent display screen.

[0016] Preferably, an inertial measurement unit and an external camera are also included to achieve real-time tracking of the user's head movement.

[0017] Preferably, the size of the inertial measurement unit is 10mm×10mm×5mm, the response time is less than 1ms to ensure real-time response, the angular velocity accuracy reaches 0.01° / s, and the acceleration accuracy reaches 0.1m / s 2 , to ensure that the user's subtle head movements are accurately captured.

[0018] Preferably, the resolution of the external camera should be no less than 720p, and 1080p is recommended to ensure that sufficiently clear environmental information is captured. The frame rate should be no less than 60 frames per second to ensure that dynamic environmental changes can be captured in real time. The field of view angle should be between 90° and 120° to ensure that the camera can cover the user's main field of view. In order not to affect the portability of the glasses, the camera needs to be a miniature design with a size of approximately 5mm×5mm.

[0019] Preferably, the light engine uses a high-brightness LED as a light source, and guides the light to the display through a reflector and a waveguide.

[0020] As attached Figure 2 As shown, the microlens array is the key to realizing the three-dimensional light field, and the sub-image array generates a multi-view three-dimensional image through optical reconstruction of the microlens array.

[0021] The three-dimensional display process of integrated imaging can be described by the light field, which represents the direction and intensity of all light in the scene. The four-dimensional function L(u,v,x,y) of the light field describes the direction (x,y) of the light on the sub-image plane under the microlens array at each position (u,v):

[0022] L(u,v,x,y)=I(x′,y′)·δ(x′-u,y′-v)

[0023] Where I(x',y') is the intensity distribution of the sub-image array, and δ(x'-u,y'-v) is the Dirac function, which represents the reconstruction of the light direction. When reproducing the light field through the microlens array, the system reproduces the three-dimensional information by redirecting the light.

[0024] During the recording phase of the integrated imaging system, each microlens captures light from different angles in the scene, and the direction of these rays is related to the depth and position of the object. o ,y o ,z o ) is projected onto the microlens array plane (u, v) and the sensor plane (x′, y′):

[0025]

[0026] Where f is the focal length of the microlens, z ois the depth of the object (i.e., the distance from the object to the microlens array), x o and o It is the horizontal coordinate of the object in the scene. The sub-image array information on the transparent display module can be obtained from it.

[0027] The microlens array projects the light from the sub-image into the observer's eyes, reconstructing the light field in the original scene. This can be described by a similar geometric relationship to the re-projection of light from the sub-image plane into the observer's eyes:

[0028]

[0029] Among them, x r and r is the actual position of the object in the observed 3D scene.

[0030] The relationship between the depth of 3D display and the parallax angle θ is expressed by the geometric optics formula:

[0031]

[0032] Where Z is the depth of the object in the scene, d is the diameter of the microlens, and f is the focal length of the microlens.

[0033] In order to project a three-dimensional object to the near-eye end for display, the size of the displayed image needs to be appropriately reduced. Assuming that the reproduced three-dimensional image is reduced by m times, the size of the pixels in the sub-image array in the system and the diameter and focal length of the reproduced microlens array need to be adjusted to be reduced by m times compared to the acquisition microlens array and the acquisition pixel size to ensure that the reduced image maintains its three-dimensional stereoscopic effect.

[0034] Beneficial effects: The present invention sets up an integrated imaging display module, adopts a microlens array as the core component of the integrated imaging system, is used to reproduce a three-dimensional image, reconstructs a light field through the lens array, and generates a multi-viewpoint three-dimensional image;

[0035] A transparent display module is provided, and a transparent display screen is combined with an integrated imaging module to realize the superposition of a three-dimensional virtual image and a real scene. The transparent display screen has a high light transmittance, so that the user can clearly see the virtual image while watching the real scene;

[0036] Design optical waveguide systems to guide virtual image light into the user's field of view through holographic waveguides or free-form optical devices, while reducing the size and improving the portability of the device;

[0037] A head tracking and parallax adjustment module is designed, which uses an inertial measurement unit and an external camera to track the user's head movement in real time, and dynamically adjusts the display of three-dimensional images through a parallax adjustment algorithm to ensure that the virtual image is consistent with the real environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The present invention is a flowchart of the method.

[0039] Figure 2 Schematic diagram of the integrated imaging three-dimensional display process.

[0040] Figure 3 Design diagram of the microlens array in the integrated imaging system.

[0041] Figure 4 Schematic diagram of the optical path of the display screen and the optical waveguide module.

[0042] Figure 5 Schematic diagram of the overall structure of AR glasses. DETAILED DESCRIPTION

[0043] The present invention will be further explained below in conjunction with the accompanying drawings and a specific embodiment. Figure 1 As shown, the AR glasses based on integrated imaging three-dimensional display provided in this embodiment include an imaging display module, an optical waveguide system and a head tracking and parallax adjustment module; the imaging display module includes a micro display and a micro lens array; the optical waveguide system includes an input coupler, a waveguide body and an output coupler; the head tracking and parallax adjustment module includes a head camera and an inertial measurement unit.

[0044] The design method of this embodiment mainly includes the following parts: 1. Micro display selection; 2. Waveguide body design; 3. Input and output coupler design; 4. Micro lens array design; 5. Inertial measurement unit, external camera selection; 6. Power module selection; 7. Overall assembly; 8. Alignment and calibration of display screen and micro lens array.

[0045] Required components: Micro-LED display, microlens array, holographic waveguide, head-mounted frame, inertial measurement unit, external camera, power module.

[0046] 1. Microdisplay selection

[0047] The present invention uses a Micro-LED display with a resolution of not less than 480p as the image source of the AR glasses, and the image source emits a collimated light beam. In this embodiment, the light source size is 0.4cc and the luminous flux is 6lm.

[0048] 2. Waveguide body design

[0049] Select glass with a refractive index of 1.53. Use a multi-layer waveguide structure, each layer has a different refractive index, to reduce dispersion and improve image quality. The total thickness of the waveguide is about 1.0mm to ensure sufficient strength and light transmittance.

[0050] 3. Input and output coupler design

[0051] A diffraction grating is placed in front of the transparent Micro-LED display. The grating is designed to direct the light from the screen vertically into the in-coupling region. A diffraction grating is placed at the end of the waveguide. The grating is designed to direct the light vertically out of the waveguide to the out-coupling grating and diffuse it into the user's field of view. After the collimated light beam of the display module passes through the in-coupling grating, it is transmitted by total reflection in the waveguide. The in-coupling grating structure maintains symmetry to ensure that the incident light is incident on the human eye.

[0052] 4. Microlens array design

[0053] Design a microlens array with a focal length of 2mm and a diameter of 1mm, so that the microlenses are arranged in a regular rectangular array and aligned along the horizontal and vertical axes. Fix the array in front of the display, 1mm away from the display, to ensure that each microlens corresponds to a pixel area. To achieve the superposition of 3D virtual images and real scenes, the light transmittance of the transparent display module is about 70%, allowing users to clearly see the 3D virtual image while watching the real scene.

[0054] 5. Inertial measurement unit and external camera selection

[0055] The size of the inertial measurement unit is 10mm×10mm×5mm, and its response time is less than 1ms to ensure real-time response. The angular velocity accuracy reaches 0.01° / s and the acceleration accuracy reaches 0.1m / s. 2 To ensure accurate capture of the user's subtle head movements. The resolution of the external camera should be no less than 720p, the frame rate should be no less than 60 frames per second, the field of view should be between 90° and 120°, and the size should be 5mm×5mm.

[0056] 6. Power module selection

[0057] The power module is a rechargeable battery, and an intelligent power management mechanism is introduced to optimize power consumption and extend battery life.

[0058] 7. Overall assembly

[0059] Assemble all components into the head-mounted frame according to the design drawings and perform preliminary debugging.

[0060] 8. Alignment and calibration of display screen and microlens array

[0061] In order to ensure the transmission efficiency of light and the brightness and clarity of the image, laser or high-precision mechanical device calibration is used to adjust the up, down, left, and right positions of the microlens array so that the optical axis of the microlens is strictly aligned with the pixel array, ensuring that the viewpoint information reconstructed by the light passing through each microlens array must correspond one-to-one with the light field information of the display screen to avoid optical deviation and aberration.

[0062] Figure 3 The design of a microlens array in an integrated imaging system is shown for 3D image reconstruction in augmented reality (AR) glasses:

[0063] Microlens array: An array consisting of multiple microlenses, each of which is responsible for capturing or reconstructing light from a different angle to produce a three-dimensional image. These lenses are arranged in a regular grid to ensure coverage of the user's field of view.

[0064] The transmission path of light:

[0065] Incident light: The light emitted from the display screen passes through the lens array and enters each lens unit, forming a light field reconstruction process.

[0066] Reconstructed image: After passing through the microlens array, the light is reconstructed into a virtual 3D image that the user can see.

[0067] Figure 4 On a three-dimensional level, it shows the process of light starting from the MicroLED screen, passing through the input-output coupling grating and the optical waveguide system to reach the microlens array, and then projecting the image of the three-dimensional object into the human eye through the action of the microlens array.

[0068] Figure 5 The overall structure diagram of augmented reality (AR) glasses based on transparent MicroLED screen and holographic waveguide technology is shown:

[0069] Transparent OLED / MicroLED Screen: Used to display virtual images and maintain high transparency to ensure that users can see the real world.

[0070] Holographic waveguide system: guides the image displayed on the screen to the user's eyes, and realizes the transmission of virtual images through internal reflection and light guiding structure.

[0071] Micro-lens array: cooperates with the display screen to reconstruct virtual three-dimensional images and achieve stereoscopic effects through parallax and angle adjustment.

[0072] Sensors: used for head tracking and environmental perception to ensure that the virtual image dynamically adjusts the viewing angle.

[0073] Camera / Depth Sensor: Used to capture the user's field of view and surrounding environment to enhance the interaction between virtual objects and the real environment.

[0074] Light Path: Display light is transmitted from the screen through the waveguide system to the microlens array and finally reaches the user's eyes.

[0075] User's eye: The final recipient of the image, who sees the effect of superimposing the virtual image and the real scene.

[0076] Reality View: A view of the real world seen through a transparent display.

[0077] This design ensures that users can clearly see the superimposed virtual three-dimensional images when observing the real scene.

[0078] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An AR glasses based on integrated imaging three-dimensional display, characterized in that: It includes a transparent display screen, a collimation system, an optical waveguide system, and a microlens array; the transparent display screen processes a real scene image into a sub-image array; the collimation system converts the sub-image array into parallel light beams; and the optical waveguide system guides the parallel sub-image array to the microlens array; The microlens array optically reconstructs the sub-image array to generate a multi-viewpoint three-dimensional image; The real scene image and multi-viewpoint three-dimensional image are superimposed and enter the human eye.

2. The AR glasses based on integrated imaging three-dimensional display according to claim 1, characterized in that: The transparent display screen is a Micro-LED display screen with a resolution of no less than 480p.

3. The AR glasses based on integrated imaging three-dimensional display according to claim 1, characterized in that: The optical waveguide system adopts holographic waveguide material.

4. The AR glasses based on integrated imaging three-dimensional display according to claim 1, characterized in that: The focal length of the microlens array is 2 mm, the diameter is 0.5 mm, and the microlenses are arranged in a regular rectangular array, 1 mm away from the transparent display screen.

5. The AR glasses based on integrated imaging three-dimensional display according to claim 1, characterized in that: The optical material used in the microlens array is PMMA or optical glass, with a light transmittance of 70%, and an anti-reflection coating is arranged on the surface of the microlens.

6. The AR glasses based on integrated imaging three-dimensional display according to claim 1, characterized in that: The refractive index of the microlens material is consistent to reduce the dispersion of light of different wavelengths when passing through the microlens.

7. The AR glasses based on integrated imaging three-dimensional display according to claim 1, characterized in that: The optical axis of the microlens array is aligned with the sub-image array derived from the optical waveguide system, ensuring that the viewpoint information reconstructed by light passing through each microlens array corresponds one-to-one to the light field information of the transparent display screen.

8. The AR glasses based on integrated imaging three-dimensional display according to claim 1, characterized in that: It also includes an inertial measurement unit and an external camera to enable real-time tracking of the user's head movements.

9. The AR glasses based on integrated imaging three-dimensional display according to claim 8, characterized in that: The size of the inertial measurement unit is 10mm×10mm×5mm, and its response time is less than 1ms to ensure real-time response. The angular velocity accuracy reaches 0.01° / s and the acceleration accuracy reaches 0.1m / s. 2 , to ensure that the user's subtle head movements are accurately captured.

10. The AR glasses based on integrated imaging three-dimensional display according to claim 8, characterized in that: The resolution of the external camera is not less than 720p, the frame rate is not less than 60 frames per second, the field of view angle is between 90° and 120°, and the size is 5mm×5mm.

Images