Display module and handheld AR device

Abstract

The utility model discloses a display module and handheld AR equipment, and relates to the technical field of optical display.The display module comprises an optical engine, a space sensor, an optical waveguide arranged on the light emitting side of the optical engine and a controller connected with the space sensor and the optical engine, the optical engine emits image light beams, and the space sensor is arranged on the optical waveguide; the image light beam is emitted to form an image after passing through the optical waveguide, the spatial sensor detects the spatial position state of the optical waveguide, and the controller acquires the spatial position state and adjusts the emitting direction of the image light beam of the optical engine according to the spatial position state. According to the display module and the handheld AR equipment provided by the invention, the displacement of the display image when the handheld AR equipment shakes can be reduced, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of optical display technology, and in particular, to a display module and a handheld AR device. Background Art

[0002] Augmented reality (AR) technology is a technology that calculates the position and angle of the image emitted by the light engine system (also called a projector or light engine) in real time and adds the corresponding image. Augmented reality technology can realize the virtual world on the screen and interact with the real world. That is, physical information that is difficult to experience within the time and space of the real world (such as visual information, sound, or touch, etc.) can be simulated by computers and then superimposed, and virtual information can be applied to the real world. Because augmented reality technology makes it possible for the virtual world to interact with the real world, it has been widely used in AR devices. Common AR devices include AR glasses, car-mounted AR-HUD and handheld AR devices.

[0003] Among them, handheld AR devices are similar to mobile phones and tablets, but the difference is that conventional mobile phones and tablets display real images on the surface of the tablet screen, which are close to the human eye and are not sensitive to hand-held shaking, and shaking will not cause obvious viewing discomfort. The image displayed by handheld AR devices is a virtual image at a long distance (greater than 2m), and a slight angular shaking will be magnified by the long image distance, causing obvious displacement of the virtual image and reducing the user experience. Utility Model Content

[0004] The purpose of this application is to provide a display module and a handheld AR device, which can reduce the displacement of the displayed image when the handheld AR device is shaken and improve the user experience.

[0005] On the one hand, an embodiment of the present application provides a display module, including a light engine, a spatial sensor, an optical waveguide arranged on the light emitting side of the light engine, and a controller connected to the spatial sensor and the light engine respectively. The light engine emits an image light beam, and the image light beam is emitted after passing through the optical waveguide to form an image. The spatial sensor detects the spatial position state of the optical waveguide, and the controller obtains the spatial position state and adjusts the emission direction of the image light beam of the light engine according to the spatial position state.

[0006] As an implementable manner, the light engine includes an image source element, the image source element includes a display area for display, a partial area in the display area displays to form an image light beam, and a controller controls the position of the partial area in the display area to control the angle of the image light beam relative to the optical waveguide.

[0007] As an implementable manner, the display module further includes a driving device connected to the controller, and the driving device is connected to the light engine to change the emission direction of the image light beam of the light engine.

[0008] As an implementable manner, the spatial sensor includes a fisheye camera and an image processor connected to the fisheye camera, the image processor is connected to the controller, and the image processor calculates the moving direction and distance of the optical waveguide according to the image taken by the fisheye camera.

[0009] As an implementable manner, the spatial sensor is a gyroscope.

[0010] As an implementable manner, a coupling-in grating is provided in the coupling-in region of the optical waveguide, and a coupling-out grating is provided in the coupling-out region of the optical waveguide. The coupling-out grating includes one of a one-dimensional grating, a two-dimensional grating, or a double-sided one-dimensional grating.

[0011] As an implementable manner, the coupling-in grating and the coupling-out grating are both surface relief gratings and volume holographic gratings.

[0012] As an implementable manner, the image source element includes one of a silicon-based liquid crystal screen, a liquid crystal display screen, a digital micromirror, an organic electric laser display, and a micro light emitting diode display.

[0013] As an implementable manner, the light engine further includes a collimating element disposed on the light emitting side of the image source element.

[0014] Another aspect of an embodiment of the present application provides a handheld AR device, comprising the above-mentioned display module.

[0015] The beneficial effects of the embodiments of the present application include:

[0016] The display module provided by the present application includes a light engine, a spatial sensor, an optical waveguide arranged on the light-emitting side of the light engine, and a controller connected to the spatial sensor and the light engine respectively. The light engine emits an image beam, and the image beam is emitted after passing through the optical waveguide to form an image. The spatial sensor detects the spatial position state of the optical waveguide, and the controller obtains the spatial position state and calculates the offset information of the optical waveguide based on the spatial position state, and then calculates the compensation information based on the offset information, and then controls the emission direction of the image beam of the light engine based on the compensation information. During the propagation process in the optical waveguide, the angle of the image beam remains unchanged, so that the direction of the image beam emitted by the display module is compensated, so that the image beam emission position is the same, and the jitter of the virtual image formed by the image beam is reduced. Therefore, the display module of the embodiment of the present application can reduce the displacement of the displayed image when the handheld AR device is jittered, and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 One of the structural schematic diagrams of a display module provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of the structure of an optical waveguide provided in an embodiment of the present application;

[0020] Figure 3 The structure of a display module provided in the embodiment of the present application is the second intention;

[0021] Figure 4 A schematic diagram showing the direction of the image beam emission before and after the display module is shaken;

[0022] Figure 5 A schematic diagram of the structure of an image source element provided in an embodiment of the present application;

[0023] Figure 6 A schematic diagram of the position of an outgoing light beam of an image source element provided in an embodiment of the present application;

[0024] Figure 7 A schematic diagram of the structure of a light engine provided in an embodiment of the present application.

[0025] Icons: 100 - display module; 110 - light engine; 111 - image source element; 112 - collimation element; 113 - display area; 114 - partial display area; 120 - optical waveguide; 121 - coupling grating; 122 - coupling grating; 130 - spatial sensor; 140 - controller. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0027] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0029] In the description of this application, it should be noted that the terms "center", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the application is usually placed when in use. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific position, be constructed and operated in a specific position, and therefore cannot be understood as a limitation on this application. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0030] The present application embodiment provides a display module 100, such as Figure 1 , Figure 2 and Figure 3 As shown, it includes a light engine 110, an optical waveguide 120 arranged on the light emitting side of the light engine 110, a spatial sensor 130 arranged on the optical waveguide 120, and a controller 140 connected to the spatial sensor 130 and the light engine 110 respectively. The light engine 110 emits an image light beam, and the image light beam is emitted through the optical waveguide 120 to form an image. The spatial sensor 130 detects the spatial position state of the optical waveguide 120, and the controller 140 obtains the spatial position state and adjusts the emission direction of the image light beam of the light engine 110 according to the spatial position state.

[0031] The display module 100 provided in the embodiment of the present application is applied to a handheld AR device. The image formed by the handheld AR device in the prior art is a virtual image at a long distance. Since the imaging distance is long, when the handheld AR device has a small jitter offset angle, the offset of the virtual image formed by the AR device is large. The display module 100 of the embodiment of the present application includes a light engine 110 and an optical waveguide 120 arranged on the light output side of the light engine 110. The light engine 110 emits an image light beam. The image light beam is coupled into the optical waveguide 120 from the coupling-in region of the optical waveguide 120 and propagates to the coupling-out region of the optical waveguide 120 through total reflection in the optical waveguide 120, and is coupled out from the coupling-out region to form an output image. In order to prevent a large image offset when the handheld AR device is jittered, the embodiment of the present application also arranges a spatial sensor 130 and a controller 140 on the optical waveguide 120. The spatial sensor 130 and the controller 140 are arranged on the optical waveguide 120. The sensor 130 detects the spatial position state of the optical waveguide 120, and the controller 140 obtains the spatial position state, and calculates the offset information of the optical waveguide 120 according to the spatial position state, and then calculates the compensation information according to the offset information, and then controls the emission direction of the image beam of the light engine 110 according to the compensation information. During the propagation process in the optical waveguide 120, the angle of the image beam remains unchanged, so that the direction of the image beam emitted by the display module 100 is compensated, so that the image beam emission position is the same, and the jitter of the virtual image formed by the image beam is reduced. Therefore, the display module 100 of the embodiment of the present application can reduce the displacement of the displayed image when the handheld AR device is shaken, and improve the user experience.

[0032] Specifically, the offset information may include an offset direction, an offset angle, and an offset distance. For example, Figure 4 As shown, when the handheld AR device shakes, the optical waveguide 120 tilts inward, and when it shakes from the solid line position to the dotted line position, assuming that the tilt angle of the optical waveguide 120 detected by the spatial sensor 130 is α, the optical axis of the image light beam is also tilted by α. At this time, the optical axis of the image light beam needs to be tilted in the opposite direction by α. The tilting α in the opposite direction is used as compensation information to control the emission of the image light beam of the light engine 110, so that when the handheld AR device shakes, the emission position and emission angle of the image light beam remain unchanged, thereby avoiding image displacement and improving user experience.

[0033] The display module 100 provided in the present application includes a light engine 110, an optical waveguide 120 disposed on the light emitting side of the light engine 110, a spatial sensor 130 disposed on the optical waveguide 120, and a controller 140 connected to the spatial sensor 130 and the light engine 110 respectively. The light engine 110 emits an image beam, and the image beam passes through the optical waveguide 120 and then emits an image. The spatial sensor 130 detects the spatial position state of the optical waveguide 120, and the controller 140 obtains the spatial position state and calculates the offset information of the optical waveguide 120 according to the spatial position state, and then calculates the compensation information according to the offset information, and then controls the emission direction of the image beam of the light engine 110 according to the compensation information. During the propagation process in the optical waveguide 120, the angle of the image beam remains unchanged, so that the direction of the image beam emitted by the display module 100 is compensated, so that the image beam emission position is the same, and the virtual image jitter formed by the image beam is reduced. Therefore, the display module 100 of the embodiment of the present application can reduce the displacement of the displayed image when the handheld AR device is jittered, and improve the user experience.

[0034] Optional, such as Figure 5 and Figure 6 As shown, the light engine 110 includes an image source element 111, and the image source element 111 includes a display area 113 for display. A partial area of ​​the display area 113 (hereinafter referred to as the partial display area 114) displays to form an image light beam, and the controller 140 controls the position of the partial display area 114 in the display area 113 to control the angle of the image light beam relative to the optical waveguide 120.

[0035] The image source element 111 includes a relatively large display area 113, and the display area 113 includes a plurality of pixels arranged in a matrix, wherein a plurality of interconnected pixels form a partial display area 114. When the image source element 111 is working, the partial display area 114 displays, for example, Figure 5 Area 1 and Area 2 in the chart are displayed in different time periods, such as Figure 6 As shown, the angle between the optical axis of the image light beam emitted from area 1 and the optical waveguide 120 is different from the angle between the optical axis of the image light beam emitted from area 2 and the optical waveguide 120 , thereby achieving a change in the angle at which the image light beam enters the optical waveguide 120 .

[0036] The controller 140 of the embodiment of the present application controls the position of the partial display area 114 within the display area so as to realize the control of the angle of the image light beam relative to the optical waveguide 120. Since software control is adopted, the response speed is fast, and the jitter of the virtual image formed by the image light beam cannot be felt within the resolution range of the human eye, thereby improving the experience.

[0037] In one achievable manner of the embodiment of the present application, the display module 100 further includes a driving device connected to the controller 140 , and the driving device is connected to the light engine 110 to change the emission direction of the image light beam of the light engine 110 .

[0038] After the controller 140 obtains the compensation information through calculation based on the information of the spatial sensor 130 , it controls the driving device to drive the emission angle of the light engine 110 according to the compensation information, thereby changing the information of the image light beam relative to the light waveguide 120 .

[0039] When a driving device is used, since the driving device drives the light engine 110 as a whole, there is no need to display in a partial area of ​​the display area 113 of the image source element 111, that is, the image source element 111 only needs to display in a partial area, thereby reducing the area of ​​the image source element 111 and reducing costs.

[0040] Optionally, the spatial sensor 130 includes a fisheye camera and an image processor connected to the fisheye camera, the image processor is connected to the controller 140, and the image processor calculates the moving direction and distance of the optical waveguide 120 according to the image taken by the fisheye camera.

[0041] When the spatial sensor 130 includes a fisheye camera and an image processor, the fisheye camera captures images of the external environment in real time, the image processor acquires and processes the images, and transmits the processed image information to the controller 140, and the controller 140 acquires the offset information of the optical waveguide 120 according to the information of two adjacent images. In practical applications, the image processor and the controller 140 can be integrated into the same electronic module.

[0042] In addition, the number of fisheye cameras is not limited in the embodiment of the present application, and can be one fisheye camera or a combination of multiple fisheye cameras. When a single fisheye camera is working, it consumes less energy, generates less heat, and has lower requirements on the controller 140 and hardware; multiple fisheye cameras can capture a larger range of scenes. When the jitter range is large, that is, the offset is large, the offset of the optical waveguide 120 is calculated more accurately based on the images captured by multiple fisheye cameras. Those skilled in the art can set the number of fisheye cameras according to actual needs.

[0043] In one implementable manner of the embodiment of the present application, the spatial sensor 130 is a gyroscope.

[0044] The gyroscope is a mechanical sensor with high detection accuracy. Specifically, the structure of the gyroscope is not limited in the embodiment of the present application, as long as it can detect the offset information of the optical waveguide 120. For example, a six-axis gyroscope can be used. The six-axis gyroscope can detect the angular velocity and acceleration in three directions. The controller 140 calculates the offset information of the optical waveguide 120 according to the angular velocity and acceleration in the three directions.

[0045] Optional, such as Figure 2 As shown, a coupling-in region of the optical waveguide 120 is provided with a coupling-in grating 121, and a coupling-out region of the optical waveguide 120 is provided with a coupling-out grating 122. The coupling-out grating 122 includes one of a one-dimensional grating, a two-dimensional grating, or a double-sided one-dimensional grating.

[0046] The coupling-in region is provided with a coupling-in grating 121 for coupling-in the image light beam, and the coupling-out region is provided with a coupling-out grating 122 for coupling-out the light beam. When the image light beam does not need to be expanded, a one-dimensional grating is provided in the coupling-out region; when the image light beam needs to be expanded, a two-dimensional grating or a double-sided one-dimensional grating is provided. Specifically, the two-dimensional grating is located on the same side of the optical waveguide 120, and the processing steps are reduced. The double-sided one-dimensional grating is a one-dimensional grating provided on opposite sides of the optical waveguide 120, and the projections of the one-dimensional gratings on the surface of the optical waveguide 120 overlap. Compared with the two-dimensional grating, the one-dimensional grating has high manufacturing accuracy and a high yield rate.

[0047] In one achievable manner of the embodiment of the present application, the coupling-in grating 121 and the coupling-out grating 122 are both one of a surface relief grating and a volume holographic grating.

[0048] Among them, the holographic grating is the interference of two beams with specific wavefront shapes, forming interference fringes of alternating light and dark on the recording plane. The interference fringes are recorded with a holographic recording medium, and the holographic grating is obtained after processing. The volume holographic grating is formed after multiple exposures. In other words, the volume holographic grating can be understood as multiple holographic gratings integrated in the same holographic material, which has the advantage of a simple preparation method.

[0049] The surface relief grating is amplitude modulated and is formed by pressing a molded plate, including a vertical surface relief grating and an inclined surface relief grating. Those skilled in the art can select the type of grating according to actual conditions.

[0050] Optionally, the image source element 111 includes one of a silicon-based liquid crystal screen, a liquid crystal display screen, a digital micromirror, an organic electro-laser display, and a micro light-emitting diode display.

[0051] Among them, the silicon-based liquid crystal screen, liquid crystal display screen, and digital micromirror do not emit light by themselves, and during use, a backlight source needs to be provided on the back of the above-mentioned image source element 111. Organic electric laser display and micro-light emitting diode display emit light by themselves.

[0052] The image source element 111 may adopt various structural elements, and those skilled in the art may select them according to actual conditions to improve the adaptability of the display module 100 .

[0053] In one possible implementation of the embodiment of the present application, Figure 7 As shown, the light engine 110 further includes a collimating element 112 disposed on the light emitting side of the image source element 111 .

[0054] The collimating element 112 is used to collimate the light beam to shrink the light beam, thereby reducing the divergence of the light beam, thereby preventing the light beam from deviating from the light path due to excessive divergence and causing light loss. The specific structure of the collimating element 112 is not limited in the present embodiment, as long as it can collimate the light beam. For example, it can be as follows Figure 7 When the collimating convex lens is set, the distance between the collimating convex lens and the image source element 111 can be equal to the focal length of the collimating convex lens. For example, the focal length is as follows: Figure 6 The F in it can also be a Fresnel lens.

[0055] The present application also discloses a handheld AR device. The handheld AR device includes the same structure and beneficial effects as the display module 100 in the aforementioned embodiment. The structure and beneficial effects of the display module 100 have been described in detail in the aforementioned embodiment and will not be repeated here.

[0056] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A display module, characterized in that: The invention comprises a light engine, a spatial sensor, an optical waveguide arranged on the light emitting side of the light engine, and a controller respectively connected to the spatial sensor and the light engine, wherein the light engine emits an image light beam, and the image light beam is emitted after passing through the optical waveguide to form an image, the spatial sensor detects the spatial position state of the optical waveguide, and the controller obtains the spatial position state and adjusts the emission direction of the image light beam of the light engine according to the spatial position state.

2. The display module according to claim 1, characterized in that: The light engine comprises an image source element, the image source element comprises a display area for display, a partial area in the display area displays and forms the image light beam, and the controller controls the position of the partial area in the display area to control the angle of the image light beam relative to the optical waveguide.

3. The display module according to claim 1, characterized in that: It also includes a driving device connected to the controller, and the driving device is connected to the light engine to change the emission direction of the image light beam of the light engine.

4. The display module according to claim 1, characterized in that: The spatial sensor includes a fisheye camera and an image processor connected to the fisheye camera. The image processor is connected to a controller. The image processor calculates the moving direction and distance of the optical waveguide according to the image captured by the fisheye camera.

5. The display module according to claim 1, characterized in that: The spatial sensor is a gyroscope.

6. The display module according to claim 1, characterized in that: The coupling-in region of the optical waveguide is provided with a coupling-in grating, and the coupling-out region of the optical waveguide is provided with a coupling-out grating, and the coupling-out grating includes one of a one-dimensional grating, a two-dimensional grating, or a double-sided one-dimensional grating.

7. The display module according to claim 6, characterized in that: The coupling-in grating and the coupling-out grating are both one of a surface relief grating and a volume holographic grating.

8. The display module according to claim 2, characterized in that: The image source element includes one of a silicon-based liquid crystal screen, a liquid crystal display screen, a digital micromirror, an organic electric laser display, and a micro light emitting diode display.

9. The display module according to claim 8, characterized in that: The light engine further includes a collimating element disposed on the light emitting side of the image source element.

10. A handheld AR device, characterized in that: A display module comprising any one of claims 1 to 9.

Images