Near-to-eye display method and near-to-eye display equipment

Abstract

The invention relates to the technical field of near-to-eye display, and provides a near-to-eye display method and near-to-eye display device.According to the method, a high-resolution original electronic image is generated through a micro display screen, accurate information content is provided for a user, and the quality and definition of an information source are ensured. The collimating projection lens converts divergent light of the micro display screen into parallel light beams, disordered scattering of light is reduced, and images of the micro display screen are imaged at an infinite distance after passing through the collimating lens. Parallel light beams are directionally regulated and controlled through micro-structure devices such as a micro-cube-corner prism array or an orthogonal reflector array, the parallel light beams are converged to a small-size exit pupil area, on-demand display of image information is achieved, an image can be seen only when the sight line of a user is aligned, and therefore continuous interference of the information on the normal view field of the user is avoided, and the user experience is improved. The comfort and concentration degree of visual experience are improved, the information transmission effect is optimized, and more natural, comfortable and efficient man-machine interaction experience is provided for near-eye display equipment.

Description

Technical Field

[0001] This invention relates to the field of near-eye display technology, and more particularly to a near-eye display method and near-eye display device. Background Technology

[0002] Near-eye display devices, such as augmented reality (AR) glasses and virtual reality (VR) glasses, integrate virtual information with the real world by overlaying electronic images onto the user's field of vision.

[0003] Traditional near-eye display solutions typically employ optical transmission-based overlay technology, utilizing pupil dilation to accommodate users with varying interpupillary distances and ensure the human eye receives the electronic image during eye movement. However, when electronic images are continuously superimposed on the real field of vision, excessive, irrelevant, or invalid information can become visual noise, severely interfering with the user's observation of the real world. Furthermore, the human eye must constantly switch focus between the electronic image and the real environment; this continuous adjustment leads to visual fatigue, reducing visual comfort and concentration, resulting in low content display efficiency and poor information delivery in near-eye display devices.

[0004] Therefore, improving the display efficiency of near-eye display devices has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a near-eye display method and a near-eye display device, aiming to solve the technical problem that visual interference exists in the display mode of near-eye display devices in the related art, resulting in low content display efficiency and poor information transmission effect.

[0006] In a first aspect, this application provides a near-eye display method, comprising:

[0007] A raw electronic image is generated using a microdisplay, the raw electronic image containing text, image, or video information elements;

[0008] The divergent light emitted by each pixel in the microdisplay is collimated by a collimating projection lens to obtain a parallel beam of light;

[0009] By using microstructure devices, the orientation angle of the parallel beam is adjusted so that the parallel beams from different directions converge within a preset exit pupil region to form a small exit pupil region, in order to display the original electronic image in the small exit pupil region. The microstructure devices include a micro-corner pyramid prism array or an orthogonal mirror array.

[0010] Secondly, this application also provides a near-eye display device, comprising:

[0011] Microdisplays are used to generate raw electronic images;

[0012] A collimating lens is used to convert the divergent light emitted by each pixel on the microdisplay into a parallel beam;

[0013] Microstructured devices are used to modulate a parallel beam of light into diffracted light emitted at a specific angle to form a virtual image in the user's field of vision.

[0014] The exit pupil region is used to accommodate the user's pupil, allowing the diffracted light modulated by the microstructure device to accurately enter the human eye.

[0015] This application provides a near-eye display method and device. The method generates a high-resolution original electronic image containing text, images, or video through a microdisplay, providing users with accurate information content, ensuring the diversity and richness of information, and guaranteeing the quality and clarity of the information source. A collimating projection lens converts the diverging light from the microdisplay into a parallel beam, and the microdisplay image is projected at infinity after passing through the collimating lens. The parallel beam is directionally controlled by microstructure devices such as micro-corner pyramid prism arrays or orthogonal mirror arrays, converging the beam into a small exit pupil area. This enables on-demand display of image information; the image is only visible when the user's gaze is aligned, thus avoiding continuous interference with the user's normal field of vision, improving visual comfort and focus, optimizing information transmission, and providing a more natural, comfortable, and efficient human-computer interaction experience for near-eye display devices. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a near-eye display device according to an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of the structure of the micro-corner pyramid prism array and the corner pyramid prism unit provided in the embodiments of this application;

[0019] Figure 3 This is a schematic diagram illustrating the structural effect of the microstructure device composed of a micro-conical prism array and a beam splitter provided in the embodiments of this application.

[0020] Figure 4 This is a schematic diagram of the structure of an orthogonal mirror array provided in an embodiment of this application;

[0021] Figure 5This is a schematic diagram of the structure of the orthogonal reflecting surface in the orthogonal reflecting mirror array provided in the embodiments of this application;

[0022] Figure 6 This is a schematic diagram illustrating the structural effect of the orthogonal mirror array provided in the embodiments of this application;

[0023] Figure 7 This is a schematic diagram illustrating the structural principle of an orthogonal mirror array provided in an embodiment of this application.

[0024] Figure 8 This is a flowchart illustrating a first embodiment of a near-eye display method provided in this application.

[0025] Figure 9 This is a flowchart illustrating a second embodiment of a near-eye display method provided in this application.

[0026] Figure 10 This is a schematic diagram of the structure of a first embodiment of a near-eye display device provided in this application;

[0027] Figure 11 This is a schematic block diagram of the communication structure of a near-eye display device provided in an embodiment of this application.

[0028] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0031] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0032] The near-eye display method provided in this application is mainly applied to near-eye display devices, such as augmented reality (AR) glasses, virtual reality (VR) glasses, and smart head-mounted helmets.

[0033] like Figure 1As shown, the near-eye display device provided in this application embodiment includes a microdisplay, a collimating lens, microstructure devices, and an exit pupil region.

[0034] This application employs a three-tiered optical architecture: a microdisplay, a collimating projection lens, and microstructure devices. The microdisplay generates the original electronic image, which is then converted into parallel light by the collimating projection lens. In traditional collimation systems, the parallel light emitted is mutually divergent, requiring the human eye to be precisely aligned to receive the complete image, and the viewing range is extremely limited.

[0035] like Figure 1 As shown, this application introduces a microstructure device to directionally control aligned parallel light. This microstructure device allows parallel light from various angles to reconverge while maintaining parallelism, forming a small exit pupil region. The electronic image can only be viewed when the human eye rotates to a specific angle, achieving on-demand display of information.

[0036] Microdisplays are used to generate raw electronic images. As an image source, a microdisplay generates raw electronic images containing text, images, or video information. Depending on the application scenario and cost requirements, technologies such as liquid crystal displays (LCDs), organic light-emitting diode displays (OLEDs), or micro-electro-mechanical systems (MEMS) displays can be selected.

[0037] The collimating lens is positioned on the light-emitting side of the microdisplay to convert the divergent light emitted by each pixel on the microdisplay into a parallel beam.

[0038] The focal length, aperture, and other parameters of the collimating lens can be optimized based on the size and display distance of the microdisplay to ensure proper light collimation. Specifically, a shorter focal length reduces the size of the optical system but results in a larger light divergence angle, affecting display quality; a longer focal length improves the parallelism of light rays but increases the system's size and weight. Therefore, a trade-off must be struck between size and display quality. Aperture size also affects collimation. A larger aperture increases light throughput and brightness but may increase the light divergence angle; a smaller aperture improves parallelism but reduces brightness. Aperture size needs to be optimized according to the specific requirements of the application scenario.

[0039] Microstructured devices are used to control the orientation angle of aligned parallel beams, causing parallel beams from different directions to converge within a preset exit pupil region, forming a small exit pupil area. Microstructured devices can be implemented using either micro-corner pyramid prism arrays or orthogonal mirror arrays.

[0040] In one embodiment, the microstructure device may employ a micro-pyramidal prism array, which consists of at least one periodically arranged pyramidal prism unit.

[0041] like Figure 2 As shown, the micro-cornerstone prism array consists of multiple tiny cornerstone prism units arranged periodically, each with three mutually perpendicular reflecting surfaces. When a parallel beam of light is incident on the micro-cornerstone prism array, the light undergoes three reflections within each cornerstone prism unit, causing the direction of the outgoing beam to be opposite to the incident direction. In this way, the originally divergent parallel beams are redirected and converged into a specific spatial region by the beam splitter, forming a small exit pupil region.

[0042] It is important to note that a micro-pyramidal prism array alone cannot effectively converge parallel light rays after their direction has been reversed to a small exit pupil region. The micro-pyramidal prism array needs to be used in conjunction with a beam splitter to redirect and converge divergent parallel beams to a specific spatial area. The beam splitter is a key auxiliary component for achieving the small exit pupil function of the micro-pyramidal prism array. It effectively guides and focuses the light rays processed by the prism array to a predetermined small exit pupil position, ensuring that the human eye can only receive an image when aligned at a specific angle. The introduction of the beam splitter not only improves the efficiency of the optical system but also avoids disordered scattering of light in space. The micro-pyramidal prism array and the beam splitter can be two independent combined structures, or they can be bonded together into a single structure using prisms.

[0043] Specifically, the main function of a micro-cornerstone prism array is to reverse the propagation direction of the outgoing beam relative to the incident angle after three reflections of the incident beam. However, relying solely on the cornerstone prism array itself is insufficient to effectively "converge" these originally divergent parallel light rays from different perspectives into a narrow "small exit pupil area." A beam splitter plays a crucial role in this process, effectively guiding and focusing the light processed by the cornerstone prism array to a predetermined small exit pupil position. This ensures that the human eye can only receive the image when aligned at a specific angle, achieving on-demand display. Therefore, the beam splitter is a key component ensuring the successful formation of a small exit pupil and on-demand information display by the micro-cornerstone prism array. It enables precise spatial focusing of light, avoiding disordered scattering of light in space.

[0044] like Figure 3As shown, in an optical system consisting of a microdisplay, a collimating lens, and a cornerstone prism array, a beam splitter is placed in the optical path to split and converge the light rays processed by the cornerstone prism array, ultimately forming a compact small exit pupil region for human observation.

[0045] This embodiment achieves efficient directional control and precise spatial focusing of the light beam in a near-eye display system through the innovative collaborative design of a micro-corner pyramid prism array and a beam splitter. The micro-corner pyramid prism array utilizes the triple reflection characteristics of three mutually perpendicular reflective surfaces to completely reverse the propagation direction of the incident light beam, laying the foundation for subsequent light redirection. The beam splitter, as a key auxiliary component, effectively converges the parallel light rays, which were originally diverging after the direction reversal, into a narrow exit pupil area through precise beam guidance and spatial focusing, forming a compact display area visible only at a specific eye angle. This dual mechanism of "direction reversal + precise focusing" improves the efficiency of the optical system, avoids disordered light scattering, and ensures that the user can only receive the image at a specific angle, thereby reducing visual interference and enhancing the comfort and focus of the visual experience.

[0046] Furthermore, the micro-conical prism array and beam splitter can be flexibly combined as independent modules or integrated into a single structure through bonding processes, further optimizing the structure of the optical module and making it more compact and efficient. This design flexibility further optimizes the system integration, optical efficiency, and structural application adaptability.

[0047] In another embodiment, the microstructure device can also employ an orthogonal mirror array, which consists of two sets of mutually perpendicular micromirrors. Each set of mirrors modulates the angle of the incident parallel light in an orthogonal direction. This structure achieves the reconvergence of parallel beams by precisely controlling the propagation components of light in the X and Y directions. Specifically, when a parallel beam is incident on the orthogonal mirror array, the first set of micromirrors modulates the angle of light in the X-axis direction, and the second set of micromirrors modulates the angle of light in the Y-axis direction. The modulated outgoing beam direction vector remains unchanged in the Z-axis component, while the X-axis and Y-axis components have opposite signs to the incident beam direction vector. This specific light modulation method can converge parallel light from different viewing angles into a small exit pupil area, achieving a display result with a small exit pupil area.

[0048] Orthogonal mirror arrays offer a different optical principle from micro-pyramidal prism arrays for achieving directional control and refocusing of light. For example... Figure 4 and Figure 5As shown, an orthogonal mirror array can be implemented using a micro-mirror grid, with each grid cell containing two mutually perpendicular reflecting surfaces. Through its unique structure, the orthogonal mirror array can precisely reverse the propagation components of the incident beam in the X and Y directions while keeping the Z-direction component unchanged.

[0049] The light modulation method using an orthogonal reflector array can also converge parallel light rays from different viewing angles into a narrow exit pupil area, thereby achieving a small exit pupil display. For example, an array composed of micro-mirror grids can be designed, where each grid unit contains two mutually perpendicular reflective surfaces. When light is incident, after two reflections, its propagation direction in the X and Y directions will be reversed, thus achieving the re-convergence of light rays.

[0050] like Figure 6 and Figure 7 As shown, the orthogonal mirror array consists of two sets of mutually perpendicular micromirrors. Each set of mirrors modulates the angle of the incident parallel light in an orthogonal direction. Taking the direction perpendicular to the mirror array as the Z-direction, the Z-axis component of the modulated outgoing beam direction vector remains unchanged, while the X-axis and Y-axis components have opposite signs. Through precise angle control, the orthogonal mirror array achieves the reconvergence of parallel light.

[0051] This embodiment achieves efficient directional control and refocusing of parallel light beams through an orthogonal mirror array, effectively solving the problem of inaccurate light focusing in near-eye display devices. The orthogonal mirror array uses two sets of mutually perpendicular micromirrors to modulate the angle of the incident parallel light in orthogonal directions, precisely reversing the propagation components of the light in the X and Y directions while keeping the Z-direction component unchanged. This concentrates parallel light rays from different viewing angles into a narrow exit pupil area, achieving a small exit pupil display. This design improves the efficiency of the optical system, avoids disordered light scattering, reduces interference with the user's normal field of vision, and enhances visual comfort and focus. Furthermore, the orthogonal mirror array has a compact structure and is easy to integrate, making it suitable for near-eye display devices with strict requirements on size and weight, such as smart glasses, providing a more flexible optical solution for near-eye display technology.

[0052] In one embodiment, the microstructure device is bonded to other optical elements in the near-eye display device to achieve a compact optical structure, wherein the other optical elements include prisms or lenses.

[0053] Both micro-conical prism arrays and orthogonal mirror arrays can be cemented with other prisms or lenses to achieve compact optical structures and higher space utilization. Specifically, cementing microstructure devices with other prisms or lenses can significantly optimize the optical module design of smart glasses.

[0054] Gluing eliminates air gaps between optical components, thereby reducing light energy loss due to air-glass interface reflections and improving the overall efficiency of the optical system. Gluing allows multiple optical components to form a compact, robust whole, effectively reducing the size and weight of the optical module, which is crucial for wearable devices such as smart glasses with strict size and weight requirements. Smaller size and higher space utilization contribute to improved wearing comfort and aesthetics, while also enhancing the system's mechanical stability and reducing the risk of optical performance degradation due to vibration or impact.

[0055] For example, in actual product design, a micro-conical prism array, which is a microstructure device, can be directly glued to the exit surface of a collimating projection lens, or glued to a reflective prism used for optical path folding, thereby forming a seamless optical component, reducing the length of the entire optical path and improving the transmission efficiency of light.

[0056] This embodiment achieves a compact optical structure and higher space utilization by bonding microstructured devices (such as micro-pyramidal prism arrays or orthogonal mirror arrays) with other optical elements (such as prisms or lenses), significantly optimizing the optical module design of near-eye display devices. This bonding method not only eliminates air gaps between optical elements, reducing light energy loss and improving the overall efficiency of the optical system, but also allows multiple optical elements to form a compact and robust whole, effectively reducing the size and weight of the optical module. This is crucial for wearable devices such as smart glasses, which have strict requirements for size and weight, helping to improve the product's wearing comfort and aesthetics, while also enhancing the system's mechanical stability and reducing the risk of optical performance degradation due to vibration or impact.

[0057] The exit pupil region accommodates the user's pupil, allowing diffracted light modulated by the microstructured devices to accurately enter the eye. As the optical interface between the user's pupil and the displayed image, its size and position directly affect the user's visual experience. The design of the exit pupil region must ensure that the user can clearly see the displayed image at specific angles, while avoiding continuous interference with the real field of vision. The size and position of the exit pupil region should be optimized based on the design of the microstructured devices. For example, the position and size of the exit pupil region can be precisely controlled by adjusting the parameters of a micro-corner pyramid prism array or an orthogonal mirror array.

[0058] This embodiment optimizes the design of the exit pupil area to ensure that users can clearly see the displayed image at specific angles, while avoiding continuous interference with the real field of vision, thus further enhancing the user's visual experience.

[0059] To further improve the adaptability of the exit pupil area and the user experience, this application also introduces an eye-tracking module for real-time detection of the user's pupil coordinates and feeding this information back to the driving unit of the microstructure device. The eye-tracking module employs eye-tracking technology based on the pupil-corneal reflection (PCCR) principle. It illuminates the eye with an infrared light source, forming a reflective point (Pulchin spot) on the corneal surface. Simultaneously, a camera captures the relative positional changes between the pupil center and the reflective point, thereby calculating the direction of gaze.

[0060] Specifically, the system acquires real-time images of the user's eyes through an eye-tracking module and uses an infrared light source to enhance pupil contrast. When the infrared light source is coaxial with the optical axis, the pupil appears as a bright spot (bright pupil effect); when the light source deviates from the optical axis, the pupil appears as a dark area (dark pupil effect). Through a bright pupil-dark pupil difference algorithm, the relative position of the pupil center and the corneal reflection point can be accurately located.

[0061] Image processing algorithms are used to identify the center position of the user's pupil and determine the pupil coordinates. Specifically, edge detection and threshold segmentation algorithms are used to locate the eye contour, and ellipse fitting, grayscale analysis, or machine learning models (such as convolutional neural networks) are used to accurately identify the pupil center coordinates. The reflection points of infrared light sources are combined to eliminate ambient light interference and enhance the stability of feature point recognition.

[0062] Based on the pupil's center position and a pre-defined geometric model, the user's gaze direction is calculated. Specifically, an eyeball geometric model is established, simplifying the eyeball to a sphere, and a three-dimensional spatial coordinate system is created by combining physiological parameters such as corneal curvature and pupil position. The direction of the gaze in three-dimensional space is determined by using the spatial vector between the pupil center and the corneal reflection point, combined with head position compensation. Converting the gaze direction into the coordinates of the fixation point on the display screen requires pre-calibration (e.g., nine-point calibration) to establish a personalized mapping model.

[0063] The pupil coordinates detected by the eye-tracking module are fed back to the driving unit of the microstructure device in real time. A preset exit pupil area can be set according to user needs and display content to define the position that the user's pupil needs to reach to activate the display.

[0064] The detected pupil coordinates are compared with a preset exit pupil area to determine if the user's pupil is within this range. When the pupil coordinates are detected to be within the preset exit pupil area, the drive unit dynamically adjusts the diffraction parameters of the microstructure device based on the pupil coordinates to ensure that the exit pupil area always covers the effective pupil range. For example, by changing the reflection angle or diffraction angle of the micro-corner pyramid prism array or orthogonal mirror array, the position and size of the exit pupil area are optimized to ensure that the user can clearly see the displayed image. This dynamic adjustment mechanism not only improves the efficiency of information acquisition but also reduces visual fatigue during use. When the pupil coordinates are detected to be outside the preset exit pupil area, the user's pupil coordinates are monitored again.

[0065] The dynamic adjustment of the diffraction parameters of microstructured devices can be achieved by changing the reflection or diffraction angle of the microstructured devices, thereby optimizing the display effect. Specifically, for micro pyramidal prism arrays, the tilt angle of the prism units is adjusted by piezoelectric or electromagnetic drive; for orthogonal mirror arrays, the deflection angle of the micromirrors is controlled by microelectromechanical systems (MEMS).

[0066] In addition, when the user's gaze direction is detected to deviate from the preset angle range and the deviation duration exceeds the preset duration threshold (usually set to 1-3 seconds), the micro-display can be turned off to reduce interference with the user's normal gaze and reduce power consumption.

[0067] This embodiment significantly improves the user experience and display effect of near-eye display devices by introducing an eye-tracking module and a dynamic adjustment mechanism. The eye-tracking module utilizes the pupil-corneal reflection principle, accurately detecting the user's pupil coordinates and gaze direction using an infrared light source and camera, and can operate stably even in complex ambient light conditions. Combined with image processing algorithms and machine learning models, it can accurately locate the pupil center and calculate the gaze direction using a preset geometric model, achieving personalized calibration. By feeding back the pupil coordinates to the drive unit of the microstructure device in real time, the system dynamically adjusts the diffraction parameters of the microstructure device, optimizing the position and size of the exit pupil area to ensure that the user can clearly see the displayed image at a specific angle, while reducing visual fatigue. Furthermore, when the user's gaze deviates from the preset range for a certain period of time, the system automatically shuts off the display screen to reduce power consumption and visual interference. This intelligent dynamic adjustment mechanism not only improves the efficiency of information acquisition but also enhances the adaptability and user-friendliness of the device, making it particularly suitable for near-eye display devices with high visual experience requirements.

[0068] The near-eye display device provided in this application utilizes small exit pupil near-eye display technology to form a small exit pupil area, enabling on-demand display of electronic information. This avoids continuous interference from electronic images on the real field of vision, improving visual comfort and focus. Simultaneously, combined with eye-tracking technology, the exit pupil area can be dynamically adjusted according to the user's gaze direction, ensuring that the user can quickly and accurately obtain information when needed, further improving information acquisition efficiency and user experience. Furthermore, the compact microstructure device design of this application reduces optical design complexity, decreases module size and power consumption, and achieves high space utilization, making it suitable for smart glasses scenarios involving prolonged wear and high-frequency interaction.

[0069] Please refer to Figure 8 , Figure 8 This is a flowchart illustrating a first embodiment of a near-eye display method provided in this application.

[0070] like Figure 8 As shown, the near-eye display method includes steps S101 to S103.

[0071] S101. Generate an original electronic image through a micro-display screen, wherein the original electronic image contains text, image, or video information elements.

[0072] In one embodiment, the microdisplay is one of the core components of the near-eye display device, used to generate raw electronic images containing text, images, or video information. For example, the raw electronic image may be navigation information, notification messages, video call footage, real-time data, etc., depending on the application scenario and user needs.

[0073] Microdisplays can employ technologies such as liquid crystal displays (LCDs), organic light-emitting diode (OLEDs), or microelectromechanical systems (MEMS) displays, selected based on application scenarios and cost requirements. For example, LCDs control light transmittance by manipulating the arrangement of liquid crystal molecules to display images, suitable for general near-eye display devices, such as ordinary AR glasses or low-cost VR devices. OLEDs are self-emissive, requiring no backlight, while OLEDs offer higher contrast and faster response times, making them suitable for high-resolution and high-dynamic-range content display, ideal for AR glasses, high-end VR devices, or smart helmets with high display quality requirements. MEMS displays utilize microelectromechanical technology for image display, featuring high resolution and low power consumption. MEMS displays are suitable for wearable devices with strict size and power consumption requirements, suitable for lightweight, low-power smart glasses or helmets.

[0074] To ensure image clarity, the resolution of the microdisplay can be set to at least 1920×1080 pixels. Simultaneously, to avoid image flicker, the refresh rate of the microdisplay can be set to at least 60Hz.

[0075] The microdisplay can control the display of the original electronic image through a driving circuit. This driving circuit is responsible for converting the input image data into pixel signals on the microdisplay and ensuring the real-time updating of the original electronic image.

[0076] In this embodiment, different technologies such as LCD, OLED, or MEMS are flexibly selected to configure the microdisplay based on application scenarios and cost requirements, meeting diverse needs ranging from ordinary AR glasses to high-end VR devices. By setting high resolution and high refresh rate, the clarity and smoothness of the displayed image are ensured, avoiding image flicker and providing users with a high-quality visual experience. In addition, the efficient driving circuit can update image data in real time, further optimizing the display effect. These designs not only improve display quality but also enhance the adaptability and user-friendliness of the device, making it suitable for various near-eye display devices with high requirements for display effects.

[0077] S102. The divergent light emitted by each pixel in the micro-display screen is collimated by a collimating projection lens to obtain a parallel beam.

[0078] In one embodiment, the collimating projection lens converts the divergent light from each pixel on the microdisplay into parallel light. Theoretically, the parallel light transmitted through the collimating lens can be directly projected into the human eye, but in reality, the light emitted from the collimating lens exhibits a divergent effect, resulting in a very limited viewing range. Therefore, microstructure devices are needed for further optical path control.

[0079] Generally, the focal length, aperture size, and other parameters of a collimating projection lens can be optimized according to the size of the microdisplay and the display distance.

[0080] Specifically, the focal length of a collimating lens determines the convergence and divergence of light. A shorter focal length can reduce the size of the optical system, but may result in a larger divergence angle, affecting the display effect; a longer focal length can improve the parallelism of light, but will increase the size and weight of the system. For example, for a microdisplay with a diagonal length of 1 inch, the focal length of the collimating projection lens can be designed to be around 10mm.

[0081] Aperture size (usually expressed as an F-number, such as F / 2.8) affects the amount of light passing through and the divergence angle. A larger aperture increases the amount of light passing through and improves display brightness, but may lead to a larger divergence angle; a smaller aperture improves the parallelism of light rays, but reduces display brightness. For example, for the aforementioned 1-inch microdisplay, the aperture size of the collimating projection lens can be designed to be around F / 2.8. This design can minimize the divergence angle of light while ensuring sufficient brightness.

[0082] This embodiment achieves high-quality original electronic image generation and efficient optical path conversion through the coordinated design of an optimized microdisplay and a collimating projection lens. The collimating projection lens, through precise focal length and aperture parameter optimization, efficiently converts the divergent light from the microdisplay into a parallel beam, ensuring good light parallelism while controlling the size of the optical system, thus improving the image quality, visual comfort, and system reliability of near-eye display devices.

[0083] S103. Using microstructure devices, the orientation angle of the parallel beam is adjusted so that the parallel beams in different directions converge within a preset exit pupil area to form a small exit pupil area, so as to display the original electronic image in the small exit pupil area. The microstructure device includes a micro-corner pyramid prism array or an orthogonal mirror array.

[0084] Microstructured devices are used to directionally control the aligned parallel light, causing parallel light from various angles to reconverge and form a small exit pupil region.

[0085] In one embodiment, the microstructure device may employ a micro pyramidal prism array.

[0086] A micro-pyramidal prism array consists of multiple tiny pyramidal prism units arranged periodically. Each pyramidal prism unit has three mutually perpendicular reflecting surfaces. The incident parallel light undergoes three reflections internally, changing its propagation direction so that the angle of the outgoing beam is opposite to the incident angle. When combined with a beam splitter, the micro-pyramidal prism array can redirect and converge diverging parallel beams into a specific spatial region.

[0087] For example, the size of the micro-conical prism array can be designed according to the diameter of the exit beam and the size of the exit pupil area of ​​the collimating projection lens. For instance, for a collimating projection lens with an exit beam diameter of 5mm, the size of the micro-conical prism unit can be designed to be approximately 1mm × 1mm.

[0088] For example, the reflective surface of the cornerstone prism unit can be coated with a high-reflectivity material (such as aluminum or silver coating) to reduce light loss and improve the efficiency of the optical system.

[0089] In one specific embodiment, when a parallel light beam is incident on the micro-cornerstone prism array, the light undergoes three reflections within each cornerstone prism unit, causing the direction of the outgoing beam to be opposite to the incident direction. Combined with a beam splitter, these direction-reversed parallel beams can be redirected and converged into a specific spatial region, forming a small exit pupil region. The beam splitter (such as a beam splitter mirror) is used to split and converge the light beam processed by the cornerstone prism array, ultimately forming a compact, small exit pupil region. The beam splitter can effectively guide and concentrate the originally divergent light to a predetermined small exit pupil position, ensuring that the human eye can only receive an image when aligned at a specific angle.

[0090] This embodiment utilizes a micro-corner pyramid prism array as a microstructure device to achieve efficient directional control of aligned parallel beams. This allows parallel beams from different directions to converge within a preset exit pupil region, forming a small exit pupil area. This design not only significantly reduces disordered light scattering and improves the efficiency of the optical system, but also ensures that the user can only receive the image when aligned at a specific angle. This effectively avoids continuous interference from electronic images to the user's normal field of vision, significantly enhancing visual comfort and focus.

[0091] In another embodiment, the microstructure device may also employ an orthogonal mirror array.

[0092] An orthogonal mirror array consists of two sets of mutually perpendicular micromirrors. Each set of mirrors modulates the angle of the incident parallel light in an orthogonal direction. If the direction perpendicular to the orthogonal mirror array is taken as the Z direction, and the plane parallel to the orthogonal mirror array is taken as the XY plane, then the z-component of the direction vector of the outgoing beam modulated by the orthogonal mirror array is the same as the z-component of the direction vector of the incident beam, while the x and y components have opposite signs to the x and y components of the incident beam.

[0093] For example, the size and spacing of the orthogonal reflector array can be designed according to the diameter of the exit beam and the size of the exit pupil area of ​​the collimating projection lens. For instance, for a collimating projection lens with an exit beam diameter of 5mm, the size of the micromirrors can be designed to be approximately 1mm × 1mm, with a spacing of approximately 0.5mm.

[0094] For example, the reflective surface of a micromirror can also be coated with a high-reflectivity coating material (such as aluminum or silver coating) to reduce light loss.

[0095] In one specific embodiment, when a parallel beam of light is incident on an orthogonal reflector array, the first set of micromirrors modulates the light beam along the X-axis, and the second set of micromirrors modulates the light beam along the Y-axis. After these two modulations, the direction vector of the outgoing beam remains unchanged in the Z-axis component, while the X-axis and Y-axis components have opposite signs to the incident beam. This specific light modulation method can converge parallel light rays from different viewing angles into a small exit pupil area, thereby achieving a small exit pupil display.

[0096] This embodiment utilizes an orthogonal mirror array as a microstructure device to achieve efficient directional control of parallel light beams. This allows parallel beams from different directions to converge within a preset exit pupil region, forming a small exit pupil area. This design not only significantly reduces disordered light scattering and improves the efficiency of the optical system, but also ensures that the user can only receive the image when aligned at a specific angle. This effectively avoids continuous interference from electronic images to the user's normal field of vision, significantly enhancing the comfort and focus of the visual experience.

[0097] Understandably, whether the microstructure device uses a micro-pyramidal prism array or an orthogonal mirror array, it can be bonded with other optical elements (such as prisms or lenses) to achieve a compact optical structure and higher space utilization.

[0098] For example, microstructured devices can be directly glued to the exit surface of a collimating projection lens, or glued to a reflective prism used for optical path folding, forming a seamless optical assembly that can reduce the length of the entire optical path and improve the transmission efficiency of light.

[0099] This embodiment achieves a compact optical structure and higher space utilization by bonding microstructure devices (whether micro pyramidal prism arrays or orthogonal mirror arrays) with other optical elements (such as prisms or lenses), reducing the length of the entire optical path and improving the transmission efficiency of light.

[0100] In one embodiment, the preset exit pupil area can be the edge of the display interface of the near-eye display device, or other areas that do not affect the main display area (such as the 135° forward viewing area). The display screen in the small exit pupil area can be constantly lit. The user will only see the content displayed in the small exit pupil area when their eyes move into the viewing angle range corresponding to the preset exit pupil area. If the user's eyes move to a viewing angle range outside the preset exit pupil area, they will not see the content displayed in the small exit pupil area. In other words, the content displayed in the small exit pupil area will not interfere with the user's vision.

[0101] This embodiment ensures that the small-sized exit pupil area does not interfere with the user's normal line of sight by setting the exit pupil area at the edge of the display interface or in another area that does not affect the main display area. The user can only see the displayed content when their eyes move within the line of sight corresponding to the preset exit pupil area, thus realizing on-demand display of information and significantly improving the comfort and focus of the visual experience.

[0102] This embodiment achieves precise angle control and spatial convergence of parallel light beams through innovative microstructure device design, effectively solving the technical problem of continuous interference between electronic images and the real field of vision in traditional near-eye display devices. A micro-corner cone prism array reverses the beam direction through triple reflection, and combined with a beam splitter, precisely converges the diverging light to a specific spatial region. An orthogonal mirror array, through angle modulation in orthogonal directions, reverses the X and Y components of the incident beam while maintaining the Z-axis component unchanged. Both schemes can form a compact, small exit pupil region, ensuring that the electronic image is displayed only when the user's line of sight is aligned, thus avoiding visual interference and reducing system power consumption. Further optimization of the optical structure's compactness through adhesive bonding technology enhances display comfort and provides reliable technical support for the miniaturization and lightweight design of wearable devices such as smart glasses.

[0103] In one embodiment, based on the above Figure 9 As shown in the embodiment, this application further provides a near-eye display method based on eye tracking. For example... Figure 8 As shown, before step S101, the procedure further includes:

[0104] S201. Real-time monitoring of the user's gaze direction using an eye-tracking algorithm;

[0105] In one embodiment, an eye-tracking module can acquire real-time images of the user's eyes, and an image processing algorithm can be used to identify the center position of the user's pupils and determine the pupil coordinates. Based on the pupil coordinates and a preset geometric model, the user's gaze direction can be calculated.

[0106] Furthermore, real-time images of the user's eyes are acquired through an eye-tracking module; the center position of the user's pupils is identified using an image processing algorithm to determine the pupil coordinates; and the user's gaze direction is calculated based on a preset geometric model and the pupil coordinates.

[0107] For example, the eye-tracking module can use an infrared camera or other high-precision imaging device to obtain a clear image of the user's eyes. For instance, because infrared light is harmless to the human eye and can provide a clear image of the pupil, an infrared camera can be used to capture the reflection of the pupil under infrared light, thereby more accurately identifying the pupil position.

[0108] The eye-tracking module captures real-time images of the user's eyes at a high frame rate. These images are then transmitted to the image processing unit for further analysis using image processing algorithms. These algorithms identify the center position of the pupil from the captured eye images and calculate the user's gaze direction. Techniques such as edge detection and template matching can be employed to accurately identify the pupil's center position.

[0109] In one embodiment, the pupil contour of the user's pupil is extracted based on an edge detection algorithm and a template matching method; the centroid of the pupil contour is calculated to obtain the pupil coordinates.

[0110] Specifically, edge detection algorithms (such as Canny edge detection) are used to identify the outline of the pupil, and a predefined pupil template is used for matching to further improve the recognition accuracy. The center position of the pupil is determined by calculating the centroid of the pupil outline.

[0111] Edge detection is a technique in image processing used to identify points of brightness variation in an image; these points typically correspond to the contours of objects. For example, Canny edge detection is a widely used edge detection algorithm, widely applied in pupil recognition due to its good performance in noise suppression and edge detection accuracy.

[0112] In the Canny edge detection process, a Gaussian filter is first used to smooth the image, reducing noise and preventing noisy points from being misidentified as edges. The kernel size and standard deviation (σ) of the Gaussian filter can be adjusted according to the noise level of the image.

[0113] Then, using the Sobel or Prewitt operator, the gradient intensity and direction of each pixel in the image are calculated; areas with strong gradients are typically edges. For example, the Sobel operator calculates the gradient in the horizontal direction... and vertical gradient To highlight the edges. The gradient strength is calculated based on the horizontal and vertical gradients. and gradient direction .

[0114] For each pixel, check its two neighboring points along the gradient direction. If the gradient strength of the current point is not the maximum value of the two neighboring points, mark the current point as a non-edge point, thereby refining the edge and ensuring that the edge is only one pixel wide.

[0115] By setting two thresholds, high and low, strong and weak edges can be distinguished, avoiding false positives. For example, a high threshold can be set. and a low threshold The gradient strength is higher than Points with gradient strength below a certain value are considered strong edge points. Points with gradients between 0 and 1 are considered non-edge points, while points with gradient strengths in between are considered weak edge points. Generally, we set... , .

[0116] Use either Depth-First Search (DFS) or Breadth-First Search (BFS) algorithms to connect edge points and form complete edges. If a weak edge point is connected to a strong edge point, the weak edge point is retained; otherwise, it is marked as a non-edge point.

[0117] Define a standard pupil template, such as a circular template, with a diameter equal to the average pupil diameter (e.g., 50 pixels) and its center as the template's centroid, used to match the pupil region in an image. For example, a standard pupil template can be generated by collecting a large number of user eye images, extracting typical pupil features, and then applying this template. The pupil template can be a binary image where the pupil region is set to 1 and the background to 0.

[0118] For each possible image region, a template is slid across the image using Normalized Cross-Correlation (NCC) or other similarity metrics. The normalized cross-correlation score between the pupil template and the image region is calculated, and the location with the highest score is considered the pupil location. Post-processing is performed on the matching results, such as removing isolated points and smoothing edges. Morphological operations (such as dilation and erosion) can be combined to optimize the matching results and further improve the accuracy of pupil recognition. For example, morphological dilation can be used to fill small holes within the pupil region, and erosion can be used to remove noise points at the edges.

[0119] Finally, calculating the centroid of the pupil contour allows for accurate location of the pupil's center. Specifically, a contour extraction algorithm (such as the findContours function in OpenCV) is used to extract all closed contours from the results of the edge detection and template matching described above. Contours that match pupil characteristics (such as circles, areas, etc.) are then selected to determine the pupil contour.

[0120] For the extracted pupil contour, the average coordinates of all contour points are calculated to obtain the centroid position. For example, for a given contour, its centroid... It can be calculated using the following formula:

[0121]

[0122] in, It is the number of points on the outline. It is the first in the outline The coordinates of the points.

[0123] The user's gaze direction is calculated based on the pupil center position and a preset geometric model. The geometric model can be optimized according to the user's eye structure and the relative position of the device to improve the accuracy of the gaze direction calculation.

[0124] To calculate the direction of gaze, a simplified geometric model of the human eye is first needed to describe its structure. Typically, the human eye can be approximated as a sphere, with the pupil located on its surface. The average radius of the human eye is approximately 12 mm. This value can be adjusted for specific applications, such as through user calibration to obtain a more precise value. The center position of the pupil is represented in the image as two-dimensional coordinates. .

[0125] This embodiment achieves real-time and accurate monitoring of the user's gaze direction through an eye-tracking module and advanced image processing algorithms. It uses an infrared camera to capture eye images and combines them with technologies such as Canny edge detection and template matching to accurately identify the pupil center position. Then, it calculates the user's gaze direction using a preset geometric model. This process not only improves the accuracy and stability of pupil recognition but also enhances the accuracy of gaze direction calculation through optimized geometric models and user calibration. This high-precision eye-tracking technology provides a smart interaction method for near-eye display devices, dynamically adjusting the displayed content according to the user's gaze, reducing visual interference, lowering power consumption, and significantly improving user experience and device adaptability.

[0126] The relative position of the image acquisition device (such as an eye-tracking camera) to the eye is another important parameter in the geometric model. Assume the distance between the camera and the eye is... The angle between the optical axis of the camera and the optical axis of the eye is The camera distance (D) can be determined experimentally during the device design phase. For example, in AR glasses, the distance between the camera and the eyes might be 20-30mm. The included angle ( This can be determined through the mechanical design of the device and user calibration. Ideally, the camera's optical axis is perpendicular to the eye's optical axis, i.e. .

[0127] To facilitate calculations, a coordinate system needs to be established to describe the pupil position and gaze direction. For example, a camera coordinate system can be constructed, with the camera's optical center as the origin, the optical axis as the Z-axis, the horizontal direction as the X-axis, and the vertical direction as the Y-axis. Alternatively, an eye coordinate system can be constructed, with the eye's spherical center as the origin, the optical axis as the Z-axis, the horizontal direction as the X-axis, and the vertical direction as the Y-axis.

[0128] The coordinates of the two-dimensional image of the pupil Mapped into three-dimensional space. Assuming the pupil is located on the surface of the eyeball, its three-dimensional coordinates are... It can be calculated using the following formula:

[0129]

[0130] in, The angular position of the pupil in the image can be calculated using the following formula:

[0131]

[0132] The direction of gaze can be calculated using the three-dimensional coordinates of the pupil and the coordinates of the eye's center of gravity. Let's assume the eye's center of gravity is... Line of sight vector It can be represented as:

[0133]

[0134] Normalize the line-of-sight vector to obtain the unit line-of-sight vector. :

[0135]

[0136] The direction of sight is usually expressed in angles, and the horizontal angle of sight direction can be calculated using the following formula. and vertical angle :

[0137]

[0138]

[0139] By establishing a simplified geometric model and combining the pupil center position with preset parameters, the user's gaze direction can be accurately calculated.

[0140] This embodiment achieves real-time and accurate monitoring of the user's gaze direction by combining a high-precision eye-tracking algorithm with an optimized geometric model. Employing high-precision imaging equipment such as infrared cameras and image processing technology, the pupil center position can be accurately identified. By establishing a simplified eyeball geometric model and considering the transformation relationship between the camera coordinate system and the eye coordinate system, the two-dimensional pupil coordinates are mapped to three-dimensional space, and the precise gaze direction vector is calculated. Post-processing techniques such as normalized cross-correlation matching and morphological optimization significantly improve the accuracy and robustness of pupil recognition.

[0141] S202. When the line of sight is detected to be within a preset angle range, the micro display screen is activated to generate an original electronic image through the micro display screen.

[0142] In one embodiment, a preset angle range is set according to the application scenario and user needs. This angle range is typically expressed as the horizontal and vertical angles along the line of sight.

[0143] For example, in high-precision display scenarios (such as surgical assistance or precision operations), the preset angle range can be relatively small, such as ±5° for the horizontal angle and ±5° for the vertical angle. In ordinary display scenarios (such as daily navigation or message reminders), the preset angle range can be relatively large, such as ±15° for the horizontal angle and ±15° for the vertical angle.

[0144] Furthermore, a preset angle range is set; the line of sight is compared with the preset angle range to determine whether the line of sight is within the preset angle range; when the line of sight is within the preset angle range, an activation signal is sent to the micro-display screen to activate the micro-display screen to generate the original electronic image.

[0145] In one embodiment, a preset angle range can be set according to user needs and the displayed content to define the position that the user's line of sight needs to reach to activate the display. The preset angle range can be adjusted according to the application scenario. For example, in scenarios requiring high-precision display, the preset angle range can be smaller; in scenarios where display precision requirements are not high, the preset angle range can be larger.

[0146] In one embodiment, a preset angle range can be set at the edge of the user's field of vision, such as a direction 45° to 60° to the right (left, upper right, lower right, upper left, lower left, etc.) in the direction of direct view. Within the selected direction, the specific position is determined according to the size of the preset angle range (such as the range of horizontal and vertical angles). For example, right side: horizontal angle range is [45°, 60°], vertical angle range is [−15°, 15°]. Or, upper left: horizontal angle range is [−60°, −45°], vertical angle range is [−15°, −30°].

[0147] The calculated line of sight (horizontal angle) and vertical angle The system compares the user's gaze with a preset angle range to determine if the user's gaze is within that range. If the user's gaze is within the preset angle range, the micro-display is activated to display a small exit pupil; otherwise, the system continues to monitor the user's gaze direction.

[0148] When the user's line of sight is within a preset angle range, a signal is sent to the micro-display screen to activate the displayed content. The activation signal can be sent via a control module, which can dynamically adjust the display content and parameters of the micro-display screen according to the user's line of sight to optimize the user experience.

[0149] Specifically, the system continuously acquires the user's gaze direction to ensure the control module can respond promptly to changes in the user's gaze. A high-frame-rate infrared camera is used to capture real-time images of the user's eyes, and eye-tracking algorithms (such as Canny edge detection and template matching) are used to calculate the user's gaze direction (horizontal angle). and vertical angle The calculated viewing direction is compared with a preset angle range. When the user's viewing direction enters the preset angle range, an activation signal is generated using a control module (such as a microcontroller or FPGA). The activation signal can be a simple level signal or a data packet containing specific instructions. The activation signal is transmitted from the control module to the microdisplay. Upon receiving the activation signal, the microdisplay switches from standby mode to working mode and immediately starts the display function.

[0150] Based on the user's gaze direction and the application scenario, the system generates corresponding display content. The control module dynamically adjusts the position, size, and type of the displayed content according to the user's gaze direction. The displayed content can include navigation information, notification messages, video call footage, etc. For example, when the user's gaze enters a preset angle range, navigation information is displayed, and the position of the information is adjusted according to the user's gaze point.

[0151] In one embodiment, the brightness and contrast of the displayed content can be adjusted according to the user's line of sight. When the user's line of sight is close to the edge of the screen, the brightness of the displayed content is automatically adjusted to make it clearer.

[0152] This embodiment achieves precise, on-demand control of the microdisplay through an intelligent gaze-activated mechanism. Based on different application scenarios, a preset angle range is adaptively set, and the microdisplay is intelligently activated by comparing the calculated gaze direction with the preset range in real time. This gaze-direction-based intelligent activation strategy not only significantly reduces system power consumption and avoids interference from invalid information in the main field of view, but also ensures the best visual experience through adaptive display parameter adjustments, providing an efficient and reliable human-computer interaction solution for near-eye display devices.

[0153] In one embodiment, in order to reduce interference with the user's normal line of sight and reduce power consumption, the micro-display can be automatically turned off after the user's line of sight deviates from a preset angle range for a certain period of time.

[0154] Specifically, when the user's gaze is within a preset angle range, the content of the original electronic image is displayed through a small exit pupil area. While the user is viewing the content displayed in this small exit pupil area, i.e., when the user's gaze is within the preset angle range, the system continues to detect the user's gaze direction in real time and compares the calculated gaze direction with the preset angle range.

[0155] In one embodiment, when the user's line of sight is detected to deviate from the preset angle range and the deviation duration is greater than a preset duration threshold, the micro-display is turned off to reduce interference with the user's normal line of sight and reduce power consumption.

[0156] For example, the maximum allowable time for a user's gaze to deviate from a preset angle range can be defined as a preset duration threshold, based on user habits and device power consumption requirements. For instance, the preset duration threshold can be set to 1 second, meaning that the micro-display will automatically turn off when the user's gaze deviates from the preset angle range for more than 1 second.

[0157] The preset duration threshold can be dynamically adjusted based on the user's real-time needs and usage scenarios. For example, in surgical assistance scenarios, the preset duration threshold is shortened to 0.5 seconds; in daily navigation scenarios, the preset duration threshold is extended to 2 seconds. Personalized preset duration thresholds can also be optimized through user-defined calibration. If users report a desire for the display to turn off faster, the preset duration threshold will be automatically shortened to adapt to individual differences.

[0158] Specifically, the system monitors the user's gaze direction in real time. When the user's gaze direction deviates from a preset angle range, a timer is started. When the deviation duration exceeds a preset duration threshold, a control module (such as a microcontroller or FPGA) generates a shutdown signal. The shutdown signal can be a simple level signal or a data packet containing specific instructions.

[0159] The shutdown signal is transmitted from the control module to the micro display screen. After receiving the shutdown signal, the micro display screen switches from the working state to the standby state or turns off completely, turning off the display content to reduce interference with the user's normal line of sight and reduce power consumption.

[0160] This embodiment achieves precise power consumption management and visual interference control for the microdisplay through an intelligent timeout shutdown mechanism. A timing function is activated when the user's gaze deviates from a preset angle range. Once the deviation exceeds a dynamically adjustable preset threshold, a shutdown signal is automatically generated, switching the microdisplay from operating mode to standby mode. This adaptive threshold adjustment strategy, based on usage scenarios and personalized needs, ensures timely information display while effectively preventing continuous interference from ineffective displays on the main field of vision. Through the coordinated operation of real-time gaze monitoring and intelligent shutdown control, system power consumption is reduced while maintaining a good user experience, providing an efficient energy-saving solution for near-eye display devices, particularly suitable for applications such as smart glasses that require prolonged wear.

[0161] This embodiment provides a near-eye display method. This method generates high-resolution raw electronic images containing text, images, or videos through a microdisplay, providing users with accurate information content, ensuring the diversity and richness of information, and guaranteeing the quality and clarity of the information source. A collimating projection lens converts the divergent light from the microdisplay into a parallel beam, reducing disordered light scattering and effectively eliminating ghosting and blurring caused by light scattering. This improves the efficiency and display quality of the optical system and avoids light interference with non-target areas. Microstructure devices, through directional control of the parallel beam, converge it to a small exit pupil area, enabling on-demand display of image information. The image is only visible when the user's gaze is aligned, thus avoiding continuous interference with the user's normal field of vision, improving visual comfort and focus, optimizing information transmission, and providing a more natural, comfortable, and efficient human-computer interaction experience for near-eye display devices.

[0162] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a first embodiment of a near-eye display device provided in this application. The near-eye display device is used to perform the aforementioned near-eye display method.

[0163] like Figure 10 As shown, the near-eye display device 300 includes: an image generation module 301, a beam collimation module 302, and an angle adjustment module 303.

[0164] Image generation module 301 is used to generate a raw electronic image through a micro display screen, the raw electronic image containing text, image or video information elements;

[0165] The beam collimation module 302 is used to collimate the divergent light of each pixel in the microdisplay through the collimation projection lens to obtain a parallel beam.

[0166] Angle adjustment module 303 is used to adjust the directional angle of the parallel beam through a microstructure device, so that the parallel beams in different directions converge within a preset exit pupil area to form a small exit pupil area, so as to display the original electronic image in the small exit pupil area. The microstructure device includes a micro-corner pyramid prism array or an orthogonal reflector array.

[0167] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the device and each module described above can be referred to the corresponding processes in the aforementioned near-eye display method embodiments, and will not be repeated here.

[0168] The apparatus provided in the above embodiments can be implemented as a computer program, which can be used in, for example... Figure 11 It runs on the near-eye display device shown.

[0169] Please see Figure 11 , Figure 11 This is a schematic block diagram illustrating the structure of a near-eye display device provided in an embodiment of this application. The near-eye display device may be a server.

[0170] See Figure 11 The near-eye display device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0171] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any near-eye display method.

[0172] The processor provides computing and control capabilities to support the operation of the entire near-eye display device.

[0173] Internal memory provides an environment for the execution of computer programs in non-volatile storage media, which, when executed by a processor, enable the processor to perform any near-eye display method.

[0174] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the near-eye display device to which the present application is applied. A specific near-eye display device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0175] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0176] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:

[0177] A raw electronic image is generated using a microdisplay, the raw electronic image containing text, image, or video information elements;

[0178] The divergent light emitted by each pixel in the microdisplay is collimated by a collimating projection lens to obtain a parallel beam of light;

[0179] By using microstructure devices, the orientation angle of the parallel beam is adjusted so that the parallel beams from different directions converge within a preset exit pupil region to form a small exit pupil region, in order to display the original electronic image in the small exit pupil region. The microstructure devices include a micro-corner pyramid prism array or an orthogonal mirror array.

[0180] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement any of the near-eye display methods provided in the embodiments of this application.

[0181] The computer-readable storage medium can be an internal storage unit of the near-eye display device described in the foregoing embodiments, such as the hard disk or memory of the near-eye display device. Alternatively, the computer-readable storage medium can be an external storage device of the near-eye display device, such as a plug-in hard disk, SmartMediaCard (SMC), SecureDigital (SD) card, or FlashCard equipped on the near-eye display device.

[0182] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A near-eye display method, characterized in that, The method includes: A raw electronic image is generated using a microdisplay, the raw electronic image containing text, image, or video information elements; The divergent light emitted by each pixel in the microdisplay is collimated by a collimating projection lens to obtain a parallel beam of light; By using microstructure devices, the orientation angle of the parallel beam is adjusted so that the parallel beams from different directions converge within a preset exit pupil region to form a small exit pupil region, in order to display the original electronic image in the small exit pupil region. The microstructure devices include a micro-corner pyramid prism array or an orthogonal mirror array.

2. The near-eye display method according to claim 1, characterized in that, Before generating the original electronic image via the microdisplay, wherein the original electronic image contains text, image, or video information elements, the method further includes: The user's gaze direction is monitored in real time using an eye-tracking algorithm; When the line of sight is detected to be within a preset angle range, the micro-display is activated to generate an original electronic image through the micro-display.

3. The near-eye display method according to claim 2, characterized in that, The method of real-time monitoring of the user's gaze direction using an eye-tracking algorithm includes: Real-time images of the user's eyes are captured using an eye-tracking module; Image processing algorithms are used to identify the center position of the user's pupils and determine the pupil coordinates. The user's gaze direction is calculated based on the preset geometric model and the pupil coordinates.

4. The near-eye display method according to claim 3, characterized in that, The step of using image processing algorithms to identify the center position of the user's pupil and determine the pupil coordinates includes: Based on edge detection algorithms and template matching methods, the pupil contour of the user's pupil is extracted; Calculate the centroid of the pupil outline to obtain the pupil coordinates.

5. The near-eye display method according to claim 2, characterized in that, The step of activating the micro-display when the line of sight is detected to be within a preset angle range includes: Set a preset angle range; Compare the line of sight with the preset angle range to determine whether the line of sight is within the preset angle range; When the line of sight is within the preset angle range, an activation signal is sent to the micro display screen to activate the micro display screen.

6. The near-eye display method according to claim 2, characterized in that, After monitoring the user's gaze direction in real time using an eye-tracking algorithm, the method further includes: When the user's gaze direction is detected to deviate from the preset angle range and the deviation duration exceeds the preset duration threshold, the micro-display is turned off to reduce interference with the user's normal gaze and reduce power consumption.

7. A near-eye display device, characterized in that, The near-eye display device includes: Microdisplays are used to generate raw electronic images; A collimating lens is used to convert the divergent light emitted by each pixel on the microdisplay into a parallel beam; A microstructure device is used to modulate a parallel beam of light into diffracted light emitted at a specific angle to form a virtual image in the user's field of vision; the microstructure device is bonded to other optical elements in the near-eye display device to achieve a compact optical structure, wherein the other optical elements include prisms or lenses; The exit pupil region is used to accommodate the user's pupil, allowing the diffracted light modulated by the microstructure device to accurately enter the human eye.

8. The near-eye display device according to claim 7, characterized in that, The near-eye display device also includes an eye-tracking module for real-time detection of the user's pupil coordinates; The pupil coordinates detected by the eye-tracking module are fed back to the driving unit of the microstructure device in real time. When the pupil coordinates are detected to be within a preset exit pupil region, the driving unit dynamically adjusts the diffraction parameters of the microstructure device according to the pupil coordinates to ensure that the exit pupil region always covers the effective pupil range.

9. The near-eye display device according to claim 7, characterized in that, The microstructure device includes a micro pyramidal prism array, which is composed of at least one periodically arranged pyramidal prism unit; Each of the cornerstone prism units reflects the parallel beam three times through three mutually perpendicular reflecting surfaces, so that the outgoing direction of the parallel beam is opposite to the incident direction, thereby achieving the reorientation and convergence of the parallel beam.

10. The near-eye display device according to claim 7, characterized in that, The microstructure device includes an orthogonal mirror array, which consists of two sets of mutually perpendicular micromirrors; The first set of micromirrors is used to modulate the angle of the parallel beam in the X-axis direction, and the second set of micromirrors is used to modulate the angle of the parallel beam in the Y-axis direction. The direction vector of the outgoing beam after being modulated by the orthogonal mirror array remains unchanged in the Z-axis component, while the X-axis and Y-axis components have opposite signs to the X-axis and Y-axis components of the incident beam direction vector.

Images