Near-to-eye display device and method and computer readable storage medium

Abstract

The invention provides a near-to-eye display device and method and a computer readable storage medium, and belongs to the technical field of near-to-eye display.The near-to-eye display device comprises an image acquisition module, a projection module connected with the image acquisition module and an eyepiece connected with the projection module, and the projection module comprises a micro-display and a curtain; the image acquisition module is configured to acquire an external environment image; the projection module is configured to generate a virtual reality image based on the external environment image, display the virtual reality image on the micro-display, and project a first magnified image obtained by magnifying the virtual reality image displayed on the micro-display by a first preset multiple on the curtain; and the eyepiece is configured to project a second magnified image obtained by magnifying the magnified image on the curtain by a second preset multiple to a preset distance. At least one of the field angle, the exit pupil distance and the eye box range can be flexibly selected and increased according to user requirements.

Description

Technical Field

[0001] The present disclosure relates to the field of near-eye display technology, and more particularly to a near-eye display device, method, and computer-readable storage medium. Background Art

[0002] Near-eye display devices use optical technology to guide the video image light emitted by the microdisplay to the user's pupil, allowing the user to observe virtual, enlarged video images, providing the user with intuitive, visual images, videos, and text information. It can be used in outdoor, simulated driving, training, demonstration, teaching, training, medical, flight and other scenarios.

[0003] However, existing near-eye display devices generally have the following problems: limited field of view, limited eye box range and insufficient pupil distance. A too small field of view will seriously affect the user's immersive experience; a too small eye box range will cause a slight movement of the user's head or eyeball to cause a significant change in the imaging effect; a too small pupil distance will cause the user to put the eyepiece optical system close to the eye when wearing the device, affecting the wearing comfort and making it difficult to adapt to users wearing glasses. Summary of the invention

[0004] The present disclosure provides a near-eye display device, method and computer-readable storage medium; the device can flexibly select to increase at least one of the field of view angle, exit pupil distance and eye box range according to user needs.

[0005] The technical solution of the present disclosure is achieved as follows: In a first aspect, the present disclosure provides a near-eye display device, which includes: an image acquisition module, a projection module connected to the image acquisition module, and an eyepiece connected to the projection module, wherein the projection module includes: a projection controller, a microdisplay and a screen; the image acquisition module is configured to acquire an external environment image; the projection module is configured to generate a virtual reality image based on the external environment image through the projection controller and display it on the microdisplay, and to project a first enlarged image after enlarging the virtual reality image displayed on the microdisplay by a first preset multiple onto the screen through the projection controller; the eyepiece is configured to project a second enlarged image after enlarging the enlarged image on the screen by a second preset multiple to a preset distance.

[0006] In a second aspect, the present disclosure provides a near-eye display method, which includes: acquiring an external environment image; generating a virtual reality image based on the external environment image; projecting a first enlarged image after enlarging the virtual reality image by a first preset multiple onto a screen; and projecting a second enlarged image after enlarging the enlarged image on the screen by a second preset multiple to a preset distance.

[0007] In a third aspect, the present disclosure provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the near-eye display method as described in the second aspect.

[0008] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing a program or instruction, which, when executed by a processor, implements the steps of the near-eye display method as described in the second aspect.

[0009] In a fifth aspect, the present disclosure provides a computer program product, wherein the computer program product includes a computer program or instructions. When the computer program product runs on a processor, the processor executes the computer program or instructions to implement the steps of the near-eye display method as described in the second aspect.

[0010] In a sixth aspect, the present disclosure provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the near-eye display method as described in the first aspect.

[0011] The present disclosure provides a near-eye display device, which includes: an image acquisition module, a projection module connected to the image acquisition module, and an eyepiece connected to the projection module, wherein the projection module includes: a projection controller, a microdisplay and a screen; the image acquisition module is configured to acquire an external environment image; the projection module is configured to generate a virtual reality image based on the external environment image through the projection controller and display it on the microdisplay, and, through the projection controller, a first magnified image after the virtual reality image displayed on the microdisplay is magnified by a first preset multiple is projected on the screen; the eyepiece is configured to project a second magnified image after the magnified image on the screen is magnified by a second preset multiple to a preset distance for easy viewing by human eyes. In the present disclosure, the size of the displayed image is increased by projection design, so that the field of view angle, the exit pupil distance and the eye box range are correspondingly increased, and the user can flexibly choose to increase at least one of the field of view angle, the exit pupil distance and the eye box range according to needs. In addition, increasing the displayed image by projection design will not increase the weight and size of the near-eye display device too much (the screen is thinner), and the screen price is lower. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of the mutual constraints between the exit pupil distance, the exit pupil diameter and the field of view angle provided by the present invention; Figure 2 A schematic diagram of the relationship between exit pupil diameter and eye box range provided for the present disclosure; Figure 3This is one of the structural schematic diagrams of a near-eye display device provided by the present disclosure; Figure 4 The second structural schematic diagram of a near-eye display device provided by the present disclosure; Figure 5 A schematic diagram of the change of the field of view angle when the exit pupil distance and the exit pupil diameter remain unchanged provided by the present disclosure; Figure 6 A schematic diagram of the change of the exit pupil distance provided by the present disclosure when the exit pupil diameter and the field of view angle remain unchanged; Figure 7 A schematic diagram of the change of the exit pupil diameter when the field of view angle and the exit pupil distance remain unchanged provided by the present disclosure; Figure 8 The image displayed on the screen when the eye point focus position provided by the present disclosure is in the middle of the screen; Fig. 9 The image displayed on the screen when the eye point focus position provided by the present disclosure is on the left side of the screen; Fig.10a A schematic diagram of realizing fused reality using an optical synthesizer provided in the present disclosure; Fig.10b A schematic diagram of realizing fused reality through a non-see-through curtain provided for the present disclosure; Fig.10c A schematic diagram of realizing fused reality through a perspective curtain provided for the present disclosure; Fig.11 A schematic flow chart of a near-eye display method provided by the present invention. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the present disclosure to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of the present application.

[0014] The terms "first", "second", etc. in the specification of this application are used to distinguish similar objects, rather than to describe a specific order or precedence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the present disclosure can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more.

[0015] In near-eye display devices, the size of the display screen, pupil distance, eye box range and field of view are mutually constrained. The pupil distance refers to the distance between the human eye and the eyepiece, and the eye box range (eyebox) refers to a conical area between the near-eye display optical module and the eyeball. In this area, the display content is complete and clearest. Figure 1 As shown, the mutual constraints among exit pupil distance, exit pupil diameter and field of view angle are shown.

[0016] Figure 1 The display screen 103 shown in FIG. 1 is placed within the focal plane of the eyepiece 102, and the focus 104 of the eyepiece 102 has a focal length of , the width of the display screen 103 is , Gao Wei , the diagonal is , the corresponding field of view ω It includes horizontal field angle, vertical field angle and diagonal field angle. Since the eyepiece is generally circular, only the height of the display screen is used as an example for calculation. The height of the virtual image of the display screen 103 through the eyepiece 102 is , for human eyes 101, the corresponding vertical field of view is: , is the exit pupil diameter, is the pupil distance, is the virtual image distance, and when there is no vignetting, the aperture of the eyepiece 102 is: .

[0017] like Figure 2 As shown in Figure 1, it is the corresponding relationship between the exit pupil diameter and the eye box range. When the other parameters are fixed, the larger the exit pupil diameter, the larger the eyebox area.

[0018] From the above, we can see that the exit pupil distance, exit pupil diameter and field of view angle restrict each other, that is, the exit pupil distance, eyebox and field of view angle restrict each other. If you want to increase the field of view angle and keep the eyebox unchanged, you need to reduce the exit pupil distance; if you want to increase the field of view angle and keep the exit pupil distance unchanged, you need to reduce the eyebox; if you want to increase the exit pupil distance and keep the field of view angle unchanged, you need to reduce the eyebox; if you want to increase the exit pupil distance and keep the eyebox unchanged, you need to reduce the field of view; if you want to increase the eyebox and keep the field of view angle unchanged, you need to reduce the exit pupil distance; if you want to increase the eyebox and keep the exit pupil distance unchanged, you need to reduce the field of view.

[0019] Therefore, for the exit pupil distance, eyebox and field of view angle, according to different needs, when keeping one of the three unchanged, if one of the remaining two increases, the other one needs to be reduced. It is impossible to increase one of the three without affecting the other two, or to increase two at the same time without affecting the remaining one. Based on this, the present disclosure aims to provide a near-eye display device that can optimize the exit pupil distance, field of view angle and eyebox. The near-eye display device provided by the present disclosure is described in detail below in conjunction with the accompanying drawings through specific embodiments and their application scenarios.

[0020] like Figure 3 As shown, a near-eye display device is provided in the present disclosure, and the device includes: an image acquisition module 30, a projection module 40 connected to the image acquisition module 30, and an eyepiece 50 connected to the projection module 40. The projection module 40 includes: a projection controller 401, a micro display 402 and a screen 403.

[0021] Among them, the image acquisition module 30 is configured to obtain an external environment image; the projection module 40 is configured to generate a virtual reality image based on the external environment image through a projection controller 401 and display it on a micro display 402, and, through the projection controller 401, a first enlarged image of the virtual reality image displayed on the micro display 402 is magnified by a first preset multiple and projected onto a screen 403; the eyepiece 50 is configured to project a second enlarged image after the enlarged image on the screen 403 is magnified by a second preset multiple to a preset distance for easy viewing by human eyes.

[0022] The image acquisition module 30 includes at least one camera, each of which includes a lens and an image sensor. The lens is used to focus the ambient light to obtain an ambient light signal, and the image sensor is used to convert the ambient light signal into a digital image signal, that is, an external environment image.

[0023] The projection module 40 may be a miniaturized projection device, such as a projection device using a micro electromechanical system (MEMS), a reflective projection device based on liquid crystal on silicon (LCoS), etc. The projection technology used by the projection module 40 may be a projection technology of a picture generation unit (PGU), holographic projection, laser beam scanning projection, etc. The specific selected projection device and projection technology are not limited in this disclosure.

[0024] Since the size of the near-eye display device should be as small as possible, the projection module 40 adopts a short-throw or ultra-short-throw design to project the image on the microdisplay 402 onto the screen 403. The projection distance is generally 10-20 mm, thereby ensuring the compactness of the entire near-eye display device.

[0025] In order to make the size of the near-eye display device smaller, the micro display 402 is arranged on the side of the screen 403 to ensure that the micro display 402 and the screen 403 are arranged more compactly when the projection distance remains unchanged.

[0026] The size of the microdisplay 402 is smaller than that of the screen 403, and the screen size can be selected to be 2-4 inches to be suitable for head-mounted devices. The screen 403 can be made of materials with good light scattering properties such as high-reflectivity polyvinyl chloride, metal, and fiberglass to ensure uniform display of the image and to ensure sufficient brightness with little light loss. In addition, a special projection screen with a microlens array (similar to a gain screen) can also be used to further improve light utilization. In order to obtain a larger field of view, the screen 403 can adopt a slightly curved design. The slightly curved screen can reduce edge distortion while further expanding the field of view.

[0027] The main task of the eyepiece 50 is to magnify the enlarged image projected on the screen 403 again and project it to the user's eyes. The eyepiece 50 should be designed with aspherical optics to minimize aberrations and distortions. The eyepiece 50 can be made of plastic or glass materials with high light transmittance, and use multi-layer coating technology to reduce reflection and increase transmittance. The magnification of the eyepiece 50 is designed according to the requirements of the screen size and the field of view. Usually, the second preset multiple is about 1.5 to 10 times according to different requirements, ensuring that the clarity of the image is guaranteed while providing a large field of view. The virtual image distance is changed by adjusting the spacing and focal length of the eyepiece 50 lens group. In order to reduce the user's visual fatigue, the virtual image distance is generally designed to be about 2 meters to simulate the user's experience of watching from a distance.

[0028] For example, Figure 4 As shown in FIG. 1 , it is a schematic diagram of the structure of a near-eye display device, taking a projection module 40 including: a PGU 404 and a screen 403 as an example, wherein the PGU 404 includes a projection controller 401 and a micro display 402. The PGU 404 generates a virtual reality image on the micro display 402 and projects it onto the screen 403, and the eyepiece 50 further magnifies the image on the screen and displays it at the virtual image distance. The outer shadow part in the figure is a closed cavity to ensure that the projection process will not be affected by external light.

[0029] For example, Figure 5 As shown, the dotted line shows the height of the curtain 403, compared to Figure 1 , when the pupil distance and pupil diameter remain unchanged, the field of view angle changes from ω becomes , it can be seen from the figure that the field of view increases. Figure 6 As shown, compared to Figure 1, when the field of view angle and exit pupil diameter remain unchanged, the exit pupil distance changes from becomes , it can be seen from the figure that the pupil distance increases. Figure 7 As shown, compared to Figure 1 , when the field of view angle and pupil distance remain unchanged, the exit pupil diameter (corresponding to the eye box range) changes from becomes In addition, if the field of view angle, exit pupil distance and exit pupil diameter need to be increased at the same time, the field of view angle can be increased to greater than ω and less than If the angle is increased, the corresponding exit pupil distance and exit pupil diameter will also increase.

[0030] It should be noted that Figures 5 to 7 The schematic diagram of the change of the remaining parameter when the exit pupil diameter, the exit pupil distance or the field of view angle remain unchanged is only given as an example. Other situations (such as the situation where the exit pupil diameter remains unchanged, and the exit pupil distance and the field of view angle change simultaneously) are similar and will not be repeated here.

[0031] In the present disclosure, the size of the displayed image is increased by projection design, so that the field of view angle, exit pupil distance and eye box range are correspondingly increased, and compared with directly increasing the size of the micro display, increasing the displayed image by projection design will not increase the weight and size of the near-eye display device too much (the screen is thinner), and in addition, the price of the screen and micro display of the same size is lower. Therefore, by using the projection method, the user can flexibly choose to increase at least one of the field of view angle, exit pupil distance and eye box range as needed, such as: keeping two parameter values ​​of the field of view angle, exit pupil distance and eye box range unchanged, increasing one parameter value, or keeping one parameter value unchanged and increasing the other two parameter values.

[0032] In some embodiments of the present application, the near-eye display device further includes: a magnification ratio adjustment module connected to the projection module; the magnification ratio adjustment module is configured to determine a first preset multiple according to an observation distance.

[0033] Since the distance between the curtain and the eyepiece is fixed, the magnification and virtual image distance of the eyepiece are fixed. If the virtual image distance is set to the optimal observation distance, that is, the observation target is at the dotted line distance, the size of the observation target observed by the near-eye device is the same as the size of the observation target observed by the naked eye; but if the observation target is at a position greater than the virtual image distance, the observation target observed by the near-eye device is smaller than the observation target observed by the naked eye, that is, the observation target is reduced; but if the observation target is at a position less than the virtual image distance, the observation target observed by the near-eye device is larger than the observation target observed by the naked eye, that is, the observation target is magnified. When the user wears the near-eye device, as the distance to the observation target changes, the observation target will fluctuate, which will cause the user to feel different from reality and increase the sense of dizziness.

[0034] Therefore, the present disclosure determines the first preset multiple according to the observation distance. Specifically, when the observation distance is greater than the standard observation distance, the first preset multiple is determined to be greater than 1; when the observation distance is less than the standard observation distance, the first preset multiple is determined to be less than 1; and the larger the observation distance, the larger the first preset multiple, and the smaller the observation distance, the smaller the first preset multiple. Wherein, at the standard observation distance, the size of the observation target observed by the near-eye display device is the same as the size of the observation target observed by the naked eye.

[0035] In the present disclosure, the first preset magnification factor that the projection needs to be magnified is determined based on the observation distance, which can ensure that the visually perceived magnification ratio is 1:1 at different observation positions, that is, ensuring that the external environment observed using a near-eye display device is consistent with that observed by the naked eye, while also reducing dizziness.

[0036] In some embodiments, the magnification ratio adjustment module includes an eye tracker, which is configured to determine the observation distance according to the eye focus position.

[0037] The eye tracker uses a camera to capture the image of the eye, and uses image processing technology to identify the center position of the pupil and the reflected light spot (Purkin's spot) on the cornea. These feature points remain in the same relative position when the eyeball moves, providing a stable reference for calculating the direction of sight. By locating the position of the pupil and Purkin's spot on the eye image in real time, the corneal reflection vector is calculated, and the user's sight direction is estimated based on the eye point focus position using a geometric model. The sight direction is calculated using the geometric model, and the straight-line distance from the eye to the eye point focus position is the observed distance. This model takes into account the spatial relationship between the eye tracker, the user's eye, and the observed target.

[0038] In some embodiments, the observation distance may be determined independently of the eye tracker. The observation distance is determined by triangulation, infrared ranging, ultrasonic ranging, etc. The first preset multiple is determined according to the observation distance. This method has a lower accuracy than the distance measurement method of the eye tracker, but the algorithm is simpler, so that the efficiency of determining the observation distance is higher and the delay is smaller.

[0039] In some embodiments, the eye tracker is further configured to determine the observation angle according to the eye focus position; the projection module 40 is specifically configured to generate virtual reality images of various definition based on the external environment image and the observation angle and display them on the micro display 402, and finally enlarge the virtual reality image displayed on the micro display 402 by a first preset multiple and project the enlarged image onto the screen 403.

[0040] When observing the external environment with the naked eye, although the field of view is large, users usually focus on the environmental information within a certain range of the observation target, rather than all the information in the entire field of view. Therefore, in order to better match naked eye observation, an image can be divided into multiple sub-areas, and different clarity can be set for different sub-areas in an image according to the observation angle.

[0041] For example, Figure 8 As shown, when the eye point focus position is in the middle of the curtain 403, the arrow direction is the line of sight, the determined observation angle is 90 degrees, and the user's eye point focus position is on the letter "D", so the letter "D" and the letters "C" and "E" within a certain range from the letter "D" are determined to be the targets that the user needs to focus on, and the clarity of other parts decreases as the distance from the eye point focus position increases. Fig. 9 As shown, when the eye point focus position is on the left side of the curtain 403, the determined observation angle is 110 degrees, and the user's eye point focus position is on the letter "B", so the letter "B" and the letters "A" and "C" within a certain range from the letter "B" are determined to be the targets that the user needs to focus on, and the clarity of other parts decreases as the distance from the eye point focus position increases. In this way, the information of the user's focus point can be highlighted, and the information of the non-focus point can be weakened, which is closer to naked eye observation, thereby improving the user experience.

[0042] In some embodiments, the eye tracker is further configured to determine display parameters based on the number of blinks; the projection module 40 is specifically configured to project a first magnified image of the virtual reality image displayed on the micro display 402 by a first preset multiple onto the screen according to the display parameters through the projection controller 401.

[0043] Since long-term use of near-eye display devices can easily cause visual fatigue, and in order to improve user experience, the color saturation, contrast, brightness, etc. of the image displayed on the screen 403 are all fixedly set to ensure that even if the brightness of the external environment is reduced, such as on a cloudy day, the external environment seen through the near-eye display device is bright and clear, thereby further accelerating user visual fatigue.

[0044] Therefore, in the present disclosure, the number of times the user blinks is determined by an eye tracker. If the number of blinks exceeds a threshold value, the display parameters are adjusted, such as lowering the brightness, lowering the contrast, adjusting the color temperature, reducing blue light, etc. After adjusting for a certain period of time, the number of blinks within the user's preset time is counted again. If the number of blinks is less than the threshold value, the original display parameters are restored for display. If it is still greater than or equal to the threshold value, the adjusted display parameters continue to be displayed.

[0045] In some embodiments, display parameters may also be adjusted according to the external light intensity to ensure that the currently displayed image is closer to the actual external environment.

[0046] In some embodiments of the present disclosure, the image acquisition module 30 includes: a night vision camera; the night vision camera is configured to acquire images of the external environment with a light intensity less than a light intensity threshold.

[0047] The near-eye display devices in related technologies are usually used in indoor bright environments. However, in some scenarios, such as assisted driving, the near-eye devices are required to collect outdoor environments. However, there may be various situations in the outdoor environment, the most common of which is insufficient illumination. At this time, the naked eye may not be able to clearly identify the various targets in the external environment. At this time, the night vision camera is used to collect images under weak illumination, so that users can see the external environment clearly through the near-eye display device in the case of insufficient illumination.

[0048] In some embodiments, the screen 403 is an opaque screen; the projection module 40 is specifically configured to generate a virtual reality image based on the external environment image and the virtual image through the projection controller 401 and display it on the micro display 402, and finally project a first enlarged image after enlarging the virtual reality image displayed on the micro display 402 by a first preset multiple onto the screen 403.

[0049] In order to combine elements of the physical world with the digital world and provide a more immersive experience, in the present disclosure, after the actual external environment image of the outside world is collected, the virtual image is aligned with the objects in the external environment image in three-dimensional space to ensure that the size, position and direction of the objects in the virtual image match those in the real world.

[0050] If the image acquisition module includes multiple cameras (such as a night vision camera), multimodal image fusion technology is used to fuse the images acquired by the multiple cameras with the virtual image to improve the accuracy and quality of the image while retaining the complementary information between the images.

[0051] In some embodiments, the screen 403 is a transparent screen; the projection module 40 is specifically configured to determine the compensation parameters according to the light loss of the screen through the projection controller 401; the first enlarged image after enlarging the virtual reality image displayed on the micro display 402 by a first preset multiple is projected onto the screen after compensation according to the compensation parameters, so that the external environment observed through the transparent screen is clearer.

[0052] like Fig.10a As shown in the related art, the light of the real external environment enters the user's eyes through the optical synthesizer 90, and the virtual image also enters the user's eyes through the optical synthesizer 90, so that the user sees the external environment and the virtual image at the same time, and the user perceives that the virtual image and the external environment are integrated. Fig.10b As shown, when the screen 403 provided by the present disclosure is not transparent, the real external environment is obtained through the image acquisition module 30 to obtain the external environment image, and then the virtual image is fused with the external environment image and displayed on the screen 403, and the user can also perceive that the virtual image is fused with the external environment. However, since the degree of transparency of the optical synthesizer 90 is usually difficult to achieve no light loss, when the light loss is more serious, the average light efficiency level may be only 0.3%-1%, which means that only a small part of the light can be transmitted from the outside to the user's eyes, resulting in insufficient image clarity, and due to the different diffraction angles of light of different wavelengths, the RGB ratio within the field of view angle and the eye box range is uneven, that is, a rainbow effect occurs, which distracts the user's attention and reduces the image quality. In addition, light loss and the difference in the refractive index of the material will affect the uniformity of the color, affecting the field of view and color uniformity.

[0053] Therefore, in order to solve the problems of poor image quality and lack of clarity caused by light loss, such as Fig.10c As shown, the curtain 403 is a transparent curtain, and the user can observe the real external environment through the curtain 403. In addition, the image acquisition module 30 also synchronously acquires the external real environment to obtain the external environment image. After the compensation parameters (such as brightness compensation, clarity compensation, saturation compensation, contrast compensation, etc.) are determined according to the light loss of the curtain, the virtual reality image is compensated according to the compensation parameters and displayed on the curtain 403. At this time, the user can see the superposition of the external environment image and the compensated virtual reality image through the curtain 403, so that the final displayed image has higher quality and higher clarity.

[0054] In addition, after obtaining the external environment image, such as Fig.10cAs shown in the figure, the virtual image can also be projected onto the screen 403 to achieve the fusion of the virtual image and the real environment image; it can also be that after the external environment image and the virtual image are fused, the fused virtual reality image is directly displayed on the screen 403 according to the compensation parameters.

[0055] Compared with an opaque screen, since the external environment can be seen through the screen, the virtual reality image projected on the screen compensates for the external environment. Therefore, the requirements for the reflection gain and resolution of the transparent screen are reduced.

[0056] Based on the same concept as the above-mentioned near-eye display device, the present disclosure also provides a near-eye display method. Fig.11 , which is a flow chart of a near-eye display method shown in the present disclosure, the method includes the following steps S1001 to S1004. In step S1001, an external environment image is acquired.

[0057] In step S1002, a virtual reality image is generated based on the external environment image.

[0058] In step S1003, a first magnified image after the virtual reality image is magnified by a first preset multiple is projected onto a screen.

[0059] In step S1004, a second enlarged image obtained by enlarging the enlarged image on the screen by a second preset multiple is projected to a preset distance for easy viewing by human eyes.

[0060] In this way, by increasing the size of the displayed image, the field of view angle, the exit pupil distance and the eye box range are correspondingly enlarged, and the user can flexibly choose to increase at least one of the field of view angle, the exit pupil distance and the eye box range as needed.

[0061] In some embodiments, before the first magnified image after the virtual reality image is magnified by the first preset multiple is projected onto the screen in the above step S1003, the first preset multiple is determined according to the observation distance.

[0062] In this way, the image projected on the screen observed by the end user is the same as the image of the external environment observed by the naked eye, that is, the device's visual magnification ratio is 1:1, and dizziness can also be avoided.

[0063] In some embodiments, before determining the first preset magnification according to the observation distance, the observation distance is determined according to the eye point focus position. The observation distance determined according to the eye point focus position is more accurate, so that the first preset magnification determined thereby is also more accurate.

[0064] In some embodiments, the near-eye display method further includes: determining the observation angle according to the eye point focus position; the above step S1002 generates a virtual reality image based on the external environment image, specifically generating virtual reality images of various resolutions based on the external environment image and the observation angle. In this way, the information of the user's focus point can be highlighted, and the information of the non-focus point can be weakened, which is closer to naked eye observation, thereby improving the user experience.

[0065] In some embodiments, the near-eye display method further includes: determining display parameters according to the number of blinks; the above step S1003 magnifies the virtual reality image by a first preset multiple and projects the first magnified image on the screen, specifically, magnifies the virtual reality image by a first preset multiple and projects the first magnified image on the screen according to the display parameters. In this way, when it is determined that the user's eyes are tired, the display parameters can be changed to relieve the user's eyestrain.

[0066] In some embodiments, the step S1001 of acquiring an external environment image includes acquiring an external environment image whose illumination is less than an illumination threshold, so as to ensure that various objects in the external environment can still be seen through the near-eye display device in a low illumination environment.

[0067] In some embodiments, the screen is a non-transparent screen; the above step S1002 generates a virtual reality image based on the external environment image, specifically generates a virtual reality image based on the external environment image and the virtual image. In this way, elements of the physical world and the digital world can be combined to provide users with a more immersive experience.

[0068] In some embodiments, the screen is a perspective screen; the above step S1003 projects the first enlarged image after the virtual reality image is enlarged by a first preset multiple onto the screen, specifically, the compensation parameter is determined according to the light loss of the screen; the first enlarged image after the virtual reality image displayed on the micro display is enlarged by a first preset multiple and then compensated according to the compensation parameter and projected onto the screen, so that the external environment observed through the perspective screen is clearer, thereby ensuring that the clarity and quality of the final displayed image are higher.

[0069] It should be noted that the executor of the above-mentioned near-eye display method can be the above-mentioned near-eye display device of the present application, or it can be a functional module and / or functional entity in the near-eye display device that can implement the near-eye display method, and the embodiments of the present application are not limited thereto.

[0070] The present disclosure also provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the near-eye display method described in the above embodiments.

[0071] The present disclosure also provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes to implement the near-eye display method described in the above-mentioned embodiments.

[0072] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned near-eye display method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0073] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0074] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices, servers and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0075] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0076] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0077] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0078] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by a general or special-purpose computer.

[0079] It should be noted that the technical solutions described in the present disclosure can be combined arbitrarily without conflict.

[0080] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A near-eye display device, characterized in that: The near-eye display device comprises: an image acquisition module, a projection module connected to the image acquisition module, and an eyepiece connected to the projection module, wherein the projection module comprises: a projection controller, a micro display and a screen; An image acquisition module is configured to acquire an image of the external environment; The projection module is configured to generate a virtual reality image based on an external environment image through a projection controller and display the image on a micro display, and to project a first magnified image after magnifying the virtual reality image displayed on the micro display by a first preset multiple onto a screen through the projection controller; The eyepiece is configured to project a second magnified image obtained by magnifying the first magnified image by a second preset multiple to a preset distance.

2. The near-eye display device according to claim 1, characterized in that: The near-eye display device further includes: an amplification ratio adjustment module connected to the projection module; The magnification ratio adjustment module is configured to determine the first preset multiple according to the observation distance.

3. The near-eye display device according to claim 2, characterized in that: The magnification ratio adjustment module includes: an eye tracker; The eye tracker is configured to determine the observation distance according to the eye focus position.

4. The near-eye display device according to claim 3, characterized in that: The eye tracker is further configured to determine the observation angle according to the eye focus position; The projection module is specifically configured to generate virtual reality images of various definition based on the external environment image and the observation angle and display them on the micro display.

5. The near-eye display device according to claim 3, characterized in that: The eye tracker is further configured to determine display parameters based on the number of blinks; The projection module is specifically configured to project a first magnified image after magnifying the virtual reality image displayed on the micro display by a first preset multiple through the projection controller onto the screen according to the display parameters.

6. The near-eye display device according to any one of claims 1 to 5, characterized in that: The image acquisition module includes: a night vision camera; The night vision camera is configured to collect images of the external environment with a light intensity less than a light intensity threshold.

7. The near-eye display device according to any one of claims 1 to 5, characterized in that: The curtain is an opaque curtain; The projection module is specifically configured to generate a virtual reality image based on the external environment image and the virtual image through a projection controller and display it on the micro display.

8. The near-eye display device according to any one of claims 1 to 5, characterized in that: The curtain is a perspective curtain; The projection module is specifically configured to determine the compensation parameter according to the light loss of the screen through the projection controller; A first magnified image after magnifying the virtual reality image displayed on the microdisplay by a first preset multiple is projected onto the screen after compensation according to the compensation parameter, so that the external environment observed through the perspective screen is clearer.

9. A near-eye display method, characterized in that: The method comprises: Acquire external environment images; Generate virtual reality images based on external environment images; Projecting a first magnified image after magnifying the virtual reality image by a first preset multiple onto a screen; A second magnified image after the magnified image on the screen is magnified by a second preset multiple is projected to a preset distance.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the near-eye display method according to claim 9 are implemented.

Images