Near-to-eye display equipment and image acquisition method and device of near-to-eye display equipment

Abstract

The invention relates to the technical field of near-to-eye display, and provides near-to-eye display equipment, and an image acquisition method and device of the near-to-eye display equipment, and the near-to-eye display equipment comprises a lens which comprises a display side; the optical fiber light supplementing module comprises an optical fiber and an optical fiber input module, and the optical fiber input module is connected with the light input end of the optical fiber; the optical fiber light supplementing module is used for providing incident light for the optical fiber through the optical fiber input module and providing emergent light for the display side of the lens through the optical fiber, so that the accuracy of the near-to-eye display equipment during eye movement tracking of the eyes of a user is improved, and the eyes of the user are prevented from being damaged.

Description

Technical Field

[0001] The present application relates to the field of near-eye display technology, and in particular to a near-eye display device, and an image acquisition method and apparatus for a near-eye display device. Background Art

[0002] At present, light-emitting diodes (LEDs), such as LED lamp beads, are usually set on the lenses of near-eye display devices to provide light to the user's eyes through the LED lamp beads, so as to track the user's eyes. Usually, in order to improve the accuracy of eye tracking, multiple LED lamp beads need to be arranged. However, the large size and wiring of LED lamp beads will obscure the information of the real scene outside. At the same time, since LED lamp beads generate a lot of heat and electromagnetic radiation, it is easy to cause damage to the user's eyes. Summary of the invention

[0003] The main purpose of the present application is to provide a near-eye display device, an image acquisition method and an apparatus for a near-eye display device, aiming to solve the technical problem that the near-eye display device has poor accuracy when tracking the user's eyes, causing damage to the user's eyes.

[0004] In a first aspect, the present application provides a near-eye display device, comprising:

[0005] A lens, the lens comprising a display side;

[0006] A fiber optic fill light module, the fiber optic fill light module comprises an optical fiber and an optical fiber input module, the optical fiber input module is connected to the light input end of the optical fiber; the fiber optic fill light module is used to provide incident light to the optical fiber through the optical fiber input module, and to provide outgoing light to the display side of the lens through the optical fiber.

[0007] In a second aspect, the present application further provides an image acquisition method for a near-eye display device, the image acquisition method comprising:

[0008] Providing outgoing light to the display side of the lens of the near-eye display device through the optical fiber fill light module of the near-eye display device, so as to provide outgoing light to the eyes of the user wearing the near-eye display device;

[0009] A target image is obtained by photographing the eyes of the user.

[0010] In a third aspect, the present application further provides an image acquisition device for a near-eye display device, the image acquisition device comprising:

[0011] Providing a module for providing outgoing light to the display side of the lens of the near-eye display device through the optical fiber fill light module of the near-eye display device, so as to provide the outgoing light to the eyes of the user wearing the near-eye display device;

[0012] The acquisition module is used to acquire a target image obtained by photographing the user's eyes.

[0013] The present application provides a near-eye display device, an image acquisition method and device for the near-eye display device, wherein the near-eye display device comprises: a lens, the lens comprises a display side; an optical fiber fill-light module, the optical fiber fill-light module comprises an optical fiber and an optical fiber input module, the optical fiber input module is connected to the optical input end of the optical fiber; the optical fiber fill-light module is used to provide an outgoing light to the optical fiber through the optical fiber input module, and to provide an outgoing light to the display side of the lens through the optical fiber. Based on the setting of the optical fiber fill-light module, when the eye movement of the user wearing the near-eye display device is tracked, the optical fiber fill-light module can provide an outgoing light to the optical fiber through the optical fiber input module, and provide an outgoing light to the display side of the lens through the optical fiber, which is equivalent to providing an outgoing light to the user's eye. The near-eye display device can capture the user's eye to obtain a target image. The target image can be used for the near-eye display device to track the eye movement information of the user's eye. For example, the target image includes a light spot formed by the user's eye reflecting the outgoing light, and the user's eye movement information can be determined based on the target image. Increasing the number of optical fibers is conducive to improving the accuracy of the near-eye display device when tracking the user's eye movement. In addition, the optical fiber fill light module provides outgoing light to the user's eyes. Since optical fiber basically does not generate heat or electromagnetic radiation, it can avoid damage to the user's eyes caused by the heat and electromagnetic radiation generated by LED lamp beads, such as increasing eye fatigue, dry eyes, tears and other problems, which is beneficial to improving the user's experience of near-eye display devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 is a structural schematic diagram of a near-eye display device provided in an embodiment of the present application;

[0016] Figure 2 is an application scenario diagram of a near-eye display device involved in an embodiment of the present application;

[0017] Figure 3 is an application scenario diagram of a near-eye display device involved in another embodiment of the present application;

[0018] Figure 4 It is a flowchart of an image acquisition method of a near-eye display device provided in an embodiment of the present application;

[0019] Figure 5 is a schematic block diagram of a target image involved in an embodiment of the present application;

[0020] Figure 6 is a schematic block diagram of a target image involved in another embodiment of the present application;

[0021] Figure 7 It is a schematic block diagram of an image acquisition device of a near-eye display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0024] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0025] See also Figure 1 , Figure 1 1 is a schematic diagram of the structure of a near-eye display device 100 provided in an embodiment of the present application. The near-eye display device 100 may include augmented reality (AR) glasses, virtual reality (VR) glasses, AR helmets, VR helmets, etc., which are not limited here.

[0026] like Figure 1 As shown, the near-eye display device 100 includes a lens 110, and the lens 110 includes a display side; a fiber optic fill light module 120, and the fiber optic fill light module 120 includes an optical fiber 121 and a fiber optic input module 123, and the fiber optic input module 123 is connected to the light input end of the optical fiber 121; the fiber optic fill light module 120 is used to provide incident light to the optical fiber 121 through the fiber optic input module 123, and to provide outgoing light to the display side of the lens 110 through the optical fiber 121.

[0027] In some embodiments, the fiber optic fill light module 120 can provide incident light to the optical fiber 121 through the optical fiber input module 123 , and the optical fiber 121 can use the incident light to provide outgoing light to the display side of the lens 110 .

[0028] For example, the optical fiber input module 123 provides incident light to the optical fiber 121 , and the incident light can be emitted after being propagated through the optical fiber 121 , refracted and collimated by the directional focusing module 122 , so as to provide output light to the display side of the lens 110 .

[0029] In some embodiments, when a user wears the near-eye display device 100, the display side of the lens 110 of the near-eye display device 100 corresponds to the side of the lens 110 close to the user's eye.

[0030] For example, the user's eye movement information can be determined through the near-eye display device 100. For example, the optical fiber 121 in the optical fiber fill light module 120 provides outgoing light to the display side of the lens 110. Since the display side of the lens 110 is the side of the lens 110 close to the user's eyes, the outgoing light can be emitted to the user's eyes. In the case where the outgoing light is emitted to the user's eyes, if a target image obtained by photographing the user's eyes is obtained, the target image may include a light spot formed by the user's eyes reflecting the outgoing light. Based on this, the target image can track the user's eyes. For example, based on the target image, the position offset of the light spot relative to the user's pupil is determined; based on the position offset, the user's eye movement information is determined.

[0031] When the optical fiber fill light module 120 is used to provide the outgoing light to the display side of the lens 110, the optical fiber 121 in the optical fiber fill light module 120 will not generate heat or electromagnetic interference in the process of propagating light, and can avoid causing damage to the user's eyes. Secondly, since the size of the optical fiber 121 is small, such as the diameter of a single-mode optical fiber can include 10 microns, etc., when the outgoing light is provided to the display side of the lens 110 through the optical fiber 121, the user cannot see the optical fiber 121, so that the setting of the optical fiber 121 will not block the light provided to the user's eyes by the real scene outside. Furthermore, the number of optical fibers 121 in the optical fiber fill light module 120 can be set. For example, by increasing the number of optical fibers 121, the accuracy of the near-eye display device 100 when tracking the user's eyes can be improved, while avoiding causing damage to the user's eyes. In addition, the optical fiber 121 can collimate the outgoing light provided to the display side of the lens 110, so that the outgoing light has a smaller divergence angle and improves the signal-to-noise ratio of the outgoing light. Based on this, the provision of the optical fiber fill light module 120 in the near-eye display device 100 is beneficial to improving the eye tracking accuracy of the near-eye display device 100 while helping to avoid harming the user's eyes.

[0032] Exemplarily, the fiber optic fill light module 120 further includes a directional focusing module 122 , which is connected to the light emitting end of the optical fiber 121 and is used to emit the light provided by the optical fiber 121 to provide the emitted light to the display side of the lens 110 .

[0033] For example, the fiber optic fill light module 120 can propagate light from the light incident end of the optical fiber 121 to the light emitting end of the optical fiber 121 through the optical fiber 121. Since the directional focusing module 122 is connected to the light emitting end of the optical fiber 121, the optical fiber 121 can provide light to the directional focusing module 122. The directional focusing module 122 can emit the light provided by the optical fiber 121 to provide the output light to the display side of the lens 110. For example, the light provided by the optical fiber 121 is emitted after being deflected and collimated by the directional focusing module 122 to provide the output light to the display side of the lens 110.

[0034] In some embodiments, the directional focusing module 122 can be arranged at the light emitting end of the optical fiber 121 (equivalent to the end of the optical fiber). For example, the optical fiber 121 can be arranged along the lens bracket 130 located at the edge of the lens 110, such as a lens frame. Among them, the optical fiber 121 can be arranged inside the lens frame, and the lens frame is provided with an output light channel corresponding to the optical fiber 121. For example, a through hole is provided on the lens frame, and the through hole can be used to lead the optical fiber 121 from the inside of the lens frame to the outside of the lens frame. The optical fiber 121 can provide output light to the display side of the lens 110 through the through hole. By arranging the directional focusing module 122 at the light emitting end of the optical fiber 121, the directional focusing module 122 can deflect and collimate the output light provided by the optical fiber 121 to provide output light to the display side of the lens 110.

[0035] In other embodiments, the directional focusing module 122 may be arranged on the display side of the lens 110. For example, the optical fiber 121 may be arranged inside the lens 110. Among them, an outgoing light channel corresponding to the optical fiber 121 is arranged on the lens 110. For example, a first through hole of the outgoing light channel is arranged on the edge end face of the lens 110, and a second through hole of the outgoing light channel is arranged on the display side of the lens 110, and a channel for accommodating the optical fiber 121 is provided between the second through hole and the second through hole. Through the first through hole, the optical fiber 121 can be buried inside the lens 110. Through the second through hole, the optical fiber 121 can be led out of the lens 110. Based on this, the directional focusing module 122 can be arranged at the position where the second through hole is located on the display side of the lens 110, or the directional focusing module 122 can be arranged on the display side of the lens 110, and the directional focusing module 122 can deflect and collimate the outgoing light provided by the optical fiber 121 to provide the outgoing light to the display side of the lens 110. Of course, it is not limited to this. For example, the non-display side of the lens 110 (i.e., the surface opposite to the display side of the lens 110) may be provided with a first through hole of the outgoing light channel, and the display side of the lens 110 may be provided with a second through hole of the outgoing light channel, and a channel for accommodating the optical fiber 121 may be provided between the first through hole and the second through hole. Based on this, a directional focusing module 122 may be provided at the position where the second through hole is located on the display side of the lens 110, or a directional focusing module 122 may be provided on the display side of the lens 110, and the directional focusing module 122 may deflect and collimate the outgoing light provided by the optical fiber 121 to provide the outgoing light to the display side of the lens 110. This is not limited here.

[0036] The directional focusing module 122 includes but is not limited to a lens, a metasurface, a Fresnel lens, and an optical antenna. The directional focusing module 122 may also use active optical devices, such as an optical phased array, a liquid lens, a phase change material, and a liquid crystal device.

[0037] In some embodiments, since the display side of the lens 110 is close to the user's eyes, when the directional focusing module 122 provides the output light to the display side of the lens 110, the directional focusing module 122 can focus the output light to the user's eyes. For example, the directional focusing module 122 can include a directional focusing lens. The directional focusing lens can perform directional and focusing processing on the light provided by the optical fiber 121 to focus the light provided by the optical fiber 121 to the user's eyes.

[0038] Based on setting a directional focusing module 122 at the light emitting end of the optical fiber 121, the directional focusing module 122 can direct and focus the outgoing light provided to the display side of the lens 110, so that the outgoing light can be emitted to the corresponding position of the user's eye for subsequent tracking of the user's eye movement information, which is beneficial to the subsequent calibration of the human eye information and improves the eye movement tracking accuracy of the near-eye display device 100.

[0039] See also Figure 2 , Figure 2 This is an application scenario diagram of the near-eye display device 100 involved in an embodiment of the present application.

[0040] like Figure 2 As shown, the number of optical fibers 121 in the optical fiber fill light module 120 can be one, and the number of directional focusing modules 122 can be one. The optical fiber input module 123 is connected to the optical input end of the optical fiber 121, and the optical output end of the optical fiber 121 is connected to the directional focusing module 122. The optical fiber input module 123 provides an optical signal to the optical fiber 121. The optical signal from the optical fiber input module 123 is transmitted to the setting position of the directional focusing module 122 through the optical fiber 121. After the optical signal is transmitted to the setting position of the directional focusing module 122, the directional focusing module 122 can perform directional and focusing processing on the optical signal provided by the optical fiber 121, and then provide outgoing light to the display side of the lens 110 to focus the optical signal on the user's eyes.

[0041] See also Figure 3 , Figure 3 This is an application scenario diagram of a near-eye display device 100 involved in another embodiment of the present application.

[0042] like Figure 3 As shown, the number of optical fibers 121 in the optical fiber fill light module 120 can be multiple, and the number of directional focusing modules 122 can be multiple. The optical fiber 121 input module is connected to the light input end of each optical fiber 121, and the light output end of each optical fiber 121 is connected to a directional focusing module 122. The setting positions of each directional focusing module 122 are different. The optical fiber input module 123 provides an optical signal to the optical fiber 121. The optical signals from the optical fiber 121 input module are each transmitted to the setting position of each directional focusing module 122 through a different optical fiber 121. Then, each directional focusing module 122 can respectively direct and focus the optical signal provided by the optical fiber 121, and provide different output light to the display side of the lens 110 to focus the optical signal to different positions of the user's eyes.

[0043] For example, the connection method between the optical fiber 121 and the directional focusing module 122 can be designed according to the number of optical fibers 121 and the number of directional focusing modules 122 .

[0044] In some embodiments, at least one directional focusing module 122 is connected to the light output end of an optical fiber 121 .

[0045] For example, the number of directional focusing modules 122 and the number of optical fibers 121 may correspond one to one, that is, one directional focusing module 122 is connected to the light output end of one optical fiber 121. In the case where the directional focusing modules 122 are set at different positions, if the optical signals are transmitted to different directional focusing modules 122 through different optical fibers 121, different directional focusing modules 122 can provide different optical signals to the display side of the lens 110 from different setting positions, so that different optical signals can be focused on different positions of the user's eyes.

[0046] In some embodiments, at least one directional focusing module 122 is connected to the light output ends of the plurality of optical fibers 121 , and the light output end of each optical fiber 121 is connected to a different position of the directional focusing module 122 .

[0047] For example, the number of directional focusing modules 122 and the number of optical fibers 121 may be in a one-to-many relationship, that is, one directional focusing module 122 is connected to the light output ends of multiple optical fibers 121. For example, the directional focusing module 122 may include a directional focusing lens array. A plurality of directional focusing lenses may be provided in the directional focusing lens array. Among them, the position interval between two directional focusing lenses in the directional focusing lens array is greater than or equal to the average diameter of each optical fiber 121. The light output end of each optical fiber 121 may be respectively connected to a directional focusing lens in the directional focusing lens array, so that the light output end of each optical fiber 121 is connected to a different position of the directional focusing module 122. In the case where the light output end of each optical fiber 121 is connected to a different position of the directional focusing module 122, the optical signal can be transmitted to the different positions of the directional focusing module 122 through different optical fibers 121. The directional focusing module 122 can provide different output light signals to the display side of the lens 110 from different positions, so that different light signals can be focused on different positions of the user's eyes.

[0048] In some embodiments, at least one optical fiber 121 includes multiple fiber splits, and at least one directional focusing module 122 is connected to the light output end of a fiber split of the optical fiber 121. For example, a main optical fiber can be used to split the main optical fiber by a coupler to obtain a fiber split.

[0049] For example, the number of directional focusing modules 122 and the number of fiber splittings of the optical fiber 121 may correspond one to one, that is, one directional focusing module 122 is connected to the light output end of one fiber splitting of the optical fiber 121. In the case where the setting positions of the directional focusing modules 122 are different, if the optical signals are transmitted to different directional focusing modules 122 through different fiber splittings, different directional focusing modules 122 can provide different optical signals to the display side of the lens 110 from different setting positions, so that different optical signals can be focused on different positions of the user's eyes.

[0050] In some embodiments, at least one optical fiber 121 includes multiple fiber optic bundles, at least one directional focusing module 122 is connected to the light output ends of the multiple fiber optic bundles of the optical fiber 121, and the light output end of each fiber optic bundle is connected to a different position of the directional focusing module 122.

[0051] For example, the number of directional focusing modules 122 and the number of fiber splits of the optical fiber 121 can be in a one-to-many relationship, that is, one directional focusing module 122 is connected to the light output ends of multiple fiber splits of the optical fiber 121. For example, the directional focusing module 122 may include a directional focusing lens array. A plurality of directional focusing lenses may be provided in the directional focusing lens array. Among them, the position interval between two directional focusing lenses in the directional focusing lens array is greater than or equal to the average diameter of each fiber split. The light output end of each fiber split can be connected to a directional focusing lens in the directional focusing lens array, so that the light output end of each fiber split is connected to a different position of the directional focusing module 122. In the case where the light output ends of each fiber split of the optical fiber 121 are connected to different positions of the directional focusing module 122, the optical signal can be transmitted to different positions of the directional focusing module 122 through different fiber splits. The directional focusing module 122 can provide different optical signals to the display side of the lens 110 from different positions, so that different optical signals can be focused on different positions of the user's eyes.

[0052] In this way, according to the number of optical fibers 121 and the number of directional focusing modules 122, the connection method between the optical fiber 121 and the directional focusing module 122 can be flexibly designed. Based on the different connection methods between the optical fiber 121 and the directional focusing module 122, the light can be focused to different positions of the user's eyes through the respective setting positions of different directional focusing modules 122, or the light can be focused to different positions of the user's eyes through different positions of the same directional focusing module 122, so as to provide for subsequent tracking of the user's eye movement information, which is beneficial to improving the convenience and flexibility of the near-eye display device 100 for tracking the user's eyes. Accordingly, when the light is focused to different positions of the user's eyes through the directional focusing module 122, the target image obtained by photographing the user's eyes includes light spots formed by the reflection of light at different positions of the user's eyes, and multiple light spots can be integrated to track the user's eye movement information, which is beneficial to improving the accuracy of the near-eye display device 100 for tracking the user's eyes.

[0053] In some embodiments, the interval between the connection positions of two optical fibers 121 connected to the same directional focusing module 122 is greater than or equal to the average diameter of the two optical fibers 121 .

[0054] For example, the directional focusing module 122 may include a directional focusing lens array. A plurality of directional focusing lenses are arranged in the directional focusing lens array, and the position interval between any two directional focusing lenses is greater than or equal to the average diameter of each optical fiber 121. Based on the setting of the position interval between the directional focusing lenses, sufficient positions can be reserved for directional focusing lenses with adjacent positions in the directional focusing lens array to respectively connect to the light output ends of different optical fibers 121. In the case where the same directional focusing module 122 is connected to the light output ends of multiple optical fibers 121, the interval between the connection positions where the light output ends of each optical fiber 121 are connected to the directional focusing module 122 is greater than or equal to the average diameter of each optical fiber 121. Accordingly, the interval between the connection positions of two optical fibers 121 connected to the same directional focusing module 122 is greater than or equal to the average diameter of the two optical fibers 121.

[0055] In some embodiments, the interval between the connection positions of two optical fiber bundles connected to the same directional focusing module 122 is greater than or equal to the average diameter of the two optical fiber bundles.

[0056] For example, the directional focusing module 122 may include a directional focusing lens array. A plurality of directional focusing lenses are arranged in the directional focusing lens array, and the position interval between any two directional focusing lenses is greater than or equal to the average diameter of each fiber optic beam. Based on the setting of the position interval between the directional focusing lenses, sufficient positions can be reserved for adjacent directional focusing lenses in the directional focusing lens array to respectively connect to the light output ends of different fiber optic beams. In the case where the same directional focusing module 122 is connected to the light output ends of multiple fiber optic beams, the interval between the connection positions where the light output ends of each fiber optic beam are connected to the directional focusing module 122 is greater than or equal to the average diameter of each fiber optic beam. Accordingly, the interval between the connection positions of two fiber optic beams connected to the same directional focusing module 122 is greater than or equal to the average diameter of the two fiber optic beams.

[0057] In some embodiments, the directional focus module 122 is disposed on the display side of the lens 110 .

[0058] For example, when the directional focusing module 122 is set on the display side of the lens 110, the optical fiber input module 123 can provide an optical signal to the optical fiber 121. The optical signal from the optical fiber input module 123 can be transmitted to a specified position on the display side of the lens 110 through the optical fiber 121, that is, the setting position of the directional focusing module 122 on the display side of the lens 110. Based on this, after the directional focusing module 122 performs directional and focusing processing on the optical signal provided by the optical fiber 121, the optical signal can be focused on the corresponding position of the user's eye. For example, different directional focusing modules 122 can each perform directional and focusing processing on the optical signal provided by the optical fiber 121, and focus the optical signal to different positions of the user's eye. For another example, after receiving the optical signal provided by the optical fiber 121, different positions of the same directional focusing module 122 can perform directional and focusing processing on the optical signal provided by the optical fiber 121, and focus the optical signal to different positions of the user's eye.

[0059] Among them, the color of the directional focusing module 122 can be the same or similar to the color of the lens 110. Then, setting the directional focusing module 122 on the display side of the lens 110 will not affect the appearance of the near-eye display device 100, and will not block the light provided to the user's eyes by the real external scene.

[0060] In some embodiments, the directional focusing module 122 is disposed on the display side close to the edge of the lens 110 .

[0061] In the prior art, when LED lamp beads are used to provide illumination to the user's eyes, the LED lamp beads are arranged on the lens 110. Since the color of the LED lamp beads is different from or not similar to the color of the lens 110, the arrangement of the LED lamp beads is likely to affect the user's visual experience of the near-eye display device 100. For example, the LED lamp beads may affect the appearance of the near-eye display device 100, and too many LED lamp beads may block the light provided to the user's eyes by the real scene outside.

[0062] Based on this, the setting position of the directional focusing module 122 on the display side can be limited to be close to the edge of the lens 110. Taking the color of the optical fiber 121 as an example, since the position of the directional focusing module 122 on the display side is close to the edge of the lens 110, it is possible to avoid setting the optical fiber 121 on the lens 110, and the user will not see the optical fiber 121 on the lens 110, so that the setting position of the optical fiber 121 will not interfere with the user's viewing of the real scene outside. In addition, since the color of the directional focusing module 122 is the same as or similar to the color of the lens 110, the setting position of the directional focusing module 122 on the display side will not interfere with the user's viewing of the real scene outside. In this way, it is conducive to improving the user's visual experience of the near-eye display device 100.

[0063] In some embodiments, the directional focusing module 122 is disposed in at least one of an upper region, a lower region, a left region, and a right region of the display side.

[0064] For example, the directional focusing module 122 can be set in at least one of the upper area, the lower area, the left area, and the right area of ​​the display side of the lens 110. Based on the different setting positions of the directional focusing module 122 on the display side of the lens 110, the directional focusing modules 122 at different setting positions can provide outgoing light to the display side of the lens 110, so as to focus on different positions of the user's eyes. When the user's eyes are tracked by the near-eye display device 100, the light spots corresponding to the light focused on different positions of the user's eyes can be comprehensively determined to determine the user's eye movement information. In this way, it is beneficial to improve the setting flexibility of the directional focusing module 122 while improving the eye tracking accuracy of the near-eye display device 100.

[0065] In some embodiments, the near-eye display device 100 further includes a lens holder 130 , and at least a portion of the optical fiber 121 is disposed on the lens holder 130 .

[0066] For example, the lens holder 130 may include any area of ​​the near-eye display device 100 except the lens 110, which is not limited here. Since the optical fiber 121 is highly flexible and can be easily bent or customized into various shapes, the shape of the optical fiber 121 can be designed according to the shape of the lens holder 130, so that at least a portion of the optical fiber 121 can be set on the lens holder 130. For example, at least a portion of the optical fiber 121 can be set on the surface of the lens holder 130, or can be set inside the lens holder 130, which is not limited here.

[0067] In this way, based on the fact that at least part of the optical fiber 121 is set on the lens bracket 130, the user will not see the optical fiber 121 on the lens 110, so that the setting position of the optical fiber 121 will not interfere with the user's viewing of the real scene outside, which is beneficial to improving the user's visual experience of the near-eye display device 100.

[0068] Exemplarily, the display side of the lens 110 includes a coupling-in region 111 and a coupling-out region 112 ; the coupling-in region 111 is used to couple the light provided by the optical module 140 into the lens 110 ; the coupling-out region 112 is used to couple out the light propagated by total reflection in the lens 110 .

[0069] like Figure 1 As shown, the display side of the lens 110 may include an incoupling region 111 and an outcoupling region 112. Figure 2 or Figure 3As shown, the optical module 140 can provide light to the coupling-in area 111. The coupling-in area 111 couples the light provided by the optical module 140 into the lens 110 to diffract the light and obtain diffracted light. When the diffraction angle of the diffracted light meets the total internal reflection condition of the lens 110, the diffracted light can be transmitted in the lens 110 until it is emitted to the user's eyes through the coupling-out area 112. For example, the light provided by the optical module 140 to the coupling-in area 111 is determined based on the virtual image provided by the optical module 140. After the coupling-out area 112 couples out the light that is totally reflected and propagated in the lens 110, the user can see the virtual image, thus completing the near-eye display of the virtual image.

[0070] Exemplarily, the near-eye display device 100 further includes a camera module 150. The camera module 150 may be disposed on the lens 110 or on the lens bracket 130, which is not limited herein.

[0071] like Figure 2 As shown, the camera module 150 can shoot the eyes of the user wearing the near-eye display device 100. When the near-eye display device 100 provides outgoing light to the user's eyes through the optical machine fill light module, the target image obtained by the camera module 150 shooting the user's eyes may include a light spot formed by the user's eyes reflecting the outgoing light. Based on this, the target image can be used to subsequently determine the user's eye movement information.

[0072] See also Figure 4 , Figure 4 1 is a flow chart of an image acquisition method for a near-eye display device provided in an embodiment of the present application. The image acquisition method can be applied to a near-eye display device, which may include AR glasses, VR glasses, AR helmets, VR helmets, etc., without limitation.

[0073] like Figure 4 As shown, the image acquisition method of the near-eye display device includes steps S101 to S102.

[0074] S101. Provide outgoing light to a display side of a lens of a near-eye display device through an optical fiber fill light module of the near-eye display device, so as to provide outgoing light to eyes of a user wearing the near-eye display device.

[0075] For example, the optical fiber in the optical fiber fill light module provides outgoing light to the display side of the lens. Since the display side of the lens is the side of the lens close to the eyes of the user wearing the near-eye display device, the outgoing light can be emitted to the user's eyes. After the user's eyes reflect the outgoing light emitted to the eyes, the user's eyes can form a light spot corresponding to the outgoing light. Based on this, the user's eye movement information can be determined according to the light spot corresponding to the outgoing light.

[0076] S102: Acquire a target image obtained by photographing the user's eyes.

[0077] For example, the near-eye display device may include a camera module. The camera module may be used to photograph the user's eyes. In some embodiments, when the near-eye display device provides outgoing light to the user's eyes through the optical fiber fill light module, the user's eyes may be photographed by the camera module to obtain a target image. The target image may include a light spot formed by the user's eyes reflecting the outgoing light. Figure 5 As shown, when an optical fiber or an optical fiber splitting beam of an optical fiber provides outgoing light to the user's eyes, the acquired target image may include a light spot. The light spot may be located at any position of the user's eyes, such as within the corresponding position range of any one of the user's pupil, cornea, and sclera, without limitation. Figure 6 As shown, when multiple optical fibers or multiple optical fiber splits of optical fibers provide outgoing light to the user's eyes, the acquired target image may include multiple light spots. The multiple light spots may be located at any position of the user's eyes, such as within the position range corresponding to at least one of the user's pupil, cornea, and sclera, without limitation herein.

[0078] In some embodiments, after acquiring the target image obtained by photographing the user's eyes, it also includes: determining, based on the target image, a position offset of a light spot corresponding to the outgoing light on the target image relative to the user's pupil; and determining the user's eye movement information based on the position offset.

[0079] For example, image recognition can be performed on the target image to determine the position of the user's pupil in the target image and the position of the light spot in the target image. Based on the position of the pupil in the target image and the position of the light spot in the target image, the position offset of the light spot relative to the user's pupil is determined. For example, the position offset in the X-axis direction can be obtained by subtracting the X-axis coordinate of the light spot in the coordinate system corresponding to the target image from the X-axis coordinate of the pupil in the coordinate system corresponding to the target image. The position offset in the Y-axis direction can be obtained by subtracting the Y-axis coordinate of the light spot in the coordinate system corresponding to the target image from the Y-axis coordinate of the pupil in the coordinate system corresponding to the target image. This is not limited here.

[0080] In the case where the target image includes multiple light spots, the position offset of each light spot relative to the user's pupil is determined respectively, and the user's eye movement information is determined according to the average value of the position offset of each light spot relative to the user's pupil. Of course, it is not limited to this. In the case where the target image includes multiple light spots, the average value of the positions of all light spots in the target image can be determined. According to the average value of the positions of all light spots in the target image and the position of the pupil in the target image, the position offset of the light spot relative to the user's pupil is determined, thereby determining the user's eye movement information, which is not limited here.

[0081] In some implementations, when determining the user's eye movement information, factors such as pupil movement speed, acceleration, duration, etc. may be comprehensively considered. For example, the user's eye movement information may be determined through target images collected at different time frames.

[0082] For example, when the difference between the position offsets corresponding to the target images of at least one continuous adjacent time frame is less than or equal to the first preset difference, it is determined that the user's eyes are in a gaze state. For example, it can be determined that the user is gazing or focusing on the position of the light spot on the target image.

[0083] For example, when the position offset corresponding to each target image of at least one continuous adjacent time frame is greater than a first preset difference and less than or equal to a second preset difference, it is determined that the user's eyes are in a moving tracking state. For example, it can be determined that the user is tracking or observing a moving object at the position of the light spot on the target image.

[0084] For example, when the position offset corresponding to each of the target images of at least one continuous adjacent time frame is greater than the second preset difference, it is determined that the user's eyes are in a scanning state. For example, it can be determined that the user is scanning different positions on the screen of the near-eye display device or in the virtual environment.

[0085] When the user's eye movement information is determined, the user's eye movement information can be applied to different scenarios, such as designing a human-computer interaction interface for a near-eye display device, learning the user's psychology of using the near-eye display device, etc., without limitation here.

[0086] The above-mentioned embodiment provides an image acquisition method for a near-eye display device, and the image acquisition method includes: providing outgoing light to the display side of the lens of the near-eye display device through the fiber optic fill light module of the near-eye display device, so as to provide outgoing light to the eyes of a user wearing the near-eye display device to obtain a target image obtained by photographing the eyes of the user, so as to improve the eye movement tracking accuracy of the near-eye display device, improve the accuracy of determining the user's eye movement information, and avoid harming the user's eyes.

[0087] See also Figure 7 , Figure 7 It is a schematic block diagram of an image acquisition device of a near-eye display device provided in an embodiment of the present application.

[0088] like Figure 7 As shown, the image acquisition device of the near-eye display device includes: a providing module 210 and an acquisition module 220.

[0089] A module 210 is provided, which is used to provide outgoing light to the display side of the lens of the near-eye display device through the optical fiber fill light module of the near-eye display device, so as to provide outgoing light to the eyes of the user wearing the near-eye display device.

[0090] The acquisition module 220 is used to acquire a target image obtained by photographing the user's eyes.

[0091] It should be noted that technicians in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned image acquisition method embodiment of the near-eye display device, and will not be repeated here.

[0092] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0093] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the image acquisition method of the near-eye display device of the present application.

[0094] The computer-readable storage medium may be an internal storage unit of the near-eye display device described in the aforementioned embodiment, such as a hard disk or memory of the near-eye display device. The computer-readable storage medium may also be an external storage device of the near-eye display device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the near-eye display device.

[0095] It should be understood that the terms used in this specification of the application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification of the application and the attached claims, unless the context clearly indicates otherwise, the singular forms of "one", "an" and "the" are intended to include plural forms. It should also be understood that the term "and / or" used in the specification of the application and the attached claims refers to any combination of one or more of the items listed in association and all possible combinations, and includes these combinations. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements includes not only those elements, but also includes other elements that are not clearly listed, or also includes elements inherent to such process, method, article or system. In the absence of more restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or system including the element.

[0096] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A near-eye display device, characterized in that: include: A lens, the lens comprising a display side; A fiber optic fill light module, the fiber optic fill light module comprises an optical fiber and an optical fiber input module, the optical fiber input module is connected to the light input end of the optical fiber; the fiber optic fill light module is used to provide incident light to the optical fiber through the optical fiber input module, and to provide outgoing light to the display side of the lens through the optical fiber.

2. The near-eye display device according to claim 1, characterized in that: The optical fiber fill light module also includes a directional focusing module, which is connected to the light emitting end of the optical fiber and is used to emit the light provided by the optical fiber to provide the emitted light to the display side of the lens.

3. The near-eye display device according to claim 2, characterized in that: Include at least one of the following: At least one of the directional focusing modules is connected to the light output end of one of the optical fibers; At least one of the directional focusing modules is connected to the light output ends of the plurality of optical fibers, and the light output ends of the optical fibers are connected to different positions of the directional focusing module; At least one of the optical fibers comprises a plurality of optical fiber splits, and at least one of the directional focusing modules is connected to a light output end of one of the optical fiber splits of the optical fiber; At least one of the optical fibers includes multiple fiber bundles, at least one of the directional focusing modules is connected to the light output ends of the multiple fiber bundles of the optical fiber, and the light output ends of the fiber bundles are connected to different positions of the directional focusing module.

4. The near-eye display device according to claim 3, characterized in that: Include at least one of the following: The interval between the connection positions of two optical fibers connected to the same directional focusing module is greater than or equal to the average diameter of the two optical fibers; The interval between the connection positions of two optical fiber split beams connected to the same directional focusing module is greater than or equal to the average diameter of the two optical fiber split beams.

5. The near-eye display device according to claim 2, characterized in that: The directional focusing module is arranged on the display side of the lens.

6. The near-eye display device according to claim 5, characterized in that: The directional focusing module is disposed on the display side close to the edge of the lens.

7. The near-eye display device according to claim 5, characterized in that: The directional focusing module is arranged in at least one of the upper area, the lower area, the left area and the right area of ​​the display side.

8. The near-eye display device according to any one of claims 1 to 7, characterized in that: The near-eye display device also includes a lens bracket, and at least a portion of the optical fiber is arranged on the lens bracket.

9. A method for acquiring an image of a near-eye display device, characterized in that: The image acquisition method comprises: Providing outgoing light to the display side of the lens of the near-eye display device through the optical fiber fill light module of the near-eye display device, so as to provide outgoing light to the eyes of the user wearing the near-eye display device; A target image is obtained by photographing the eyes of the user.

10. An image acquisition device for a near-eye display device, characterized in that: The image acquisition device comprises: Providing a module for providing outgoing light to the display side of the lens of the near-eye display device through the optical fiber fill light module of the near-eye display device, so as to provide the outgoing light to the eyes of the user wearing the near-eye display device; The acquisition module is used to acquire a target image obtained by photographing the user's eyes.

Images