Virtual scene display method and intelligent glasses

Abstract

The invention provides a virtual scene display method and intelligent glasses, relates to the technical field of optics, and aims to solve the problem that the real scene depth and the virtual scene depth are inconsistent. The virtual scene display method provided by the invention comprises the steps of obtaining a depth value of a real scene, obtaining an interpupillary distance between a left eye and a right eye, and obtaining an actual parallax angle for observing the real scene according to the depth value and the interpupillary distance; the initial parallax angle of the virtual scene is obtained, a parallax angle difference value is obtained according to the initial parallax angle and the actual parallax angle, and the virtual scene positions projected to the left eye and the right eye are moved according to the parallax angle difference value, so that the virtual parallax angle of the virtual scene is consistent with the actual parallax angle of the real scene. In the embodiment provided by the invention, when the virtual parallax angle and the actual parallax angle are basically the same, the user can simultaneously observe the clear real scene and the clear virtual scene under the condition that the eyes are not adjusted to be gathered and scattered, so that the bad problems that the eyes are not fused and the visual fatigue is caused can be effectively avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical technology, in particular to a virtual scene display method and smart glasses. BACKGROUND

[0002] Currently, the fusion position of the augmented reality (AR) glasses is aligned to a fixed fusion position by active alignment (AA) before leaving the factory, for example, the depth of the virtual scene is actively aligned to a position of about 4m. However, the AR glasses need to consider the application scenarios of daily wearing. In actual use, the human eyes will fixate on real scenes at different distances, and the depths of the real scenes at different distances are different. If the depth of the real scene and the depth of the virtual scene are inconsistent, the human eyes will switch back and forth between the real scene and the virtual scene, and the eyes need to be adjusted frequently, which can easily cause binocular non-fusion, visual fatigue and discomfort and other adverse problems.

[0003] Therefore, how to improve the consistency of the depth of the real scene and the depth of the virtual scene has become a technical problem to be solved. SUMMARY

[0004] The present application provides a virtual scene display method and smart glasses which can effectively improve the inconsistency of the depth of the real scene and the depth of the virtual scene.

[0005] In a first aspect, the present application provides a virtual scene display method, which can include: obtaining a depth value of a real scene, and obtaining a pupil distance between a left eye and a right eye. An actual parallax angle of observing the real scene is obtained according to the depth value and the pupil distance. The method further includes: obtaining an initial parallax angle of a virtual scene. A parallax angle difference value is obtained according to the initial parallax angle and the actual parallax angle. The position of the virtual scene projected to the left eye and the right eye is moved according to the parallax angle difference value, so that the virtual parallax angle of the virtual scene is consistent with the actual parallax angle of the real scene. Alternatively, it can be understood that the actual parallax angle of the user's eyes is kept unchanged when observing the real scene at a specific real scene depth. That is, the actual parallax angle corresponds to the real scene depth. Correspondingly, the virtual parallax angle is also kept unchanged when the user's eyes observe the virtual scene at a specific virtual scene depth. That is, the virtual parallax angle corresponds to the virtual scene depth. Therefore, when observing the images at the same real scene depth and virtual scene depth, the virtual parallax angle and the actual parallax angle are basically the same. That is, when the virtual parallax angle and the actual parallax angle are basically the same, the user's eyes can observe clear real scene and virtual scene at the same time without adjusting the convergence of the eyes, thereby effectively avoiding adverse problems such as binocular non-fusion and visual fatigue.

[0006] In actual application, the real scene depth of different regions in the real scene can be different. For example, assuming that the real scene is a tree, the tree is a three-dimensional landscape in space, and the distance between different regions in the tree and the human eyes is different, so the real scene depth of different regions in the real scene is different.

[0007] In an example, the acquiring the depth value of the real scene in the method above can include: acquiring the depth value of a plurality of regions of the real scene.

[0008] It can be understood that the depth values of different regions in the real scene can be different, and thus the parallax angles of the two eyes when observing different regions in the real scene are different. By acquiring the depth values of different regions in the real scene, a more accurate actual parallax angle can be obtained.

[0009] That is, the actual parallax angle corresponding to the region observed by the two eyes can be obtained by inferring the region of the real scene observed by the two eyes, so that the virtual scene is moved to a position more accurately, so that the virtual parallax angle of the virtual scene is consistent with the actual parallax angle of the real scene.

[0010] When inferring the region of the real scene observed by the two eyes, a plurality of different methods can be used.

[0011] For example, in an example provided in the present application, the observation direction of the user can be acquired. According to the observation direction, a direct observation region in the plurality of regions that is directly opposite to the observation direction is obtained. According to the depth value of the direct observation region, the depth values of other regions, and the interpupillary distance, the actual parallax angle is calculated.

[0012] Alternatively, it can be understood that by acquiring the general observation direction of the user, the actual observation region (i.e., the direct observation region) of the user can be determined. In actual application, the depth value of the direct observation region can be used as the depth value of the real scene. Alternatively, the depth values of the direct observation region and other regions near the direct observation region can be reasonably calculated and used as the depth value of the real scene. Then, the actual parallax angle is calculated in combination with the interpupillary distance.

[0013] When calculating the depth value of the real scene, the depth value of the direct observation region can be given a larger weight, and the depth values of other regions can be given a smaller weight, so as to obtain the depth value of the real scene. The direct observation region and other regions can be calculated according to a normal distribution from the center to the edge of the direct observation region. Alternatively, other step weights can be used for calculation.

[0014] In an example, the actual observation region of the two eyes when observing the real scene can also be determined by the actual gaze direction of the eyes.

[0015] For example, the method can include: acquiring the gaze direction of the eyes, obtaining a gaze region in the plurality of regions according to the gaze direction, and obtaining the actual parallax angle of the observed real scene according to the depth value of the gaze region and the interpupillary distance.

[0016] In a specific application, the depth value of the gaze region can be taken as the real scene depth value, or a larger weight can be applied to the gaze region and a smaller weight can be applied to other regions around the gaze region according to a normal distribution or the like.

[0017] In obtaining the gaze direction of the eye, the actual gaze direction of the left eye can be obtained to infer the actual gaze direction of the two eyes. Alternatively, the actual gaze direction of the right eye can be obtained to infer the actual gaze direction of the two eyes. Alternatively, the actual gaze directions of the left eye and the right eye can be obtained to obtain the actual gaze direction of the two eyes.

[0018] In an example, the moving the virtual scene position projected to the left eye according to the left parallax value and the moving the virtual scene position projected to the right eye according to the right parallax value in the above method can include: adjusting the display position of the optical system to move the virtual scene position projected to the left eye according to the left parallax value. Adjusting the focal length of the optical system to move the virtual scene position projected to the right eye according to the right parallax value, so as to adjust the fusion position of the virtual scene.

[0019] In a second aspect, the application further provides an intelligent glasses, comprising: an optical system, a depth ranging element, a pupil distance ranging element and a processor.

[0020] The optical system is used to display a virtual scene to the left eye and the right eye. The depth ranging element is used to obtain a depth value of a real scene. The pupil distance ranging element is used to obtain a pupil distance between the left eye and the right eye. The processor is used to obtain an actual parallax angle of the observed real scene according to the depth value and the pupil distance, and obtain a left parallax value of the virtual scene projected to the left eye and a right parallax value of the virtual scene projected to the right eye according to the initial parallax angle and the actual parallax angle. The processor is further used to move the virtual scene position projected to the left eye by the optical system according to the left parallax value, and move the virtual scene position projected to the right eye by the optical system according to the right parallax value, so as to make the virtual parallax angle of the virtual scene consistent with the actual parallax angle of the real scene.

[0021] Alternatively, it can be understood that when the user wears the intelligent glasses and observes a real scene at a specific real scene depth, the actual parallax angle remains unchanged. That is, the actual parallax angle corresponds to the real scene depth. Correspondingly, when the user's two eyes observe a virtual scene at a specific virtual scene depth, the virtual parallax angle also remains unchanged. That is, the virtual parallax angle corresponds to the virtual scene depth. Therefore, when observing the same image at the same real scene depth and virtual scene depth, the virtual parallax angle and the actual parallax angle are basically the same. That is, when the virtual parallax angle and the actual parallax angle are basically the same, the user's two eyes can observe clear real scene and virtual scene at the same time without adjusting the eye convergence and divergence, thereby effectively avoiding the problems of binocular non-fusion, visual fatigue and the like.

[0022] In actual application, the real scene depth of different regions in the real scene can be different. For example, assuming that the real scene is a tree, the tree is a three-dimensional landscape in space, and the distance between different regions in the tree and the human eye is different, so the real scene depth of different regions in the real scene is different.

[0023] In an example, the depth value of the real scene of the real scene of the depth measurement element equipped in the smart glasses can also be obtained.

[0024] It can be understood that the depth value of different regions in the real scene can be different, so the parallax angle of the eyes when observing different regions in the real scene is different. By obtaining the depth value of different regions in the real scene, a more accurate actual parallax angle can be obtained.

[0025] That is, the actual parallax angle corresponding to the region of the real scene observed by the eyes can be obtained by inferring the region of the real scene observed by the eyes, so that the virtual scene position projected to the left eye and the right eye is moved more accurately, and the virtual parallax angle of the virtual scene is consistent with the actual parallax angle of the real scene.

[0026] When inferring the region of the real scene observed by the eyes, it can be obtained by a variety of different methods.

[0027] For example, in an example, a posture sensor can be equipped in the smart glasses. The posture sensor is used to obtain the observation direction of the smart glasses. The processor is also used to obtain a direct observation region in the multiple regions that is directly opposite the observation direction according to the observation direction, and to calculate the actual parallax angle according to the depth value of the direct observation region, the depth value of the other regions, and the interpupillary distance.

[0028] Or it can be understood that in actual use, the smart glasses will move synchronously with the head when the head is in motion. The posture sensor can infer the posture of the user's head by detecting the posture of the smart glasses, so as to infer the observation direction of the user.

[0029] In an example, the actual observation region of the eyes when observing the real scene can also be determined by the actual gaze direction of the eyes.

[0030] For example, the smart glasses also include an eye movement tracking element. The eye movement tracking element is used to obtain the gaze direction of the eyes, and the processor is also used to obtain a gaze region in the multiple regions according to the gaze direction, and to obtain the actual parallax angle of the observed real scene according to the depth value of the gaze region and the interpupillary distance.

[0031] In specific application, the depth value of the gaze region can be taken as the real scene depth value, or the gaze region can be given a larger weight according to normal distribution and the like, and other regions around the gaze region can be given a smaller weight.

[0032] In acquiring the gaze direction of the eye, the actual gaze direction of the left eye can be acquired to infer the actual gaze direction of the binoculars. Alternatively, the actual gaze direction of the right eye can also be acquired to infer the actual gaze direction of the binoculars. Alternatively, the actual gaze direction of the left eye and the right eye can also be acquired to obtain the actual gaze direction of the binoculars. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A stereoscopic structure schematic diagram of an intelligent glasses provided by an embodiment of the present application;

[0034] Figure 2 An imaging schematic diagram of an intelligent glasses provided by an embodiment of the present application;

[0035] Figure 3 A flowchart of a virtual scene display method provided by an embodiment of the present application;

[0036] Figure 4 An imaging schematic diagram of an intelligent glasses provided by an embodiment of the present application;

[0037] Figure 5 Another imaging schematic diagram of an intelligent glasses provided by an embodiment of the present application;

[0038] Figure 6 An imaging schematic diagram of an intelligent glasses provided by an embodiment of the present application;

[0039] Figure 7 Another imaging schematic diagram of an intelligent glasses provided by an embodiment of the present application;

[0040] Figure 8 A schematic diagram of a real scene observation area provided by an embodiment of the present application;

[0041] Figure 9 Another schematic diagram of a real scene observation area provided by an embodiment of the present application;

[0042] Figure 10 A structure block diagram of an intelligent glasses provided by an embodiment of the present application;

[0043] Figure 11 A structure schematic diagram of an intelligent eye provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the accompanying drawings.

[0045] In order to facilitate the understanding of the virtual scene display method provided by the embodiments of the present application, the application scenario thereof will be introduced first.

[0046] The virtual scene display method provided by the embodiments of the present application can be applied in an augmented reality (AR) device. For example, the method can be applied in an electronic device such as AR glasses, an AR helmet, mixed reality (MR) glasses or an MR helmet, which combines digital content and a real scene together.

[0047] To facilitate understanding of the technical solutions of the present application, in the following examples, the electronic device is taken as an example of AR glasses for illustrative description.

[0048] As shown in Figure 1 , Fig. 1 is a structural schematic diagram of AR glasses provided by an embodiment of the present application. Figure 1

[0049] The AR glasses 10 include a frame 11, lenses and an optical system (not shown in the figure). The frame 11 includes a frame 111, a temple 112 and a temple 113. The temples 112 and 113 are important components of the AR glasses and can form a mechanical main structure of the AR glasses 10 together with the frame 111. A user can wear the AR glasses 10 through the structural frame formed by the temples 112, 113 and the frame 111. In the example shown in Fig. 1, the AR glasses 10 include two lenses, namely a lens 121 and a lens 122. Figure 1 The main function of the frame 111 is to provide an effective mounting space for the lenses 121 and 122. The lenses 121 and 122 can be plane lenses, concave lenses, convex lenses or other types of lenses. After the user wears the AR glasses 10, the frame 111 can be located in front of the eyes, the lenses 121 and 122 are respectively opposite to the left eye and the right eye of the user, and the eyes can observe the real scene in the environment through the lenses 121 and 122. The temples 112 and 113 can be respectively arranged above the ears of the user to ensure the stability of wearing. Of course, in some examples, the lenses 121 and 122 can also be a larger lens, a part of which can be used as the lens 121 and the other part can be used as the lens 122. In addition, the frame 11 can also be configured with a nose pad to ensure the stability of wearing the AR glasses 10, which is not described herein.

[0050] ​In addition, the AR glasses 10 provided in this application are also equipped with an optical system. The optical system generates virtual images and projects them to the user's left and right eyes respectively. In practical applications, the optical system may include an image transmitter such as a projector for generating images. The optical system may also include optical elements such as lenses and diffractors for processing images by focusing, refracting, etc. In summary, in the optical system, the image transmitter converts digital or analog signals into light signals, and the optical elements process the light signals by focusing and refracting them before transmitting them to the user's left and right eyes, thereby displaying a virtual image to the user. In specific configurations, the optical system can adopt commonly used types; this application does not limit the specific structural type or display principle of the optical system.

[0051] For example, such as Figure 2 As shown, in one example provided in this application, the virtual image generated by the optical system can be displayed in lenses 121 and 122. Specifically, the left virtual image can be projected onto the user's (wearer's) left eye through lens 121, and the right virtual image can be projected onto the user's (wearer's) right eye through lens 122. After the left and right virtual images are fused at the virtual image depth position, the user can observe a stereoscopic virtual image through both eyes. In addition, light from the external environment can also be projected onto the user's left eye through lens 121 and lens 122, allowing the user to observe the real scene in the external environment through lenses 121 and 122.

[0052] In summary, when users wear AR glasses 10, they can observe the real scene in the environment through the lenses, or they can observe virtual images through the optical system.

[0053] In practical applications, to ensure the imaging accuracy and effect of the optical system, it is necessary to correct the position, orientation, and other parameters of the components such as the image transmitter and lenses. During correction, the fusion position (i.e., virtual image depth) of the optical system needs to be determined so that the optical system can produce a clear virtual image at that fusion position. For example, before leaving the factory, the AR glasses 10 are actively aligned (AA) to a fixed fusion position, such as actively aligning the virtual image fusion depth to approximately 4m. Of course, in other examples, the virtual image fusion depth can also be actively aligned to approximately 6m or 10m. Here, the virtual image fusion depth refers to the distance of the virtual image perceived by the user's eyes when observing the virtual image fusion.

[0054] Since the AR glasses 10 have determined the virtual scene depth before leaving the factory, when the real scene depth observed by the user is consistent with the virtual scene depth, such as both the virtual scene depth and the real scene depth being about 4m, the human eye can simultaneously observe the real scene and the virtual scene fusion image more clearly. However, in the actual use scene of the user, the human eye will fixate on real scenes at different distances, and the real scene depths at different distances are different. If the real scene depth and the virtual scene depth are inconsistent, the human eye will switch back and forth between the real scene and the virtual scene fusion image, and the accommodation of the eyes needs to be adjusted frequently, which is easy to cause ghosting, binocular non-fusion image, visual fatigue and discomfort, and other adverse problems.

[0055] For example, as shown in Figure 2 In an example provided by the present application, the real scene depth is about 2m, and the virtual scene depth is about 4m. After the user wears the AR glasses 10, when observing the virtual scene fusion image, the user cannot observe the clear real scene. Correspondingly, when observing the real scene, the user cannot observe the clear virtual scene fusion image. Therefore, in order to be able to observe the clear real scene and virtual scene fusion image, the accommodation of the eyes needs to be adjusted, which is easy to cause visual fatigue. In addition, when observing the real scene, ghosting of the virtual scene will occur, that is, the virtual scene does not fuse, which affects the use experience.

[0056] Therefore, the embodiment of the present application provides a virtual scene display method and intelligent glasses which can effectively improve the inconsistency problem of the real scene depth and the virtual scene depth.

[0057] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0058] As shown in Figure 3 In an example provided by the present application, the virtual scene display method can include:

[0059] Step S100: obtaining the depth value of the real scene.

[0060] Step S200: obtaining the interpupillary distance between the left eye and the right eye.

[0061] Step S300: obtaining the actual parallax angle of observing the real scene according to the depth value and the interpupillary distance.

[0062] Step S400: obtaining the initial parallax angle of the virtual scene.

[0063] Step S500: obtaining the parallax angle difference value according to the initial parallax angle and the actual parallax angle.

[0064] Step S600: moving the virtual scene position projected to the left eye and the right eye according to the parallax angle difference value, so that the virtual parallax angle of the virtual scene is consistent with the actual parallax angle of the real scene.

[0065] Specifically, please refer to Figure 3 and Figure 4 .

[0066] In step S100, the depth value of the real scene can be understood as the distance L between the real scene and the human eye. In actual application, the distance L can be measured by a depth ranging element capable of detecting distance, such as a radar or a time of flight (TOF) sensor. The distance L can be actually measured or obtained by combining measurement and calculation.

[0067] For example, the depth ranging element 13 can be installed in the frame 11, and the depth ranging element 13 can measure the actual distance between the depth ranging element 13 and the real scene. In addition, the wearing data can also be stored in the memory of the smart glasses 10, and the distance between the depth ranging element 13 and the human eye can be included in the wearing data, so that the actual distance between the human eye and the real scene can be obtained. The distance between the depth ranging element 13 and the human eye included in the wearing data can be set before the factory. For example, the distance between the depth ranging element 13 and the human eye can be measured when different wearers wear the smart glasses 10, and the average of the plurality of different distances can be used as the distance between the depth ranging element 13 and the human eye.

[0068] Alternatively, in some examples, other ranging elements can also be provided in the frame 11, so as to measure the actual distance between the wearer's eye and the ranging element 13, and the distance L between the real scene and the human eye can be calculated by combining the distance between the ranging element 13 and the depth ranging element.

[0069] In step S200, the interpupillary distance IPD between the left eye and the right eye can be measured by the interpupillary distance ranging element installed in the frame 11. The device for measuring the interpupillary distance can be reasonably selected according to the currently commonly used type, which is not described here.

[0070] In step S300, the actual parallax angle of the observed real scene can be obtained according to the depth value and the interpupillary distance, which can specifically include that the depth ranging element 13 can send the detected depth value signal to the processor. The interpupillary distance ranging element can send the detected interpupillary distance signal to the processor, and the processor can perform corresponding calculation according to the depth value signal and the interpupillary distance signal, so as to obtain the field of view angle θ1 of the two eyes when observing the real scene.

[0071] For example,

[0072] In step S400, the initial parallax angle θ0 of the virtual scene can be obtained, which can specifically include that the initial parallax angle θ0 can be included in the wearing data.

[0073] In actual applications, in order to ensure the imaging accuracy and effect of the optical system, the position, posture and other parameters of the image emitter, lens and other components in the optical system need to be corrected. When the correction is performed, the fusion image position (i.e. virtual depth) of the optical system needs to be determined, so that the optical system can generate a clear virtual depth fusion image at the fusion image position. For example, the AR glasses 10 will be actively aligned (AA) to a certain fixed fusion image position before leaving the factory, such as aligning the depth of the virtual depth fusion image to a position of about 4 m, that is, the initial parallax angle θ0 is one of the parameters determined before the AR glasses leave the factory and stored in the memory.

[0074] Therefore, in this step, the processor can directly obtain the initial parallax angle θ0 stored in the memory.

[0075] In step S500, the parallax angle difference Δθ is obtained according to the initial parallax angle and the actual parallax angle, and specifically is:

[0076] Δθ = θ0- θ1.

[0077] In step S600, the virtual depth positions projected to the left eye and the right eye are moved according to the parallax angle difference, so that the virtual parallax angle of the virtual depth is consistent with the actual parallax angle of the real depth, and specifically can include: adjusting the imaging position of the left virtual depth and the imaging position of the right virtual depth according to the parallax angle difference Δθ.

[0078] For example, please refer to Figure 4 and Figure 5 . Figure 4 is a schematic diagram of imaging before adjustment, Figure 5 is a schematic diagram of imaging after adjustment. The imaging position of the left virtual depth on the lens 121 can be adjusted, so that the line of sight of the left eye when observing the left virtual depth is consistent with the line of sight when observing the real depth. Correspondingly, the imaging position of the right virtual depth on the lens 122 can be adjusted, so that the line of sight of the right eye when observing the right virtual depth is consistent with the line of sight when observing the real depth.

[0079] In addition, please refer to Figure 6 and Figure 7 . Figure 6 is a schematic diagram of imaging before adjustment, Figure 7 is a schematic diagram of imaging after adjustment. The imaging position of the left virtual depth can be adjusted, so that the line of sight of the left eye when observing the left virtual depth is consistent with the line of sight when observing the real depth. Correspondingly, the imaging position of the right virtual depth can be adjusted, so that the line of sight of the right eye when observing the right virtual depth is consistent with the line of sight when observing the real depth.

[0080] It should be noted that in the above examples, the human eye observing the real depth and the virtual depth at the front position is exemplarily described. When the human eye observes the real depth and the virtual depth at the left front or the right front, it is also applicable. It should be noted that in the above examples, the human eye observing the real depth and the virtual depth at the front position is exemplarily described. When the human eye observes the real depth and the virtual depth at the left front or the right front, it is also applicable.

[0081] Or it can be understood that when the user's eyes are in the same depth of the real scene when looking at or looking at the real scene, the parallax angle θ1 is basically the same. That is, the real scene depth and the parallax angle θ1 are consistent. Correspondingly, when the user's eyes are in the same depth of the virtual scene when looking at or looking at the virtual scene, the parallax angle θ0 is also basically the same. That is, the virtual scene depth and the parallax angle θ0 are consistent. Therefore, after adjusting the parallax angle θ0 of the virtual scene to be the same as the parallax angle θ1 of the real scene, the real scene depth and the virtual scene depth can be kept consistent, which can avoid the user adjusting the convergence of the eyes, thereby preventing visual fatigue, ghosting, virtual scene not blending, and other adverse conditions.

[0082] In order to facilitate understanding of the technical solutions of the present application, in the following examples, the user's eyes looking at the real scene and the virtual scene will be exemplarily illustrated.

[0083] When adjusting the positions of the left virtual scene and the right virtual scene, the imaging position of the optical system on the lens 121 and the lens 122 can be changed to achieve the adjustment. Wherein, the imaging position refers to the positions of the left virtual scene and the right virtual scene, rather than the position of the entire display area.

[0084] In the specific adjustment, the moving distance of the left virtual scene can be x1, wherein,

[0085] The moving distance of the right virtual scene can be x2, wherein,

[0086] Wherein, f is the focal length of the optical system.

[0087] In an example, the obtaining of the depth value of the real scene in the above step S100 can specifically include: obtaining the depth values of multiple regions of the real scene.

[0088] For example, in actual application, the real scene depths of different regions in the real scene can be different. For example, assuming that the real scene is a tree, the tree is a three-dimensional landscape in space, and the distances between different regions in the tree and the human eye are different, so the real scene depths of different regions in the real scene are different.

[0089] It can be understood that the depth values of different regions in the real scene can be different, so the parallax angles of the eyes when observing different regions in the real scene are different. By obtaining the depth values of different regions in the real scene, more accurate actual parallax angles can be obtained. When calculating the real scene depth value, the depth value of the directly observed region can be given a larger weight, and the depth values of other regions can be given smaller weights, so as to obtain the real scene depth value. Wherein, the directly observed region and the other regions can be calculated according to the normal distribution from the center to the edge of the directly observed region. Or, other step weights can also be used for calculation.

[0090] For example, such as Figure 8 As shown, in one example provided in this application, the real-world scene can be divided into multiple regions. For example, it can include multiple regions such as 1, 2, 3, 4, etc. The real-world depth value can specifically be... Here, k corresponds to different regions in the real scene. x is the depth value of region k, and a is the corresponding weight. It should be noted that in practical applications, k can specifically be a region in a depth sensor (such as a TOF sensor).

[0091] In addition, in practical applications, the actual parallax angle corresponding to the real scene observed by both eyes can be obtained by inferring the area of ​​the real scene observed by both eyes. This allows for more precise movement of the virtual scene projected onto the left and right eyes, making the virtual parallax angle of the virtual scene consistent with the actual parallax angle of the real scene.

[0092] There are many different methods for inferring the area of ​​the real scene observed by both eyes.

[0093] For example, in one example provided in this application, the user's observation direction can be obtained. Based on the observation direction, the observation area directly opposite the observation direction is obtained from multiple regions. The actual parallax angle is calculated based on the depth value of the observation area, the depth values ​​of other regions, and the interpupillary distance.

[0094] Alternatively, by obtaining the user's approximate observation direction, the user's actual observation area (i.e., the area directly in front of them) can be roughly determined. In practical applications, the depth value of this area directly in front of them can be used as the real-world depth value. Alternatively, the depth values ​​of this area and other nearby areas can be reasonably calculated and used as the real-world depth value. Then, the actual parallax angle can be calculated by combining this with the interpupillary distance.

[0095] For example, in one embodiment provided in this application, an inertial measurement unit (IMU) configured in the frame can measure the attitude of the smart glasses. In actual use, the smart glasses move synchronously with the user's head. The attitude sensor can infer the user's head attitude by detecting the attitude of the smart glasses, and thus infer the user's observation direction. It is understood that the IMU sensor can also be replaced with other attitude sensors capable of acquiring the observation direction; this application does not limit the specific type of sensor.

[0096] like Figure 8 As shown, this is the area directly in front of the observation point detected by the IMU sensor.

[0097] like Figure 9As shown, at this time, the IMU sensor detects that the AR glasses (i.e., the user's head) is deflected to a certain angle to the right, and thus, the observation area is offset to the right by a certain distance.

[0098] It should be noted that when the IMU is used to obtain the observation direction of the user, the data obtained may not be very accurate, and thus, in some examples, the observation direction can also be obtained by detecting the actual gaze direction of the human eye.

[0099] For example, obtaining the observation direction can specifically include: obtaining the gaze direction of the eye, obtaining the gaze area in the multiple areas according to the gaze direction, and obtaining the actual parallax angle of the observed real scene according to the depth value of the gaze area and the interpupillary distance.

[0100] In a specific application, the depth value of the gaze area can be used as the real scene depth value, or a larger weight can be applied to the gaze area according to a normal distribution, and a smaller weight can be applied to other areas around the gaze area.

[0101] In a specific application, the depth value of the gaze area can be used as the real scene depth value, or a larger weight can be applied to the gaze area according to a normal distribution, and a smaller weight can be applied to other areas around the gaze area.

[0102] In obtaining the gaze direction of the eye, the actual gaze direction of the left eye can be obtained to infer the actual gaze direction of the two eyes. Alternatively, the actual gaze direction of the right eye can be obtained to infer the actual gaze direction of the two eyes. Alternatively, the actual gaze direction of the left eye and the right eye can be obtained to obtain the actual gaze direction of the two eyes.

[0103] In an example, the method described above can include: moving the virtual scene position projected to the left eye according to the left parallax value by adjusting the display position of the optical system. Moving the virtual scene position projected to the right eye according to the right parallax value by adjusting the focal length of the optical system, so as to adjust the fusion position of the virtual scene.

[0104] In addition, as shown in the Figure 10 The present application also provides a smart glasses. The smart glasses can include: an optical system, a depth-of-field ranging element, an interpupillary distance ranging element, and a processor.

[0105] The optical system is used to display the virtual scene to the left eye and the right eye, the depth-of-field ranging element is used to obtain the depth value of the real scene, and the interpupillary distance ranging element is used to obtain the interpupillary distance between the left eye and the right eye. The processor is used to obtain the actual parallax angle of the observed real scene according to the depth value and the interpupillary distance, and obtain the left parallax value of the virtual scene projected to the left eye and the right parallax value of the virtual scene projected to the right eye according to the initial parallax angle and the actual parallax angle. The processor is also used to move the position of the virtual scene projected to the left eye by the optical system according to the left parallax value, and move the position of the virtual scene projected to the right eye by the optical system according to the right parallax value, so that the virtual parallax angle of the virtual scene is consistent with the actual parallax angle of the real scene.

[0106] Alternatively, it can be understood that the actual parallax angle of the user's eyes is kept unchanged when the user wears the smart glasses and observes the real scene at a specific depth of the real scene. That is, the actual parallax angle corresponds to the depth of the real scene. Correspondingly, the virtual parallax angle of the user's eyes is also kept unchanged when the user observes the virtual scene at a specific depth of the virtual scene. That is, the virtual parallax angle corresponds to the depth of the virtual scene. Therefore, when observing the images at the same depth of the real scene and the virtual scene, the virtual parallax angle and the actual parallax angle are basically the same. That is, when the virtual parallax angle and the actual parallax angle are basically the same, the user's eyes can observe the clear real scene and virtual scene at the same time without adjusting the ocular vergence, thereby effectively avoiding the problems of binocular non-fusion image and visual fatigue.

[0107] In actual application, the depths of different regions in the real scene can be different. For example, assuming that the real scene is a tree, the tree is a three-dimensional landscape in space, and the distances between different regions in the tree and the human eyes are different, so the depths of different regions in the real scene are different.

[0108] In an example, the depth-of-field ranging element provided in the smart glasses can also obtain the depth values of multiple regions of the real scene.

[0109] It can be understood that the depth values of different regions in the real scene can be different, so the parallax angles of the eyes when observing different regions in the real scene are different. By obtaining the depth values of different regions in the real scene, a more accurate actual parallax angle can be obtained.

[0110] That is, the actual parallax angle corresponding to the region of the real scene observed by the eyes can be obtained by inferring the region of the real scene observed by the eyes, so that the virtual scene projected to the left eye and the right eye is moved more accurately, and the virtual parallax angle of the virtual scene is consistent with the actual parallax angle of the real scene.

[0111] When inferring the region of the real scene observed by the eyes, a variety of different methods can be used.

[0112] For example, in one instance, a posture sensor can be incorporated into the smart glasses. The posture sensor is used to determine the viewing direction of the smart glasses. The processor is also used to determine the directly opposite viewing area from multiple regions based on the viewing direction, and to calculate the actual parallax angle based on the depth value of the directly opposite viewing area, the depth values ​​of other regions, and the interpupillary distance.

[0113] Alternatively, it can be understood that in actual use, the smart glasses move in sync with the user's head movements. The posture sensor can detect the posture of the smart glasses to infer the posture of the user's head, and thus infer the user's viewing direction.

[0114] In one example, the actual observation area of ​​the eyes when observing a real scene can also be determined by the actual direction of the eyes' gaze.

[0115] For example, such as Figure 11 As shown, in one example provided in this application, the smart glasses also include an eye-tracking element 14. The eye-tracking element is used to acquire the gaze direction of the eyes, and the processor is also used to obtain the gaze area among multiple regions based on the gaze direction, and to obtain the actual parallax angle of the observed scene based on the depth value of the gaze area and the interpupillary distance. In specific settings, multiple eye-tracking elements 14 can be arranged around the left lens 121 and the right lens 122 to achieve better detection results.

[0116] In practical applications, the depth value of the gaze area can be used as the real-world depth value, or a larger weight can be applied to the gaze area according to a normal distribution, while other areas around the gaze area are given smaller weights.

[0117] When determining the gaze direction, the actual gaze direction of both eyes can be inferred by obtaining the actual gaze direction of the left eye. Alternatively, the actual gaze direction of both eyes can be inferred by obtaining the actual gaze direction of the right eye. Or, the actual gaze direction of both eyes can be obtained by obtaining the actual gaze directions of both the left and right eyes.

[0118] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of different embodiments are consistent and can be referenced by each other. The technical features of different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0119] In this application, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural.

[0120] It can be understood that various digital numbers involved in the embodiments of the present application are only distinguished for the convenience of description, and are not used to limit the scope of the embodiments of the present application. The size of the serial number of the above processes does not mean the order of execution, and the execution order of the processes should be determined according to its function and inherent logic.

Claims

1. A virtual scene display method characterized by, The method comprises: obtaining a depth value of a real scene; obtaining a pupil distance between a left eye and a right eye; obtaining an actual parallax angle of the real scene according to the depth value and the pupil distance; obtaining an initial parallax angle of a virtual scene; obtaining a parallax angle difference value according to the initial parallax angle and the actual parallax angle; and moving a position of the virtual scene projected to the left eye and the right eye according to the parallax angle difference value, so that a virtual parallax angle of the virtual scene is consistent with the actual parallax angle of the real scene.

2. The virtual stage display method according to claim 1, characterized by, The obtaining of the depth value of the real scene comprises obtaining depth values of multiple regions of the real scene.

3. The virtual stage display method according to claim 2, wherein The obtaining of the actual parallax angle of the real scene according to the depth value and the pupil distance comprises: obtaining an observation direction; obtaining a direct observation region of the multiple regions that is directly opposite to the observation direction according to the observation direction; and calculating the actual parallax angle according to the depth value of the direct observation region, the depth values of other regions, and the pupil distance.

4. The virtual stage display method according to claim 3, wherein The method further comprises: applying a larger weight to the depth value of the direct observation region and a smaller weight to the depth values of other regions.

5. The virtual stage display method according to any one of claims 2 to 4, characterized in that, The obtaining of the actual parallax angle of the real scene according to the depth value and the pupil distance comprises: obtaining a gaze direction of the eye; obtaining a gaze region of the multiple regions according to the gaze direction; and obtaining the actual parallax angle of the real scene according to the depth value of the gaze region and the pupil distance.

6. The virtual stage display method according to claim 5, wherein The obtaining of the gaze direction of the eye comprises: obtaining an actual gaze direction of the left eye and / or the right eye.

7. The virtual stage display method according to any one of claims 1 to 6, characterized by, The moving of the position of the virtual scene projected to the left eye according to the left parallax value and the moving of the position of the virtual scene projected to the right eye according to the right parallax value comprise: adjusting a display position of the virtual scene of an optical system to move the position of the virtual scene projected to the left eye according to the left parallax value; and adjusting the display position of the virtual scene of the optical system to move the position of the virtual scene projected to the right eye according to the right parallax value.

8. An intelligent eyewear, characterized in that, The smart glasses comprise: an optical system configured to display a virtual scene to a left eye and a right eye; a depth distance measuring element configured to obtain a depth value of a real scene; a pupil distance measuring element configured to obtain a pupil distance between the left eye and the right eye; a processor configured to obtain an actual parallax angle of the real scene according to the depth value and the pupil distance, and to obtain a parallax angle difference value according to the initial parallax angle and the actual parallax angle; and the processor is further configured to move a position of the virtual scene projected to the left eye and the right eye by the optical system according to the parallax angle difference value, so that a virtual parallax angle of the virtual scene is consistent with the actual parallax angle of the real scene.

9. The smart glasses of claim 8, wherein, The depth distance measuring element is configured to obtain depth values of multiple regions of the real scene.

10. The smart glasses of claim 9, wherein, The smart glasses further comprise a posture sensor configured to obtain an observation direction of the smart glasses; the processor is further configured to obtain a direct observation region of the multiple regions that is directly opposite to the observation direction according to the observation direction, and to calculate the actual parallax angle according to the depth value of the direct observation region, the depth values of other regions, and the pupil distance.

11. The smart glasses of any one of claims 9-10, wherein, The smart glasses further comprise an eye movement tracking element; the eye movement tracking element is configured to obtain a gaze direction of the eye; the processor is further configured to obtain a gaze region of the multiple regions according to the gaze direction, and to obtain the actual parallax angle of the real scene according to the depth value of the gaze region and the pupil distance.

Images