Virtual display device and virtual display method

Abstract

The embodiments of the present application disclose a virtual display device and a virtual display method, which can avoid problems such as visual convergence adjustment conflicts generated in the process of providing virtual display functions, thereby avoiding visual fatigue of the human eye caused in the process of providing virtual display functions to users. The specific solution is: the first optical display module is used to implement the following functions under the control of the processor: displaying a first object, the convergence depth of the first object is a first convergence depth, and the imaging surface depth of the first optical display module is the first depth. Displaying a second object, the convergence depth of the second object is a second convergence depth, and the imaging surface depth of the first optical display module is the second depth. When the first convergence depth and the second convergence depth are different, the optical display module adjusts the first depth to be different from the second depth.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence, and in particular to a virtual display device and a virtual display method. Background Art

[0002] Display technologies based on augmented reality (AR), virtual reality (VR) or mixed reality (MR) technologies can provide users with display solutions that are close to the real scene experience, and are therefore attracting widespread attention.

[0003] However, the current solutions for providing AR, VR or MR displays (i.e., displays based on AR, VR or MR technologies) are not perfect, thereby damaging the user experience. Summary of the Invention

[0004] The virtual display device and method provided by embodiments of the present application can avoid issues such as vergence-accommodation conflict (VAC) that arise during the provision of virtual display functionality, thereby preventing visual fatigue in the user's eyes. This solution can also adjust the optical power used when displaying images to the user based on the user's ability to recognize images. This allows users with myopia or astigmatism to use the virtual display functionality provided by the virtual display device with their naked eyes.

[0005] In order to achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a virtual display device is provided, which is used to provide a user with a display function of a virtual three-dimensional environment. The virtual display device includes:

[0007] A processor, and a first optical display module; the first optical display module is used to display an image to a first human eye under the control of the processor, and the first human eye is either one of the eyes of the user; the first optical display module is used to implement the following functions under the control of the processor: display a first object, the convergence depth of the first object is the first convergence depth, and the imaging surface depth of the first optical display module is the first depth; display a second object, the convergence depth of the second object is the second convergence depth, and the imaging surface depth of the first optical display module is the second depth; wherein, the first object and the second object, the first object and the second object are included in the virtual three-dimensional environment, and when the first convergence depth and the second convergence depth are different, the optical display module adjusts the first depth to be different from the second depth.

[0008] Based on this solution, an example of a virtual display device is provided. In this example, the virtual display device can adjust the virtual image plane of the display to a depth that is similar to or the same as the convergence depth when the user observes the virtual object when displaying an image to the user's human eyes. In this way, the focusing depth and the convergence depth can be matched in the process of providing the virtual display function to the user (that is, the depth of the first virtual image plane is similar to or the same as the first convergence depth). The problem of VAC caused by the difference between the focusing depth and the convergence depth is avoided. It should be noted that the virtual three-dimensional environment in this example may refer to the entire virtual environment constructed by the virtual display device based on the display data. In other embodiments, the virtual three-dimensional environment may also include objects in the real environment. For example, the virtual display device can collect relevant information of objects in the real environment in real time, and display the real environment corresponding to this information to the user through a display screen. That is to say, in this scenario, the virtual three-dimensional environment may include a part of the objects in the real environment. In some sights, in this scenario, the virtual three-dimensional environment may also include a part of the fictional objects. It should be noted that in this example, the virtual display device can determine that the depth of the first virtual image plane is close to the first depth of convergence when the depth difference between the depth of the first virtual image plane and the first depth of convergence does not exceed 5%. Of course, the above-mentioned 5% threshold setting can be flexibly adjusted, for example, it can be set to 6%, 8%, etc.

[0009] In one possible design, when the first convergence depth is greater than the second convergence depth, the first depth is greater than the second depth; when the first convergence depth is less than the second convergence depth, the first depth is less than the second depth. Based on this solution, a changing relationship between the convergence depth and the focusing depth (such as the first depth and the second depth) in this example is provided. In this example, the smaller the convergence depth of the first object to be displayed, the smaller the corresponding focusing depth. Similarly, the greater the convergence depth of the object to be displayed, the greater the corresponding focusing depth. In this way, the focusing depth and the convergence depth can be approached or matched, thereby alleviating or solving the VAC problem.

[0010] In a possible design, the virtual display device also includes: a first eye tracking module, which is used to perform eye tracking on the first human eye under the control of the processor. The processor is used to control the first eye tracking module to perform eye tracking on the gaze point of the first human eye, and when the convergence depth of the gaze point of the first human eye is different, the first optical display module is configured to adjust the depth of the imaging surface to be different. Based on this solution, a specific solution for determining the convergence depth of the user is provided. In this example, the virtual display device can track the user's eyes through the eye tracking module, thereby obtaining some data of the user's eyes during the observation process. For example, data such as the gaze point of the human eye and the line of sight of the human eye. Based on this data, the virtual display device can determine the line of sight angle of the user's eyes when observing the object in the current virtual three-dimensional environment, thereby obtaining the current convergence depth.

[0011] In one possible design, when the user's diopter is different, the first optical display module is configured to adjust the depth of the imaging surface to be different. Based on this solution, another implementation of this example is provided. In this implementation, if the user's diopter is different, the first optical display module can provide the user with an imaging depth corresponding to the diopter. This makes it possible to use the virtual display function provided by the virtual display device with the naked eye even if the user's eyes have ametropia. In some embodiments, the user's diopter may include at least one of the following: myopia, hyperopia, and astigmatism.

[0012] In one possible design, the virtual display device further includes: a second optical display module, the second optical display module is used to display images to a second human eye under the control of the processor, the second human eye is one of the user's two eyes different from the first human eye; the first optical display module is used to implement the following functions under the control of the processor: display the first object, display the imaging plane depth of the first object as a third depth, the depth of the first object in the virtual three-dimensional environment as a second depth, and the first depth is similar to or the same as the third depth. Based on this solution, a display solution for the other human eye of the user's two eyes is provided. In this example, the virtual display device can provide a display with a focusing depth and a convergence depth matching the second human eye of the user through the second optical display module. It should be noted that since the user's convergence depth is generated when observing an object with both eyes, the solution provided in the first aspect illustrates that when the virtual display device displays images to the user's two eyes, the corresponding convergence depths can be the same.

[0013] In one possible design, the first optical module includes a first zoom module. The optical power of the first zoom module is adjustable. Under the control of the processor, the first zoom module is configured to adjust the optical power of the first zoom module to a first optical power when displaying the first object, so that the first optical module with the first optical power can be imaged at the first depth; or, under the control of the processor, the first zoom module is configured to adjust the optical power of the first zoom module to a second optical power when displaying the second object, so that the first optical module with the second optical power can be imaged at the second depth. Based on this solution, a specific example of a method for adjusting the depth of a virtual image plane is provided. In this example, in a virtual display device, the optical module responsible for displaying an image to the user's eyes (such as the first optical module corresponding to the first eye) may include a zoom module with the ability to adjust the optical power. By adjusting the optical power of the zoom module, the optical power of the virtual display device can be adjusted. It will be understood that different optical powers can correspond to different depths of the virtual image plane. Therefore, by adjusting the optical power, the focus depth can be adjusted. For example, the VAC problem can be solved by adjusting the optical power to a certain value so that the depth of the virtual image surface matches the convergence depth.

[0014] In one possible design, the first optical display module is further configured to display a third object at a first virtual position and a second virtual position in the virtual three-dimensional environment under the control of the processor. The first and second virtual positions are at different distances from the user's eyes in the three-dimensional environment. Based on this solution, another mechanism for using the virtual display device provided in an embodiment of the present application is provided. In this example, the virtual display device can present objects at different depths to the user. This allows the user's eyes to naturally control the contraction and extension of the ciliary muscles when observing the objects at different depths, thereby exercising the ciliary muscles. This can avoid visual fatigue and further prevent myopia. In some embodiments, the third object displayed at the first virtual position can be the same as the third object displayed at the second virtual position. In other embodiments, the object displayed at the first virtual position can be different from the object displayed at the second virtual position. When displaying images at different depths to the user's eyes, the virtual display device can also adjust the matching of the focus depth and the convergence depth in the corresponding scene according to the solution in the aforementioned example, thereby exercising the ciliary muscles while avoiding the problem of VAC.

[0015] In one possible design, when the current display scene is a preset scene, the virtual display device displays the third object at the first virtual position and the second virtual position, respectively. The preset scene includes at least one of the following: an advertisement playback scene and a display resource loading scene. Based on this solution, examples of possible scenarios for ciliary muscle training are provided. In this example, the virtual display device can display objects at different depths to the user in certain specific scenarios. These specific scenarios can be scenes that are not displayed when the user is using the virtual display function. For example, before displaying content to the user, when playing an advertisement, the virtual display device can display images at different depths to the user's eyes in conjunction with the advertisement content, thereby avoiding visual fatigue. For another example, when loading display resources such as videos or images, the virtual display device can display images at different depths to the user in conjunction with the content of the display resource to be displayed (or other preset content), thereby exercising the ciliary muscles and avoiding visual fatigue. In some embodiments of the present application, the virtual display device can also display different depths for each control when presenting different options to the user. This allows the user to see objects at different depths as they determine the options and make their selections, thereby avoiding visual fatigue.

[0016] In a possible design, the first optical display module is also used to display a first detection image to the first human eye under the control of the processor. When displaying the first detection image, the number of pixels of the opening used to display the first detection image is a first number. The processor is also used to receive a first recognition feedback, which is an instruction input by the user when observing the first detection image using the first human eye, and the first recognition feedback is used to indicate whether the user can identify the opening direction of the first detection image. Based on this solution, an example of a solution for judging the user's human eye's ability to recognize an image is provided. It is understandable that the virtual display device can determine the user's ability to recognize the image based on the feedback input by the user by displaying the detection image in a virtual three-dimensional environment and guiding the user to observe the detection image. In some examples, the user's ability to recognize the image may include the degree of myopia of the user's human eye, etc. In some embodiments, the detection image is taken as an example of a letter with an opening. To accurately determine the degree of myopia, the virtual display device can determine the number of pixels corresponding to the required visual score value to display to the user based on the visual score value of each pixel on the display screen, and thus determine the size of the letters to be displayed. In this way, the letter image displayed to the user with the aforementioned number of openings can accurately determine the degree of myopia based on whether the user can identify the direction of the openings. It can be understood that the smaller the number of openings that the user can identify, the better the user's vision and the corresponding lower the degree of myopia.

[0017] In one possible design, when the first number of pixels displays the opening of the first detection image to the user, the visual score of the opening of the first detection image observed by the first person's eye is the first visual score. If the first recognition feedback indicates that the user cannot identify the direction of the opening in the first detection image, the processor is configured to determine that the myopia of the first person's eye is the myopia corresponding to the first visual score. Based on this solution, a specific example of determining the user's myopia is provided. It will be understood that when determining the myopia, the size of the smallest clearly visible letter can correspond to the user's myopia. Different letter sizes also have corresponding visual scores for a person's eye in a fixed relative position. Therefore, in this example, the virtual display device can determine the user's myopia based on the visual score corresponding to the opening with the smallest number of pixels that the user can identify. For example, if the user determines that they cannot identify the direction of the opening of the current letter, they can input feedback indicating that they cannot identify it. Based on this feedback, the virtual display device can determine that the user cannot identify the direction of the opening with the current number of pixels. The opening direction corresponds to a certain visual score value, and therefore, the user's myopia can be determined with reference to the visual score value according to the virtual display device. It should be noted that, in this example, the first recognition feedback can be input to the virtual display device by the user through a remote control, voice, gestures, etc. In other embodiments, the virtual display device can also determine that the user cannot recognize the opening direction of the letters of the current size when the user does not input the correct recognition feedback in a preset market. In addition, in some embodiments, in order to make the measurement of the myopia of the user's human eyes more accurate, the virtual display device can show the user another detection image that is nearly or identical to the current size after the user inputs the feedback that the current image cannot be recognized, and provide it to the user for recognition. When the user cannot recognize the detection image of the current size for many times, the virtual display device can accurately determine that the user cannot recognize the current detection image.

[0018] In a possible design, when the first recognition feedback indicates that the user can identify the opening direction of the first detection image, the first optical display module is also used to, under the control of the processor, show the second detection image to the first human eye. When displaying the second detection image, the number of pixels of the opening for displaying the second detection image is a second number. The second number is less than the first number. The processor is also used to receive a second recognition feedback, which is an instruction input by the user when observing the second detection image using the first human eye, and the second recognition feedback is used to indicate whether the user can identify the opening direction of the second detection image. Based on this solution, another example of determining the user's ability to recognize graphics is provided. In this example, the virtual display device can show the user another detection image (such as the second detection image) that is different from the size of the first detection image when the user can recognize the first detection image. In some embodiments, the second detection image can be smaller than the first detection image. In combination with the aforementioned solution, the virtual display device can show a second detection image with a smaller number of pixels corresponding to a smaller visual score value. In this way, the human eye can recognize a letter image with a smaller opening, thereby enabling the virtual display device to more accurately measure the recognition ability of the human eye.

[0019] In one possible design, when the second number of pixels displays the opening of the second detection image to the user, the visual score of the opening of the second detection image observed by the first person's eye is the second visual score. When the second recognition feedback indicates that the user cannot recognize the opening direction of the second detection image, the processor is used to determine that the myopia degree of the first person's eye is the myopia degree corresponding to the second visual score. Based on this scheme, an example of a scheme for determining the myopia degree based on the second detection image is provided. In this example, the virtual display device can determine the user's myopia degree by determining the visual score value corresponding to the minimum detection image that the user can recognize.

[0020] In one possible design, the first optical display module is used to adjust the optical focal length of the first optical display module to the initial optical focal length under the control of the processor before displaying the first detection image to the first human eye, so that the first optical display module is imaged at the farthest distance. Based on this solution, an example of the initial setting of a virtual display device in the process of displaying a detection image to a user is provided. In this example, the virtual display device can adjust the optical focal length so that the virtual image that the human eye can see is at the farthest distance. In this way, the light incident on the human eye can be close to parallel light. It is understandable that for a human eye without myopia, the user's human eye can accurately converge parallel light on the retina. In this way, based on the solution provided in this example, the virtual display device can provide a normal display for a human eye with normal vision. At the same time, it can also achieve degree measurement for a human eye with myopia.

[0021] In one possible design, when the opening of the third inspection image displayed by the first optical display module to the first human eye is a third number of pixels, and when the received recognition feedback indicates that the user cannot identify the opening direction of the fourth detection image, the optical focal length of the virtual image displayed by the first optical display module to the first human eye is a second optical focal length. The second optical focal length corresponds to a third visual score value, which is a visual score value corresponding to the third number of pixels. Based on this solution, an example of a solution for providing a naked-eye virtual display experience to myopic users is provided. It is understandable that the human eye of a myopic user cannot smoothly converge the normally displayed image on the retina, but instead converges the image plane between the retina and the pupil. In this way, if the image is displayed normally, the myopic user will not be able to see the displayed content clearly with the naked eye. Through the solution provided in this example, the virtual image plane in the process of displaying the image to the user's human eye can be adjusted according to the number of pixels of the minimum opening that the user can recognize (such as adjusting the virtual image plane by adjusting the optical focal length). In this way, myopic users can image the adjusted virtual image plane on the retina of their human eyes, thereby achieving the effect that myopic users can clearly see the displayed content with the naked eye.

[0022] In one possible design, the first optical display module includes a rotation mechanism configured to rotate the first optical display module perpendicular to the optical axis under the control of the processor. When the first optical display module displays the first image to the user, the optical power direction of the first optical display module coincides with the axis of astigmatism of the first eye. Based on this solution, a display solution that can match the eyes of users with astigmatism is provided. It is understood that for users with astigmatism, their eyes cannot clearly see the displayed image at certain incident angles. In other words, a person with astigmatism can only clearly see images where the incident light coincides with the axis of astigmatism. In this example, the virtual display device can flexibly adjust the optical power direction of the first optical display module to achieve the effect of aligning the angle of light incident on the virtual display device with the axis of astigmatism of the eye. This allows users with astigmatism to clearly see the image displayed by the virtual display device. Thus, even if a user has astigmatism, the virtual display device, according to this solution, can provide the user with a naked-eye viewing experience.

[0023] In one possible design, the processor is further used to determine the astigmatism degree of the first human eye according to the rotation of the rotating mechanism when the optical power direction of the first optical display module coincides with the astigmatism axis of the first human eye. Based on this solution, an example of a solution for measuring the astigmatism degree of a user is provided. It can be understood that the astigmatism degree of the user's human eye is related to the astigmatism axis of the user. In this example, the astigmatism axis when the user can see the image clearly can be determined by adjusting the optical power direction of the system, and the astigmatism degree of the user can be determined based on the astigmatism axis. In some embodiments, the virtual display device can show the user an astigmatism degree measurement card, and determine the user's astigmatism axis based on the mark (such as the corresponding number) corresponding to the angle that cannot be seen clearly indicated by the user's feedback, thereby determining the astigmatism degree of the user's human eye based on the astigmatism axis.

[0024] In one possible design, the processor is further configured to obtain a user characteristic of the current user before controlling the first optical display module to display the first image, and the processor is specifically configured to control the first optical display module to display a first image corresponding to the user characteristic of the current user. When displaying the first image, the optical power of the first optical display module matches the myopia and / or astigmatism of the first eye of the current user, and the myopia and / or astigmatism of the first eye of the current user is indicated by virtual display information corresponding to the user characteristic of the current user. Based on this solution, an example of a solution for providing a naked-eye virtual experience to a corresponding user is provided. In this example, since the myopia and / or astigmatism of different users' eyes are different, the virtual display device can identify the current user based on the user characteristic of the current user, and determine the user's ability to recognize the image based on the stored current myopia and / or astigmatism. The virtual display device can then adjust the optical power to display the image to the user, so that the optical power of the displayed image matches the current user's ability to recognize the image. In this way, even if the user has myopia or astigmatism, he or she can still see the image displayed by the virtual display device clearly with naked eyes.

[0025] In one possible design, the user feature includes any one of the following features: fingerprint information of the current user. Iris feature of the current user. Account information of the current user. Identification of the current user, which is different for different users. Based on this solution, an example of user features is provided. It can be understood that the virtual display device can distinguish different users through different user features. In this example, some examples of user features are provided. For example, user features can include biometric information, such as fingerprint, iris, voiceprint and other information. For another example, the user feature can be the user's account information, such as a user name, a user nickname, etc. For another example, the user feature can also be a pre-set user identifier, which can identify different users.

[0026] In one possible design, the virtual display device stores a correspondence between different user features and corresponding virtual display information. The processor is used to search for a matching table entry from the correspondence based on the user features of the current user, and if there is a matching table entry, determine that the virtual display information corresponding to the current user is the virtual display information stored in the matching table entry. The virtual display information includes the myopia degree and / or astigmatism degree of the corresponding user. Based on this solution, an example of a solution is provided in which a virtual display device determines a corresponding display strategy based on a user. In this example, the virtual display device can identify the current user based on the user features of the user. Then, the virtual display device can search for the virtual display information corresponding to the user features of the user from the correspondence relationship stored locally (or stored in the cloud) based on the features of the current user. If it can be found, it indicates that it can be displayed according to the virtual display information. Exemplarily, the virtual display device can adjust the optical focal length based on the virtual display information so as to provide a naked eye display experience to the user's eyes at the corresponding virtual image surface position.

[0027] In one possible design, when there is no matching item corresponding to the user characteristics of the current user in the corresponding relationship, the first optical display module is further used to display a first image under the control of the processor, and the optical power when displaying the first image is the optical power that matches the myopia and / or astigmatism of the first eye of the current user, and the myopia and / or astigmatism of the first eye are automatically determined by the processor, or determined under the instruction of the user, or manually input by the user. Based on this solution, another solution for determining a display strategy based on the characteristics of the current user is provided. In this example, the virtual display device can measure the image recognition ability of the current user according to the solution provided in the above example, such as determining the user's myopia and / or astigmatism. In this way, the corresponding naked eye display experience can be presented to the user based on the user's image recognition ability. In other embodiments of the present application, the determination of the user's myopia and / or astigmatism can be performed with the authorization or instruction of the user. In other embodiments of the present application, the user's myopia and / or astigmatism can also be input by the user. It should be noted that before the virtual display device obtains the user's image recognition capability on its own, it can search for virtual display information corresponding to the user characteristics of the current user from the corresponding relationship stored locally or in the cloud. If so, it can be executed according to the aforementioned scheme, that is, display according to the virtual display information. If not, the identifier cannot directly obtain the current user's myopia and / or astigmatism, so the virtual display device can obtain the user's myopia and / or astigmatism according to the scheme in this example.

[0028] In one possible design, the processor is further used to store the correspondence between the myopia degree and / or astigmatism degree of the first person's eye and the user characteristics of the current user. Based on this solution, an example of a solution for storing an updated user's image recognition capability is provided. In this example, the virtual display device can determine that the current user is a new user when it cannot find the table item corresponding to the current user in the correspondence stored locally or from the cloud. The obtained correspondence between the image recognition capability of the current user and the user characteristics of the current user can then be stored, so that when a virtual display is subsequently provided to the user, there is no need to measure the user's image recognition capability again.

[0029] In a second aspect, a virtual display device is provided, which may have the components of the virtual display device provided in the first aspect. In this example, the first optical display module is further configured to display a third object at a first virtual position and a second virtual position in the virtual three-dimensional environment under the control of the processor. The first virtual position and the second virtual position are at different distances from the user's eyes in the three-dimensional environment. It will be understood that in the description of the first aspect, technical means for solving the VAC problem can be combined to achieve the effect of exercising the ciliary muscles. In this example, the virtual display device can also achieve the effect of exercising the ciliary muscles simply by displaying objects at different depths.

[0030] In one possible design, when the current display scene is a preset scene, the virtual display device displays the third object at the first virtual position and the second virtual position, respectively. The preset scene includes at least one of the following scenes: an advertisement playing scene, and a display resource loading scene.

[0031] A third aspect provides a virtual display device, which may have the composition of the virtual display device provided in the first aspect. In this example, the first optical display module is also used to, under the control of the processor, show the first detection image to the first human eye. When displaying the first detection image, the number of pixels of the opening for displaying the first detection image is a first number. The processor is also used to receive a first recognition feedback, which is an indication input by the user when observing the first detection image using the first human eye, and the first recognition feedback is used to indicate whether the user can identify the opening direction of the first detection image.

[0032] In this example, the virtual display device can display a detection image in a virtual 3D environment and guide the user to observe the detection image, thereby determining the user's image recognition ability based on the user's input feedback. In some examples, the user's image recognition ability can include, for example, the user's myopia degree. This allows for eye examination of the user. It should be noted that this solution can support autonomous eye examination by the virtual display device to determine the user's myopia degree. In other implementations, this solution can also be used to support remote vision.

[0033] In one possible design, when the first number of pixels displays the opening of the first detection image to the user, a visual acuity score of the opening of the first detection image observed by the first person's eye is a first visual acuity score. If the first recognition feedback indicates that the user cannot identify the direction of the opening of the first detection image, the processor is configured to determine that the myopia degree of the first person's eye is the myopia degree corresponding to the first visual acuity score.

[0034] In a possible design, when the first recognition feedback indicates that the user is able to identify the opening direction of the first detection image, the first optical display module is also used to, under the control of the processor, show the second detection image to the first human eye. When displaying the second detection image, the number of pixels of the opening used to display the second detection image is a second number. The second number is less than the first number. The processor is also used to receive a second recognition feedback, which is an indication input by the user when observing the second detection image using the first human eye, and the second recognition feedback is used to indicate whether the user is able to identify the opening direction of the second detection image.

[0035] In one possible design, when the second number of pixels displays the opening of the second detection image to the user, a visual acuity score of the opening of the second detection image observed by the first person's eye is a second visual acuity score. If the second recognition feedback indicates that the user cannot identify the direction of the opening of the second detection image, the processor is configured to determine that the myopia degree of the first person's eye is the myopia degree corresponding to the second visual acuity score.

[0036] In one possible design, the first optical display module is used to adjust the optical focal length of the first optical display module to an initial optical focal length under the control of the processor before displaying the first detection image to the first human eye, so that the first optical display module forms an image at the farthest distance.

[0037] In one possible design, when the opening of the third inspection image presented by the first optical display module to the first eye is a third number of pixels, and when the received recognition feedback indicates that the user cannot identify the opening direction of the fourth inspection image, the optical power of the virtual image presented by the first optical display module to the first eye is a third optical power. The third optical power corresponds to a third visual fraction value, which is the visual fraction value corresponding to the third number of pixels.

[0038] In a fourth aspect, a virtual display device is provided. This virtual display device may include the components of the virtual display device provided in the first aspect. In this example, the first optical display module includes a rotation mechanism configured to rotate the first optical display module perpendicular to the optical axis under the control of the processor. When the first optical display module displays the first image to the user, the optical power direction of the first optical display module coincides with the astigmatism axis of the first person's eye.

[0039] Based on this solution, a display solution is provided that can match the human eyes of users with astigmatism. It is understandable that for users with astigmatism, their eyes cannot see the displayed image clearly at a certain incident angle. In other words, a human eye with astigmatism can only see clearly the image where the incident light coincides with the astigmatism axis of the human eye. In this example, the virtual display device can flexibly adjust the optical focal length direction of the first optical display module to achieve the effect of coinciding the angle of the light incident on the virtual display device with the astigmatism axis of the human eye. In this way, users with astigmatism can also see the effect of the image displayed by the virtual display device.

[0040] In a possible design, the processor is further configured to determine the astigmatism degree of the first human eye based on the rotation of the rotating mechanism when the optical power direction of the first optical display module coincides with the astigmatism axis of the first human eye.

[0041] In a fifth aspect, a virtual display device is provided, which may have the composition of the virtual display device provided in the first aspect. In this example, the processor is also used to obtain the user characteristics of the current user before controlling the first optical display module to display the first image, and the processor is specifically used to control the first optical display module to display the first image corresponding to the user characteristics of the current user. Wherein, when displaying the first image, the optical focal length of the first optical display module is matched with the myopia and / or astigmatism of the first eye of the current user, and the myopia and / or astigmatism of the first eye of the current user is indicated by the virtual display information corresponding to the user characteristics of the current user.

[0042] Based on this solution, an example of a solution for providing a naked-eye virtual experience to a corresponding user is provided. In this example, since different users have different degrees of myopia and / or astigmatism, the virtual display device can identify the current user based on the user characteristics of the current user and determine the user's image recognition ability based on the stored current corresponding myopia and / or astigmatism. The virtual display device can then adjust the optical focal length to display the image to the user, so that the optical focal length of the displayed image can match the current user's image recognition ability. In this way, even if the user has myopia or astigmatism, they can still see the image displayed by the virtual display device clearly with the naked eye.

[0043] In one possible design, the user feature includes any one of the following features: fingerprint information of the current user; iris features of the current user; account information of the current user; and an identifier of the current user, where the identifiers of different users are different.

[0044] In one possible design, the virtual display device stores a correspondence between different user characteristics and corresponding virtual display information. The processor is configured to search for a matching entry in the correspondence based on the user characteristics of the current user. If a matching entry exists, the virtual display information corresponding to the current user is determined to be the virtual display information stored in the matching entry. The virtual display information includes the corresponding user's myopia degree and / or astigmatism degree.

[0045] In one possible design, when there is no matching item corresponding to the user characteristics of the current user in the corresponding relationship, the first optical display module is also used to display a first image under the control of the processor, and the optical focal length when displaying the first image is an optical focal length that matches the myopia and / or astigmatism of the first eye of the current user, and the myopia and / or astigmatism of the first eye are automatically determined by the processor, or determined under the user's instructions, or manually input by the user.

[0046] In a possible design, the processor is further configured to store a correspondence between the myopia degree and / or astigmatism degree of the first person's eye and the user characteristics of the current user.

[0047] It should be noted that the functions corresponding to the virtual display devices provided by the above-mentioned second aspect, third aspect, fourth aspect and fifth aspect and any possible implementation thereof may be implemented separately or in combination with each other. For example, the functions corresponding to the virtual display devices provided by the second aspect and any possible implementation thereof may be integrated into the same device with the functions corresponding to the virtual display devices provided by the third aspect and / or the fourth aspect and / or the fifth aspect and any possible implementation thereof. For another example, the functions corresponding to the virtual display devices provided by the third aspect and any possible implementation thereof may be integrated into the same device with the functions corresponding to the virtual display devices provided by the fourth aspect and / or the fifth aspect and any possible implementation thereof. For another example, the functions corresponding to the virtual display devices provided by the fourth aspect and any possible implementation thereof may be integrated into the same device with the functions corresponding to the virtual display devices provided by the fifth aspect and any possible implementation thereof.

[0048] In a sixth aspect, a virtual display method is provided, which is applied to a virtual display device as provided in the first aspect and any possible implementation thereof, and the virtual display method is used to provide a user with a display function of a virtual three-dimensional environment. The virtual display method includes: a first optical display module displays a first object, the convergence depth of the first object is a first convergence depth, and the imaging surface depth of the first optical display module is a first depth. The first optical display module displays a second object, the convergence depth of the second object is a second convergence depth, and the imaging surface depth of the first optical display module is a second depth. The first object and the second object are included in the virtual three-dimensional environment, and when the first convergence depth and the second convergence depth are different, the optical display module adjusts the first depth to be different from the second depth. In one possible design, the virtual display method also includes: the processor controls the first eye tracking module in the virtual display device to perform eye tracking on the first person's eye. When the eye tracking results of the first person's eye are different, the second depth is different.

[0049] In a possible design, when the first convergence depth is greater than the second convergence depth, the first depth is greater than the second depth. When the first convergence depth is less than the second convergence depth, the first depth is less than the second depth.

[0050] In one possible design, the processor controls the first eye tracking module of the first optical display module to perform eye tracking on the gaze point of the first person's eye. When the convergence depth of the gaze point of the first person's eye is different, the first optical display module is configured to adjust the depth of the imaging surface differently.

[0051] In a possible design, when the user's diopter is different, the first optical display module is configured to adjust the depth of the imaging plane to be different.

[0052] In one possible design, the virtual display device further includes: a second optical display module, configured to, under control of the processor, display an image to a second eye of the user, the second eye being a different eye from the first eye. The method further includes: the first optical display module, under control of the processor, performing the following functions: displaying the first object, displaying the first object at a third depth, the depth of the first object in the virtual three-dimensional environment being the second depth, the first depth being similar to or the same as the third depth.

[0053] In one possible design, the first optical module includes a first zoom module. The optical focal length of the first zoom module is adjustable. The method further includes: the processor controlling the first zoom module to adjust the optical focal length of the first zoom module to a first optical focal length when displaying the first object, so that the first optical module with the first optical focal length can form an image at the first depth. Alternatively, the processor controlling the first zoom module to adjust the optical focal length of the first zoom module to a second optical focal length when displaying the second object, so that the first optical module with the second optical focal length can form an image at the second depth.

[0054] In one possible design, the method further includes: the first optical display module, under control of the processor, displaying a third object at a first virtual position and a second virtual position in the virtual three-dimensional environment, wherein the first virtual position and the second virtual position are at different distances from the user's eyes in the three-dimensional environment.

[0055] In one possible design, when the current display scene is a preset scene, the virtual display device displays the third object at the first virtual position and the second virtual position, respectively. The preset scene includes at least one of the following scenes: an advertisement playing scene, and a display resource loading scene.

[0056] In one possible design, the method further includes: the first optical display module, under control of the processor, displaying a first detection image to the first human eye. When displaying the first detection image, the number of pixels of the opening displaying the first detection image is a first number. The processor is further configured to receive first recognition feedback, the first recognition feedback being an indication input by the user when viewing the first detection image using the first human eye, the first recognition feedback being used to indicate whether the user can recognize the opening direction of the first detection image.

[0057] In one possible design, when the first optical display module displays the opening of the first detection image to the user using the first number of pixels, a visual acuity score of the opening of the first detection image observed by the first person's eye is a first visual acuity score. If the first recognition feedback indicates that the user cannot identify the direction of the opening of the first detection image, the method further includes: determining, by the processor, a myopia degree of the first person's eye as a myopia degree corresponding to the first visual acuity score.

[0058] In one possible design, the method further includes: when the first recognition feedback indicates that the user is able to identify the opening direction of the first detection image, the first optical display module, under the control of the processor, displays the second detection image to the first human eye. When displaying the second detection image, the number of pixels of the opening used to display the second detection image is a second number. The second number is less than the first number. The processor receives a second recognition feedback, which is an indication input by the user when observing the second detection image using the first human eye, and the second recognition feedback is used to indicate whether the user is able to identify the opening direction of the second detection image.

[0059] In one possible design, when the first optical display module displays the opening of the second detection image to the user using the second number of pixels, a visual acuity score of the opening of the second detection image observed by the first person's eye is a second visual acuity score. If the second recognition feedback indicates that the user cannot identify the direction of the opening of the second detection image, the method further includes: determining, by the processor, that the myopia degree of the first person's eye is a myopia degree corresponding to the second visual acuity score.

[0060] In one possible design, the method further includes: before the first optical display module displays the first detection image to the first human eye, under the control of the processor, adjusting the optical focal length of the first optical display module to the initial optical focal length, so that the first optical display module is imaged at the farthest distance.

[0061] In one possible design, when the opening of the third inspection image presented by the first optical display module to the first eye is a third number of pixels, and when the received recognition feedback indicates that the user cannot identify the opening direction of the fourth inspection image, the optical power of the virtual image presented by the first optical display module to the first eye is a third optical power. The third optical power corresponds to a third visual fraction value, which is the visual fraction value corresponding to the third number of pixels.

[0062] In one possible design, the first optical display module includes a rotation mechanism configured to rotate the first optical display module in a direction perpendicular to the optical axis under control of the processor. The method further includes: when the first optical display module displays the first image to the user, the optical power direction of the first optical display module coincides with the astigmatism axis of the first eye.

[0063] In a possible design, the processor determines the astigmatism degree of the first human eye according to the rotation of the rotating mechanism when the optical power direction of the first optical display module coincides with the astigmatism axis of the first human eye.

[0064] In one possible design, the processor is further configured to obtain user characteristics of the current user before controlling the first optical display module to display the first image. The processor is specifically configured to control the first optical display module to display the first image corresponding to the user characteristics of the current user. When displaying the first image, the optical power of the first optical display module matches the myopia and / or astigmatism of the current user's first eye, as indicated by virtual display information corresponding to the user characteristics of the current user.

[0065] In one possible design, the user feature includes any one of the following features: fingerprint information of the current user; iris features of the current user; account information of the current user; and an identifier of the current user, where the identifiers of different users are different.

[0066] In one possible design, the virtual display device stores a correspondence between different user characteristics and corresponding virtual display information. The processor searches for a matching entry in the correspondence based on the current user's user characteristics. If a matching entry exists, the processor determines that the virtual display information corresponding to the current user is the virtual display information stored in the matching entry. The virtual display information includes the user's myopia and / or astigmatism.

[0067] In one possible design, when there is no matching item corresponding to the user characteristics of the current user in the corresponding relationship, the first optical display module, under the control of the processor, displays a first image, and the optical focal length when displaying the first image is an optical focal length that matches the myopia and / or astigmatism of the first eye of the current user, and the myopia and / or astigmatism of the first eye are automatically determined by the processor, or determined under the user's instructions, or manually input by the user.

[0068] In one possible design, the processor stores a correspondence between the myopia degree and / or astigmatism degree of the first person's eye and the user characteristics of the current user.

[0069] In a seventh aspect, a virtual display device is provided. The virtual display device includes one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions; when the one or more processors execute the computer instructions, the virtual display device implements the functions of the virtual display device of the first aspect and any of the various possible designs; or implements the functions of the virtual display device of the second aspect and any of the various possible designs; or implements the functions of the virtual display device of the third aspect and any of the various possible designs; or implements the functions of the virtual display device of the fourth aspect and any of the various possible designs; or implements the functions of the virtual display device of the fifth aspect and any of the various possible designs.

[0070] In an eighth aspect, a chip system is provided, which includes an interface circuit and a processor; the interface circuit and the processor are interconnected by lines; the interface circuit is used to receive signals from a memory and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, a virtual display device provided with the chip system implements the functions of the virtual display device of the first aspect and any one of various possible designs; or implements the functions of the virtual display device of the second aspect and any one of various possible designs; or implements the functions of the virtual display device of the third aspect and any one of various possible designs; or implements the functions of the virtual display device of the fourth aspect and any one of various possible designs; or implements the functions of the virtual display device of the fifth aspect and any one of various possible designs.

[0071] In the ninth aspect, a computer-readable storage medium is provided, which includes computer instructions. When the computer instructions are executed, the functions of the virtual display device of the first aspect and any one of the various possible designs are realized; or the functions of the virtual display device of the second aspect and any one of the various possible designs are realized; or the functions of the virtual display device of the third aspect and any one of the various possible designs are realized; or the functions of the virtual display device of the fourth aspect and any one of the various possible designs are realized; or the functions of the virtual display device of the fifth aspect and any one of the various possible designs are realized.

[0072] It should be understood that the technical solutions provided in the above-mentioned second to ninth aspects, and their technical features can all correspond to the shooting methods provided in the first aspect and its possible designs, so the beneficial effects that can be achieved are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is a schematic diagram of a convergence angle;

[0074] Figure 2 A schematic diagram of the composition of a human eye;

[0075] Figure 3 A schematic diagram of the imaging mechanism of the human eye;

[0076] Figure 4 A schematic diagram of the imaging mechanism of VR glasses;

[0077] Figure 5A This is a schematic diagram of the imaging mechanism of another type of VR glasses;

[0078] Figure 5B This is a schematic diagram of the imaging mechanism of another type of VR glasses;

[0079] Figure 6A A schematic diagram of the composition of a wearable device provided in an embodiment of the present application;

[0080] Figure 6B A schematic diagram of the composition of an optical display module provided in an embodiment of the present application;

[0081] Figure 7 A schematic diagram of the composition of a folding optical lens assembly provided in an embodiment of the present application;

[0082] Figure 8 A schematic diagram of the operation of a folding optical lens assembly provided in an embodiment of the present application;

[0083] Figure 9 A schematic diagram of the components of VR glasses provided in an embodiment of the present application;

[0084] Figure 10 A schematic diagram of imaging of VR glasses provided in an embodiment of the present application;

[0085] Figure 11 A flowchart of a virtual display method provided in an embodiment of the present application;

[0086] Figure 12 A schematic diagram of imaging of another VR glasses provided in an embodiment of the present application;

[0087] Figure 13 A schematic diagram of a display effect provided in an embodiment of the present application;

[0088] Figure 14 is a schematic diagram of another imaging mechanism of the human eye;

[0089] Figure 15 is a schematic diagram of another imaging mechanism of the human eye;

[0090] Figure 16A schematic diagram of a vision chart provided in an embodiment of the present application;

[0091] Figure 17 A schematic diagram of a visual score provided in an embodiment of the present application;

[0092] Figure 18 A flowchart of another virtual display method provided in an embodiment of the present application;

[0093] Figure 19 A schematic diagram of an imaging mechanism provided in an embodiment of the present application;

[0094] Figure 20 A schematic diagram of an imaging mechanism provided in an embodiment of the present application;

[0095] Figure 21 A schematic diagram of an imaging mechanism provided in an embodiment of the present application;

[0096] Figure 22 A schematic diagram of a detection image provided in an embodiment of the present application;

[0097] Figure 23 A schematic diagram of an input method for recognition feedback provided in an embodiment of the present application;

[0098] Figure 24 A schematic diagram of a display mechanism provided in an embodiment of the present application;

[0099] Figure 25 A schematic diagram of a display mechanism provided in an embodiment of the present application;

[0100] Figure 26 A schematic diagram of an interactive scene of VR glasses provided in an embodiment of the present application;

[0101] Figure 27 A flowchart of another virtual display method provided in an embodiment of the present application;

[0102] Figure 28 A schematic diagram of a red-green balancing method provided in an embodiment of the present application;

[0103] Figure 29 A schematic diagram of another imaging mechanism provided in an embodiment of the present application;

[0104] Figure 30 A schematic diagram of another imaging mechanism provided in an embodiment of the present application;

[0105] Figure 31 A schematic diagram of an astigmatism detection chart provided in an embodiment of the present application;

[0106] Figure 32A flowchart of another virtual display method provided in an embodiment of the present application;

[0107] Figure 33 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

[0108] Figure 34 A schematic diagram of the composition of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0109] Below, some terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0110] (1) At least one of the embodiments of the present application includes one or more; wherein, more means greater than or equal to two. In addition, it should be understood that in the description of the present application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as expressing or implying relative importance, nor can they be understood as expressing or implying order. For example, the first object and the second object do not represent the importance of the two, or the order of the two, but are for distinguishing objects.

[0111] In the embodiments of the present application, "and / or" is a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0112] (2) Virtual Reality (VR) technology is a means of human-computer interaction created with the help of computer and sensor technology. VR technology integrates a variety of scientific and technological technologies such as computer graphics technology, computer simulation technology, sensor technology, and display technology to create a virtual environment. The virtual environment includes three-dimensional realistic images generated by computers and played in real time to provide users with visual perception; in addition to the visual perception generated by computer graphics technology, there are also auditory, tactile, force, movement and other perceptions, and even smell and taste, which are also called multi-perception; in addition, the user's head rotation, eyes, gestures, or other human behavior movements can be detected, and the computer processes the data corresponding to the user's movements, responds to the user's movements in real time, and feeds back to the user's five senses respectively, thereby forming a virtual environment. For example, when a user wears a VR wearable device, he can see the VR game interface and interact with the VR game interface through gestures, handles, and other operations, as if he were in the game.

[0113] (3) Augmented Reality (AR) technology refers to the process of superimposing computer-generated virtual objects on real-world scenes to enhance the real world. In other words, AR technology requires capturing real-world scenes and then adding a virtual environment to the real world.

[0114] Therefore, the difference between VR and AR lies in that AR creates a completely virtual environment, where users see only virtual objects. AR, on the other hand, overlays virtual objects onto the real world, encompassing both real-world and virtual objects. For example, a user wearing transparent glasses can see their surroundings through them, while also displaying virtual objects on the glasses. This allows the user to see both real and virtual objects.

[0115] (4) Mixed Reality (MR) technology introduces real-world scene information (or real-world scene information) into a virtual environment, building a bridge of interactive feedback between the virtual environment, the real world, and the user, thereby enhancing the realism of the user experience. Specifically, real objects are virtualized (for example, using a camera to scan real objects for 3D reconstruction and generate virtual objects), and the virtualized real objects are introduced into the virtual environment, so that users can see the real objects in the virtual environment.

[0116] It should be noted that the technical solutions provided in the embodiments of the present application can be applied to electronic devices using technologies such as VR, AR, or MR. Electronic devices can provide users with virtual display functions through AR, VR, or MR technologies. This allows users to experience the stereoscopic visual experience of virtual scenes through the virtual display function without being in the actual environment.

[0117] In order to clearly explain the virtual display function provided to users through AR, VR or MR technology, the following first briefly explains the mechanism of human eye vision.

[0118] It is understandable that in actual scenes, when a user views an object, the human eye can obtain light signals in the actual scene and process the light signals in the brain to achieve visual perception. The light signals in the actual scene may include reflected light from different objects and / or light signals directly emitted by the light source. Since the light signals in the actual scene can carry relevant information about each object in the actual scene (such as size, position, color, etc.), the brain can obtain information about the objects in the actual scene, that is, obtain visual perception, by processing the light signals.

[0119] It should be noted that since the human eyes (such as the left eye and the right eye) have slightly different viewing angles when viewing the same object, the scenes seen by the left eye and the right eye are actually different. For example, the left eye can obtain the light signal of a two-dimensional image of the plane where the human eye's focus is located, which is perpendicular to the left eye's line of sight (hereinafter referred to as the left eye image). Similarly, the right eye can obtain the light signal of a two-dimensional image of the plane where the human eye's focus is located, which is perpendicular to the right eye's line of sight (hereinafter referred to as the right eye image). The left eye image is slightly different from the right eye image. The brain can obtain relevant information about different objects in the current scene by processing the light signals of the left eye image and the right eye image.

[0120] In addition, users can also obtain the depth of different objects in the actual scene to obtain a stereoscopic visual experience. Stereoscopic visual experience is also called binocular stereo vision.

[0121] For example, the brain can determine the depth of an object (ie, depth of field) by using the vergence distance and the accommodation distance of the two eyes when viewing an object in a real scene.

[0122] The brain can Figure 1 The mechanism shown in Figure 2 determines the convergence depth. Figure 1 As shown in the figure, when observing an object in a real scene, the left and right eyes can be moved toward the object by controlling the muscles near the eyes. The brain can determine the depth of the object by obtaining the convergence angle of the two eyes, that is, the convergence depth. As an example, Figure 1 As shown, when observing Figure 1 When observing an object as shown, the convergence angle can be the angle between the two lines of sight at the location of the observed object, as shown in the figure. It will be understood that the closer the observed object is to the human eye, the larger the convergence angle and the smaller the depth of convergence. Conversely, the farther the observed object is from the human eye, the smaller the convergence angle and the larger the depth of convergence.

[0123] In addition, the brain can also judge the depth of the object based on the zoom depth. Figure 2 as well as Figure 3 The zoom depth is explained. Figure 2 The figure shows the composition of the human eye. Figure 2 As shown, the human eye may include a lens and ciliary muscle, as well as a retina located at the fundus.

[0124] The lens can act as a zoom lens to converge the light entering the human eye. In order to converge the incident light onto the retina at the fundus of the human eye, the scene in the actual scene can be clearly imaged on the retina. The ciliary muscle can be used to adjust the shape of the lens. For example, the ciliary muscle can adjust the refractive power of the lens by contracting or relaxing, thereby adjusting the focal length of the lens. As a result, objects at different distances in the actual scene can be clearly imaged on the retina through the lens. As an example, refer to Figure 3 , which shows how the ciliary muscle adjusts the lens when the human eye observes objects at different distances. Figure 3 As shown in (a) in the figure, when the human eye observes an object at a distance, the object is taken as a non-light source. The reflected light from the surface of the object can be close to parallel light. At this time, the ciliary muscle can control the state of the lens to be as follows: Figure 3 In the state shown in (a), if the ciliary muscle is relaxed, the lens is controlled to be flat and the refractive power is small, so that parallel incident light can pass through the lens and converge on the retina at the fundus. When the human eye observes a relatively close object, combined with Figure 3 In (b), the object is a non-light source. The reflected light from the surface of the object can be Figure 3 The light path shown in (b) enters the human eye. At this time, the ciliary muscle can change the state of the lens to Figure 3 In the state shown in (b), the ciliary muscle contracts, the lens bulges, and the diopter increases, so that Figure 3 The incident light shown in (b) can pass through the lens and then converge on the retina at the fundus.

[0125] That is, the contraction or relaxation of the ciliary muscles varies when the human eye observes objects at different distances. This allows the brain to determine the depth of an object based on the current contraction or relaxation of the ciliary muscles when the eye is clearly observing the object. This depth is called zoom depth.

[0126] Currently, electronic devices can use AR, VR or MR technology, combined with the above-mentioned human eye vision generation mechanism, to display virtual scenes to users and provide virtual display functions.

[0127] For example, the electronic device that provides a virtual display function through AR, VR or MR technology is taken as an example of VR glasses with VR display function. Figure 4 Schematic diagram of a VR glasses. Figure 4As shown, two display screens (such as display screen 1 and display screen 2) can be provided in the VR glasses, and each display screen has a display function. Each display screen can be used to display corresponding content to one eye of the user (such as the left eye or the right eye) through the corresponding eyepiece. For example, on display screen 1, the corresponding left-eye image in the virtual scene can be displayed. The light of the left-eye image can pass through eyepiece 1 and converge at the left eye, so that the left eye sees the left-eye image. Similarly, on display screen 2, the corresponding right-eye image in the virtual scene can be displayed. The light of the right-eye image can pass through eyepiece 2 and converge at the left eye, so that the right eye sees the right-eye image.

[0128] Thus, the brain can fuse the left-eye image and the right-eye image, so that the user can see objects in the virtual scene corresponding to the left-eye image and the right-eye image.

[0129] It should be noted that due to the convergence effect of the eyepiece, Figure 5A As shown in the figure, the image seen by the human eye is actually the image displayed on the corresponding display screen. Figure 5A For example, the left-eye image seen by the left eye can be the virtual image corresponding to the left-eye image on the virtual image plane. For another example, the right-eye image seen by the right eye can be the virtual image corresponding to the finite image on the virtual image plane.

[0130] It's understandable that when observing objects in a real scene, the depth of the object as judged by the brain based on the vergence depth and zoom depth can be consistent. However, when the depth of the object indicated by the vergence depth and zoom depth are inconsistent, visual fatigue occurs, affecting the user's visual experience. In this example, the inconsistency between the depth of the object indicated by the vergence depth and zoom depth can also be referred to as vergence accommodation conflict (VAC).

[0131] In VR glasses, Figure 4 or Figure 5A When the solution shown shows a virtual scene to the user, the depth of the object indicated by the convergence depth and the zoom depth may be inconsistent.

[0132] For example, combined Figure 5B When the human eye observes the corresponding display screen, in order to see the image on the display screen clearly, the ciliary muscles will adjust the lenses of both eyes so that the image on the virtual image plane can be focused on the retina through the lenses. Therefore, the zoom distance can be the distance from the virtual image plane to the human eye (such as Figure 5B The depth shown is 1). However, the objects in the virtual scene displayed to the user by VR glasses are often not on the virtual image surface. For example, in the virtual scene, the observed object is Figure 5BThe user can rotate his eyeballs to focus his eyes on the triangle in the virtual scene (such as the dotted triangle 501). Figure 5B As shown by the symbol. In this way, the convergence depth should be the depth of the observed object (such as the dotted triangle 501) in the virtual scene. For example, the convergence depth can be as follows Figure 5B Depth 2 shown.

[0133] As you can see, Depth 1 and Depth 2 are not consistent. This can cause the brain to be unable to accurately judge the depth of the observed object, leading to visual fatigue and other effects that affect the user experience. If this continues for a long time, it can also have a serious impact on the user's vision.

[0134] In order to solve the above problems, the embodiments of the present application provide a virtual display device and a virtual display method, which can avoid VAC that occurs in the process of providing virtual display functions to users through AR, VR or MR technology, thereby avoiding the resulting visual fatigue and improving the user's visual experience.

[0135] The following describes in detail the solution provided in the embodiments of the present application with reference to examples and drawings.

[0136] For example, please refer to Figure 6A , taking the virtual display device as a wearable device as an example, a structural diagram of a wearable device provided by an embodiment of the present application is shown. Figure 6A As shown, the wearable device 100 may include a processor 110, a memory 120, a sensor module 130 (which can be used to obtain the user's posture), a microphone 140, a button 150, an input and output interface 160, a communication module 170, a camera 180, a battery 190, an optical display module 1100 and an eye tracking module 1200, etc.

[0137] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the wearable device 100. In other embodiments of the present application, the wearable device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0138] The processor 110 is generally used to control the overall operation of the wearable device 100 and may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a video processing unit (VPU) controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0139] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0140] In some embodiments of the present application, the processor 110 can be used to control the optical focal length of the wearable device 100. For example, the processor 110 can be used to control the optical focal length of the optical display module 1100 to achieve the function of adjusting the optical focal length of the wearable device 100. For example, the processor 110 can adjust the relative positions between the various optical devices (such as lenses, etc.) in the optical display module 1100 so that the optical focal length of the optical display module 1100 is adjusted, thereby adjusting the position of the corresponding virtual image surface when the optical display module 1100 is imaged to the human eye. This achieves the effect of controlling the optical focal length of the wearable device 100.

[0141] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, a serial peripheral interface (SPI) interface, etc.

[0142] In some embodiments, the processor 110 may render different objects based on different frame rates, for example, rendering near objects at a high frame rate and rendering distant objects at a low frame rate.

[0143] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses.

[0144] A UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the communication module 170. For example, the processor 110 communicates with the Bluetooth module in the communication module 170 via the UART interface to implement Bluetooth functionality.

[0145] The MIPI interface can be used to connect the processor 110 with the display screen, camera 180 and other peripheral devices in the optical display module 1100 .

[0146] The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 180, the display screen in the optical display module 1100, the communication module 170, the sensor module 130, the microphone 140, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc. Optionally, the camera 180 can capture images including real objects, and the processor 110 can fuse the images captured by the camera with virtual objects, and the images obtained by the optical display module 1100 can be fused with reality. This example can be seen in Figure 9 The application scenarios shown are not repeated here.

[0147] The USB interface is an interface that complies with USB standards and specifications, and can specifically be a Mini USB interface, a Micro USB interface, a USB Type-C interface, etc. The USB interface can be used to connect a charger to charge the wearable device 100, and can also be used to transfer data between the wearable device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. This interface can also be used to connect other electronic devices, such as mobile phones. The USB interface can be USB 3.0, which is compatible with high-speed display port (DP) signal transmission and can transmit high-speed audio and video data.

[0148] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely a schematic illustration and does not constitute a structural limitation on the wearable device 100. In other embodiments of the present application, the wearable device 100 may also adopt a different interface connection method from the above embodiments, or a combination of multiple interface connection methods.

[0149] In addition, the wearable device 100 may include a wireless communication function. For example, the wearable device 100 may receive a rendered image from another electronic device (such as a VR host or VR server) for display, or receive an unrendered image and then the processor 110 renders and displays the image. The communication module 170 may include a wireless communication module and a mobile communication module. The wireless communication function may be implemented through an antenna (not shown), a mobile communication module (not shown), a modem processor (not shown), and a baseband processor (not shown).

[0150] Antennas are used to transmit and receive electromagnetic wave signals. Wearable device 100 may include multiple antennas, each of which can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antenna can be used in conjunction with a tuning switch.

[0151] The mobile communication module can provide wireless communication solutions for the wearable device 100, including second-generation (2G) networks, third-generation (3G) networks, fourth-generation (4G) networks, and fifth-generation (5G) networks. The mobile communication module may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), etc. The mobile communication module can receive electromagnetic waves from an antenna, filter, amplify, and perform other processing on the received electromagnetic waves, and transmit them to a modem processor for demodulation. The mobile communication module can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation through the antenna. In some embodiments, at least some of the functional modules of the mobile communication module may be provided in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module may be provided in the same device as at least some of the modules of the processor 110.

[0152] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker, etc.) or displays an image or video through a display screen in the optical display module 1100. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module or other functional modules.

[0153] The wireless communication module can provide wireless communication solutions applied to the wearable device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module can be one or more devices that integrate at least one communication processing module. The wireless communication module receives electromagnetic waves via an antenna, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna.

[0154] In some embodiments, the antenna of the wearable device 100 is coupled to the mobile communication module so that the wearable device 100 can communicate with a network and other devices via wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS) and / or satellite-based augmentation system (SBAS).

[0155] Wearable device 100 implements display functionality through a GPU, optical display module 1100, and an application processor. The GPU is a microprocessor for image processing that connects the optical display module 1100 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0156] The memory 120 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the wearable device 100 by running the instructions stored in the memory 120. The memory 120 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the wearable device 100 (such as audio data, a phone book, etc.), etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0157] The wearable device 100 can implement audio functions such as music playback and recording through the audio module, speaker, microphone 140, headphone jack, and application processor.

[0158] The audio module is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module can also be used to encode and decode audio signals. In some embodiments, the audio module can be provided in the processor 110, or some functional modules of the audio module can be provided in the processor 110.

[0159] The speaker, also known as a "horn," is used to convert audio electrical signals into sound signals. The wearable device 100 can listen to music or make hands-free calls through the speaker.

[0160] Microphone 140, also known as a "microphone" or "microphone," is used to convert sound signals into electrical signals. The wearable device 100 can be provided with at least one microphone 140. In other embodiments, the wearable device 100 can be provided with two microphones 140, which can not only collect sound signals but also implement noise reduction. In other embodiments, the wearable device 100 can also be provided with three, four, or more microphones 140 to collect sound signals, reduce noise, identify sound sources, implement directional recording functions, and the like.

[0161] The headphone jack is used to connect wired headphones. The headphone jack can be a USB port, a 3.5 mm Open Mobile Terminal Platform (OMTP) standard port, or a CTIA (Cellular Telecommunications Industry Association of the USA) standard port.

[0162] In some embodiments, the wearable device 100 may include one or more buttons 150 that can control the wearable device and provide the user with access to functions on the wearable device 100. The button 150 can be in the form of a button, a switch, a dial, and a touch or near-touch sensing device (such as a touch sensor). Specifically, for example, the user can turn on the optical display module 1100 of the wearable device 100 by pressing a button. The button 150 includes a power button, a volume button, etc. The button 150 can be a mechanical button. It can also be a touch button. The wearable device 100 can receive button input and generate key signal input related to user settings and function control of the wearable device 100.

[0163] In some embodiments, the wearable device 100 may include an input / output interface 160, which may connect other devices to the wearable device 100 through appropriate components. The components may include, for example, audio / video jacks, data connectors, and the like.

[0164] The optical display module 1100 is used to present images to the user under the control of the processor. Using one or more optical devices, such as reflectors, transmissive mirrors, or optical waveguides, the optical display module 1100 can convert real-pixel images into near-eye projection for virtual image display, enabling a virtual interactive experience or a combined virtual and real-world interactive experience. For example, the optical display module 1100 receives image data information sent by the processor and presents the corresponding image to the user.

[0165] In some embodiments, the wearable device 100 may further include an eye tracking module 1200, which is configured to track eye movements and thereby determine the gaze point of the eye. For example, image processing techniques may be used to locate the pupil position, obtain the coordinates of the pupil center, and thereby calculate the gaze point of the eye.

[0166] Combination pair Figure 6A As shown in the illustration of the wearable device 100, the virtual display device provided in the embodiment of the present application has the function of automatically adjusting the optical focal length. In some embodiments, this function can be implemented by the optical display module 1100.

[0167] For example, please refer to Figure 6B, is a schematic diagram of the composition of an optical display module provided in an embodiment of the present application. The virtual display device can be used to support AR, VR or MR technology to provide a virtual display function. In a specific implementation, the virtual display device can be a head-mounted display (HMD) device, such as AR, VR or MR glasses, AR, VR or MR helmets, or AR, VR or MR all-in-one machines. Or the virtual display device can also be included in the above-mentioned head-mounted virtual display device. It should be noted that, in some embodiments, the virtual display device can also be used to support the implementation of mixed reality (MR) technology.

[0168] like Figure 6B As shown, the optical display module may include an eyepiece 601 , a zoom module 602 , and a display screen 603 .

[0169] Among them, the eyepiece 601 can be an optical device or device group such as a Fresnel lens, and / or an aspheric lens. The eyepiece 601 can be used to project light from the display screen 603 into the user's eyes. In some embodiments, the eyepiece 601 can have a positive optical focal length. As a result, the light from the display screen 603 can be converged into the human eye through the eyepiece 601 in a smaller space. Exemplarily, in different implementations, when the distance from the human eye to the display screen 603 is different, the display screen 603 can be projected to different virtual image distance positions by adjusting the distance between the eyepiece 601 and the display screen 603. The virtual image distance position can be the position where the user's human eye is located.

[0170] As an example, Figure 7 The schematic diagram of a pancake folding optical lens assembly provided by an embodiment of the present application is shown. The pancake folding optical lens assembly (hereinafter referred to as the pancake lens assembly) can be used to achieve the following Figure 6B The function of the eyepiece is shown. Figure 7 As shown, the Pancake lens assembly may include at least five optical components (e.g., 701-705). The specific implementation of any optical component in 701-705 may be an optical lens with a corresponding function, or the corresponding function of the component may be achieved by optical coating on adjacent lenses or other optical components.

[0171] exist Figure 7 In the example of , the order of the optical components of the Pancake lens group from the object side to the image side can be 701-702-703-704-705. Figure 7Specific examples of various optical components are also shown. In this example, 701 can be a polarizer (P). 702 can be a quarter-wave plate (QWP). 703 can be implemented by a beam splitter (BS). 704 can be a quarter-wave plate. 705 can be implemented by a polarization reflector (PR).

[0172] The following examples illustrate the light processing mechanism of the Pancake lens group during operation. Figure 8 , taking the example of the object side light being emitted by the display screen of the optical display module and the image side light entering the human eye.

[0173] As shown in the figure, after the incident light enters 701, it can be transmitted in the order of 702-703-704-705.

[0174] Exemplarily, light can be modulated into linear polarization after passing through 701. In some embodiments, the modulation direction of 701 can be set to the y-axis direction. Thus, after passing through 701, the incident light can be modulated into linear polarization in the y-axis direction. Then, the linear polarization can pass through 702, thereby adjusting it to rotationally polarized light. Exemplarily, if the fast axis direction of 702 is 45° to the y-axis, the linear polarization can be adjusted into right-handed polarized light after passing through 702. The right-handed polarized light can be incident on 703. Due to the semi-transparent and semi-reflective characteristics of 703, part of the right-handed polarized light can be transmitted through 703, while the other part can be reflected by 703. The right-handed polarized light transmitted through 703 can be incident on 704. When the fast axis direction of 704 is the same as that of 702, the right-handed polarized light transmitted through 703 can directly penetrate 704 and be incident on 705. The right-handed polarized light incident on 705 can be modulated into linear polarization along the x-direction and reflected on the surface of 705.

[0175] The light reflected by 705 can pass through 704 - 703 and be reflected at 703 .

[0176] For example, the light reflected on surface 705 can pass through 704 and be modulated into right-handed polarized light, and a portion of the right-handed polarized light can be reflected on surface 703. It should be noted that after reflection on surface 703, the right-handed polarized light can be modulated into left-handed polarized light.

[0177] The left-handed polarized light reflected by 703 can be emitted from the Pancake lens group through 704-705 and finally enter the human eye.

[0178] For example, after passing through 704, the left-handed polarized light can be modulated into linearly polarized light with a polarization direction along the y-axis. This linearly polarized light along the y-axis can then be emitted from the Pancake lens assembly through 705 and enter the human eye. It will be appreciated that in some embodiments, the polarization transmission characteristics of 705 can be set to transmit linearly polarized light along the y-axis, thereby ensuring that linearly polarized light along the y-axis can be smoothly emitted from 705.

[0179] In this way, light can be folded and transmitted through the Pancake lens assembly in the order of 701-702-703-704-705-704-703-704-705, thus achieving the effect of folding the light path. Therefore, it is possible to achieve long light path transmission in a small space (such as inside an optical display module such as VR glasses).

[0180] In some embodiments, the zoom module 602 can adjust the distance between one or more optical components (e.g., lenses) in the eyepiece 601 to change the optical power, thereby adjusting the depth of the virtual surface. For example, the zoom module 602 can pancake the relative positions of one or more optical components in the lens assembly to change the optical power, thereby adjusting the depth of the virtual surface.

[0181] It should be noted that the above Figure 7 or Figure 8 The composition shown is merely an example. In other embodiments of the present application, the Pancake lens assembly may include more or fewer optical components. Furthermore, the functions implemented by the optical lenses in the above examples may also be implemented by other optical components, such as by applying corresponding optical coatings to adjacent optical lenses.

[0182] It can be understood that, combined with the above description of the eyepiece in the optical display module, Figure 7 or Figure 8 Based on the description of the Pancake lens assembly, the optical power of the lens assembly can be increased by folding the optical path, thereby achieving the effect of shortening the focal length. This can shorten the length of the lens barrel (i.e., the optical display module) along the optical axis. This allows the optical display module to meet the miniaturization requirements of virtual display devices.

[0183] In various embodiments of the present application, Figure 6B The specific implementation of the eyepiece shown can be different. For example, the eyepiece can use the above example with Figure 7 or Figure 8 For another example, the eyepiece can also use other optical lenses or lens groups to achieve the purpose of projecting the light from the display screen to the human eye.

[0184] In such Figure 6BIn the optical display module shown, the display screen 603 may include, but is not limited to, a liquid crystal display (LCD), an organic light-emitting diode (OLED), a micro-light-emitting diode (Micro-LED), a quantum dot light-emitting diode (QLED), and other display components. In the embodiment of the present application, the display screen serves as the image source of the optical display module and can be used to display the image to be displayed.

[0185] In the optical display module, the function of the zoom module 602 can be implemented by a component with automatic zoom function, such as a mechanical zoom mechanism, a liquid crystal zoom device, a liquid zoom device, an Alvarez lens, etc. In some embodiments of the present application, the zoom module 602 can achieve the purpose of zooming the optical display module by changing its own optical focal length.

[0186] Exemplarily, in some embodiments, combined with the aforementioned Figure 6A Description, based on this Figure 6B The processor in the virtual display device can be used to control the zoom module to adjust the optical focal length. By adjusting the optical focal length, the distance from the virtual image plane to the user's eyes is adjusted when the optical display module provides a virtual display to the user, thereby achieving a match between the convergence depth and the focusing depth. In other embodiments, the processor can also achieve matching of the optical display module with the user's myopia during the display process. In other embodiments, the processor can also be used to control the rotation of the zoom module to match the axis position corresponding to the user's astigmatism degree, thereby matching the user's astigmatism degree.

[0187] In the following examples of this application, the operations controlled and executed by the optical display module / VR glasses can all be completed by the processor and will not be repeated below.

[0188] It should be noted that Figure 6BThe composition of the optical display module shown is only a logical illustration. In a specific implementation, the number of eyepieces, zoom modules and display screens can be flexibly set according to different needs. For example, in some embodiments, the optical display module may include two eyepieces, two zoom modules, and two display screens. Among them, one eyepiece, one zoom module and one display screen can constitute a sub-optical display module for providing a virtual display function to one of the user's eyes (such as the left eye or the right eye). For example, in other embodiments, the eyepieces that provide the virtual display function for the left eye and the right eye can also be integrated on the same component, so that only one eyepiece can be included in the optical display module. Similarly, in other embodiments, the zoom module and / or the display screen can also be integrated on one or more components, rather than being independently arranged. For example, the zoom component can be integrated with the eyepiece in the same component (such as the eyepiece lens assembly). For example, the eyepiece can be integrated in the zoom module.

[0189] Furthermore, in different embodiments of the present application, the specific positions of the eyepiece, zoom module, and display screen within the optical display module may vary. For example, in some embodiments, the order of the positions of the aforementioned components within the optical path, along the line of sight of the human eye, may be: eyepiece, zoom module, display screen. In other embodiments, the zoom module may be positioned between the eyepiece and the eye. That is, along the line of sight of the human eye, the order of the positions of the various components may be: zoom module, eyepiece, display screen.

[0190] As an example, Figure 9 A specific implementation example of a virtual display device having the functions related to the optical display module described above is shown. In this example, the virtual display device can be a VR glasses. The VR glasses can include two groups of Figure 6B In some embodiments, the optical display module is composed of Figure 9 The composition of the VR glasses shown in the figure can also be used for Figure 6A Another division of wearable devices in the industry, related modules can also realize corresponding functions.

[0191] For example, the VR glasses may include a first display component consisting of a display screen L, a zoom module L, and an eyepiece L. The first display component can be used to provide a virtual display function to the user's left eye. The VR glasses may also include a second display component consisting of a display screen R, a zoom module R, and an eyepiece R. The second display component can be used to provide a virtual display function to the user's right eye. It should be noted that, in the embodiments of the present application, the display component may also be referred to as an optical display module. For example, the first display component may be referred to as a first optical display module. For another example, the second display component may also be referred to as a second optical display module.

[0192] It should be noted that in some embodiments of the present application, the VR glasses may also include other components (such as an eye tracking system) for tracking the user's eye movements. The eye tracking system can determine the user's gaze point position (or determine the user's line of sight direction) through methods such as video eye diagram method, photodiode response method, or pupil corneal reflection method, thereby achieving user eye tracking.

[0193] In some embodiments, the eye tracking system may include one or more near-infrared light-emitting diodes (LEDs) and one or more near-infrared cameras. Figure 9 In different examples, the near-infrared LED can be arranged around the eyepiece to fully illuminate the human eye. In some embodiments, the central wavelength of the near-infrared LED can be 850nm or 940nm. The eye tracking system can obtain the user's line of sight direction by the following method: the human eye is illuminated by a near-infrared LED, and the near-infrared camera captures the image of the eyeball. Then, based on the position of the reflection point of the near-infrared LED on the cornea and the center of the pupil in the eyeball image, the optical axis direction of the eyeball is determined, and finally the user's line of sight direction is obtained through a calibration procedure in which the user participates.

[0194] It should be noted that, in some embodiments of the present application, a corresponding eye tracking system can be set for each of the user's eyes (e.g. Figure 9 Infrared camera R and infrared camera L are shown in the figure to synchronously or asynchronously track the eye movements of both eyes. In other embodiments of the present application, an eye tracking system can be set up near only one eye to obtain the gaze direction of the corresponding eye through the eye tracking system. Based on the relationship between the gaze points of the two eyes (for example, when a user observes an object through both eyes, the gaze points of the two eyes are generally close or the same), combined with the distance between the two eyes of the user, the gaze direction or gaze point position of the other eye can be determined.

[0195] In the embodiment of the present application, Figure 6A A wearable device having the composition shown, or having Figure 9 The virtual display device (such as VR glasses) composed as shown can control the zoom module in the optical display module to adjust the virtual image surface (such as Figure 5A or Figure 5BThe position of the virtual image plane (shown as a virtual image plane) is then adjusted. The adjusted zoom depth can be close to or equal to the vergence depth corresponding to the human eye observing objects in the virtual scene (or the zoom depth matches the vergence depth). This avoids visual fatigue caused by the difference between the vergence depth and the zoom depth, and improves the user experience.

[0196] As an example, the following combination Figure 10 In the embodiments of the present application, a virtual display device (such as VR glasses) controls a zoom module to adjust the zoom depth to match the convergence depth. This example uses a display component provided in VR glasses as an example. The specific implementation of another display component can refer to this example and will not be described in detail here.

[0197] For example, in some embodiments, when a user observes a nearby object in a virtual scene (for example, the corresponding convergence depth when observing the object can be in the range of Figure 10 When the position of the virtual image plane 1 shown in (a) is reached, the VR glasses can control the variable value module to adjust its own optical power (such as adjusting it to optical power A) so that the position of the virtual image plane corresponding to the display component falls as shown in (a). Figure 10 In this way, the zoom depth and the convergence depth are matched when the user observes nearby objects in the virtual scene.

[0198] In other embodiments, when a user observes a distant object in a virtual scene (e.g., the corresponding convergence depth when observing the object can be as follows: Figure 10 When the position of the virtual image plane 1 shown in (b) is reached, the VR glasses can control the zoom module to adjust its own optical power (such as adjusting it to optical power B) so that the position of the virtual image plane corresponding to the display component falls as shown in (b). Figure 10 In this way, the zoom depth and the convergence depth are matched when the user observes distant objects in the virtual scene.

[0199] As a possible implementation, the optical power B may be smaller than the optical power A. For example, the optical power A may be -1D, and the optical power B may be -3D.

[0200] In order to enable those skilled in the art to more clearly understand the implementation of the virtual display method provided in the embodiment of the present application, the virtual display device that executes the virtual display method is as follows. Figure 9 Taking the VR glasses with the composition shown as an example, the implementation process of the virtual display method in this example is explained.

[0201] Exemplary, reference Figure 11 , is a flow chart of a virtual display method provided in an embodiment of the present application. Figure 11As shown, the method may include:

[0202] S1101. When presenting a virtual three-dimensional environment to a user, determine the user's line of sight through an eye tracking system.

[0203] In this example, the virtual 3D environment may include objects that are closer to the user and objects that are farther away from the user. Figure 12 Objects that are closer to the user can be Figure 12 The virtual object 1 shown in FIG. 1 and the object farther from the user may be as follows: Figure 12 It should be noted that in order to show the user objects in the virtual three-dimensional environment (such as virtual object 1 or virtual object 2), VR glasses can display different images on the display screen. Figure 12 , when displaying virtual objects of different depths to the user, the display screen needs to display different content. For example, take display screen L as an example. When displaying virtual object 1, an image including the virtual object 1 may be displayed on display screen L. The position of the virtual object 1 in the image may be near the intersection of the line of sight of the left eye when observing the virtual object 1 and the display screen L as shown in the figure (such as referred to as position 1). When displaying virtual object 2, an image including the virtual object 2 may be displayed on display screen L. The position of the virtual object 2 in the image may be near the intersection of the line of sight of the left eye when observing the virtual object 2 and the display screen L as shown in the figure (such as referred to as position 2). Obviously, when displaying virtual objects with different depths, the positions of the objects displayed on the display screen are different. For example, in Figure 12 In the scenario shown, when a virtual object 2 with a large depth is displayed, the position of the object on the display screen L (i.e., position 2) is closer to the middle of the display screen than when a virtual object 1 with a smaller depth is displayed (i.e., position 1).

[0204] In an embodiment of the present application, VR glasses can realize eye tracking by the pupil corneal reflection method and other methods described above. For example, when VR glasses are configured with eye tracking systems for both eyes, the direction of sight of the eye can be determined according to the eye tracking system of the corresponding eye. For example, taking the point P1 on the virtual object 1 as an example, when the eye observes the virtual object 1, the VR glasses can determine the direction of sight of the eye as follows: Figure 12 As shown in the line of sight when observing P1. For another example, taking the point P2 on the virtual object 2 as an example, when the human eye observes the virtual object 2, the VR glasses can determine that the direction of the human eye's line of sight can be as follows: Figure 12 The line of sight when observing P2 is shown.

[0205] S1102: Determine the current convergence depth according to the user's line of sight.

[0206] In combination with the above description, when a user observes an object in a virtual scene, he or she actually observes the projection of the corresponding object on the virtual image plane through the left eye and the right eye respectively.

[0207] For example, when a user observes virtual object 1, their left eye can view the projection of the virtual object on the corresponding virtual image plane A. The projection observed by the left eye can be perpendicular to the direction of the left eye's line of sight. Their right eye can view the projection of the virtual object on the corresponding virtual image plane A. The projection observed by the right eye can be perpendicular to the direction of the right eye's line of sight. However, if the depth of virtual image plane A (i.e., zoom depth) is inconsistent with the depth of the virtual object in the virtual scene (i.e., convergence depth), VAC will occur.

[0208] In this example, the VR glasses can determine the depth of the object currently being observed by the user in the virtual scene (i.e., the convergence depth) based on the line of sight of both eyes determined in S1101. As an example, take the user observing point P1 on virtual object 1 as an example. The VR glasses can determine that the convergence angle corresponding to the user observing point P1 is a1 based on the line of sight of the user using the left eye and the right eye when observing virtual object 1. Similarly, when the user observes point P2 on virtual object 2, the VR glasses can determine that the current convergence angle is a2.

[0209] In this way, based on the convergence angle, the VR glasses can determine the convergence depth of the current user's observation position. For example, let's continue to take the user observing point P1 as an example. Since the virtual three-dimensional environment is constructed by VR glasses, the three-dimensional coordinates of each object in the virtual three-dimensional environment are known to the VR glasses. In some embodiments, according to the angle of sight when the left eye and the right eye look at P1 in the virtual scene (such as Figure 12 As shown in a1), and the interocular distance between the left eye and the right eye, the VR glasses can calculate the distance between the user's currently observed point P1 and the user, that is, the convergence depth.

[0210] In other embodiments, the VR glasses can determine the location of the intersection of the user's gaze points in the virtual 3D environment based on the user's gaze direction obtained in S1101. This location can then be the user's gaze point in the virtual 3D environment. Based on the known 3D coordinates of the gaze point in the virtual 3D environment, combined with the user's 3D coordinates in the virtual 3D environment, the VR glasses can calculate the distance between the user and the gaze point, thereby obtaining the vergence depth.

[0211] It should be noted that in a virtual 3D environment, the user's line of sight may not necessarily intersect at a single point. Therefore, in some embodiments of the present application, the VR glasses can determine the 3D coordinates of the gaze point in the virtual 3D environment by the following method:

[0212] VR glasses can use the X coordinate and Y coordinate of the intersection of the projection of the user's binocular sight lines on the XOY plane (such as (X1, Y1)) as the projection coordinate of the user's current gaze point on the XOY plane (X1, Y1). VR glasses can also use the Z coordinate (such as Z) of the (X1, Y1) position in the virtual three-dimensional environment based on the left eye's sight line. L ), and the Z coordinate of the right eye's sight at the position (X1, Y1) (such as Z R ), determine the height of the gaze point. For example, the Z coordinate of the gaze point can be In this way, the coordinates of the gaze point in the virtual three-dimensional environment can be determined as (X1, Y1, ).

[0213] In other embodiments of the present application, when the VR glasses determine the user's dominant eye through eye tracking, the gaze point corresponding to the dominant eye can be used as the gaze point during binocular observation.

[0214] S1103 : Adjust the position of the virtual image plane according to the convergence depth so that the zoom depth matches the convergence depth.

[0215] The zoom depth and the depth of convergence match may include the zoom depth being the same as the depth of convergence, or the difference between the zoom depth and the depth of convergence being less than a preset threshold. When the zoom depth and the depth of convergence match, VAC generated during prolonged use of a virtual display device (such as VR glasses) can be avoided.

[0216] For example, VR glasses can adjust the focal length of the zoom module and adjust the position of the virtual image plane to achieve the effect of adjusting the zoom depth. Figure 10 The adjustment method shown is used to adjust the optical focal length of the zoom module, which will not be described in detail here.

[0217] Combine Figure 12 , take the user observing virtual object 1 as an example. VR glasses can adjust the virtual image surface to Figure 12 The position of the virtual image plane A shown in FIG. In this way, in order to clearly see the object on the virtual image plane A, the human eye can control the ciliary muscle to adjust the state of the lens so that the object on the virtual image plane A can be clearly imaged on the retina. At this time, the zoom depth determined by the human eye can be the distance from the human eye to the virtual image plane A. Correspondingly, the user can control the rotation of both eyes to focus the line of sight on point P1 in the virtual three-dimensional environment. Since point P1 is on the virtual image plane A, or point P1 is close to the virtual image plane A, the convergence depth determined by the user based on the rotation of both eyes can also be (or close to) the distance from the human eye to the virtual virtual image plane A. This achieves the matching of the zoom depth and the convergence depth.

[0218] Similarly, when the user observes the virtual object 2, the VR glasses can adjust the virtual image surface to Figure 12 The position of the virtual image plane B is shown, thereby achieving the matching of the zoom depth and the convergence depth.

[0219] In this way, when users observe objects in a virtual scene through VR glasses, they will not experience visual fatigue caused by the inconsistency between the zoom depth and the convergence depth. This can achieve the effect of providing users with virtual display functions while improving the user's visual experience and avoiding damage to eyesight.

[0220] It should be noted that, in some embodiments of the present application, in order to provide users with a better visual experience, the virtual display device can also be used in accordance with the following Figure 11 In addition to completing the matching of the zoom depth and the convergence depth (i.e., executing S1103), the method shown in FIG. 1 further blurs different objects in the currently displayed virtual three-dimensional environment according to the zoom depth (or convergence depth), thereby highlighting that the user can obtain a more intuitive stereoscopic experience when observing objects in the virtual three-dimensional environment.

[0221] For example, combined Figure 12 ,refer to Figure 13 In some embodiments, taking the example of a virtual object 1 being observed nearby, the VR glasses can blur the distant view according to the depth of the virtual object 1. Figure 13 As shown in (a) of FIG. , it can be seen that in this image, since the distant view is blurred, the user can have a clearer observation experience of the currently observed virtual object 1, thereby simulating the out-of-focus blur experience when the user observes the real scene. Similarly, when the user observes the distant virtual object 1, the VR glasses can blur the near view, thereby showing the user Figure 13 The image shown in (b) in the figure achieves the effect of highlighting the distant view.

[0222] It should be noted that when the virtual display device performs blurring processing on the distant view / near view, the degree of blurring processing can be determined based on the difference between the depth corresponding to the object to be processed (such as zoom depth or convergence depth) and the depth of the object currently being observed by the user. For example, take the blurring processing of the distant view by VR glasses as an example. When the depth corresponding to the object in the distant view differs greatly from the depth of the object in the near view currently being observed by the user (such as greater than the depth threshold), the object in the distant view is highly blurred. Correspondingly, when the depth corresponding to the object in the distant view differs little from the depth of the object in the near view currently being observed by the user (such as less than the depth threshold), the VR glasses can appropriately reduce the degree of blurring processing of the object in the distant view, thereby increasing the sense of hierarchy of the image observed by the user. In this example, the degree of blurring processing is determined by the depth threshold as an example. In other embodiments of the present application, multiple depth thresholds can also be preset in the VR glasses, each depth threshold corresponding to a degree of blurring processing, thereby achieving the effect of multi-layer blurring processing based on depth differences.

[0223] In conjunction with the above description, when a user observes a nearby object for a long time, the ciliary muscle will contract for a long time. This can seriously affect the user's vision, such as causing pseudomyopia or even true myopia.

[0224] For example, combined Figure 3 ,refer to Figure 14 .like Figure 3 As explained in the description, when the user uses the human eye to observe distant objects, the light incident on the human eye is close to parallel light, and the light path of the light in the human eye is as follows Figure 3 When a user uses their eyes to observe a nearby object, they can contract the ciliary muscle to adjust the refractive power of the lens. The light path in the human eye is as follows: Figure 3 When observing a close object for a long time, the ciliary muscle is in a contracted state for a long time. At this time, if the human eye observes an object at a different distance (such as a distant object), the ciliary muscle may not be able to relax in time, causing the refractive power of the lens to remain at a high level. At this time, after the light enters the human eye, the light path is as follows: Figure 15 As shown in the figure, it can be seen that after the light passes through the lens, it cannot focus on the retina. Therefore, a clear image cannot be formed on the retina. The user cannot see the corresponding object clearly. After a period of time, the ciliary muscle gradually relaxes, the refractive power of the lens is adjusted accordingly, and the light path after passing through the lens gradually returns to the normal state. Figure 3 The user will gradually be able to see objects clearly when the state shown in (a) is reached. This short period of inability to see objects clearly is called pseudomyopia. If the user's eyes are in a state of pseudomyopia for a long time, it may develop into true myopia, that is, the ciliary muscle cannot adjust the refractive power of the lens to return to the state shown in (a). Figure 3In the state shown in (a), after the parallel light enters the human eye, its focusing position is between the lens and the retina.

[0225] The virtual display device provided in the embodiments of the present application can control the zoom module to adjust the depth of the virtual image plane, thereby flexibly and clearly displaying both nearby and distant objects to the user without causing VAC. In some embodiments of the present application, the virtual display device can also provide users with ciliary muscle relaxation and exercise functions.

[0226] For example, when the virtual display device detects that the user has been observing a nearby virtual object for a long time (such as longer than a preset time), the user may be prompted to take a break, or a virtual object with a larger depth may be displayed. Figure 13 When the duration of the scene shown in (a) is longer than the preset duration, the VR glasses can remind the user that the user has been using their eyes for too long. In different implementations, the VR glasses can remind the user that the user has been using their eyes for too long by displaying prompt information (such as text information, etc.) on the display screen, or by vibration, or by voice, etc. Then, the VR glasses can display virtual objects with greater depth under the control of the user or spontaneously, such as Figure 13 In other embodiments, the virtual display device can also display the same object on the virtual image plane, and change the depth of the virtual image plane displayed by the object in the virtual environment through the focusing module, so that the user can focus on virtual image planes at different depths. In this way, the virtual display device can guide the ciliary muscle of the human eye to relax, thereby avoiding pseudomyopia. In addition, by repeatedly guiding the ciliary muscle of the human eye to relax and contract, the ciliary muscle can also be exercised to avoid muscle rigidity, thereby improving the ability to resist vision problems.

[0227] As a possible implementation, let's take VR glasses as an example of a virtual display device. VR glasses can provide the display of objects at different depths as described above during user use to protect the human eye. For example, VR glasses can adjust the optical focal length of the optical display module in the VR glasses by controlling the zoom module during the playback of advertisements or the waiting process for video loading, thereby displaying virtual objects at different depths to the user. In some embodiments, VR glasses can guide the user to control the human eye to observe the virtual objects at different depths through voice or text prompts or other human-computer interaction methods, thereby achieving the effect of adjusting the ciliary muscle of the human eye.

[0228] In light of the above, when the ciliary muscles of the human eye remain contracted for a long time, visual fatigue and even myopia may occur. For virtual display devices that provide virtual display functions (such as VR glasses), the proportion of myopic users in their user base is also increasing.

[0229] Generally speaking, in normal life, myopic users can correct their vision by wearing glasses. Figure 14 For myopic users, the focal plane of light after entering the human eye may be located between the lens and the retina. In this way, light (such as parallel light) cannot form a clear image on the retina after entering the human eye. Users can wear myopic glasses to adjust the optical path of light before entering the human eye, so that the light can form a clear image on the retina after entering the human eye. For example, refer to Figure 15 Taking the concave lens as an example, parallel light can be refracted by the concave lens before entering the human eye, so that the light can converge on the retina after passing through the lens, thus making the object emitting light form a clear image on the retina.

[0230] However, when myopic users use virtual display devices, wearing glasses is obviously not convenient enough. Therefore, in order to enable myopic users to see the images displayed on the display screen clearly when wearing the virtual display device (that is, the virtual image corresponding to the display screen can be clearly imaged on the retina of the human eye), in some implementations of virtual display devices, the virtual display device provides users with the ability to manually adjust the focal length of the optical system between the human eye and the display screen, such as by combining Figure 4 The composition of the VR glasses shown is schematic. In this implementation, the VR glasses can provide the user with a control method for adjusting the optical focal length of the eyepiece. For example, a mechanical rotation mechanism can be provided on the VR glasses, so that when the user rotates the mechanical rotation mechanism, the optical focal length of the eyepiece can be adjusted. By adjusting the optical focal length of the eyepiece, the user can clearly see the virtual image corresponding to the display screen through the human eye through manual control, that is, the virtual image of the display screen is clearly imaged on the retina of the human eye. In this example, the adjusted eyepiece can play the role of simulating myopia glasses. In this way, myopic users can use the virtual display function provided by the virtual display device without wearing myopia glasses.

[0231] It's important to note that the human eye has a certain degree of automatic adjustment when observing objects. For example, when the image of an object falls in front of or behind the retina, the eye automatically controls the ciliary muscles to adjust the concave and convex state of the lens to adjust the image position as much as possible, thereby adjusting the image position of the object to the retina.

[0232] Based on this, if the aforementioned manual adjustment solution is adopted, even if the user manually adjusts the optical power of the eyepiece so that the virtual image of the display screen can be clearly formed on the retina, due to the automatic adjustment mechanism of the human eye, the virtual image of the display screen in this state is clearly formed on the retina due to the excessive squeezing of the lens by the ciliary muscle. In other words, within the range of normal control of the lens by the ciliary muscle, the optical power of the current eyepiece cannot guarantee that the virtual image of the display screen can be clearly formed on the retina. Therefore, if the reality device provides the user with a virtual display function based on the current optical power of the eyepiece, the ciliary muscle of the user's eye will cause excessive squeezing of the lens for a long time, resulting in eye discomfort, which may seriously affect the user's eye health.

[0233] The virtual display device provided by the embodiment of the present application (such as Figures 6A-10 In the process of providing virtual display functions to users, the automatic adjustment function of the zoom device can be combined with the automatic adjustment function of the zoom device to accurately measure the myopia of the user's eyes. In this solution, the myopia of the human eye can be measured to avoid the influence of the automatic adjustment ability of the human eye. Therefore, according to the actual condition of the human eye (such as myopia), it can avoid eye discomfort when the user uses the virtual display function of the virtual display device, thereby improving the user experience.

[0234] The following is combined with the above Figures 6A-9 The description of the wearable device or virtual display device provided in the embodiment of the present application is described in detail through the accompanying drawings to describe the solution for providing virtual display according to the real situation of the user's human eyes provided in the embodiment of the present application.

[0235] For the sake of convenience, the following assumes that the user is a myopic user and the virtual display device has the following Figure 9 Take the VR glasses with the composition shown as an example.

[0236] In some embodiments of this example, before the VR glasses provide the virtual display function to the user, the image recognition capability of each eye of the user can be determined. In some embodiments, the image recognition capability of the human eye can be identified by the degree of the human eye.

[0237] It is understandable that the degree of human eyes can be determined by the internationally accepted vision chart. For example, Figure 16A visual acuity chart is shown. It may include letters used to determine the human eye's ability to recognize images, as well as the visual acuity corresponding to the letters of that size. For example, the visual acuity corresponding to the second row of letters is 4.2° (logarithmic visual acuity scale) or 0.15° (international visual acuity scale). In this example, the process of determining the eye's vision can also be called an optometry process. When using the eye chart to perform optometry on a user, the chart can be hung 5 meters away from the user. The user is guided to use their left and right eyes to determine the opening direction of the letters on the chart, respectively. The vision corresponding to the row with the smallest letter opening direction that can be determined is used as the vision for the left or right eye. For example, using the left eye to observe the eye chart, if the smallest opening direction of the letters corresponding to 5.0 (1.0) on the chart can be clearly seen, then the left eye's vision is likely 1.0. In this case, the image width of the opening of the letter corresponding to 5.0 (1.0) on the retina is approximately 5 μm, so the left eye's resolution can reach a minimum width of 5 μm. At this point, the left eye's vision is generally considered normal and not myopic.

[0238] It's understandable that for the human eye to discern the opening direction of letters, it needs to be able to distinguish the corresponding width of the opening direction based on the letter image on the retina. For example, if the human eye can clearly see the opening direction of a row of letters corresponding to 5.0 (1.0), it means that the human eye can distinguish the width of a 5μm image on the retina. The size of the letter image on the human eye is obviously related to the distance between the letter and the human eye. This is why it is important to ensure the distance between the human eye and the letter when performing an eye test using an eye chart.

[0239] Through this traditional optometry method, the degree of each eye of the user can be obtained, but it is necessary to use Figure 16 The eye chart shown here also has high requirements for the venue (for example, the venue needs to provide a distance of 5 meters between the person being tested and the eye chart). This is obviously not convenient enough. In addition, the traditional method of determining the user's ability to recognize images based on vision cannot be applied in virtual display scenarios.

[0240] To this end, the VR glasses provided in the embodiments of the present application can adopt the following scheme to determine the user's ability to recognize images.

[0241] For example, the VR glasses can sequentially display inspection images with openings of different sizes to the user on the display screen (for example, the inspection images may include Figure 16 The user's ability to recognize the image is determined by combining the user's feedback with the user's visual acuity chart (such as the letters on the eye chart shown).

[0242] In some embodiments of the present application, VR glasses can determine the user's ability to recognize images by recognizing the visual score of the image through the human eye.

[0243] The visual score value can refer to the minimum angle of the image that can be distinguished when the human eye observes the image. Figure 17 , which is a schematic diagram of visual score. The smallest opening that a user can distinguish a letter, such as 1701, is used. Then, the angle between the opening of the letter and the human eye (the sight angle shown in the figure) can be used as the visual score of the human eye.

[0244] Since the visual score is an angle indicator, the distance between the human eye and the inspection image can be adjusted freely without restriction.

[0245] After obtaining the visual score value of the human eye, the VR glasses can determine the vision of the human eye based on the correspondence between the visual score value and the human eye example, combined with the visual score value of the human eye.

[0246] As a possible implementation, the correspondence between the visual score value and human vision can be preset in the VR glasses, or the VR glasses can obtain it from the cloud when needed. In some embodiments, Table 1 shows a correspondence between the visual score value and human vision.

[0247] Table 1

[0248] Logarithmic visual acuity chart International Eye Chart Visual score 4.0 0.1 10′ 4.1 0.12 7.947′ 4.2 0.15 6.312′ 4.3 0.20 5.013′ 4.4 0.25 3.982′ 4.5 0.3 3.163′ 4.6 0.4 2.512′ 4.7 0.5 1.996′ 4.8 0.6 1.585′ 4.9 0.8 1.259′ 5.0 1.0 1′

[0249] As shown in Table 1, when the visual score that the human eye can distinguish is 10', the VR glasses can determine that the corresponding visual acuity is 4.0 (0.1). Similarly, when the visual score that the human eye can distinguish is 7.947', the VR glasses can determine that the corresponding visual acuity is 4.1 (0.12). When the visual score that the human eye can distinguish is 1.259', the VR glasses can determine that the corresponding visual acuity is 4.9 (0.8). And so on.

[0250] Please refer to Figure 18 , is a flow chart of a virtual display method provided in an embodiment of the present application. Based on this flow, VR glasses can obtain the image recognition capabilities corresponding to each of the user's eyes. Here, the image recognition capability is identified by vision as an example. As shown in the figure, the solution may include:

[0251] S1801: When a user uses VR glasses, a first detection image corresponding to a first visual score value is displayed to a first eye of the user.

[0252] The first human eye may be either the left eye or the right eye of the user. The first visual score value may be any visual score value in the corresponding relationship shown in Table 1. In order to accurately measure the visual acuity of the human eye, in some embodiments of the present application, when the VR glasses begin to detect the user's vision, the first visual score value may be the visual score value corresponding to the maximum viewing angle, such as 10' as shown in Table 1.

[0253] In this example, the VR glasses can display the first detection image on a display screen corresponding to the first person's eye (such as the first display screen), so that the first person's eye can observe the first detection image at the virtual image position corresponding to the first display screen. Figure 19 For example, the first human eye is the left eye, and the component in the VR glasses that provides the virtual display function to the left eye is the first display component. Figure 19 As shown, the VR glasses can control the display screen of the first display component to display the first detection image. In this way, the user can see the first detection image displayed at the position of the virtual image plane as shown in the figure through the left eye.

[0254] The user can observe the first detection image with a human eye (e.g., a first person's eye) to determine whether the first detection image corresponding to the first visual score value can be clearly seen. For example, the user can determine the opening direction of letters or graphics included in the first detection image and input the determination result to the VR glasses, so that the VR glasses can determine whether the first person's eye can clearly see the first detection image currently having the first visual score value.

[0255] For example, the detection image displayed by VR glasses includes a C-shaped letter. Figure 20 As shown, in S1801, the opening size of the C-shaped letter may correspond to the first visual score value.

[0256] That is to say, the VR glasses need to ensure that the opening size of the C-shaped letter seen by the user corresponds to the first visual score value of the first human eye.

[0257] In an embodiment of the present application, the VR glasses can determine the size of the C-shaped letter that needs to be displayed when displaying a C-shaped letter with a first visual score value based on the current field of view angle and the number of pixels provided for displaying the graphics in the field of view angle.

[0258] It is understood that when a display screen displays an image, it displays it in units of pixels. The VR glasses can determine the display size of the first detection image on the first display screen using the pixels per degree (PPD), thereby ensuring that the aperture of the first detection image seen by the first person's eye is the first visual fraction value.

[0259] For example, combined Figure 21 Take the viewing angle of the left eye when observing the first display screen as angle A as an example. Figure 21 As shown, the number of pixels on the display screen that participate in providing image display to the user's left eye can be N pixels. N pixels can be seen. Thus, the PPD in this scenario can be obtained according to the following formula (1).

[0260]

[0261] In this way, based on the PPD, the VR glasses can determine the number of pixels corresponding to the opening of the C-shaped letter when displaying a C-shaped letter corresponding to the first visual score value, in combination with the distance between the display screen and the user's eye (such as the first eye). Then determine the pixels corresponding to the display of the C-shaped letter. Based on this, the VR glasses can ensure that the opening size of the C-shaped letter seen by the user corresponds to the first visual score value of the first eye. For example, Figure 20 The display effect shown. It is understandable that through the above solution, the VR glasses can determine the size of the first detection image to be displayed based on the visual score value. Compared with directly calculating the size of the first detection image, the number of pixels required to display the first detection image can be more accurately determined, thereby displaying a more accurate first detection image with the first visual score value to the user.

[0262] It should be noted that, in some embodiments of the present application, the first detection image may include a letter or graphic for detecting human vision. Figure 22 . Figure 22 (a) in FIG. 1 shows an example in which a first detection image includes a letter for detecting vision. Figure 22 As shown in (a) of FIG. , the letter in the first detection image may have an opening (e.g., a gap in the letter C). The VR glasses can determine the display size of the C-shaped letter on the first display screen according to the above description, thereby displaying the C-shaped letter with the opening corresponding to the first visual score value to the user.

[0263] In other embodiments of the present application, the first detection image may include multiple letters or graphics for detecting human vision. For example, continue to take the example of the first detection image including C-shaped letters. VR glasses can display multiple C-shaped letters with openings corresponding to the first visual score value to the user at one time on the first display screen. Among them, the opening directions of two adjacent C-shaped letters can be different. In this way, the user can identify the opening directions of these multiple C-shaped letters in turn, so that the VR glasses can more accurately judge the user's vision according to the recognition results input by the user. For example, Figure 22 (b) shows an example display of a first detection image including multiple C-shaped letters. Based on this first detection image, the user can use their first eye to determine the opening direction of the corresponding letters from left to right, thereby enabling the VR glasses to more accurately determine whether the user can clearly see the detection image corresponding to the visual score value (such as the first visual score value).

[0264] In addition, to avoid mutual interference between the detection process of the first eye and the detection process of the second eye, in some embodiments of the present application, the VR glasses can display a black screen on another display screen (such as the second display screen) or turn off the second display screen while the first display screen displays the first detection image. After completing the detection of the first eye according to the solution provided in the embodiment of the present application, the VR glasses can control the second display screen to display according to a similar solution to achieve vision detection of another eye (such as the second eye).

[0265] S1802: Receive first recognition feedback from the user.

[0266] The user's first eye can see the first detection image of the opening corresponding to the first visual score value through the first display component.

[0267] In this example, the user can input first recognition feedback to the VR glasses based on the opening direction of the first detection image seen. In some embodiments, the first recognition feedback can be used to indicate the opening direction of the first detection image determined by the user.

[0268] For example, taking the example of a C-shaped letter included in the first detection image, the user can input the opening direction of the C-shaped letter into the VR glasses after seeing the C-shaped letter. That is, in this example, the first recognition feedback can be the opening direction of the C-shaped letter.

[0269] Combine Figure 23 Take the example of a user inputting the opening direction through a remote control. When the user recognizes that the opening direction of the C-shaped letter is upward through the first person's eyes. In some embodiments, the user can touch the button corresponding to "up" on the remote control (such as Figure 23 In other embodiments, the user can click the button 2301 shown in (a) of the remote control to input the first recognition feedback indicating that the opening direction is upward. Figure 23 On the screen 2302 shown in (b) in FIG, an upward sliding operation is input to input a first recognition feedback indicating that the opening direction is upward.

[0270] It should be noted that the above examples are based on the example of a user inputting the first recognition feedback via a remote control. In other embodiments of the present application, the user can also input the first recognition feedback via voice commands and / or gesture commands. The specific method of the user inputting the first recognition feedback in the embodiments of the present application is not limited.

[0271] In this way, the VR glasses can receive the recognition of the opening direction of the current first detection image by the user's first eye.

[0272] In some embodiments of the present application, in order to enable the user to correctly input the first recognition feedback, the VR glasses may guide the user to identify the first detection image and input the first recognition feedback before receiving the first recognition feedback from the user. For example, the VR glasses may guide the user to identify the first detection image and input the first recognition feedback through voice prompts, text prompts in the displayed virtual scene, or other methods (such as vibration, etc.).

[0273] S1803: Determine whether the user can recognize the first detection image based on the first recognition feedback.

[0274] In an embodiment of the present application, the VR glasses can determine the subsequent execution action based on whether the first recognition feedback is consistent with the actual opening direction of the first detection image. For example, when the first recognition feedback is inconsistent with the actual opening direction of the first detection image, it indicates that the first human eye cannot see the first display image with the first visual score value. Then the VR glasses can continue to execute the following S1804. Correspondingly, when the first recognition feedback is consistent with the actual opening direction of the first detection image, it indicates that the first human eye can see the first display image with the first visual score value. Then the VR glasses can continue to execute the following S1805.

[0275] It should be noted that, in some embodiments of the present application, when the VR glasses determine that the first recognition feedback is inconsistent with the actual opening direction of the first detection image, they again display the detection image with the first visual score value to the user, and the opening direction of the detection image may be different from the first detection image. So that the user can judge the opening direction of the detection image of the first visual score value again. After repeating the above action a preset number of times (such as three times), if the recognition feedback input by the user is still inconsistent with the opening direction of the detection image, then the VR glasses can clearly indicate that the user cannot see the image corresponding to the first visual score value, and then execute S1804. This can improve the accuracy of VR glasses in judging the user's vision.

[0276] S1804: Determine the visual acuity of the first person's eye according to the first visual score.

[0277] It is understandable that, when the first person's eye cannot identify the opening direction of the first detection image, the VR glasses can determine the vision of the first person's eye based on the first visual score value.

[0278] For example, in combination with Table 1. In some embodiments, the VR glasses can determine the visual acuity corresponding to the first visual score value based on the corresponding relationship in Table 1. Then the visual acuity can be the visual acuity of the first person's eye. For example, as shown in Table 1, when the first visual score value is 7.947', the VR glasses can determine that the visual acuity of the first person's eye is 4.1 (0.12). For another example, when the first visual score value is 1.259', the VR glasses can determine that the visual acuity of the first person's eye is 4.9 (0.8).

[0279] In this way, VR glasses can achieve the determination of the image recognition ability of the first person's eye.

[0280] S1805: Display a second detection image corresponding to the second visual score value.

[0281] The size of the second detection image corresponding to the second visual score value may be smaller than the size of the first detection image corresponding to the first visual score value, thereby achieving the purpose of gradient reduction detection of user examples.

[0282] For example, in combination with Table 1, the first visual score is 2.512' as an example. In S1803, when the VR glasses determine that the letter opening direction corresponding to the user's first recognition feedback is consistent with the actual opening direction of the first detection image. The VR glasses can display the visual score of the second detection image on the first display screen as 1.996'. In other words, the second detection image is more stringent in detecting vision than the first detection image, thereby further determining whether the user can see a smaller image (such as the second detection image) clearly when he can see the first detection image clearly.

[0283] In this step, the VR glasses determine the second detection image using a similar method as the first detection image. For example, the VR glasses can calculate the size of the second detection image based on the PPD and the second visual score, and then display the second detection image with the second visual score on the first display screen based on this size. The specific implementation is similar and will not be further described here.

[0284] S1806: Receive the user's second recognition feedback.

[0285] The second recognition feedback may be an indication of the opening direction of the second detection image inputted by the user to the VR glasses after the user observes the second detection image with the first eye. The specific execution process and implementation method thereof may refer to the first recognition feedback.

[0286] S1807: When the second recognition feedback is different from the actual opening direction of the second detection image, determine the visual acuity of the first person's eye according to the second visual score.

[0287] The specific implementation of this step is similar to the above S1804 and will not be repeated here.

[0288] It should be noted that if Figure 18 In the example shown, the user can identify the opening direction of the first detection image but cannot identify the opening direction of the second detection image. In other words, the VR glasses can provide the user with detection images with a gradient of decreasing visual score values ​​for the user to judge. The user's visual acuity can be determined based on the maximum visual score value corresponding to the inability to clearly identify the opening direction of the detection image.

[0289] As another possible implementation, continue to refer to Figure 18 . After executing 1806, the VR glasses can loop through S1801 along the dotted path when the second recognition feedback is different from the actual opening direction of the second detection image. That is, continue to display other detection images to the user. It can be understood that, in this example, the opening size corresponding to the detection image displayed to the user by the VR glasses again may be smaller than the opening size corresponding to the second detection image, thereby further increasing the difficulty for the user to identify the detection image until the maximum visual score value that cannot be recognized by the human eye is obtained.

[0290] Therefore, VR glasses can determine the user's eye's maximum visual acuity by determining the value of the eye's maximum resolution, thereby determining the eye's ability to recognize images.

[0291] In the specific implementation of the embodiment of the present application, VR glasses can control the number of detection images displayed on the display screen, so as to enhance the fun while obtaining the user's human eye's ability to recognize images, thereby achieving the effect of improving user experience.

[0292] For example, combined Figure 18 The scheme shown, refer to Figure 24 The first display screen may display the following when executing S1801, ie, displaying the first detection image: Figure 24The interface shown in 2410 is shown in FIG. As can be seen, in this example, the interface can display multiple (e.g., five) detection images (e.g., C-shaped letters) corresponding to the first visual score value. Different C-shaped letters can have different opening directions, allowing the user to determine the opening direction from left to right. For example, if the user cannot accurately identify the opening direction of the leftmost C-shaped letter in interface 2410, the VR glasses can display an interface shown in 2420 on the first display screen. This allows the user to continue to determine the opening direction of C-shaped letters with the first visual score value (e.g., letters within the area shown in 2402), thereby enabling the VR glasses to more accurately determine whether the first person's eye can recognize the first visual score value. As can be seen, on this interface 2420, the letters 2401 that the user has already identified can be hidden or not displayed, thereby achieving a visual elimination effect. This allows the user to continue to determine the opening direction of the leftmost C-shaped letter on the interface. Correspondingly, if the opening direction indicated by the first recognition feedback input by the user is consistent with the actual opening direction of the C-shaped letter, the VR glasses can display an interface shown in 2430 on the first display screen. This provides the user with a lower visual score for judgment. As can be seen on interface 2430, the position of the letter 2401, which the user has already judged, is blank. That is, the letter at that position is not displayed or is hidden, thus achieving a visual effect of elimination. The user can then continue to use their first eye to judge the opening direction of the leftmost letter displayed in the area shown in 2403, allowing the VR glasses to determine whether the user can recognize the image with the lower visual score.

[0293] Thus, through Figure 24 In the specific implementation provided by the example, VR glasses can achieve a visual elimination effect by hiding or not displaying the image of the position of the determined letters during the display of the detection image, thereby increasing the fun of the entire detection process and thus improving the user experience. At the same time, it can more easily locate the undetermined letters, improving detection efficiency.

[0294] In addition, the above example is explained by taking the example of only displaying one or more detection images with one visual score value to the user on a single interface (such as the interface of the first display screen). In other embodiments of the present application, the VR glasses can also display multiple detection images with different visual score values ​​to the first person's eye on the first display screen, and guide the user through images, voice, vibration, etc., so that the user can judge the opening direction of the image corresponding to each visual score value according to a certain process. In this way, the VR glasses can also obtain the minimum visual score value that the user can recognize.

[0295] For example, combined Figure 25 VR glasses can be used according to the above Figure 18 In the solution shown, the method for determining the letter size corresponding to each visual score value determines the size of the letter corresponding to different visual score values. And on the first display screen, the user is presented with the following information at one time: Figure 25 The VR glasses can prompt the user to judge the image of a specific row in the image to guide the user to complete the vision test. Figure 25 As shown, the VR glasses can display a prompt symbol (such as an arrow) as shown in 2501 near the row of images requiring user judgment. Through voice or text prompts, the user is guided to recognize the detection image displayed in that row and enter corresponding recognition feedback. As an example, the VR glasses can guide the user through a voice prompt: "Please determine the opening direction of the first letter on the left in the row of letters indicated by the arrow." This allows the user to determine the opening direction of the first letter on the left in the row indicated by the arrow. After the user enters recognition feedback regarding the opening direction of the letter, if the recognition feedback indicates that the user has correctly identified the letters in that row, the VR glasses can move the prompt symbol down one row (such as displaying 2502) and hide 2501 or not display it at all. This guides the user to determine the opening direction of letters with a lower visual score value. This process continues until the VR glasses determine the minimum visual score value that the user can recognize.

[0296] It is understandable that through the above Figure 18-Figure 25 As explained above, in the above embodiment, the VR glasses can provide detection images of different sizes corresponding to different visual score values ​​on the same virtual image plane. In other words, in this visual score-based detection solution, the VR glasses do not need to adjust the position of the virtual image plane or the position of the imaging plane after the light enters the human eye to detect the user's vision. Since the display size of the detection image in this visual score-based display solution corresponds to the current position of the virtual image plane (such as according to the PDD), there is no need to control the distance between the virtual image plane and the human eye to be 5m.

[0297] In other embodiments of the present application, the VR glasses can also adjust the virtual image plane to a specific position of 5m from the human eye through a zoom module. The VR glasses can display a virtual image of the same size as the international eye chart on the first display screen, and guide the user to identify the smallest identifiable letter opening in the virtual scene, and then determine the user's vision accordingly. This process is equivalent to simulating the optometry process in a real environment in a virtual scene, thereby achieving the purpose of determining the user's vision without requiring the actual site.

[0298] After completing the detection of one person's eyes, VR glasses can use a similar solution to detect another person's eyes, thereby obtaining the user's ability to recognize images with both eyes.

[0299] It should be noted that in the above Figures 18-25 The provided solution can detect the user's binocular vision (i.e., complete eye examination). This allows the user to obtain the degree of myopia in each eye. Obviously, this solution allows users to perform eye examinations through VR glasses without having to go to a professional optometrist to know the degree of myopia in both eyes.

[0300] In addition, in the above example, the user can detect the degree of myopia through the guidance of VR glasses. Figure 26 As shown, the VR glasses can also interact with the cloud (such as a server, etc.) so that the cloud can provide users with more professional and scientific optometry guidance through the VR glasses. The cloud can send instructions to the VR glasses to control the VR glasses to perform optometry on the user. In some implementations, the instructions on the cloud can be uploaded from other terminals that interact with the cloud. The instructions can be input by professional optometrists, other professionals, or determined by the cloud or other terminals according to a preset execution plan.

[0301] In an embodiment of the present application, the VR glasses can adjust the content displayed to the user based on the user's ability to recognize images (such as the degree of myopia of the user's eyes), so that myopic users can experience clear virtual display functions through the VR glasses without wearing glasses.

[0302] For example, VR glasses can be Figures 18-28 Any of the solutions, as well as any of the solutions in the aforementioned description, obtains the myopia degree of the user's eyes.

[0303] The following describes a specific solution for VR glasses to adaptively adjust the display content according to the user's myopia degree with reference to the accompanying drawings.

[0304] As a possible implementation, the following Table 2 shows a correspondence between the degree of myopia of the human eye and the optical power of VR glasses. This correspondence can be pre-set in the VR glasses or obtained from the cloud when the VR glasses are needed.

[0305] Table 2

[0306] vision Myopia optical power 4.0 650 -6.5D 4.1 600 -6D 4.2 550 -5.5D 4.3 500 -5D 4.4 450 -4.5D 4.5 400 -4D 4.6 325 -3.25D 4.7 250 -2.5D 4.8 125 -1.25D 4.9 50 -0.5D

[0307] According to Table 2, VR glasses can determine the corresponding optical power based on the user's visual acuity. For example, when the user's visual acuity of the first eye is 4.6, VR glasses can determine the optical power to be -3.25D. For another example, when the user's visual acuity of the first eye is 4.8, VR glasses can determine the optical power to be -1.25D. And so on. It should be noted that the correspondence shown in Table 2 is only an example and does not list all possible correspondences between optical power and myopia degree. In other implementations of the embodiments of the present application, Table 2 may also include more or fewer correspondences.

[0308] In this example, the VR glasses can determine the adjustment mechanism corresponding to the optical power based on a preset strategy. For example, taking the example of implementing the function of the zoom module through mechanical structure adjustment, the VR glasses can preset the correspondence between different optical powers and the relative positions of each lens in the zoom module. Therefore, after the optical power is determined, the VR glasses can control the zoom module to adjust the relative positions of each lens to the position corresponding to the optical power, thereby achieving corresponding adjustment of the optical power.

[0309] In this way, the VR glasses can adjust the distance between the virtual image plane of the display screen and the human eye to an appropriate range by adjusting the optical focal length, so that the virtual image plane can form a clear image in the user's eyes. It should be noted that because the VR glasses control the zoom module to adjust the optical focal length before the virtual image plane forms an image in the human eye, the distance between the virtual image plane and the image plane formed in the human eye (such as the virtual image distance) is matched with the user's current myopia. In this way, the image on the virtual image plane can be clearly formed on the retina of the human eye, so that the user can see the image displayed by the VR glasses with the naked eye.

[0310] In order to enable those skilled in the art to more clearly understand the solution provided by the embodiment of the present application, the following is combined with the accompanying drawings to illustrate the VR glasses according to the following example. Figure 18 The solution shown here uses an example of obtaining the user's myopia and adjusting the content displayed to the user based on the myopia. It should be noted that VR glasses can implement the following solution based on one of the user's eyes to adjust the displayed image for that eye. A similar strategy is implemented for the other eye.

[0311] It is understandable that when the human eye is not myopic or the degree of myopia is very small, the parallel light incident on the human eye can be smoothly converged on the retina of the human eye. When the virtual image plane is at the farthest distance (i.e. 0D), the light incident on the human eye is closest to parallel light. Therefore, in an embodiment of the present application, the virtual image distance corresponding to 0D can be used as a reference to display detection images of different visual scores to the user, so as to adjust to the position of the virtual image distance that matches the human eye according to the user's recognition situation. Thereby, the effect of providing the human eye with a virtual image that matches the degree of myopia of the human eye is achieved. For example, please refer to Figure 27 , is a flow chart of a virtual display solution provided in an embodiment of the present application. As shown in the figure, the solution may include:

[0312] S2701: Control the zoom module to adjust the virtual image plane to a first position. For example, the first position may be a 0D position.

[0313] The 0D position may be the farthest distance that the VR glasses can reach by controlling the zoom module to adjust the virtual image distance. In some embodiments, the processor may control the focal length of the zoom module so that the virtual image plane is adjusted to the 0D position.

[0314] It is understandable that in the subsequent adjustment process, the VR glasses may need to control the zoom module to reduce the virtual image distance so that the virtual image distance can match the user's myopia degree.

[0315] S2702: Display the detection image 1 corresponding to the first visual score value. For example, the first visual score value may be 10'. Then the detection image 1 may be a detection image of a size corresponding to 10'.

[0316] In this example, the size of the detection image 1 corresponding to the visual score of 10' can be determined and displayed by referring to the above Figure 18 The determination and display method of the first detection image in the illustrated solution will not be described in detail here.

[0317] It should be noted that the detection image 1 corresponding to 10′ can be the detection image with a larger aperture size in this solution, so it is the least difficult for the user's eyes to recognize. Through the following solution, VR glasses can gradually reduce the aperture size of the detection image, thereby achieving a step-by-step increase in recognition difficulty, thereby enabling VR glasses to more accurately determine the user's myopia degree.

[0318] It should be noted that, in the embodiments of the present application, there is no strict order of execution of S2701 and S2702. For example, in some embodiments, S2701 can be executed before S2702, that is, the VR glasses can first adjust the position of the virtual image plane through the processor, and then display the detection image 1 on the virtual image plane. In other embodiments, S2701 can also be executed simultaneously with S2702, that is, the VR glasses can display the detection image 1 at the position of the corresponding virtual image plane during the process of adjusting the position of the virtual image plane. In other embodiments, S2701 can also be executed after S2702, that is, the VR glasses can determine the size of the detection image 1 according to the first visual score value, and display it on the current virtual image plane, and then adjust the position of the virtual image plane through the processor, so as to achieve the effect of displaying the detection image 1 at the 0D position.

[0319] S2703A: Determine whether the user can recognize the detection image 1.

[0320] If the user can recognize the opening direction of the detection image 1, S2704 is executed. If the user cannot recognize the opening direction of the detection image 1, S2703B is executed.

[0321] It is understandable that, combined with Figure 18 In S1802 , the user may input recognition feedback corresponding to the detection image 1 to the VR glasses, so that the VR glasses know whether the user can recognize the opening direction of the detection image 1 .

[0322] S2703B: Control the zoom module to adjust the virtual image plane to a second position. For example, the second position may be a position corresponding to -7D.

[0323] When the user cannot identify the opening direction of the detection image 1, it means that the user has a high degree of myopia. The VR glasses can control the zoom module and adjust the virtual image distance so that the virtual image plane is at a position corresponding to -7D, so that the virtual image plane of the display screen can be imaged at a closer position in the human eye as much as possible, thereby matching the user's myopia and allowing the user to see the image on the virtual image plane clearly through the naked eye.

[0324] S2704: Display the detection image 2 corresponding to the second visual score value. For example, the second visual score value may be 7.947'. Then the detection image 2 may be a detection image of a size corresponding to 7.947'.

[0325] After the VR glasses determine that the user can recognize the detection image with a larger visual score value (such as detection image 1), the visual score value can be appropriately reduced and detection image 2 can be displayed to determine the user's myopia degree.

[0326] S2705A: Determine whether the user can recognize the detection image 2.

[0327] If the user can recognize the opening direction of the detection image 2, S2706 is executed. If the user cannot recognize the opening direction of the detection image 2, S2705B is executed.

[0328] S2705B: Control the zoom module to adjust the virtual image plane to a third position. For example, the second position may be a position corresponding to -6D.

[0329] S2706: Display the detection image 3 corresponding to the third visual score value. For example, the third visual score value may be 5′, and the detection image 3 may be a detection image of a size corresponding to 5′.

[0330] S2707A: Determine whether the user can recognize detection image 3.

[0331] If the user can recognize the opening direction of the detection image 3, S2708 is executed. If the user cannot recognize the opening direction of the detection image 3, S2707B is executed.

[0332] S2707B: Control the zoom module to adjust the virtual image plane to a fourth position. For example, the fourth position may be a position corresponding to -5D.

[0333] S2708: Display the detection image 4 corresponding to the fourth visual score. For example, the fourth visual score may be 3.2′. Then the detection image 4 may be a detection image of a size corresponding to 3.2′.

[0334] S2709A: Determine whether the user can recognize the detection image 4.

[0335] If the user can recognize the opening direction of the detection image 4, S2710 is executed. If the user cannot recognize the opening direction of the detection image 4, S2709B is executed.

[0336] S2709B: Control the zoom module to adjust the virtual image plane to a fifth position. For example, the fifth position may be a position corresponding to -4D.

[0337] S2710: Display the detection image 5 corresponding to the fifth visual score value. For example, the fifth visual score value may be 2′. Then the detection image 5 may be a detection image of a size corresponding to 2′.

[0338] S2711A: Determine whether the user can recognize the detection image 5.

[0339] If the user can recognize the opening direction of the detection image 5, S2712 is executed. If the user cannot recognize the opening direction of the detection image 5, S2711B is executed.

[0340] S2711B: Control the zoom module to adjust the virtual image plane to a sixth position. For example, the sixth position may be a position corresponding to -2.5D.

[0341] S2712: Display the detection image 6 corresponding to the sixth visual score value. For example, the sixth visual score value may be 1.5′. Then the detection image 6 may be a detection image of a size corresponding to 1.5′.

[0342] S2713A: Determine whether the user can recognize the detection image 6.

[0343] If the user can recognize the opening direction of the detection image 6, S2714 is executed. If the user cannot recognize the opening direction of the detection image 5, S2712B is executed.

[0344] S2713B: Control the zoom module to adjust the virtual image plane to the seventh position. For example, the seventh position may be a position corresponding to -1D.

[0345] S2714: Control the zoom module to adjust the virtual image plane to a first position. In combination with S2701, the first position may be a position corresponding to 0D.

[0346] After the above process, if the user can still identify the opening direction of the detection image 6 with the minimum visual score, then this indicates that the user's eyesight is good and the image on the virtual image plane can be clearly imaged on the human retina without adjusting the optical power of the system. Therefore, the VR glasses can execute S2714 to fix the virtual image plane to the 0D position, that is, to the position with the farthest virtual image distance, thereby providing a clear virtual display to the human eye.

[0347] It should be noted that the above description is based on an example of a detection image displayed to the user with a gradually decreasing size. In other embodiments of the present application, the VR glasses can also display detection images to the user with a gradually increasing size. In other embodiments of the present application, the VR glasses can also start with an intermediate size and use a dichotomy method to display detection images larger than the starting intermediate size and smaller than the starting intermediate size to the user, thereby determining the user's vision and adjusting the position of the virtual image surface accordingly.

[0348] It is understandable that in Figure 27 In the scheme shown, the optical power position of the virtual image surface corresponding to each visual score value can be obtained through the above Table 2. In different implementations, the gradient and number of the corresponding relationship in Table 2 are different, and the implementation based on this is Figure 27 The illustrated scheme may also be varied.

[0349] In this way, VR glasses can adaptively adjust the optical focal length according to the user's myopia, so that even if the user's eyes are in a myopic state, there is no need to wear glasses and the virtual display function provided by VR glasses can be used with naked eyes.

[0350] In conjunction with the aforementioned description of the human eye's automatic adjustment function, the human eye can autonomously adjust itself when recognizing images of different visual scores, attempting to identify the image's opening direction. In some embodiments of the present application, to avoid inaccurate vision detection caused by excessive adjustment of the lens by the ciliary muscle during the user's eye recognition of the image's opening direction, a red-green balance solution can be introduced in addition to the above solution.

[0351] For example, combined Figure 28 When different light rays pass through the human eye (such as the lens of the human eye) to form images, due to the different wavelengths of the light rays, the refraction paths of the light rays after passing through the lens are different. Figure 28 As shown, when white light can be focused on the retina, the focusing point of green light can be located between the lens and the retina, while the focusing point of red light can be located behind the retina.

[0352] In this example, after the VR glasses determine the user's myopia, they can further fine-tune the virtual image distance in combination with the red-green balance solution so that the virtual image can be displayed to the human eye at a virtual image distance position that matches the user's actual myopia.

[0353] For example, combined Figure 27 , after executing S2703A, the VR glasses determine that the user cannot recognize the opening direction of the detection image 1 as an example. Figure 27 As shown, the VR glasses can execute S2703B, that is, control the zoom module to adjust the virtual image plane to the -7D position. Combined with the red-green balance solution, the VR glasses can continue to control the zoom module after adjusting the virtual image plane to the position corresponding to -7D, and make fine adjustments near the position corresponding to -7D (such as fine-tuning between -6.5D and -7D) to find the position where the red and green light have the same imaging clarity on the retina. The VR glasses can use this image plane position as the fine-tuning result, so that in the process of displaying the virtual image to the user's current human eyes, the adjustment result can be used to show the user a clear virtual image.

[0354] Similarly, when executing S2705B, the VR glasses can control the zoom module to adjust the virtual image plane to the -6D position, and then fine-tune between -6D and -6.5D to obtain the position where the red light and green light have the same imaging clarity on the retina as the adjustment result. Alternatively, when executing S2707B, the VR glasses can control the zoom module to adjust the virtual image plane to the -5D position, and then fine-tune between -5D and -6D to obtain the position where the red light and green light have the same imaging clarity on the retina as the adjustment result. Alternatively, when executing S2709B, the VR glasses can control the zoom module to adjust the virtual image plane to the -4D position, and then fine-tune between -4D and -5D to obtain the position where the red light and green light have the same imaging clarity on the retina as the adjustment result. Alternatively, when executing S2711B, the VR glasses may control the zoom module to adjust the virtual image plane to a -2.5D position, and then fine-tune between -2.5D and -4D to obtain a position where the red and green light images on the retina are consistent in clarity as the adjustment result. Alternatively, when executing S2713B, the VR glasses may control the zoom module to adjust the virtual image plane to a -1D position, and then fine-tune between -1D and -2.5D to obtain a position where the red and green light images on the retina are consistent in clarity as the adjustment result.

[0355] Thus, through the above Figure 27 The coarse adjustment shown, combined with fine adjustments to the red-green balance scheme, can achieve a result that matches the user's actual myopia. This display prevents the user's eyes from over-accommodating, thus preventing problems such as eye fatigue.

[0356] It is understood that the above description uses the example of identifying a user's ability to recognize graphics using their myopia. It is also understood that for some users, in addition to myopia, they may also have astigmatism. Different astigmatism degrees can correspond to different astigmatism axis directions.

[0357] For example, combined Figure 29 as well as Figure 30 Explain astigmatism. Figure 29 As shown in , for users with astigmatism, at different angles, parallel light enters the human eye and is refracted by the lens, and the convergence position in the human eye may be different. Figure 29 As shown in (a) in the figure, after the parallel light in the XOZ plane enters the human eye, it can converge on the retina. At the same time, after the parallel light in the YOZ plane enters the human eye, its convergence point may no longer be on the retina (such as Figure 29 (b) is located between the lens and the retina). This will cause the image at some angles to be unable to be clearly imaged by the human eye. For example, combined with Figure 30 .like Figure 30 As shown in (a), when the human eye has no astigmatism, all incident light at all angles can form images evenly in the human eye, that is, the images of light at all angles in the human eye are similar. However, for human eyes with astigmatism, some angles of light may not form clear images in the human eye. For example, Figure 30 In (b), in the current coordinate system, vertical light can form an image better, and the closer the light angle is to the horizontal, the worse the imaging clarity.

[0358] However, in real scenes or virtual scenes provided by VR glasses, the light entering the human eye generally does not come from just one angle, which will cause blur when the user observes the object, affecting the user experience. In order to enable users with astigmatism to use the virtual display function provided by VR glasses with naked eyes, the embodiment of the present application also provides a solution that can determine the astigmatism degree of users with astigmatism. Then, the image displayed to the user is corrected according to the astigmatism degree, so that users with astigmatism can also use the virtual display function of VR glasses with naked eyes.

[0359] As an example, the solution can be implemented by the virtual display device provided in the embodiment of the present application. For example, the virtual display device is a device having Figure 10 Take the VR glasses composed as an example. In this example, the zoom module in the VR glasses can generate cylindrical optical power and has a rotation function. When the zoom module rotates, the zoom module can be rotated around the corresponding line of sight center (such as around the corresponding lens barrel center), so that the cylindrical optical power direction of the optical system composed of the zoom module and the eyepiece coincides with the user's astigmatism axis. In some embodiments, if the human eye also has myopia, the optical power of the zoom module can be an optical power that matches the myopia of the human eye. The specific matching and adjustment methods can refer to the solutions provided in the above embodiments and will not be repeated here.

[0360] Therefore, when the virtual display device provides a virtual display to the user, the light incident on the user's eyes can be the light that coincides with the axis of astigmatism, so that the light can form a clear image in the human eye, avoiding the problem of unclear imaging caused by astigmatism. This also allows users with astigmatism to use the virtual display solution provided by VR glasses with their naked eyes.

[0361] Similar to the aforementioned process for myopia detection, in embodiments of the present application, the virtual display device can adjust the axis position of the incident light according to the degree of astigmatism of the human eye to avoid unclear imaging caused by astigmatism. In other embodiments of the present application, combined with the aforementioned remote optometry solution, the virtual display device provided in embodiments of the present application can also be used to measure the degree of astigmatism.

[0362] For example, when measuring the astigmatism of the human eye, VR glasses can combine the current myopia of the human eye to make corresponding measurements. For example, if the myopia degree is M (unit is m -1 ) as an example. VR glasses can control the zoom module so that the virtual image plane is 5 meters away from the human eye after myopia correction.

[0363] For example, the VR glasses can determine the 5m position after myopia correction according to the following formula (2).

[0364]

[0365] Where M is the degree of myopia and V is the distance from the human eye in the virtual scene.

[0366] VR glasses can display the following on the virtual image surface: Figure 31 Astigmatism detection chart shown. VR glasses can guide users to input the angle at which they cannot see clearly. For example, the angle can be Figure 31 The angle corresponding to any one or more numbers from 1 to 12 shown. In some embodiments, the user can input relevant information about the unclear angle through voice input, text input on a remote control, etc. Based on the angle information input by the user, the VR glasses can determine the astigmatism axis direction corresponding to the digital information input by the user. For example, if the number input by the user is 1, the VR glasses can determine that the astigmatism axis direction is 60°. Similarly, if the number input by the user is 5, the VR glasses can determine that the astigmatism axis direction is 120°. And so on.

[0367] In this example, the VR glasses can control the rotation of the zoom mechanism according to the direction of the user's astigmatism axis, and then adjust the cylindrical optical power of the zoom mechanism until the user can see clearly the following two images at the same time: Figure 31 The line on the astigmatism detection chart shown in the figure is 1 / 4 of the image. At this time, the cylindrical optical power of the zoom mechanism is the user's astigmatism degree. In this way, the astigmatism of the user's eyes can be detected.

[0368] In some embodiments, combined with the implementation of the aforementioned remote optometry solution, the astigmatism detection process can also be implemented remotely. For example, the physician performing the remote detection can control the VR glasses through the cloud to guide the user to perform the astigmatism detection.

[0369] It should be noted that the above descriptions of the solutions for myopia correction and astigmatism correction are all based on the user's single eye as an example. In scenarios where correction of both eyes is required, the VR glasses can perform the corresponding steps in the above description for both eyes separately, thereby achieving corresponding adjustments based on the image recognition capabilities of each eye (such as myopia and / or astigmatism), thereby ensuring the user's naked-eye virtual display experience.

[0370] It is understandable that different users have different abilities to recognize images. In some embodiments of the present application, the VR glasses can associate the acquired user's myopia and / or astigmatism with the user information to avoid repeated detection of the same user the next time the VR glasses are used.

[0371] For example, in some embodiments, the VR glasses can generate a correspondence between the user's iris information and the degree of myopia and / or astigmatism and store it locally or in the cloud. This allows the user to determine the degree of myopia and / or astigmatism based on the iris information the next time the VR glasses are used, thereby controlling the zoom module to adjust the focal length and / or the axis position of the incident light. This provides the user with a naked-eye virtual display.

[0372] Please refer to Figure 32 , which is another virtual display method provided by an embodiment of the present application. As shown in the figure, this solution may include:

[0373] S3201. When a user uses VR glasses, extract the user's iris features.

[0374] For example, the VR glasses can capture the user's eyes through their eye tracking module, and extract iris features of the eyes by analyzing the captured images.

[0375] In the implementation of this application, VR glasses can only extract the iris features of one eye of the user, thereby realizing the confirmation of the user's identity while saving computing power.

[0376] S3202: Determine whether there is virtual display information matching the user based on the acquired iris features.

[0377] The virtual display information may include myopia degree and / or astigmatism degree.

[0378] In some embodiments, the VR glasses may store a correspondence between different users' iris features and their myopia and / or astigmatism. Thus, after obtaining the current user's iris features, the VR glasses can query the correspondence to see if there is a match corresponding to the iris features. If so, virtual display information matching the user exists. Conversely, if no match is found, virtual display information matching the user does not exist.

[0379] In other embodiments, the correspondence between different users' iris features and myopia and / or astigmatism levels can be stored in the cloud. The VR glasses can send the acquired iris features of the current user to the cloud, so that the cloud can query whether there is a match corresponding to the iris features. If the cloud determines that there is a match corresponding to the iris features, the cloud can send the VR glasses a prescription for the refractive error corresponding to the match in the aforementioned correspondence. In some embodiments, the prescription for the refractive error can include myopia and / or astigmatism. Upon receiving the myopia and / or astigmatism levels, the VR glasses can confirm that virtual display information matching the user exists, and the virtual display information matching the user can be the myopia and / or astigmatism levels received from the cloud. Accordingly, if the cloud determines that there is no match corresponding to the iris features, it can send a "not found" notification to the VR glasses, so that the VR glasses can confirm that there is no match corresponding to the iris features. Alternatively, if the cloud determines that there is no match corresponding to the iris features, the cloud can not send any information to the VR glasses. In this way, the VR glasses can determine that there is no match corresponding to the iris feature if no myopia degree and / or astigmatism degree is received within a preset time.

[0380] It should be noted that, in other embodiments of the present application, the myopia degree and / or astigmatism degree included in the virtual display information may be identified by corresponding optical power and / or axis position.

[0381] Furthermore, since the zoom module changes the optical focal length, the magnification of the entire display system will change. This means that the size of the VR content viewed by the user will change. Furthermore, the zoom module's change in optical focal length will also cause the distortion parameters of the display system to change. Therefore, in some embodiments of the present application, the virtual display information may also include distortion correction parameters and / or magnification parameters corresponding to the optical focal length during myopia adjustment.

[0382] In the case where there is virtual display information matching the user, the VR glasses may execute the following S3204.

[0383] In the absence of virtual display information matching the user, the VR glasses may execute the following S3205.

[0384] S3204: Retrieve the virtual display information, and provide a virtual display to the user according to the virtual display information.

[0385] When it is determined that there is a match, the VR glasses can retrieve the various parameters included in the virtual display information and provide a virtual display to the current user's eyes based on these parameters.

[0386] S3205. Determine the virtual display information of the user.

[0387] S3206. Provide a virtual display to the user according to the virtual display information of the user.

[0388] The process of determining the user's virtual display information can refer to the process of determining the user's myopia and the process of detecting the user's astigmatism in the above examples. Thus, the user's virtual display information can be determined. The process of providing a virtual display to the user based on the virtual display information can refer to the solution for adjusting the optical power based on the myopia and the solution for adjusting the axial position of the incident light based on the user's astigmatism in the above examples. This will not be further elaborated here.

[0389] In some embodiments, since the current user's iris features do not match any of the stored iris features, this indicates that the user may be new. In this case, to ensure the security of the user's iris features, the VR glasses may obtain the user's authorization before executing S3205. This allows the VR glasses to establish a correspondence between the current user's iris features and the corresponding virtual display information. In some implementations, the VR glasses may store this correspondence to avoid repeated detection the next time the user uses the VR glasses.

[0390] It should be noted that, in Figure 32 In the scheme shown, the example of identifying different users by their iris features is used for explanation. In other embodiments of the present application, the virtual display device can also identify different users by other means. For example, the virtual display device can identify different users by their user accounts and / or their biometric information (such as eye distance, fingerprint, voiceprint, etc.). Then, similar to the correspondence between the above-mentioned iris features and virtual display information, the virtual display device can store the corresponding correspondence between the feature information for identifying different users and the virtual display information, thereby realizing the above-mentioned correspondence between the feature information for identifying different users and the virtual display information. Figure 32 Corresponding functions.

[0391] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of electronic equipment. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0392] The embodiments of the present application can divide the functional modules of the devices involved therein according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0393] For example, Figure 33 FIG. 3 shows a schematic diagram of the composition of an electronic device 3300. The electronic device 3300 can correspond to any virtual display device in the above embodiments. Figure 33 As shown, the electronic device 3300 may include: a processor 3301 and a memory 3302. The memory 3302 is used to store computer-executable instructions. For example, in some embodiments, when the processor 3301 executes the instructions stored in the memory 3302, the electronic device 3300 may perform any of the virtual display methods described in the above embodiments.

[0394] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0395] Figure 34 A schematic diagram of the composition of a chip system 3400 is shown. The chip system 3400 can be set in any virtual display device in the above embodiments. The chip system 3400 may include: a processor 3401 and a communication interface 3402, which are used to support related devices to implement the functions involved in the above embodiments. In one possible design, the chip system also includes a memory for storing program instructions and data necessary for the virtual display device. The chip system can be composed of chips, or it can include chips and other discrete devices. It should be noted that in some implementations of the present application, the communication interface 3402 may also be referred to as an interface circuit.

[0396] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0397] The functions, actions, operations, steps, etc. in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that can be integrated with the medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0398] Although the present application has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the scope of the present embodiments. Accordingly, this specification and the drawings are merely illustrative of the present application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the scope of the present embodiments. Thus, the present application is intended to include such modifications and variations as would fall within the scope of the claims of the present application and their equivalents.

Claims

1. A virtual display device, characterized in that: The virtual display device is used to provide a user with a display function of a virtual three-dimensional environment; the virtual display device includes: a processor, and a first optical display module; the first optical display module is configured to display an image to a first eye under the control of the processor, the first eye being either one of the user's eyes; The first optical display module is used to implement the following functions under the control of the processor: Displaying a first object, where the convergence depth of the first object is a first convergence depth, and the imaging surface depth of the first optical display module is the first depth; Displaying a second object, wherein the convergence depth of the second object is a second convergence depth, and the imaging surface depth of the first optical display module is the second depth; When the first convergence depth and the second convergence depth are different, the first optical display module adjusts the first depth to be different from the second depth.

2. The virtual display device according to claim 1, wherein: When the first convergence depth is greater than the second convergence depth, the first depth is greater than the second depth; when the first convergence depth is less than the second convergence depth, the first depth is less than the second depth.

3. The virtual display device according to claim 1 or 2, characterized in that: The virtual display device further includes: a first eye tracking module, the first eye tracking module being configured to perform eye tracking on the first human eye under the control of the processor; The processor is used to control the first eye tracking module to perform eye tracking on the gaze point of the first person's eye, When the convergence depth of the gaze point of the first human eye is different, the first optical display module is configured to adjust the depth of the imaging plane to be different.

4. The virtual display device according to claim 1 or 2, characterized in that: When the diopter of the user is different, the first optical display module is configured to adjust the depth of the imaging plane to be different.

5. The virtual display device according to claim 1 or 2, characterized in that: The virtual display device further includes: a second optical display module, the second optical display module being configured to display an image to a second eye under the control of the processor, the second eye being one of the user's eyes different from the first eye; The first optical display module is used to implement the following functions under the control of the processor: The first object is displayed, and the imaging plane depth of the first object is displayed as a third depth. The depth of the first object in the virtual three-dimensional environment is a second depth, and the first depth is similar to or the same as the third depth.

6. The virtual display device according to claim 1 or 2, characterized in that: The first optical display module includes: a first zoom module; the optical focal length of the first zoom module is adjustable; The first zoom module is configured to, under the control of the processor, adjust the optical power of the first zoom module to a first optical power when displaying the first object, so that the first optical display module having the first optical power can form an image at the first depth; or The first zoom module is configured to adjust the optical focal length of the first zoom module to a second optical focal length when displaying the second object under the control of the processor, so that the first optical display module having the second optical focal length can be imaged at the second depth.

7. The virtual display device according to claim 1 or 2, characterized in that: The first optical display module is further configured to display a third object at a first virtual position and a second virtual position in the virtual three-dimensional environment under the control of the processor; The first virtual position and the second virtual position have different distances from the user's eyes in the virtual three-dimensional environment.

8. The virtual display device according to claim 7, wherein: When the current display scene is a preset scene, the virtual display device displays the third object at the first virtual position and the second virtual position respectively; The preset scenarios include at least one of the following scenarios: Advertisement playback scene, display resource loading scene.

9. The virtual display device according to claim 1 or 2, characterized in that: The first optical display module is further configured to display a first detection image to the first human eye under the control of the processor; when displaying the first detection image, the number of pixels of the opening used to display the first detection image is a first number; The processor is further configured to receive first recognition feedback, which is an indication input by the user when observing the first detection image using the first human eye, and is configured to indicate whether the user can recognize the opening direction of the first detection image.

10. The virtual display device according to claim 9, wherein: When the first optical display module uses the first number of pixels to display the opening of the first detection image to the user, a visual score value of the opening of the first detection image observed by a first person's eye is a first visual score value; In a case where the first recognition feedback indicates that the user cannot recognize the opening direction of the first detection image, The processor is used to determine that the myopia degree of the first human eye is the myopia degree corresponding to the first visual score value.

11. The virtual display device according to claim 9, wherein: In a case where the first recognition feedback indicates that the user is able to recognize the opening direction of the first detection image, The first optical display module is further configured to display a second detection image to the first human eye under the control of the processor; when displaying the second detection image, the number of pixels of the opening used to display the second detection image is a second number; and the second number is smaller than the first number. The processor is further configured to receive second recognition feedback, which is an indication input by the user when observing the second detection image using the first human eye, and the second recognition feedback is configured to indicate whether the user is able to recognize the opening direction of the second detection image.

12. The virtual display device according to claim 11, wherein: When the first optical display module uses the second number of pixels to display the opening of the second detection image to the user, the visual score of the opening of the second detection image observed by the first human eye is the second visual score; In a case where the second recognition feedback indicates that the user cannot recognize the opening direction of the second detection image, The processor is configured to determine that the myopia degree of the first human eye is the myopia degree corresponding to the second visual score value.

13. The virtual display device according to claim 9, wherein: The first optical display module is used to adjust the optical focal length of the first optical display module to an initial optical focal length under the control of the processor before displaying the first detection image to the first human eye, so that the first optical display module forms an image at the farthest distance.

14. The virtual display device according to claim 9, wherein: In a case where the opening of the third detection image displayed by the first optical display module to the first human eye is a third number of pixels, when the received recognition feedback indicates that the user cannot recognize the opening direction of the third detection image, The optical focal length of the virtual image displayed by the first optical display module to the first human eye is a third optical focal length; the third optical focal length corresponds to a third visual fraction value, and the third visual fraction value is a visual fraction value corresponding to the third number of pixels.

15. The virtual display device according to claim 1 or 2, characterized in that: The first optical display module includes a rotating mechanism, and the rotating mechanism is used to rotate the first optical display module along a direction perpendicular to the optical axis under the control of the processor; When the first optical display module displays the first image to the user, the optical power direction of the first optical display module coincides with the astigmatism axis of the first human eye.

16. The virtual display device according to claim 15, characterized in that: The processor is further configured to determine the astigmatism degree of the first human eye based on the rotation of the rotating mechanism when the optical power direction of the first optical display module coincides with the astigmatism axis of the first human eye.

17. The virtual display device according to claim 15, wherein: The processor is further configured to obtain user characteristics of the current user before controlling the first optical display module to display the first image. The processor is specifically configured to control the first optical display module to display a first image corresponding to the user characteristics of the current user; In which, when displaying the first image, the optical focal length of the first optical display module matches the myopia degree and / or astigmatism degree of the first eye of the current user, and the myopia degree and / or astigmatism degree of the first eye of the current user is indicated by the virtual display information corresponding to the user characteristics of the current user.

18. The virtual display device according to claim 17, wherein: The user feature includes any one of the following features: fingerprint information of the current user; iris feature of the current user; account information of the current user; and an identifier of the current user, where the identifiers of different users are different.

19. The virtual display device according to claim 17, wherein: The virtual display device stores a correspondence between different user characteristics and corresponding virtual display information; The processor is used to search for a matching table item from the correspondence based on the user characteristics of the current user, and if the matching table item exists, determine that the virtual display information corresponding to the current user is the virtual display information stored in the matching table item; the virtual display information includes the myopia degree and / or astigmatism degree of the corresponding user.

20. The virtual display device according to claim 19, wherein: When there is no matching item corresponding to the user feature of the current user in the corresponding relationship, The first optical display module is further configured to display the first image under the control of the processor. The optical focal length when displaying the first image is an optical focal length that matches the myopia degree and / or astigmatism degree of the first eye of the current user, and the myopia degree and / or astigmatism degree of the first eye is automatically determined by the processor, or determined under the user's instructions, or manually input by the user.

21. The virtual display device according to claim 20, wherein: The processor is further configured to store a correspondence between the myopia degree and / or astigmatism degree of the first human eye and the user characteristics of the current user.

22. A virtual display method, characterized in that: The virtual display method is applied to a virtual display device according to any one of claims 1 to 21, and is used to provide a user with a display function of a virtual three-dimensional environment; the method comprises: The first optical display module displays a first object, the convergence depth of the first object is a first convergence depth, and the imaging plane depth of the first optical display module is a first depth; The first optical display module displays a second object, the convergence depth of the second object is a second convergence depth, and the imaging plane depth of the first optical display module is the second depth; When the first convergence depth and the second convergence depth are different, the optical display module adjusts the first depth to be different from the second depth.

23. The method according to claim 22, characterized in that When the first convergence depth is greater than the second convergence depth, the first depth is greater than the second depth; when the first convergence depth is less than the second convergence depth, the first depth is less than the second depth.

24. The method according to claim 22 or 23, characterized in that The method further comprises: The processor controls the first eye tracking module of the first optical display module to perform eye tracking on the gaze point of the first human eye. When the convergence depth of the gaze point of the first human eye is different, the first optical display module is configured to adjust the depth of the imaging plane to be different.

25. The method according to claim 22 or 23, characterized in that The first optical display module includes: a first zoom module; the optical focal length of the first zoom module is adjustable; and the method further includes: The processor controls the first zoom module to adjust the optical power of the first zoom module to a first optical power when displaying the first object, so that the first optical display module with the first optical power can form an image at the first depth; or The processor controls the first zoom module to adjust the optical focal length of the first zoom module to a second optical focal length when displaying the second object, so that the first optical display module with the second optical focal length can form an image at the second depth.

26. The method according to claim 22 or 23, characterized in that The method further comprises: The processor controls the first optical display module to display a third object at a first virtual position and a second virtual position in the virtual three-dimensional environment; The first virtual position and the second virtual position have different distances from the user's eyes in the three-dimensional environment; Wherein, when the current display scene is a preset scene, the first optical display module displays the second object at the first virtual position and the second virtual position respectively; The preset scenarios include at least one of the following scenarios: Advertisement playback scene, display resource loading scene.

27. The method according to claim 22 or 23, characterized in that The method further comprises: The processor controls the first optical display module to display a first detection image to the first human eye; when displaying the first detection image, the number of pixels of the opening used to display the first detection image is a first number; The processor receives first recognition feedback, where the first recognition feedback is an indication input by the user when observing the first detection image with the first human eye, and the first recognition feedback is used to indicate whether the user can recognize the opening direction of the first detection image; When the first optical display module uses the first number of pixels to display the opening of the first detection image to the user, a visual score value of the opening of the first detection image observed by a first person's eye is a first visual score value; In a case where the first recognition feedback indicates that the user cannot recognize the opening direction of the first detection image, The processor determines that the myopia degree of the first human eye is the myopia degree corresponding to the first visual score value.

28. The method according to claim 27, characterized in that When the first recognition feedback indicates that the user is able to recognize the opening direction of the first detection image, the method further includes: The processor controls the first optical display module to display a second detection image to the first human eye; when displaying the second detection image, the number of pixels of the opening used to display the second detection image is a second number; the second number is smaller than the first number; The processor receives second recognition feedback, where the second recognition feedback is an indication input by the user when observing the second detection image with the first human eye, and the second recognition feedback is used to indicate whether the user can recognize the opening direction of the second detection image; When the first optical display module uses the second number of pixels to display the opening of the second detection image to the user, the visual score of the opening of the second detection image observed by the first human eye is the second visual score; In a case where the second recognition feedback indicates that the user cannot recognize the opening direction of the second detection image, The processor determines that the myopia degree of the first human eye is the myopia degree corresponding to the second visual score value.

29. The method according to claim 27, characterized in that In a case where the opening of the third detection image displayed by the first optical display module to the first human eye is a third number of pixels, when the received recognition feedback indicates that the user cannot recognize the opening direction of the third detection image, The processor controls the first optical display module to display a virtual image to the first human eye with an optical focal length of a third optical focal length; the third optical focal length corresponds to a third visual score value, and the third visual score value is a visual score value corresponding to the third number of pixels.

30. The method according to claim 22 or 23, characterized in that The method further comprises: The processor obtains user characteristics of the current user before controlling the first optical display module to display the first image. The processor controls the first optical display module to display the first image, including: The processor controls the first optical display module to display a first image corresponding to the user characteristics of the current user; wherein, when displaying the first image, the optical power of the first optical display module matches the myopia degree and / or astigmatism degree of the first eye of the current user, and the myopia degree and / or astigmatism degree of the first eye of the current user is indicated by virtual display information corresponding to the user characteristics of the current user; The user feature includes any one of the following features: fingerprint information of the current user; iris feature of the current user; account information of the current user; and an identifier of the current user, where the identifiers of different users are different.

31. The method according to claim 30, wherein The method stores a correspondence between different user characteristics and corresponding virtual display information; the method further includes: The processor is used to search for a matching table item from the correspondence based on the user characteristics of the current user, and if the matching table item exists, determine that the virtual display information corresponding to the current user is the virtual display information stored in the matching table item; the virtual display information includes the myopia degree and / or astigmatism degree of the corresponding user.

32. The method according to claim 31, characterized in that The method further comprises: When there is no matching item corresponding to the user feature of the current user in the corresponding relationship, The first optical display module displays the first image under the control of the processor. The optical focal length when displaying the first image is an optical focal length that matches the myopia degree and / or astigmatism degree of the first eye of the current user, and the myopia degree and / or astigmatism degree of the first eye is automatically determined by the processor, or determined under the user's instructions, or manually input by the user.

33. The method according to claim 32, characterized in that The method further comprises: The processor stores a correspondence between the myopia degree and / or astigmatism degree of the first human eye and the user characteristics of the current user.

34. A virtual display device, characterized in that The virtual display device is used to provide a display function of a virtual three-dimensional environment to the user; The virtual display device includes one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions; When the one or more processors execute the computer instructions, the virtual display device is caused to perform the virtual display method according to any one of claims 22 to 33.

35. A computer-readable storage medium, characterized in that The computer-readable storage medium includes computer instructions, and when the computer instructions are executed, the virtual display method according to any one of claims 22 to 33 is executed.

Images