Display method and device, virtual reality device, and storage medium

Abstract

The application relates to the virtual reality technical field and discloses a display method and device, a virtual reality device and a storage medium, the method is applied to a virtual reality device provided with an adjusting mechanism, the adjusting mechanism is used for adjusting a current focal length between a lens and a screen of the virtual reality device, the method comprises the following steps: acquiring eye movement information of a user, determining a current gaze point at which the user gazes at the screen of the virtual reality device based on the eye movement information; determining a current display virtual scene, and determining a current virtual target gazed at by the user according to the current display virtual scene and the current gaze point; determining current depth information of the current virtual target from eyeballs of the user according to the current display virtual scene; adjusting the current focal length according to the current depth information through the adjusting mechanism, and displaying through the adjusted screen. Compared with the existing manual adjustment through a knob mechanical structure, the application can realize automatic adjustment through the adjusting mechanism, simplifies the operation, and improves the user experience.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, and in particular to a display method, apparatus, virtual reality device and storage medium. Background Technology

[0002] Currently, with the rapid development of the internet and digital technologies, people's demand for immersive experiences is growing. Virtual Reality (VR) technology, with its unique immersiveness and interactivity, provides users with a brand-new virtual world, allowing people to experience new sensations that transcend the limitations of reality.

[0003] Current VR headsets typically achieve zooming by adjusting the distance between the VR lens and the screen using a rotary knob mechanism. However, this operation requires manual intervention from the user, making it rather cumbersome. Summary of the Invention

[0004] The main objective of this application is to provide a display method, device, virtual reality equipment, and storage medium, aiming to solve the technical problem in the prior art that requires manual adjustment of a knob mechanical structure to adjust the distance between the VR lens and the screen to achieve zoom, resulting in cumbersome operation.

[0005] To achieve the above objectives, this application provides a display method applied to a virtual reality device equipped with an adjustment mechanism, the adjustment mechanism being used to adjust the current focal length between the lens and the screen of the virtual reality device, the method comprising:

[0006] Acquire the user's eye movement information, and determine the user's current gaze point as they gaze at the screen of the virtual reality device based on the eye movement information;

[0007] Determine the currently displayed virtual scene, and determine the current virtual target that the user is looking at based on the currently displayed virtual scene and the current gaze point;

[0008] The current depth information of the current virtual target relative to the user's eyeball is determined based on the currently displayed virtual scene;

[0009] The current focal length is adjusted by the adjustment mechanism according to the current depth information, and the adjusted focal length is displayed on the screen.

[0010] In one embodiment, the step of determining the current gaze point of the user gazing at the screen of the virtual reality device based on the eye-tracking information includes:

[0011] The eye movement vector of the user gazing at the screen of the virtual reality device is determined based on the eye movement information;

[0012] Based on the eye movement vector, the horizontal and vertical coordinate values ​​of the eye movement vector in the screen coordinate system corresponding to the screen are determined according to a preset coordinate mapping relationship;

[0013] The user's current gaze point on the screen is obtained based on the horizontal and vertical coordinate values.

[0014] In one embodiment, the step of obtaining the user's current gaze point on the screen based on the horizontal coordinate value and the vertical coordinate value includes:

[0015] Based on the eye movement information, the horizontal offset of the user's left eye and the horizontal offset of the right eye are determined, and the horizontal offset of the left eye and the horizontal offset of the right eye are weighted according to a first preset weight to obtain the horizontal offset of one eye.

[0016] Based on the eye movement information, the vertical offset of the user's left eye and right eye are determined, and the vertical offset of the left eye and right eye are weighted according to the second preset weight to obtain the vertical offset of one eye.

[0017] The horizontal coordinate value is corrected by the monocular horizontal offset, and the vertical coordinate value is corrected by the monocular vertical offset;

[0018] The current gaze point of the user on the screen is obtained based on the correction result.

[0019] In one embodiment, the adjustment mechanism includes a focusing lens and a motor, the motor being connected to the focusing lens, and the focusing lens being disposed between the lens and the screen;

[0020] The step of adjusting the current focal length according to the current depth information through the adjustment mechanism and displaying the adjusted focal length on the screen includes:

[0021] The user's eye prescription is obtained, and the required lens focal length for the user is determined based on the eye prescription and the current depth information;

[0022] The target adjustment parameters of the motor are determined based on the required lens focal length.

[0023] The current focal length is adjusted by the motor according to the target adjustment parameters, and the adjusted focal length is displayed on the screen.

[0024] In one embodiment, the step of determining the target adjustment parameters of the motor based on the desired lens focal length includes:

[0025] Obtain the structural parameters of the focusing lens, and determine the target focusing position based on the required lens focal length according to the structural parameters;

[0026] Determine the current focusing position, and determine the target adjustment parameters of the motor based on the current focusing position and the target focusing position.

[0027] In one embodiment, prior to the step of acquiring the user's eye movement information, the method further includes:

[0028] Obtain the user's acceptable display level, and determine the corresponding acceptable depth variation range based on the acceptable display level;

[0029] The step of adjusting the current focal length according to the current depth information via the adjustment mechanism includes:

[0030] Determine the historical depth information corresponding to the last adjustment of the adjustment mechanism, and determine the depth change based on the historical depth information and the current depth information;

[0031] When the depth change is not within the acceptable depth change range, the current focal length is adjusted by the adjustment mechanism according to the current depth information.

[0032] In one embodiment, after the step of determining the depth change based on the historical depth information and the current depth information, the method further includes:

[0033] When the depth change is within the acceptable depth change range, the current gaze duration of the user gazing at the current virtual target is obtained;

[0034] The user's gaze duration threshold is determined based on the acceptable display level;

[0035] When the current fixation duration reaches the current fixation threshold, the current focal length is adjusted by the adjustment mechanism according to the current depth information.

[0036] Furthermore, to achieve the above objectives, this application also proposes a display device, the device comprising:

[0037] The information acquisition module is used to acquire the user's eye movement information and determine the user's current gaze point on the screen of the virtual reality device based on the eye movement information;

[0038] The scene determination module is used to determine the currently displayed virtual scene and determine the current virtual target being gazed at by the user based on the currently displayed virtual scene and the current gaze point;

[0039] A depth determination module is used to determine the current depth information of the current virtual target from the user's eyeball based on the currently displayed virtual scene;

[0040] The distance adjustment module is used to adjust the current focal length according to the current depth information through the adjustment mechanism, and to display the adjusted focal length on the screen.

[0041] In addition, to achieve the above objectives, this application also proposes a virtual reality device, the device comprising: an adjustment mechanism, a processor, a memory, a lens, and a screen, wherein the processor is connected to the memory and the adjustment mechanism, and the adjustment mechanism is used to adjust the current focal length between the lens and the screen;

[0042] The processor is configured to run a display program stored in the memory, the display program being configured to implement the steps of the display method as described above.

[0043] In addition, to achieve the above objectives, this application also proposes a storage medium storing a display program, which, when executed by a processor, implements the steps of the display method described above.

[0044] This application provides a display method, apparatus, virtual reality device, and storage medium. The method is applied to a virtual reality device equipped with an adjustment mechanism, the adjustment mechanism being used to adjust the current focal length between the lens and the screen of the virtual reality device. The method includes: acquiring eye movement information of a user; determining the current gaze point of the user looking at the screen of the virtual reality device based on the eye movement information; determining a currently displayed virtual scene; determining a currently displayed virtual target being looked at by the user based on the currently displayed virtual scene and the current gaze point; determining the current depth information of the current virtual target from the user's eyeball based on the currently displayed virtual scene; adjusting the current focal length according to the current depth information through the adjustment mechanism; and displaying the adjusted focal length on the screen.

[0045] This application incorporates an adjustment mechanism within the virtual reality device to adjust the current focal length between the lens and the screen. In practical use, the user's current gaze point on the screen is first determined based on their eye movement information. Then, the current virtual target being viewed is determined by combining this with the displayed virtual scene. Finally, the current depth information of the virtual target relative to the user's eyeball is determined. Based on this depth information, the adjustment mechanism is used to adjust the current focal length. Compared to existing manual adjustments via knobs, this application achieves automatic adjustment through the adjustment mechanism, simplifying operation and improving the user experience. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the first embodiment of the method shown in this application;

[0049] Figure 2 This is a schematic diagram of the adjustment mechanism in the first embodiment of the method shown in this application;

[0050] Figure 3 This is a schematic diagram of the structure of the adjusting mechanism in the first embodiment of the method shown in this application;

[0051] Figure 4 This is a flowchart illustrating the second embodiment of the method shown in this application;

[0052] Figure 5 This is a flowchart illustrating the third embodiment of the method shown in this application;

[0053] Figure 6 This is a structural block diagram of the first embodiment of the display device of this application.

[0054] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0056] It should be noted that the hardware operating environment involved in the embodiments of this application is a virtual reality device, which may include: a processor, such as a central processing unit (CPU), a communication bus, a user interface, a network interface, and memory. The communication bus is used to enable communication between these components. The user interface may connect to a display screen, an input unit such as a keyboard, etc., and may optionally include a standard wired interface or a wireless interface. The network interface may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. Alternatively, the memory may be a storage device independent of the aforementioned processor.

[0057] Those skilled in the art will understand that the above structure does not constitute a limitation on the virtual reality device, and may include more or fewer components, or combine certain components, or arrange different components. Since the virtual reality device in this embodiment can be VR glasses, the virtual reality device may further include: a lens and a screen. The screen can display an image, and the lens images and magnifies the displayed image, thereby projecting it onto the user's retina, allowing the user to view the image. Simultaneously, to achieve automatic adjustment of the current focal length between the lens and the screen, the virtual reality device may further include an adjustment mechanism, which can be connected to a processor. The adjustment mechanism is used to adjust the current focal length between the lens and the screen; and the processor can call a display program stored in the memory, the display program being configured to implement the steps of the display method provided in this application embodiment.

[0058] It should be noted that with the rapid development of the internet and digital technologies, people's demand for immersive experiences is growing. Virtual Reality (VR) technology, with its unique immersiveness and interactivity, provides users with a completely new virtual world, allowing people to experience new sensations that transcend the limitations of reality.

[0059] Current VR headsets typically achieve zooming by adjusting the distance between the VR lens and the screen using a rotary knob mechanism. However, this operation requires manual intervention from the user, making it rather cumbersome.

[0060] Therefore, to address the aforementioned shortcomings, this embodiment provides a display method applied to a virtual reality device equipped with an adjustment mechanism. In this embodiment, the virtual reality device includes an adjustment mechanism for adjusting the current focal length between the lens and the screen. In practical use, the user's current gaze point on the screen is first determined based on their eye movement information. Then, the current virtual target being gazed at is determined by combining this with the displayed virtual scene. Finally, the current depth information of the virtual target's distance from the user's eyeball is determined. Based on this current depth information, the adjustment mechanism is used to adjust the current focal length. Compared to existing manual adjustments via knob mechanisms, this embodiment achieves automatic adjustment through the adjustment mechanism, simplifying operation and improving the user experience.

[0061] For ease of understanding, the following is combined with Figures 1 to 5 The display method provided in the embodiments of this application will be described in detail.

[0062] This application provides a display method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the method shown in this application.

[0063] In this embodiment, the display method includes the following steps:

[0064] Step S10: Obtain the user's eye movement information, and determine the user's current gaze point on the screen of the virtual reality device based on the eye movement information.

[0065] It is understood that the method of this embodiment can be executed by a virtual reality device, specifically by a processor within the virtual reality device. The aforementioned virtual reality device can be a device such as VR glasses used for virtual reality interaction, and this embodiment does not limit this. For ease of subsequent explanation, this embodiment uses the processor in VR glasses as the executing entity of the above method for description.

[0066] It should be understood that the aforementioned eye-tracking information can be the eye movement information of the user currently wearing the VR glasses, including but not limited to eye position coordinates, corner of the eye position coordinates, eye movement area, pupil coordinates, and reflective bright spots. To obtain this eye-tracking information, the VR glasses can be equipped with an eye-tracking module, which can be a module for capturing the user's eye movements and can incorporate eye-tracking technology. In actual use, the eye-tracking module can continuously capture the user's eyes, determine the user's eye movement information based on the captured images using eye-tracking technology, and transmit this information to the processor, thereby enabling the processor to obtain the user's eye-tracking information.

[0067] It should also be understood that the aforementioned current gaze point can be the point corresponding to the location on the screen where the user is currently looking. After obtaining the aforementioned eye movement information, the processor can determine which location on the screen the user is currently looking at based on the eye movement information and use that location as the user's current gaze point.

[0068] Step S20: Determine the currently displayed virtual scene, and determine the current virtual target being gazed at by the user based on the currently displayed virtual scene and the current gaze point.

[0069] It should be noted that the aforementioned currently displayed virtual scene can be the scene corresponding to the image currently displayed on the screen. When the VR glasses are used for display, a processor for controlling the display screen can be provided, or it can be implemented directly through the processor corresponding to the execution subject in this embodiment. During display, the processor can transmit the scene to be displayed to the screen for display, thereby obtaining the aforementioned currently displayed virtual scene.

[0070] It should also be noted that the aforementioned current virtual target can be the target corresponding to the current gaze point within the currently displayed virtual scene. In simpler terms, it can be understood as the specific target the user is looking at within the current virtual scene. In actual use, once the processor determines the current gaze point, it can acquire the virtual scene currently displayed on the screen and use the content at the location corresponding to the current gaze point within the currently displayed virtual scene as the current virtual target being looked at by the user.

[0071] Step S30: Determine the current depth information of the current virtual target from the user's eyeball based on the currently displayed virtual scene.

[0072] Understandably, the aforementioned current depth information can be the information corresponding to the distance between the user's eyeball and the current virtual target. Since the content displayed by VR glasses simulates the content observed by the human eye in an actual 3D scene, the aforementioned current depth information can be understood as the information corresponding to the distance between the eyeball and the current virtual target in the 3D scene.

[0073] Meanwhile, the virtual scene displayed on the screen is generally obtained through pre-recording in an actual 3D scene, allowing the depth information of each target within the scene relative to the eyeball to be directly determined during recording. Therefore, during display, the processor can directly determine the depth information corresponding to the current virtual target, which is then used as the aforementioned current depth information.

[0074] Step S40: Adjust the current focal length according to the current depth information using the adjustment mechanism, and display the adjusted focal length on the screen.

[0075] It should be understood that the above-mentioned adjustment mechanism can be a mechanism for adjusting the current focal length between the lens and the screen. Therefore, in this embodiment, the above-mentioned adjustment mechanism can be set between the lens and the screen.

[0076] After determining the current depth information, the processor adjusts the adjustment mechanism accordingly, thereby adjusting the current focal length between the lens and the screen to a suitable focal length for observing the virtual target, and then displaying the adjusted image on the screen. Compared to existing manual adjustments using a knob mechanism, this embodiment simplifies operation and improves the user experience.

[0077] As one implementation method, refer to Figure 2 as well as Figure 3 , Figure 2 This is a schematic diagram of the adjustment mechanism in the first embodiment of the method shown in this application. Figure 3 This is a schematic diagram of the structure of the adjusting mechanism in the first embodiment of the method shown in this application, as follows: Figure 2 as well as Figure 3 As shown, in this embodiment, the adjustment mechanism may include a focusing lens and a motor, wherein the motor is connected to the focusing lens and the focusing lens is disposed between the lens and the screen.

[0078] It should be noted that the aforementioned focusing lens can be a lens used to adjust the focal length. This can be understood as different positions of the focusing lens corresponding to different diopter values. The specific form of the focusing lens is not limited in this embodiment. The aforementioned lens can be a lens used for imaging and magnification in VR glasses. In this embodiment, the lens can be a VR anti-distortion lens, but it can also be other lenses; this embodiment does not limit its application.

[0079] like Figure 2 As shown, in this embodiment, the focusing lens can be positioned between the lens and the screen, and the specific position can be set according to the actual situation. The motor can be located at the lower end of the focusing lens and fixedly connected to it. The motor can be a telescopic motor and can be connected to the processor. The motor can then control the focusing lens to move up and down, thereby changing the focal length. Of course, a telescopic rod can also be used to replace the motor, which can also achieve the same result. This embodiment does not limit this.

[0080] As another implementation, the focusing lens in this embodiment can also move left and right to change the focal length, and the motor can be set to a corresponding position to move left and right. Of course, other directions of movement can also be used to change the focal length, and the specific direction can be set according to the actual situation. This embodiment does not limit this.

[0081] It is important to emphasize that, because VR glasses contain lenses for the left and right eyes, and some users require different focal lengths for their left and right eyes, in actual use, an adjustment mechanism can be set up between the left lens and the screen, and between the right lens and the screen, for independent control. The specific structure can be similar to... Figure 2 and Figure 3 Similarly, this embodiment will not elaborate further.

[0082] Accordingly, considering that different users have different eye prescriptions, in order to improve the viewing experience, step S40 above includes:

[0083] Step S41: Obtain the user's eye prescription and determine the user's required lens focal length based on the eye prescription and the current depth information.

[0084] Understandably, after the processor determines the current depth information, it can obtain the user's eye diopter. This eye diopter can be obtained in advance through user input, or it can be obtained through other methods; this embodiment does not limit this. Furthermore, when obtaining the eye diopter, the diopter of the left eye and the diopter of the right eye can be obtained separately.

[0085] After obtaining the eye prescription, the focal length required for the user to clearly observe the current virtual target can be determined based on the eye prescription and the current depth information. The required lens focal length can be obtained through the focal length calculation formula, or it can be obtained through other methods. This embodiment does not limit this.

[0086] Step S42: Determine the target adjustment parameters of the motor based on the required lens focal length.

[0087] It should be understood that the aforementioned target adjustment parameters can be parameters used to adjust the motor so that the focusing lens reaches the desired focal length. Specifically, step S42 includes:

[0088] Step S421: Obtain the structural parameters of the focusing lens, and determine the target focusing position based on the required lens focal length according to the structural parameters;

[0089] Step S422: Determine the current focusing position, and determine the target adjustment parameters of the motor based on the current focusing position and the target focusing position.

[0090] It should also be understood that the above structural parameters can be the focal length parameters corresponding to each position of the focusing lens currently in use. For example, if the focusing range of the currently used focusing lens is 0 to 800 degrees and the adjustable distance range is 0 to 8 mm, then it can be determined that every 1 mm movement corresponds to a change of 100 degrees; or, for example, if the focusing range of the focusing lens is 0 to 600 degrees and the adjustable distance range is 0 to 6 μm, then it can be determined that every 1 μm movement corresponds to a change of 100 degrees.

[0091] The target focusing position mentioned above can be the position of the focusing lens when the required focal length needs to be achieved. The current focusing position mentioned above can be the current position of the focusing lens before adjustment.

[0092] In actual use, the processor can determine the target focusing position of the focusing lens when the lens is to be adjusted to the required focal length from the pre-stored structural parameters, and obtain the current focusing position of the focusing lens. Based on the current focusing position and the target focusing position, the amount of adjustment required by the motor can be determined as the target adjustment parameter. The processor then calls the motor interface to transmit the target adjustment parameter to the motor in the form of a control signal.

[0093] Step S43: Adjust the current focal length using the motor according to the target adjustment parameters, and display the adjusted focal length on the screen.

[0094] After receiving the control signal, the motor can move the focusing lens to the target focusing position according to the corresponding target adjustment parameters, thereby adjusting the focal length between the lens and the screen and completing the automatic adjustment.

[0095] The virtual reality device in this embodiment includes an adjustment mechanism for adjusting the current focal length between the lens and the screen. In actual use, the user's current gaze point on the screen is first determined based on their eye movement information. Then, the current virtual target being viewed by the user is determined by combining this with the displayed virtual scene. Finally, the current depth information of the virtual target relative to the user's eyeball is determined. Based on this depth information, the adjustment mechanism is used to adjust the current focal length. Compared to existing manual adjustments using knobs, this embodiment achieves automatic adjustment via the adjustment mechanism, simplifying operation and improving the user experience.

[0096] Furthermore, since manual adjustment is prone to human error, this application improves the accuracy of adjustment by using an automatic method.

[0097] refer to Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the display method of this application. Based on the first embodiment described above, a second embodiment of the display method of this application is proposed.

[0098] To obtain the user's current gaze point on the screen, such as Figure 4 As shown, in this embodiment, the step of determining the user's current gaze point on the screen of the virtual reality device based on the eye-tracking information includes:

[0099] Step S11: Determine the eye movement vector of the user gazing at the screen of the virtual reality device based on the eye movement information.

[0100] It should be noted that the aforementioned eye-tracking vector can be the vector corresponding to the user's eye movements. When the eye-tracking module obtains the aforementioned eye-tracking information, it can include the eye-tracking vector within that information. Therefore, the processor can directly obtain the user's eye-tracking vector based on the eye-tracking information.

[0101] Step S12: Based on the eye movement vector, determine the horizontal and vertical coordinate values ​​of the eye movement vector in the screen coordinate system corresponding to the screen according to the preset coordinate mapping relationship.

[0102] Understandably, the aforementioned preset coordinate mapping relationship can be the relationship between each eye-tracking vector and each horizontal and vertical coordinate value on the screen coordinate system. The aforementioned screen coordinate system can be a coordinate system established with the screen as the reference.

[0103] Before use, the user can be pre-calibrated. This involves providing a set of target points on the screen for the user to observe, and determining the eye movement vector corresponding to each target point using the eye-tracking module. Then, calibration is performed based on the eye movement vector and the coordinates of the corresponding target point, thereby obtaining the aforementioned preset mapping relationship.

[0104] Therefore, in actual use, the processor can directly query the preset coordinate mapping relationship corresponding to the user, thereby determining the horizontal and vertical coordinate values ​​of the eye movement vector in the screen coordinate system.

[0105] Step S13: Obtain the current gaze point of the user looking at the screen based on the horizontal coordinate value and the vertical coordinate value.

[0106] After obtaining the horizontal and vertical coordinate values, the processor can use the positions on the screen corresponding to these values ​​as the user's current gaze point.

[0107] Furthermore, considering that the line of sight may shift, in order to further improve the accuracy of the current gaze point, in this embodiment, step S13 includes:

[0108] Step S131: Determine the horizontal offset of the user's left eye and right eye based on the eye movement information, and weight the horizontal offset of the left eye and right eye according to the first preset weight to obtain the horizontal offset of one eye.

[0109] Step S132: Determine the vertical offset of the user's left eye and right eye based on the eye movement information, and weight the vertical offset of the left eye and right eye according to the second preset weight to obtain the vertical offset of one eye.

[0110] It should be understood that the aforementioned left-eye horizontal offset can be the horizontal distance of the user's left eye's line of sight relative to the reference point, and the aforementioned right-eye horizontal offset can be the horizontal distance of the user's right eye's line of sight relative to the reference point. Similarly, the aforementioned left-eye vertical offset can be the vertical distance of the user's left eye's line of sight relative to the reference point, and the aforementioned right-eye vertical offset can be the vertical distance of the user's right eye's line of sight relative to the reference point.

[0111] The aforementioned horizontal offset of the left eye, horizontal offset of the right eye, vertical offset of the left eye, and vertical offset of the right eye can all be obtained by the eye-tracking module using eye-tracking technology, which will not be elaborated upon in this embodiment.

[0112] For ease of understanding, the horizontal offset of the left eye is denoted as XL, the horizontal offset of the right eye is denoted as XR, the first preset weight is denoted as w, and the horizontal offset of the single eye obtained after weighting is denoted as XW. Then XW = w*XL + (1-w)*XR.

[0113] Let the vertical offset of the left eye be denoted as YL, the vertical offset of the right eye as YR, the second preset weight as v, and the vertical offset of the single eye obtained after weighting as YW. Then YW = v * YL + (1 - v) * YR.

[0114] It should be emphasized that the first preset weight and the second preset weight can be set according to the actual situation, and this embodiment does not impose any restrictions on them.

[0115] Step S133: Correct the horizontal coordinate value using the monocular horizontal offset, and correct the vertical coordinate value using the monocular vertical offset;

[0116] Step S134: Obtain the current gaze point of the user gazing at the screen based on the correction result.

[0117] After obtaining the monocular horizontal offset XW and monocular vertical offset YW, the horizontal and vertical coordinate values ​​can be corrected based on these values. If we denote the horizontal coordinate value as X, the vertical coordinate value as Y, the corrected horizontal coordinate value as X', and the corrected vertical coordinate value as Y', then X' = X + XW, and Y' = Y + YW. Finally, the position corresponding to (X', Y') can be taken as the current fixation point.

[0118] This embodiment can determine the horizontal and vertical coordinates on the screen coordinate system using the user's eye movement vector, and correct the horizontal coordinates using a single-eye horizontal offset and the vertical coordinates using a single-eye vertical offset, thereby improving the accuracy of the current gaze point.

[0119] refer to Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the display method of this application. Based on the above embodiments, a third embodiment of the display method of this application is proposed.

[0120] Considering that the focus needs to be adjusted every time the user's eye moves, but in some cases this adjustment may not be necessary and will not affect the user's observation, therefore... Figure 5 As shown, in this embodiment, before the step of obtaining the user's eye movement information, the method further includes:

[0121] Step S01: Obtain the user's acceptable display level, and determine the corresponding acceptable depth variation range based on the acceptable display level.

[0122] It should be noted that the aforementioned acceptable display level can be the level of display quality that the current wearer can accept. The aforementioned acceptable depth variation range can be the range that the user can accept without refocusing when the current depth information changes. Each acceptable display level can correspond to an acceptable depth variation range, and the specific acceptable depth variation range can be set according to the actual situation. This embodiment does not impose any restrictions on this.

[0123] This embodiment can be set with several different acceptable display levels. For example, three acceptable display levels can be set, referred to as Level 1, Level 2, and Level 3, respectively. The higher the level, the higher the display quality that the user can accept. Each acceptable display level can be set with a corresponding acceptable depth variation range. For example, the acceptable depth variation range for Level 1 is within ±0.6 meters, for Level 2 it is within ±0.4 meters, and for Level 3 it is within ±0.2 meters.

[0124] The step of adjusting the current focal length according to the current depth information via the adjustment mechanism includes:

[0125] Step S401: Determine the historical depth information corresponding to the last adjustment of the adjustment mechanism, and determine the depth change based on the historical depth information and the current depth information;

[0126] Step S402: When the depth change is not within the acceptable depth change range, the current focal length is adjusted by the adjustment mechanism according to the current depth information.

[0127] It is understandable that the aforementioned historical depth information could be the current depth information corresponding to the last adjustment made by the motor.

[0128] In actual use, after determining the corresponding acceptable depth change range, the processor calculates the difference between the historical depth information corresponding to the last adjustment and the current depth information, as the aforementioned depth change situation. Then, it determines whether the depth change situation is within the user's acceptable range, that is, whether the depth change situation is within the acceptable depth change range. If it is not, it means that the current depth change is large and requires focus adjustment, which can be done through the adjustment mechanism. If it is, it means that the current depth change is small and within the user's acceptable range, and no focus adjustment is required.

[0129] For example, if the user selects the acceptable display level as level 2, and the current depth information is 4.3 meters, while the historical depth information at the time of the last adjustment was 4.1 meters, subtracting them will give the depth change as 0.2 meters, which is within the acceptable depth change range, so no adjustment is needed.

[0130] Furthermore, considering that if a user observes a scene for an extended period, an unsuitable focal length will still affect viewing quality, this embodiment, after determining the depth change based on the historical depth information and the current depth information, further includes:

[0131] Step S4011: When the depth change is within the acceptable depth change range, obtain the current gaze duration of the user gazing at the current virtual target;

[0132] Step S4012: Determine the user's gaze duration threshold based on the acceptable display level;

[0133] Step S4013: When the current fixation duration reaches the current fixation threshold, the current focal length is adjusted by the adjustment mechanism according to the current depth information.

[0134] It should be understood that the aforementioned current gaze duration can be the duration for which the user gazes at the current virtual target. The aforementioned gaze duration threshold can be a threshold used to determine whether the user gazes at the current virtual target for an extended period of time. In this embodiment, corresponding gaze duration thresholds can be set according to different acceptable display levels, and the higher the acceptable display level, the smaller the corresponding gaze duration threshold. For example, the gaze duration threshold for level one can be 3 seconds, the gaze duration threshold for level two can be 2 seconds, and the gaze duration threshold for level three can be 1 second.

[0135] In actual use, when the processor determines that the depth change is within an acceptable range, it can obtain the current gaze duration of the user looking at the current virtual target and determine the corresponding gaze duration threshold based on the acceptable display level. Then, it determines whether the current gaze duration has reached the gaze duration threshold. If it has not reached the threshold, it means that the user does not need to observe for a long time and does not need to adjust it through the adjustment mechanism. If it has reached the threshold, it means that the user needs to observe for a long time and can adjust it through the adjustment structure.

[0136] Furthermore, this application also proposes a storage medium storing a display program, which, when executed by a processor, implements the steps of the display method described above.

[0137] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the display device of this application.

[0138] like Figure 6 As shown, the display device proposed in this application includes:

[0139] The information acquisition module 601 is used to acquire the user's eye movement information and determine the current gaze point of the user looking at the screen of the virtual reality device based on the eye movement information;

[0140] Scene determination module 602 is used to determine the currently displayed virtual scene and determine the current virtual target being gazed at by the user based on the currently displayed virtual scene and the current gaze point;

[0141] The depth determination module 603 is used to determine the current depth information of the current virtual target from the user's eyeball based on the currently displayed virtual scene;

[0142] The distance adjustment module 604 is used to adjust the current focal length according to the current depth information through the adjustment mechanism, and to display the adjusted focal length on the screen.

[0143] The virtual reality device in this embodiment includes an adjustment mechanism for adjusting the current focal length between the lens and the screen. In actual use, the user's current gaze point on the screen is first determined based on their eye movement information. Then, the current virtual target being viewed by the user is determined by combining this with the displayed virtual scene. Finally, the current depth information of the virtual target relative to the user's eyeball is determined. Based on this depth information, the adjustment mechanism is used to adjust the current focal length. Compared to existing manual adjustments using knobs, this embodiment achieves automatic adjustment via the adjustment mechanism, simplifying operation and improving the user experience.

[0144] In one embodiment, the adjustment mechanism includes: a focusing lens and a motor, the motor being connected to the focusing lens, and the focusing lens being disposed between the lens and the screen;

[0145] The distance adjustment module 604 is also used to acquire the user's eye prescription and determine the user's required lens focal length based on the eye prescription and the current depth information; determine the target adjustment parameters of the motor based on the required lens focal length; adjust the current focal length according to the target adjustment parameters by the motor, and display the adjusted focal length on the screen.

[0146] In one implementation, the distance adjustment module 604 is further configured to acquire the structural parameters of the focusing lens, and determine the target focusing position based on the required lens focal length according to the structural parameters; determine the current focusing position, and determine the target adjustment parameters of the motor based on the current focusing position and the target focusing position.

[0147] Based on the first embodiment of the display device described above, a second embodiment of the display device of this application is proposed.

[0148] In this embodiment, the information acquisition module 601 is further configured to determine the eye movement vector of the user looking at the screen of the virtual reality device based on the eye movement information; determine the horizontal and vertical coordinate values ​​of the eye movement vector in the screen coordinate system corresponding to the screen based on the eye movement vector according to a preset coordinate mapping relationship; and obtain the current gaze point of the user looking at the screen based on the horizontal and vertical coordinate values.

[0149] In one implementation, the information acquisition module 601 is further configured to: determine the user's left-eye horizontal offset and right-eye horizontal offset based on the eye-tracking information; weight the left-eye horizontal offset and right-eye horizontal offset according to a first preset weight to obtain a single-eye horizontal offset; determine the user's left-eye vertical offset and right-eye vertical offset based on the eye-tracking information; weight the left-eye vertical offset and right-eye vertical offset according to a second preset weight to obtain a single-eye vertical offset; correct the horizontal coordinate value using the single-eye horizontal offset; correct the vertical coordinate value using the single-eye vertical offset; and obtain the user's current gaze point on the screen based on the correction result.

[0150] Based on the above embodiments of the display device of this application, a third embodiment of the display device of this application is proposed.

[0151] In this embodiment, the information acquisition module 601 is further configured to acquire the user's acceptable display level and determine the corresponding acceptable depth variation range based on the acceptable display level;

[0152] The distance adjustment module 604 is also used to determine the historical depth information corresponding to the last adjustment of the adjustment mechanism, and to determine the depth change based on the historical depth information and the current depth information; when the depth change is not within the acceptable depth change range, the current focal length is adjusted by the adjustment mechanism according to the current depth information.

[0153] In one implementation, the distance adjustment module 604 is further configured to: acquire the current gaze duration of the user gazing at the current virtual target when the depth change is within the acceptable depth change range; determine the gaze duration threshold corresponding to the user based on the acceptable display level; and adjust the current focal length according to the current depth information through the adjustment mechanism when the current gaze duration reaches the current gaze threshold.

[0154] Other embodiments or specific implementations of the display device of this application can be found in the above-described method embodiments, and will not be repeated here.

[0155] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0156] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0158] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A display method, characterized in that, The method is applied to a virtual reality device equipped with an adjustment mechanism, the adjustment mechanism being used to adjust the current focal length between the lens and the screen of the virtual reality device, the method comprising: Acquire the user's eye movement information, and determine the user's current gaze point as they gaze at the screen of the virtual reality device based on the eye movement information; Determine the currently displayed virtual scene, and determine the current virtual target that the user is looking at based on the currently displayed virtual scene and the current gaze point; The current depth information of the current virtual target relative to the user's eyeball is determined based on the currently displayed virtual scene; The current focal length is adjusted by the adjustment mechanism according to the current depth information, and the adjusted focal length is displayed on the screen.

2. The method as described in claim 1, characterized in that, The step of determining the user's current gaze point on the screen of the virtual reality device based on the eye-tracking information includes: The eye movement vector of the user gazing at the screen of the virtual reality device is determined based on the eye movement information; Based on the eye movement vector, the horizontal and vertical coordinate values ​​of the eye movement vector in the screen coordinate system corresponding to the screen are determined according to a preset coordinate mapping relationship; The user's current gaze point on the screen is obtained based on the horizontal and vertical coordinate values.

3. The method as described in claim 2, characterized in that, The step of obtaining the user's current gaze point on the screen based on the horizontal and vertical coordinate values ​​includes: Based on the eye movement information, the horizontal offset of the user's left eye and the horizontal offset of the right eye are determined, and the horizontal offset of the left eye and the horizontal offset of the right eye are weighted according to a first preset weight to obtain the horizontal offset of one eye. Based on the eye movement information, the vertical offset of the user's left eye and right eye are determined, and the vertical offset of the left eye and right eye are weighted according to the second preset weight to obtain the vertical offset of one eye. The horizontal coordinate value is corrected by the monocular horizontal offset, and the vertical coordinate value is corrected by the monocular vertical offset; The current gaze point of the user on the screen is obtained based on the correction result.

4. The method as described in claim 1, characterized in that, The adjustment mechanism includes a focusing lens and a motor, wherein the motor is connected to the focusing lens, and the focusing lens is disposed between the lens and the screen; The step of adjusting the current focal length according to the current depth information through the adjustment mechanism and displaying the adjusted focal length on the screen includes: The user's eye prescription is obtained, and the required lens focal length for the user is determined based on the eye prescription and the current depth information; The target adjustment parameters of the motor are determined based on the required lens focal length. The current focal length is adjusted by the motor according to the target adjustment parameters, and the adjusted focal length is displayed on the screen.

5. The method as described in claim 4, characterized in that, The step of determining the target adjustment parameters of the motor based on the required lens focal length includes: Obtain the structural parameters of the focusing lens, and determine the target focusing position based on the required lens focal length according to the structural parameters; Determine the current focusing position, and determine the target adjustment parameters of the motor based on the current focusing position and the target focusing position.

6. The method according to any one of claims 1 to 5, characterized in that, Before the step of obtaining the user's eye movement information, the method further includes: Obtain the user's acceptable display level, and determine the corresponding acceptable depth variation range based on the acceptable display level; The step of adjusting the current focal length according to the current depth information via the adjustment mechanism includes: Determine the historical depth information corresponding to the last adjustment of the adjustment mechanism, and determine the depth change based on the historical depth information and the current depth information; When the depth change is not within the acceptable depth change range, the current focal length is adjusted by the adjustment mechanism according to the current depth information.

7. The method as described in claim 6, characterized in that, After the step of determining the depth change based on the historical depth information and the current depth information, the method further includes: When the depth change is within the acceptable depth change range, the current gaze duration of the user gazing at the current virtual target is obtained; The user's gaze duration threshold is determined based on the acceptable display level; When the current fixation duration reaches the current fixation threshold, the current focal length is adjusted by the adjustment mechanism according to the current depth information.

8. A display device, characterized in that, The device includes: The information acquisition module is used to acquire the user's eye movement information and determine the user's current gaze point on the screen of the virtual reality device based on the eye movement information; The scene determination module is used to determine the currently displayed virtual scene and determine the current virtual target being gazed at by the user based on the currently displayed virtual scene and the current gaze point; A depth determination module is used to determine the current depth information of the current virtual target from the user's eyeball based on the currently displayed virtual scene; The distance adjustment module is used to adjust the current focal length according to the current depth information through the adjustment mechanism, and to display the adjusted focal length on the screen.

9. A virtual reality device, characterized in that, The device includes: an adjustment mechanism, a processor, a memory, a lens, and a screen. The processor is connected to the memory and the adjustment mechanism, and the adjustment mechanism is used to adjust the current focal length between the lens and the screen. The processor is configured to run a display program stored in the memory, the display program being configured to implement the steps of the display method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a display program, which, when executed by a processor, implements the steps of the display method as described in any one of claims 1 to 7.

Images