Image display method, device, wearable device and storage medium
By obtaining eye rotation information and adjusting the shooting angle of the virtual camera, the visual fatigue problem caused by frequent zooming of smart glasses when viewing a three-dimensional picture is solved, and an image that is suitable for the reflection is achieved.
Patent Information
- Application Number
- CN202010731424.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-27
- Publication Date
- 2025-04-25
- Estimated Expiration
- 2040-07-27
AI Technical Summary
When viewing three-dimensional images with smart glasses, frequent zooming is required to cause visual fatigue.
By obtaining eye rotation information, the actual coordinates of the eye gaze point in the world coordinate system are determined, and the shooting angle of the virtual camera is adjusted according to the coordinates, so that the picture captured by the virtual camera is imaged on the focal plane of the eye.
Avoid frequent eye zooming, reduce visual fatigue, and generate images that are suitable for the radiation.
Smart Images

Figure CN111880654B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of intelligent wearable devices, and in particular to an image display method, an apparatus, a wearable device and a storage medium. Background Art
[0002] Smart wearable devices can process information efficiently with the assistance of computers and networks. Smart wearable devices may include smart watches, smart glasses, smart bracelets, etc. Smart glasses are usually combined with virtual reality technology or augmented reality technology to display specific images through lenses for human eyes to observe. In related technologies, when using smart glasses to watch three-dimensional images, the human eye needs to achieve zooming. For example, when the human eye looks at near and far, the eye achieves zooming through the changes in the tension of the surrounding muscles, so long-term viewing will cause fatigue.
[0003] Therefore, how to generate an image that is compatible with the convergence and avoid frequent zooming is a technical problem that those skilled in the art currently need to solve. Summary of the invention
[0004] The purpose of the present application is to provide an image display method, an apparatus, a wearable device and a storage medium, which can generate an image adapted to convergence and avoid frequent zooming.
[0005] In order to solve the above technical problems, the present application provides an image display method, which includes:
[0006] Acquire eyeball rotation information, and determine the actual coordinates of the eye gaze point in the world coordinate system according to the eyeball rotation information;
[0007] Determine the virtual coordinates of the eye gaze point in a virtual camera coordinate system according to the real coordinates; wherein the virtual camera coordinate system is a coordinate system of a virtual space where the virtual camera is located;
[0008] The shooting angle of the virtual camera is adjusted according to the virtual coordinates, and the picture shot by the virtual camera after the shooting angle is adjusted is displayed as the target image.
[0009] Optionally, determining the real coordinates of the eye gaze point in the world coordinate system according to the eye movement information includes:
[0010] Determine the left eye perspective and the right eye perspective according to the eye movement information;
[0011] The real coordinates of the eye gaze point in the world coordinate system are determined according to the left eye viewing angle, the right eye viewing angle and the pupil distance.
[0012] Optionally, the virtual camera includes a first virtual camera and a second virtual camera, wherein the first virtual camera is used to capture a target image observed by a left eye; and the second virtual camera is used to capture a target image observed by a right eye.
[0013] Optionally, adjusting the shooting angle of the virtual camera according to the virtual coordinates includes:
[0014] Determining the camera coordinates of the virtual camera in the virtual space, and adjusting the shooting angle of the virtual camera according to the positional relationship between the camera coordinates and the virtual coordinates;
[0015] The shooting direction of the virtual camera after adjusting the shooting direction is the same as a preset direction, and the preset direction is the direction in which the camera coordinates point to the virtual coordinates.
[0016] Optionally, adjusting the shooting angle of the virtual camera according to the virtual coordinates includes:
[0017] Determining an original shooting angle of the virtual camera;
[0018] Determining a target shooting angle of the virtual camera according to the virtual coordinates;
[0019] The virtual camera is controlled to rotate from the original shooting angle to the target shooting angle.
[0020] Optionally, obtaining eye movement information includes:
[0021] The eyeball rotation information is obtained by using an eyeball tracking device; wherein the eyeball rotation information includes the angle of the eyeball rotation within a preset period and the direction of the eyeball rotation.
[0022] Optionally, determining the virtual coordinates of the eye gaze point in the virtual camera coordinate system according to the real coordinates includes:
[0023] The coordinates are transformed according to the transformation relationship between the world coordinate system and the virtual camera coordinate system to obtain the virtual coordinates of the eye gaze point in the virtual camera coordinate system.
[0024] The present application also provides an image display device, the device comprising:
[0025] A real coordinate determination module, used for obtaining eyeball rotation information and determining the real coordinates of the eye gaze point in the world coordinate system according to the eyeball rotation information;
[0026] A virtual coordinate determination module, used to determine the virtual coordinates of the eye gaze point in a virtual camera coordinate system according to the real coordinates; wherein the virtual camera coordinate system is a coordinate system of a virtual space where the virtual camera is located;
[0027] The image display module is used to adjust the shooting angle of the virtual camera according to the virtual coordinates, and display the picture shot by the virtual camera after the shooting angle is adjusted as the target image.
[0028] The present application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps of the above-mentioned image display method are implemented.
[0029] The present application also provides a wearable device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of executing the above-mentioned image display method are implemented.
[0030] The present application provides an image display method, comprising: obtaining eye movement information, and determining the real coordinates of the eye gaze point in the world coordinate system according to the eye movement information; determining the virtual coordinates of the eye gaze point in the virtual camera coordinate system according to the real coordinates; wherein the virtual camera coordinate system is the coordinate system of the virtual space where the virtual camera is located; adjusting the shooting angle of the virtual camera according to the virtual coordinates, and displaying the picture taken by the virtual camera after adjusting the shooting angle as the target image.
[0031] The present application determines the real coordinates of the eye gaze point in the world coordinate system based on the eye rotation information, and determines the virtual coordinates of the eye gaze point in the virtual camera coordinate system. The image observed by the eye is the image taken by the virtual camera in the virtual space. Therefore, by adjusting the shooting angle of the virtual camera according to the virtual coordinates, the image taken by the virtual camera can be imaged on the focal plane of the eye. The glasses do not need to adjust the focal length, avoiding visual fatigue caused by changes in the tension of the muscles around the eyes. The virtual camera shooting angle adjustment method in the present application can truly simulate the eye movement state, can generate images that are adapted to the convergence, and avoid visual fatigue caused by frequent zooming of the eyes. The present application also provides an image display device, a wearable device and a storage medium, which have the above-mentioned beneficial effects and are not repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0033] Figure 1 A flowchart of an image display method provided in an embodiment of the present application;
[0034] Figure 2A schematic diagram of a principle for determining an eye gaze point provided in an embodiment of the present application;
[0035] Figure 3 A schematic diagram of a method for characterizing shooting angles of a virtual camera provided in an embodiment of the present application;
[0036] Figure 4 A schematic diagram of the structure of an image display device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0038] See below Figure 1 , Figure 1 A flowchart of an image display method provided in an embodiment of the present application.
[0039] Specific steps may include:
[0040] S101: Acquire eyeball rotation information, and determine the actual coordinates of the eye gaze point in the world coordinate system according to the eyeball rotation information;
[0041] Among them, this embodiment can be applied to wearable devices such as AR (Augmented Reality) glasses, VR (Virtual Reality) glasses, VR helmets, etc. that provide image display functions through display devices. The above-mentioned display device can be a pair of glasses lenses or a display screen.
[0042] The eyeball rotation information mentioned in this step is information describing the direction and angle of eyeball rotation, and the eyeball rotation information may include the rotation information of one or two eyeballs, and the eyeball gaze point may be determined based on the eyeball rotation information. As a feasible implementation, this embodiment may use an eyeball tracking device to obtain the eyeball rotation information; wherein the eyeball rotation information includes the angle of the eyeball rotation within a preset period and the direction of the eyeball rotation.
[0043] See also Figure 2 , Figure 2A schematic diagram of the principle of determining the eye gaze point provided in an embodiment of the present application, this embodiment can calculate the eye gaze point based on the line of sight direction of both eyes and the distance between both eyes combined with triangular geometric features. The process of determining the eye gaze point in this implementation can be converted into a side and two angles of a known triangle, and the position of the third vertex is calculated. This embodiment can calculate the line of sight direction of both eyes based on eye rotation information. The world coordinate system is a coordinate system in the real space where the user is located, and can be a Cartesian coordinate system established with a certain part of the user's body (such as the tip of the nose or the top of the head) as the origin.
[0044] S102: Determine the virtual coordinates of the eye gaze point in the virtual camera coordinate system according to the real coordinates;
[0045] Among them, the wearable device used in this embodiment can run a three-dimensional engine, which refers to a core component that can realize an interactive real-time image application. Through the three-dimensional engine, a three-dimensional model can be created in a virtual space, and the three-dimensional model in the virtual space can be photographed by a virtual camera to obtain the image that the user needs to observe, so that the user can experience an immersive and realistic feeling through the virtual camera shooting picture in the virtual space. Common three-dimensional engines may include Unity3D, Unreal Engine, Frost Engine, etc. Unity3D is a multi-platform comprehensive game engine that can create interactive content such as three-dimensional video games, architectural visualization, real-time three-dimensional animation, etc. Unity is similar to Director, Blendergame engine, Virtools or Torque Game Builder, etc., which use interactive graphical development environments as the primary method. The image display in the virtual space created by Unity3D is based on the principles of computer graphics. The size of the display screen is limited according to the clipping window, and the nearest and farthest distances of the scenery that the user can see are determined by determining the near clipping plane and the far clipping plane. Without changing the shooting angle of the virtual camera, what the user wants to see and the focus of attention are obtained by the user by adjusting the eyes to change to the focal plane and focus.
[0046] In this step, the virtual camera coordinate system is the coordinate system of the virtual space where the virtual camera is located. In this embodiment, the position of the eye focus point in the virtual space, that is, the virtual coordinates, can be determined according to the conversion relationship between the virtual camera coordinate system and the world coordinate system. Specifically, in this embodiment, the coordinates can be converted to the coordinate system according to the conversion relationship between the world coordinate system and the virtual camera coordinate system to obtain the virtual coordinates of the eye focus point in the virtual camera coordinate system. Specifically, if the ratio of the virtual space to the real world is equal, the real coordinates can be directly used as the virtual coordinates. If there is a scaling ratio between the virtual space and the real world, the real coordinates can be proportionally changed to obtain the virtual coordinates.
[0047] S103: adjusting the shooting angle of the virtual camera according to the virtual coordinates, and displaying the picture shot by the virtual camera after the shooting angle is adjusted as the target image.
[0048] Among them, this step is based on the virtual coordinates that have been determined, and the shooting angle of the virtual camera can be adjusted according to the virtual coordinates so that the focus of the virtual camera falls on the virtual coordinates. After the shooting angle of the virtual camera is adjusted, the picture taken by the virtual camera can be displayed as the target image. Since the target image is the picture taken by the virtual camera after adjusting the shooting angle, the target image is imaged on the focal plane of the user's eyes, and the user does not need to adjust the focal length by the tension of the eye muscles.
[0049] It is understandable that when a person's eyes look in different directions, there will be slight changes in the eyes. This embodiment obtains the eye gaze position based on the eye rotation information, thereby tracking the changes in the eyes in real time, predicting the user's state and needs, and responding by controlling the shooting angle of the virtual camera to achieve the purpose of controlling the device with the eyes. As a feasible implementation method, this embodiment can also use eye tracking technology to determine the eye gaze point, and calculate the position of the human eye in the AR image or VR image, adjust the shooting angle of the virtual camera to simulate the convergence of the human eye, and display the shooting picture after adjusting the shooting angle as the target image to achieve the simulation of human eye convergence in the three-dimensional engine. Convergence refers to the convergence of the binocular visual axis. When the user looks at an object, the binocular convergence makes the object fall on the corresponding position of the retina. When the position of the object changes in distance, the size of the convergence angle also changes accordingly, and this change is completed by the eye muscles. The signal of this change is transmitted to the brain, which constitutes a perception of depth information.
[0050] This embodiment determines the real coordinates of the eye gaze point in the world coordinate system based on the eyeball rotation information, and determines the virtual coordinates of the eye gaze point in the virtual camera coordinate system. The image observed by the eye is the image taken by the virtual camera in the virtual space. Therefore, by adjusting the shooting angle of the virtual camera according to the virtual coordinates, the image taken by the virtual camera can be imaged on the focal plane of the eye. The glasses do not need to adjust the focal length, avoiding visual fatigue caused by changes in the tension of the muscles around the eyes. The virtual camera shooting angle adjustment method in this embodiment can truly simulate the eye movement state, generate images that are adapted to convergence, and avoid visual fatigue caused by frequent zooming of the eyes.
[0051] As for Figure 1 For further description of the corresponding embodiment, the user usually uses both eyes to observe the image displayed by the wearable device, such as Figure 2The eyeball rotation information may include left eye rotation information and right eye rotation information, and then the left eye viewing angle is determined according to the left eye rotation information, the right eye viewing angle is determined according to the right eye rotation information, and the real coordinates of the eye gaze point in the world coordinate system are determined according to the left eye viewing angle, the right eye viewing angle and the pupil distance. Accordingly, in order to better conform to human observation habits, Figure 1 The virtual camera mentioned in the corresponding embodiment may include a first virtual camera and a second virtual camera, wherein the first virtual camera is used to capture the target image observed by the left eye; and the second virtual camera is used to capture the target image observed by the right eye.
[0052] See also Figure 3 , Figure 3 A schematic diagram of a method for determining a shooting angle characteristic of a virtual camera provided in an embodiment of the present application is shown in FIG. Figure 3 The dotted line in is the shooting direction of the virtual camera before adjusting the shooting angle, and the solid line is the shooting direction of the virtual camera after adjusting the shooting angle. In the process of adjusting the shooting angle of the virtual camera according to the virtual coordinates, the camera coordinates of the virtual camera in the virtual space can be determined first, and the shooting angle of the virtual camera can be adjusted according to the positional relationship between the camera coordinates and the virtual coordinates. Among them, the shooting direction of the virtual camera after adjusting the shooting direction is the same as the preset direction, and the preset direction is the direction in which the camera coordinates point to the virtual coordinates. It can be understood that when there are a first virtual camera and a second virtual camera corresponding to both eyes in the virtual space, respectively, the shooting angle of the first virtual camera can be adjusted according to the positional relationship between the camera coordinates of the first virtual camera and the virtual coordinates, and the shooting angle of the second virtual camera can be adjusted according to the positional relationship between the camera coordinates of the second virtual camera and the virtual coordinates.
[0053] As a feasible implementation, the specific process of adjusting the shooting angle of the virtual camera may be: determining the original shooting angle of the virtual camera before the shooting angle is adjusted, determining the target shooting angle of the virtual camera according to the virtual coordinates, and controlling the virtual camera to rotate from the original shooting angle to the target shooting angle. The target shooting angle is determined according to the virtual coordinates and the coordinates of the virtual camera.
[0054] The process described in the above embodiment is explained below through an embodiment in actual application. This embodiment describes a VR glasses display solution that simulates human eye convergence in a virtual space created by Unity3D, which may specifically include the following steps:
[0055] Step 1: Use an eye-tracking device to track the movement of the eyeballs and obtain the rotation angle φ1 of the eyeballs of both eyes;
[0056] The rotation angle can be determined by the degree of change in the angle between the current sight line direction of the eyeball and the preset direction, where the preset direction is the sight line direction when the human eye looks vertically at the VR screen.
[0057] Step 2: Calculate the gaze point of both eyes through the triangle geometric features and the rotation angle φ1, and calculate the real coordinates D(x1, y1, z1) of the gaze point in the world coordinate system, and upload the real coordinates D(x1, y1, z1) to the unity3D program;
[0058] Step 3: The Unity3D program converts the real coordinates D(x1, y1, z1) of the gaze point into the virtual coordinates d(x2, y2, z2) of the Unity3D virtual space, and then adjusts the rotation angle of the two virtual cameras in the Unity3D virtual space to φ2 according to the geometric characteristics of the triangle. The rotation angle of the two virtual cameras simulates the rotation φ2 of the human eye to realize the human eye convergence phenomenon.
[0059] The virtual coordinates are coordinates on a virtual space coordinate system, and the virtual space coordinate system may be a Cartesian coordinate system with a virtual camera in Unity3D as the coordinate origin o(0,0,0).
[0060] Step 4: VR glasses display the images captured by the virtual camera in the Unity3D virtual space. The images fall exactly on the focal plane where the user's eyes are focused. The user can clearly see the images in the Unity3D virtual space without having to readjust the eye muscles to change the focus.
[0061] In this embodiment, the rotation angle of the virtual camera is adjusted by the eyeball rotation information, the virtual camera simulates the direction of the human eye's gaze, and then the displayed picture is controlled by the direction of the virtual camera. In this embodiment, the eye's gaze point is calculated by the rotation angle of both eyes, and then the gaze point is converted into the rotation angle of the virtual camera and the picture taken after the rotation of the virtual camera is displayed. In this embodiment, the angle of the virtual camera is adjusted according to the rotation of the human eye, so that the displayed picture is always displayed in the focal plane, which can avoid visual fatigue caused by frequent zooming of the eyes.
[0062] See also Figure 4 , Figure 4 A schematic diagram of the structure of an image display device provided in an embodiment of the present application;
[0063] The device may include:
[0064] A real coordinate determination module 100, for obtaining eyeball rotation information and determining the real coordinates of the eye gaze point in the world coordinate system according to the eyeball rotation information;
[0065] A virtual coordinate determination module 200, configured to determine the virtual coordinates of the eye gaze point in a virtual camera coordinate system according to the real coordinates; wherein the virtual camera coordinate system is a coordinate system of a virtual space where the virtual camera is located;
[0066] The image display module 300 is used to adjust the shooting angle of the virtual camera according to the virtual coordinates, and display the picture shot by the virtual camera after the shooting angle is adjusted as the target image.
[0067] This embodiment determines the real coordinates of the eye gaze point in the world coordinate system based on the eyeball rotation information, and determines the virtual coordinates of the eye gaze point in the virtual camera coordinate system. The image observed by the eye is the image taken by the virtual camera in the virtual space. Therefore, by adjusting the shooting angle of the virtual camera according to the virtual coordinates, the image taken by the virtual camera can be imaged on the focal plane of the eye. The glasses do not need to adjust the focal length, avoiding visual fatigue caused by changes in the tension of the muscles around the eyes. The virtual camera shooting angle adjustment method in this embodiment can truly simulate the eye movement state, generate images that are adapted to convergence, and avoid visual fatigue caused by frequent zooming of the eyes.
[0068] Furthermore, the real coordinate determination module 100 includes:
[0069] A viewing angle determination unit, used to determine the left eye viewing angle and the right eye viewing angle according to the eyeball rotation information;
[0070] A coordinate determination unit is used to determine the real coordinates of the eye gaze point in the world coordinate system according to the left eye viewing angle, the right eye viewing angle and the pupil distance.
[0071] Furthermore, the virtual camera includes a first virtual camera and a second virtual camera, wherein the first virtual camera is used to capture a target image observed by a left eye; and the second virtual camera is used to capture a target image observed by a right eye.
[0072] Furthermore, the image display module 300 includes:
[0073] A first angle adjustment module, used to determine the camera coordinates of the virtual camera in the virtual space, and adjust the shooting angle of the virtual camera according to the positional relationship between the camera coordinates and the virtual coordinates;
[0074] The shooting direction of the virtual camera after adjusting the shooting direction is the same as a preset direction, and the preset direction is the direction in which the camera coordinates point to the virtual coordinates.
[0075] Furthermore, the image display module 300 includes:
[0076] The second angle adjustment module is used to determine the original shooting angle of the virtual camera; it is also used to determine the target shooting angle of the virtual camera according to the virtual coordinates; it is also used to control the virtual camera to rotate from the original shooting angle to the target shooting angle.
[0077] Furthermore, the real coordinate determination module 100 includes:
[0078] An eye tracking unit is used to obtain the eye movement information using an eye tracking device; wherein the eye movement information includes the angle of the eye rotation within a preset period and the direction of the eye rotation.
[0079] Furthermore, the virtual coordinate determination module 200 is specifically a module for performing coordinate system transformation on the coordinates according to the transformation relationship between the world coordinate system and the virtual camera coordinate system to obtain the virtual coordinates of the eye gaze point in the virtual camera coordinate system.
[0080] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.
[0081] The present application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0082] The present application also provides a wearable device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the wearable device may also include various network interfaces, power supplies and other components.
[0083] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.
[0084] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
Claims
1. An image display method, characterized in that: include: Acquire eyeball rotation information, and determine the real coordinates of the eye gaze point in the world coordinate system according to the eyeball rotation information; the eyeball rotation information includes the rotation angle of the eyeball; The rotation angle is determined by the degree of change in the angle between the current sight direction of the eyeball and the preset direction. The preset direction is the sight direction when the human eye looks vertically at the VR screen. Determine the virtual coordinates of the eye gaze point in a virtual camera coordinate system according to the real coordinates; wherein the virtual camera coordinate system is a coordinate system of a virtual space where the virtual camera is located; Adjusting the shooting angle of the virtual camera according to the virtual coordinates, and displaying the picture taken by the virtual camera after adjusting the shooting angle as the target image; Wherein, adjusting the shooting angle of the virtual camera according to the virtual coordinates includes: Determining the camera coordinates of the virtual camera in the virtual space, and adjusting the shooting angle of the virtual camera according to the positional relationship between the camera coordinates and the virtual coordinates; Wherein, the shooting direction of the virtual camera after adjusting the shooting direction is the same as the preset direction, and the preset direction is the direction in which the camera coordinates point to the virtual coordinates; The virtual camera includes a first virtual camera and a second virtual camera, wherein the first virtual camera is used to capture a target image observed by the left eye; and the second virtual camera is used to capture a target image observed by the right eye; Accordingly, adjusting the shooting angle of the virtual camera according to the positional relationship between the camera coordinates and the virtual coordinates includes: The shooting angle of the first virtual camera is adjusted according to the positional relationship between the camera coordinates of the first virtual camera and the virtual coordinates, and the shooting angle of the second virtual camera is adjusted according to the positional relationship between the camera coordinates of the second virtual camera and the virtual coordinates, so as to realize the human eye convergence phenomenon.
2. The image display method according to claim 1, characterized in that: Determining the real coordinates of the eye gaze point in the world coordinate system according to the eye movement information includes: Determine the left eye perspective and the right eye perspective according to the eye movement information; The real coordinates of the eye gaze point in the world coordinate system are determined according to the left eye viewing angle, the right eye viewing angle and the pupil distance.
3. The image display method according to claim 1, characterized in that: Adjusting the shooting angle of the virtual camera according to the virtual coordinates includes: Determining an original shooting angle of the virtual camera; Determining a target shooting angle of the virtual camera according to the virtual coordinates; The virtual camera is controlled to rotate from the original shooting angle to the target shooting angle.
4. The image display method according to claim 1, characterized in that: The obtaining of eye movement information comprises: The eyeball rotation information is obtained by using an eyeball tracking device; wherein the eyeball rotation information includes the angle of the eyeball rotation within a preset period and the direction of the eyeball rotation.
5. The image display method according to any one of claims 1 to 4, characterized in that: Determining the virtual coordinates of the eye gaze point in the virtual camera coordinate system according to the real coordinates includes: The real coordinates are transformed into a coordinate system according to the transformation relationship between the world coordinate system and the virtual camera coordinate system to obtain the virtual coordinates of the eye gaze point in the virtual camera coordinate system.
6. An image display device, characterized in that: include: A real coordinate determination module, used to obtain eyeball rotation information, and determine the real coordinates of the eye gaze point in the world coordinate system according to the eyeball rotation information; the eyeball rotation information includes the rotation angle of the eyeball; The rotation angle is determined by the degree of change in the angle between the current sight direction of the eyeball and the preset direction. The preset direction is the sight direction when the human eye looks vertically at the VR screen. A virtual coordinate determination module, used to determine the virtual coordinates of the eye gaze point in a virtual camera coordinate system according to the real coordinates; wherein the virtual camera coordinate system is a coordinate system of a virtual space where the virtual camera is located; An image display module, used for adjusting the shooting angle of the virtual camera according to the virtual coordinates, and displaying the picture shot by the virtual camera after adjusting the shooting angle as the target image; Wherein, the image display module comprises: A first angle adjustment module, used to determine the camera coordinates of the virtual camera in the virtual space, and adjust the shooting angle of the virtual camera according to the positional relationship between the camera coordinates and the virtual coordinates; Wherein, the shooting direction of the virtual camera after adjusting the shooting direction is the same as the preset direction, and the preset direction is the direction in which the camera coordinates point to the virtual coordinates; The virtual camera includes a first virtual camera and a second virtual camera, wherein the first virtual camera is used to capture a target image observed by the left eye; and the second virtual camera is used to capture a target image observed by the right eye; Correspondingly, the process of the first angle adjustment module adjusting the shooting angle of the virtual camera according to the positional relationship between the camera coordinates and the virtual coordinates includes: The shooting angle of the first virtual camera is adjusted according to the positional relationship between the camera coordinates of the first virtual camera and the virtual coordinates, and the shooting angle of the second virtual camera is adjusted according to the positional relationship between the camera coordinates of the second virtual camera and the virtual coordinates, so as to realize the human eye convergence phenomenon.
7. A wearable device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the image display method according to any one of claims 1 to 5 when calling the computer program in the memory.
8. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the steps of the image display method as described in any one of claims 1 to 5 are implemented.
Citation Information
Patent Citations
Information processing device, information processing method, and program
US20180205888A1