Display method of VR device and electronic device

Abstract

The application provides a display method of a VR device and an electronic device, and is used for improving the display effect of the VR device. The method comprises the following steps: after the VR device starts a screen material acquisition function, every specified time length, the eyeball data of a user is acquired by using a preset eyeball tracking algorithm; when the screen material is acquired, for any screen material, the position coordinates of the pupil center point in the eyeball data corresponding to the screen material and the position coordinates of the reference point in the preset standard eyeball data in the VR device are used to obtain the offset parameter of the screen material; if it is determined that the offset parameter of the screen material is in a specified range, the adjustment parameter of the screen material is obtained according to the eyeball data of the screen material and the standard eyeball data; the position and size of the screen material are adjusted by using the adjustment parameter to obtain a target screen material; and the target screen material is rendered by using a preset rendering mode, and the rendered screen material is displayed.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, and more particularly to a display method and electronic device for a VR device. Background Technology

[0002] VR (Virtual Reality) is a brand-new practical technology that emerged in the 20th century. Virtual reality technology encompasses computer science, electronic information, and simulation technology. Its basic implementation relies primarily on computer technology, utilizing and integrating the latest advancements in 3D graphics, multimedia, simulation, display, and server technologies. With the help of computers and other equipment, it creates a realistic 3D virtual world offering a multi-sensory experience, including visual, tactile, and olfactory sensations, thus giving those within the virtual world a sense of immersion.

[0003] With the rapid development of the internet, VR devices are becoming increasingly popular and widespread. Due to the differences between current VR products and the real world, sharing content often relies on screenshot or screen recording functions. However, in actual use, due to differences in user wearing habits or individual facial dimensions, screenshots or screen recordings exported to the display device may not be centered, exhibiting tilt or angular misalignment. Figure 1 As shown, the screen materials are displayed screenshots or screen recordings, from... Figure 1 As can be seen, the screen material is tilted and angularly offset. This results in a poor display effect on VR devices. Summary of the Invention

[0004] This application provides a display method and electronic device for VR devices, which adjusts screen materials such as screenshots or screen recordings to avoid problems such as non-centering, tilting, or angle shift when displaying the screen materials, thereby improving the display effect of VR devices.

[0005] In a first aspect, embodiments of this application provide a display method for a VR device, the method comprising:

[0006] When the VR device activates the screen material acquisition function, it acquires the user's eye data using a preset eye-tracking algorithm at specified intervals. The eye data includes the position coordinates of the center point of the pupil and the left and right eye identifiers. The screen material acquisition function includes a screenshot function or a screen recording function.

[0007] Once screen footage is acquired, for any given screen footage, based on the coordinates of the pupil center point in the corresponding eye data and the coordinates of a reference point in the preset standard eye data of the VR device, an offset parameter for the screen footage is obtained. This offset parameter characterizes whether an offset occurs between the screen footage and the reference point. The screen footage includes left-eye and / or right-eye screen footage, and the left and right eye identifiers in the standard eye data are the same as those in the eye data.

[0008] If the offset parameter of the screen material is determined to be within a specified range, then the adjustment parameters of the screen material are obtained based on the eye data corresponding to the screen material and the standard eye data; and,

[0009] The position and size of the screen material are adjusted using the aforementioned adjustment parameters to obtain the target screen material;

[0010] The target screen material is rendered using a preset rendering method, and the rendered screen material is then displayed.

[0011] A second aspect of this application provides an electronic device, including a processor and a memory, wherein the processor and the memory are connected via a bus;

[0012] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0013] When the VR device activates the screen material acquisition function, it acquires the user's eye data using a preset eye-tracking algorithm at specified intervals. The eye data includes the position coordinates of the center point of the pupil and the left and right eye identifiers. The screen material acquisition function includes a screenshot function or a screen recording function.

[0014] Once screen footage is acquired, for any given screen footage, based on the coordinates of the pupil center point in the corresponding eye data and the coordinates of a reference point in the preset standard eye data of the VR device, an offset parameter for the screen footage is obtained. This offset parameter characterizes whether an offset occurs between the screen footage and the reference point. The screen footage includes left-eye and / or right-eye screen footage, and the left and right eye identifiers in the standard eye data are the same as those in the eye data.

[0015] If the offset parameter of the screen material is determined to be within a specified range, then the adjustment parameters of the screen material are obtained based on the eye data corresponding to the screen material and the standard eye data; and,

[0016] The position and size of the screen material are adjusted using the aforementioned adjustment parameters to obtain the target screen material;

[0017] The target screen material is rendered using a preset rendering method, and the rendered screen material is then displayed.

[0018] According to a third aspect of the present invention, a computer storage medium is provided, the computer storage medium storing a computer program for performing the method as described in the first aspect.

[0019] In the above embodiments of this application, the offset parameters of the screen material are determined by the position coordinates of the pupil center point in the eye data corresponding to the screen material and the position coordinates of the reference point in the preset standard eye data in the VR device. When the offset parameters of the screen material are determined to be within a specified range, the adjustment parameters of the screen material are obtained according to the eye data corresponding to the screen material and the standard eye data. Then, the display position and size of the screen material are adjusted using the adjustment parameters to obtain the target screen material and display it. Thus, in the embodiments of this application, the size and position of the screen material are adjusted by using the user's eye data, which solves the problems of screen material not being centered, tilting, or angular offset caused by the difference between the user's pupil center and the screen center point, thereby improving the display effect of the VR device. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 An exemplary diagram of screen material in the prior art provided in the embodiments of this application is shown;

[0022] Figure 2 An exemplary illustration shows one of the application scenarios provided in the embodiments of this application;

[0023] Figure 3 The second example illustration shows a schematic diagram of an application scenario provided in an embodiment of this application;

[0024] Figure 4 One of the flowcharts of the display method of the VR device provided in the embodiments of this application is illustrated by way of example;

[0025] Figure 5 An exemplary diagram of the reference point of the VR device provided in an embodiment of this application is shown;

[0026] Figure 6 An exemplary flowchart illustrating the process of determining the offset parameters of screen material provided in an embodiment of this application is shown.

[0027] Figure 7 An exemplary flowchart illustrating the determination of adjustment parameters for screen materials provided in an embodiment of this application is shown;

[0028] Figure 8 An exemplary schematic diagram of the adjusted screen material provided in an embodiment of this application is shown;

[0029] Figure 9 The second schematic flowchart of the display method of the VR device provided in the embodiment of this application is illustrated by example;

[0030] Figure 10 An exemplary schematic diagram of the display device of the VR device provided in an embodiment of this application is shown;

[0031] Figure 11 An exemplary hardware structure diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0032] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0033] Based on the exemplary embodiments described in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the appended claims. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete implementation on its own.

[0034] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0035] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to be omnipresent but not exclusive; for example, a product or device comprising a series of components is not necessarily limited to those explicitly listed, but may include other components not explicitly listed or inherent to such product or device.

[0036] As used in this application, the term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0037] The following is an overview of the ideas behind the embodiments of this application.

[0038] Currently, due to the differences between VR products and the real world, sharing VR content often relies on screenshot or screen recording functions. However, in actual use, due to differences in user wearing habits or individual facial dimensions, screenshots or screen recordings often fail to display centered on the screen, exhibiting tilt or angle shifts. This results in poor display quality from VR devices.

[0039] In existing technologies, screenshots or screen recordings exported to a display device may not be centered, resulting in tilting or angular offset. This application provides a display method for VR devices. The method determines the offset parameters of the screen material by using the coordinates of the pupil center point in the eye data corresponding to the screen material and the coordinates of a reference point in the preset standard eye data of the VR device. When the offset parameters are determined to be within a specified range, adjustment parameters are obtained based on the eye data and standard eye data. These adjustment parameters are then used to adjust the display position and size of the screen material to obtain the target screen material for display. Therefore, this application uses the user's eye data to adjust the screen material, solving the problem of screen material not being centered and exhibiting tilting or angular offset due to the difference between the user's pupil center and the screen center point, thus improving the display effect of the VR device. The embodiments of this application are described in detail below with reference to the accompanying drawings.

[0040] Figure 2 An exemplary illustration shows an application scenario diagram of the display method for a VR device provided in an embodiment of this application. For example... Figure 2The application scenario illustrated uses an electronic device as the server. This scenario includes a VR device 210 and a server 220. The server 220 can be implemented using a single server or multiple servers. The server 220 can be implemented using a physical server or a virtual server.

[0041] In one possible application scenario, after the VR device 210 activates the screen material acquisition function, the server 220 acquires the user's eye data at specified intervals using a preset eye-tracking algorithm. This eye data includes the coordinates of the pupil's center point and left / right eye identifiers. The screen material acquisition function includes either a screenshot or screen recording function. When the server 220 acquires screen material, for any given screen material, based on the coordinates of the pupil's center point in the corresponding eye data and the coordinates of a reference point in the preset standard eye data of the VR device 210, an offset parameter is obtained. This offset parameter indicates whether an offset occurs between the screen material and the reference point. The screen material includes left-eye and / or right-eye screen material, and the left / right eye identifiers in the standard eye data are the same as those in the eye data. If the server 220 determines that the offset parameter of the screen material is within a specified range, it obtains the adjustment parameter of the screen material based on the eye data corresponding to the screen material and the standard eye data; and adjusts the position and size of the screen material using the adjustment parameter to obtain the target screen material; renders the target screen material using a preset rendering method, and displays the rendered screen material through the VR device 210.

[0042] like Figure 3The diagram illustrates another application scenario of this application, using a VR device as an example. This scenario includes a VR device 210 and a memory 230. When the VR device 210 activates its screen material acquisition function, it acquires the user's eye data using a preset eye-tracking algorithm at specified intervals. This eye data includes the coordinates of the pupil's center point and left / right eye identifiers. The screen material acquisition function includes either a screenshot or screen recording function. Upon acquiring screen material, for any given screen material, based on the coordinates of the pupil's center point in the corresponding eye data and the coordinates of a reference point in the standard eye data pre-stored in the memory 230 by the VR device 210, an offset parameter is obtained. This offset parameter is used to represent... The VR device 210 determines whether there is an offset between the screen material and the reference point, and the screen material includes left-eye screen material and / or right-eye screen material, and the left and right eye identifiers of the standard eye data are the same as the left and right eye identifiers of the eye data; and if the VR device 210 determines that the offset parameter of the screen material is within a specified range, then it obtains the adjustment parameter of the screen material according to the eye data corresponding to the screen material and the standard eye data; and adjusts the position and size of the screen material through the adjustment parameter to obtain the target screen material; the VR device 210 renders the target screen material using a preset rendering method and displays the rendered screen material.

[0043] in, Figure 2 The server 220 and the VR device 210 can interact via a communication network, which can be either wireless or wired communication.

[0044] For example, server 220 can access the network via cellular mobile communication technology to communicate with VR device 210, wherein the cellular mobile communication technology includes, for example, 5th Generation Mobile Networks (5G) technology.

[0045] Optionally, the server 220 can access the network and communicate with the VR device 210 via short-range wireless communication, such as Wireless Fidelity (Wi-Fi) technology.

[0046] The description in this application focuses on a single VR device 210, a single server 220, and a single memory 230. However, those skilled in the art should understand that the illustrated VR device 210, server 220, and memory 230 are intended to illustrate the operation of the VR device 210, server 220, and memory 230 involved in the technical solutions of this application, and do not imply any limitation on the number, type, or location of the VR device 210, server 220, and memory 230. It should be noted that adding additional modules to or removing individual modules from the illustrated environment will not change the underlying concept of the exemplary embodiments of this application.

[0047] It should be noted that the display method for VR devices proposed in this application is not only applicable to... Figure 2 and Figure 3 The application scenarios shown can also be applied to any device with a VR display.

[0048] The following describes an exemplary embodiment of the VR device display method of this application, in conjunction with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the methods and principles of this application, and the implementation of this application is not limited in any way in this respect.

[0049] like Figure 4 The diagram shown illustrates a display method for a VR device, which may include the following steps:

[0050] Step 401: After the VR device starts the screen material acquisition function, it acquires the user's eye data using a preset eye tracking algorithm at specified intervals. The eye data includes the position coordinates of the center point of the pupil and the left and right eye identifiers. The screen material acquisition function includes a screenshot function or a screen recording function.

[0051] In this embodiment, if the screen material acquisition function is a screenshot function, the corresponding specified duration is the screenshot duration, meaning one screenshot corresponds to one eye data point. If the screen material acquisition function is a screen recording function, the corresponding specified duration is no greater than the screen recording duration, meaning one screen recording corresponds to at least one eye data point. However, this embodiment does not limit the specified duration; the specified duration in this embodiment can be set according to actual circumstances.

[0052] It should be noted that in this embodiment, obtaining screen material involves taking screenshots or recording the content displayed on the VR device. In this embodiment, the content displayed on the VR device changes with the user's head posture or the VR device's wearing position. That is, if the same user has the same head posture and the same VR device wearing position, the content displayed on the VR device will not change with eye movement. In other words, if the same user does not change their head posture or VR device wearing position, and only their eyes move, the content displayed on the VR device will not change. However, when obtaining screen material, if the user's eye data differs from the preset standard eye data in the VR device, the position and / or size of the obtained screen material will differ from the position and / or size of the screen material corresponding to the standard eye data.

[0053] Step 402: When screen material is obtained, for any screen material, based on the position coordinates of the pupil center point in the eye data corresponding to the screen material and the position coordinates of the reference point in the preset standard eye data in the VR device, the offset parameter of the screen material is obtained. The offset parameter is used to characterize whether there is an offset between the screen material and the reference point. The screen material includes left eye screen material and / or right eye screen material. The left and right eye identifiers of the standard eye data are the same as the left and right eye identifiers of the eye data.

[0054] For example, such as Figure 5 The diagram shows a reference point in a VR device. In this embodiment, the reference point is the midpoint of the line connecting the center points of the left and right eyes of the VR device.

[0055] In one embodiment, the screen material includes screenshots or screen recordings. In this embodiment, the acquired screen material can be only the left-eye view, only the right-eye view, or both. This embodiment does not limit the acquired screen material and can set it according to actual conditions. The method for determining the offset parameters of the screen material is described below. Figure 6 The diagram shown illustrates the process for determining the offset parameters of screen assets, including the following steps:

[0056] Step 601: For any eyeball data in the screen material, based on the position coordinates of the pupil center point and the position coordinates of the reference point in the eyeball data, obtain the distance between the pupil center point and the reference point; wherein, the distance between the pupil center point and the reference point can be obtained by formula (1):

[0057]

[0058] Where d is the distance between the center point of the pupil and the reference point, x1 is the x-coordinate of the center point of the pupil, y1 is the y-coordinate of the center point of the pupil, z1 is the y-coordinate of the center point of the pupil, x2 is the x-coordinate of the reference point, y2 is the y-coordinate of the reference point, and z2 is the y-coordinate of the reference point.

[0059] Step 602: Determine whether the eye data of the screen material is a single one. If yes, proceed to step 603; otherwise, proceed to step 604.

[0060] Step 603: Determine the offset parameter of the screen material based on the distance;

[0061] Step 604: Weighted summation of multiple distances corresponding to each eyeball data to obtain the offset parameters of the screen material.

[0062] It should be noted that the weights of the multiple distances corresponding to each eyeball data in this embodiment are pre-set and can be set according to the actual situation. This embodiment does not limit the weights of each distance.

[0063] Step 403: If it is determined that the offset parameter of the screen material is within the specified range, then the adjustment parameter of the screen material is obtained based on the eye data corresponding to the screen material and the standard eye data;

[0064] To conserve computing resources, in one embodiment, if the offset parameter is not within the specified range, the screen material is not adjusted. Specifically, if the offset parameter is less than the minimum value within the specified range, it is determined that there is no offset or the offset is small, and therefore no adjustment is needed to the screen material. If the offset parameter is greater than the maximum value within the specified range, the offset is too large, and even adjustment will not achieve a good effect; therefore, in this embodiment, no adjustment is made.

[0065] It should be noted that the specified range in this embodiment can be set according to the actual situation, and this embodiment does not limit the specified range. Furthermore, the eye data in this embodiment also includes interpupillary distance.

[0066] like Figure 7 The flowchart shown illustrates the process of determining adjustment parameters for screen assets, and may include the following steps:

[0067] Step 701: For any eye data of the screen material, based on the matrix corresponding to the eye data and the matrix corresponding to the standard eye data, obtain the first intermediate adjustment parameter, wherein the matrix corresponding to the eye data is composed of the interpupillary distance, the position coordinates of the pupil center point, and the left and right eye identifiers;

[0068] The matrix corresponding to the eyeball data can be in the form of {E0 E}. x E y E z S I}, where E0 represents the interpupillary distance, E x E is the x-coordinate of the center point of the pupil. y E is the ordinate of the center point of the pupil. z S is the vertical coordinate of the center point of the pupil. I For left and right eye identification, I takes the value L or R, where S L S represents the left eye. R This represents the right eye. Furthermore, the matrix form corresponding to the standard eyeball data in this embodiment is the same as the matrix form corresponding to the eyeball data in the embodiments of this application, and will not be repeated here.

[0069] It should be noted that the form of the matrix corresponding to the eye data in this embodiment is only for illustrative purposes and does not limit the form of the matrix corresponding to the eye data in the application embodiment. The form of the matrix corresponding to the eye data in this application embodiment can be set according to the actual situation.

[0070] In one embodiment, step 701 may be specifically implemented as: subtracting the matrix corresponding to the eyeball data from the matrix corresponding to the standard eyeball data to obtain the first intermediate adjustment parameter.

[0071] Step 702: Obtain the second intermediate adjustment parameter based on the first intermediate adjustment parameter and the preset adjustment parameter;

[0072] In one embodiment, step 702 may be specifically implemented as follows: multiplying the first intermediate adjustment parameter by the preset adjustment parameter to obtain the second intermediate adjustment parameter.

[0073] In this embodiment, the preset adjustment parameters include a first preset adjustment parameter and a second preset adjustment parameter;

[0074] In one embodiment, the second intermediate adjustment parameter can be obtained in the following manner:

[0075] Multiply the preset adjustment parameter by the first preset adjustment parameter, and then multiply by the second preset adjustment parameter to obtain the second intermediate adjustment parameter. The second intermediate adjustment parameter can be obtained using formula (2):

[0076] F i =(E i -f)×C1×C2……(2);

[0077] Among them, F iE is the second intermediate adjustment parameter corresponding to eyeball data i. i The matrix corresponding to eyeball data i, f is the matrix corresponding to the standard eyeball data, C1 is the first preset adjustment parameter, and C2 is the second preset adjustment parameter.

[0078] It should be noted that the first preset adjustment parameter in this embodiment Furthermore, in this embodiment, the first preset adjustment parameters A0, A x A xy A y A xz A yz A z These are coordinate mapping correction coefficients determined based on hardware and optical parameters in different dimensions. These are pre-set parameters, and in this embodiment, R is the second preset adjustment parameter. f B x B y and B z To address the display adjustment matrix parameters related to screen size, resolution, arrangement (single screen, dual screen, etc.), and refresh direction for different devices based on their screen display characteristics, similarly, the second preset adjustment parameter is also pre-set. For example, for a dual-screen setup with a screen size of 5.46 inches, a resolution of 1920*3664, and a horizontal refresh direction, the R parameter in the second preset adjustment parameter would be... f =5.4, B x =1.0, B y =1.0, B z =0.1. The first preset adjustment parameter and the second preset adjustment parameter in this embodiment can be set according to the actual situation. This embodiment does not limit the first preset adjustment parameter and the second preset adjustment parameter.

[0079] Step 703: Determine whether the number of eye data in the screen material is one. If yes, proceed to step 704; otherwise, proceed to step 705.

[0080] Step 704: Determine the second intermediate adjustment parameter as the adjustment parameter of the screen material;

[0081] Step 705: The second intermediate adjustment parameters corresponding to each eyeball data are weighted and summed to obtain the adjustment parameters of the screen material.

[0082] It should be noted that the weights of the second intermediate adjustment parameters corresponding to each eyeball data in this embodiment are preset. This embodiment does not limit the weights of each second intermediate adjustment parameter. The second intermediate adjustment parameters in this embodiment can be set according to the actual situation.

[0083] Step 404: Adjust the position and size of the screen material using the adjustment parameters to obtain the target screen material;

[0084] The adjustment parameters include scaling parameters and displacement parameters, and the displacement parameters include moving distance and moving direction.

[0085] In one embodiment, step 404 may be specifically implemented as follows: adjusting the size of the screen material using the scaling parameter, and moving the screen material by the moving distance in the moving direction based on the displacement parameter, to obtain the target screen material.

[0086] For example, the scaling parameter can be in the form of {R xyz}, where R is the scaling parameter. When R > 1, the size of the screen material is enlarged; specifically, the enlarged size is the current size of the screen material multiplied by the scaling parameter. When R < 1, the size of the screen material is reduced; specifically, the reduced size is the current size of the screen material multiplied by the scaling parameter. x represents the distance moved along the horizontal axis, y represents the distance moved along the vertical axis, and z represents the distance moved along the vertical axis. Taking x as an example, when x > 0, the screen material moves in the positive direction along the x-axis; when x < 0, the screen material moves in the negative direction along the x-axis. The directions of movement for y and z are determined in the same way as for x, and will not be elaborated upon in this embodiment.

[0087] Step 405: Render the target screen material using the preset rendering method and display the rendered screen material.

[0088] For example, such as Figure 8 As shown, this is for Figure 1 An illustration of the adjusted screen assets, from... Figure 8 As can be seen, the adjusted screen material did not shift and was displayed in the center, improving the display effect of VR devices.

[0089] In this embodiment, the method of rendering screen materials is the existing technology. The specific rendering method can be set according to the actual situation. This embodiment does not limit the rendering method.

[0090] It should be noted that: when the screen material obtained in step 401 is the left-eye screen material, the rendered screen material is the left-eye screen material and is limited by the left eye of the VR device. When the screen material obtained in step 401 is the right-eye screen material, the rendered screen material is the right-eye screen material and is limited by the right eye of the VR device. When both the left-eye and right-eye screen materials are obtained simultaneously in step 401, the rendered screen material includes both the rendered left-eye and rendered right-eye screen materials, and the rendered left-eye screen material is displayed through the left eye of the VR device, while the rendered right-eye screen material is displayed through the right eye of the VR device.

[0091] To further connect the technical solutions in this application, the following is combined with... Figure 9 A detailed explanation may include the following steps:

[0092] Step 901: After the VR device starts the screen material acquisition function, it acquires the user's eye data using a preset eye tracking algorithm at specified intervals. The eye data includes the position coordinates of the center point of the pupil and the left and right eye identifiers. The screen material acquisition function includes a screenshot function or a screen recording function.

[0093] Step 902: When screen material is obtained, for any screen material, based on the position coordinates of the pupil center point in the eye data corresponding to the screen material and the position coordinates of the reference point in the preset standard eye data in the VR device, the offset parameter of the screen material is obtained. The offset parameter is used to characterize whether there is an offset between the screen material and the reference point. The screen material includes left eye screen material and / or right eye screen material. The left and right eye identifiers of the standard eye data are the same as the left and right eye identifiers of the eye data.

[0094] Step 903: Determine whether the number of eye data in the screen material is one. If yes, proceed to step 904; otherwise, return to step 905.

[0095] Step 904: Determine the offset parameter of the screen material based on the distance;

[0096] Step 905: Sum the multiple distances corresponding to each eyeball data using a weighted method to obtain the offset parameters of the screen material;

[0097] Step 906: Based on the matrix corresponding to the eyeball data and the matrix corresponding to the standard eyeball data, obtain the first intermediate adjustment parameter, wherein the matrix corresponding to the eyeball data is composed of the interpupillary distance, the position coordinates of the pupil center point, and the left and right eye identifiers;

[0098] Step 907: Obtain the second intermediate adjustment parameter based on the first intermediate adjustment parameter and the preset adjustment parameter;

[0099] Step 908: Determine whether the eye data of the screen material is a single one. If yes, proceed to step 909; otherwise, proceed to step 910.

[0100] Step 909: Determine the second intermediate adjustment parameter as the adjustment parameter of the screen material;

[0101] Step 910: The second intermediate adjustment parameters corresponding to each eyeball data are weighted and summed to obtain the adjustment parameters of the screen material;

[0102] Step 911: Determine whether the adjustment parameter is within the specified range. If yes, proceed to step 912; otherwise, proceed to step 914.

[0103] Step 912: Obtain the adjustment parameters of the screen material based on the eye data corresponding to the screen material and the standard eye data;

[0104] Step 913: Adjust the position and size of the screen material using the adjustment parameters to obtain the target screen material;

[0105] Step 914: Do not adjust the screen material;

[0106] Step 915: Render the target screen material using the preset rendering method and display the rendered screen material.

[0107] Based on the same inventive concept, the VR device display method described above can also be implemented by a VR device display device. The effect of this VR device display device is similar to that of the aforementioned method, and will not be described again here.

[0108] Figure 10 This is a schematic diagram of the structure of a display device for a VR device according to an embodiment of the present disclosure.

[0109] like Figure 10 As shown, the display device 1000 of the VR device disclosed herein may include an eye data acquisition module 1010, an offset parameter determination module 1020, an adjustment parameter determination module 1030, a target screen material determination module 1040, and a rendering module 1050.

[0110] The eye data acquisition module 1010 is used to acquire the user's eye data using a preset eye tracking algorithm at specified intervals after the VR device starts the screen material acquisition function. The eye data includes the position coordinates of the center point of the pupil and the left and right eye identifiers. The screen material acquisition function includes a screenshot function or a screen recording function.

[0111] The offset parameter determination module 1020 is used to, when screen material is acquired, obtain the offset parameter of the screen material for any screen material based on the position coordinates of the pupil center point in the eye data corresponding to the screen material and the position coordinates of the reference point in the preset standard eye data in the VR device. The offset parameter is used to characterize whether there is an offset between the screen material and the reference point, and the screen material includes left eye screen material and / or right eye screen material. The left and right eye identifiers of the standard eye data are the same as the left and right eye identifiers of the eye data.

[0112] The parameter adjustment determination module 1030 is used to determine the adjustment parameters of the screen material based on the eye data corresponding to the screen material and the standard eye data if the offset parameter of the screen material is determined to be within a specified range.

[0113] The target screen material determination module 1040 is used to adjust the position and size of the screen material using the adjustment parameters to obtain the target screen material;

[0114] The rendering module 1050 is used to render the target screen material using a preset rendering method and display the rendered screen material.

[0115] In one embodiment, the offset parameter determination module 1020 is specifically used for:

[0116] For any eyeball data in the screen material, the distance between the pupil center point and the reference point is obtained based on the position coordinates of the pupil center point and the position coordinates of the reference point in the eyeball data;

[0117] If the number of eyeball data in the screen material is one, then the distance is used to determine the offset parameter of the screen material;

[0118] If there are multiple eyeball data points in the screen material, then the multiple distances corresponding to each eyeball data point are weighted and summed to obtain the offset parameters of the screen material.

[0119] In one embodiment, the eye data also includes interpupillary distance;

[0120] The adjustment parameter determination module 1030 is specifically used for:

[0121] For any eye data point in the screen material, a first intermediate adjustment parameter is obtained based on the matrix corresponding to the eye data and the matrix corresponding to the standard eye data. The matrix corresponding to the eye data is composed of the interpupillary distance, the coordinates of the pupil center point, and left / right eye identifiers.

[0122] The second intermediate adjustment parameter is obtained based on the first intermediate adjustment parameter and the preset adjustment parameter;

[0123] If the number of eye data in the screen material is one, then the second intermediate adjustment parameter is determined as the adjustment parameter of the screen material;

[0124] If there are multiple eye data points in the screen material, the second intermediate adjustment parameters corresponding to each eye data point are weighted and summed to obtain the adjustment parameters of the screen material.

[0125] In one embodiment, the adjustment parameter determination module 1030 executes the matrix based on the eyeball data and the matrix based on the standard eyeball data to obtain the first intermediate adjustment parameter, specifically used for:

[0126] Subtract the matrix corresponding to the eyeball data from the matrix corresponding to the standard eyeball data to obtain the first intermediate adjustment parameter;

[0127] The adjustment parameter determination module 1030 performs the step of obtaining the second intermediate adjustment parameter based on the first intermediate adjustment parameter and the preset adjustment parameter, specifically for:

[0128] The second intermediate adjustment parameter is obtained by multiplying the first intermediate adjustment parameter by the preset adjustment parameter.

[0129] In one embodiment, the preset adjustment parameters include a first preset adjustment parameter and a second preset adjustment parameter;

[0130] The second intermediate adjustment parameter determination module 1060 is used to obtain the second intermediate adjustment parameter in the following manner:

[0131] F i =(E i -f)×C1×C2;

[0132] Among them, F i E is the second intermediate adjustment parameter corresponding to eyeball data i. i The matrix corresponding to eyeball data i, f is the matrix corresponding to the standard eyeball data, C1 is the first preset adjustment parameter, and C2 is the second preset adjustment parameter.

[0133] In one embodiment, the adjustment parameters include scaling parameters and displacement parameters, and the displacement parameters include moving distance and moving direction;

[0134] The target screen material determination module 1040 is specifically used for:

[0135] The screen material is adjusted in size using the scaling parameters and moved by the displacement parameters in the moving direction by the moving distance to obtain the target screen material.

[0136] In one embodiment, the target screen material determination module 1040 is further configured to:

[0137] If the offset parameter is not within the specified range, the screen material will not be adjusted.

[0138] In one embodiment, the screen material includes screenshots or screen recordings.

[0139] After introducing a display method and apparatus for a VR device according to an exemplary embodiment of the present invention, the electronic device according to another exemplary embodiment of the present invention will be introduced next.

[0140] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as "circuit", "module", or "system".

[0141] In some possible implementations, the electronic device according to the present invention may include at least one processor and at least one computer storage medium. The computer storage medium stores program code that, when executed by the processor, causes the processor to perform the steps in the display methods of the VR device according to various exemplary embodiments of the present invention described above. For example, the processor may perform actions such as... Figure 4 Steps 401-405 are shown in the diagram.

[0142] The following reference Figure 11 To describe an electronic device 1100 according to this embodiment of the present invention. Figure 11 The electronic device 1100 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0143] like Figure 11 As shown, the electronic device 1100 is manifested in the form of a general electronic device. The components of the electronic device 1100 may include, but are not limited to: at least one processor 1101, at least one computer storage medium 1102, and a bus 1103 connecting different system components (including the computer storage medium 1102 and the processor 1101).

[0144] Bus 1103 represents one or more of several bus structures, including a computer storage media bus or computer storage media controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0145] Computer storage medium 1102 may include readable media in the form of volatile computer storage media, such as random access computer storage medium (RAM) 1121 and / or cache storage medium 1122, and may further include read-only computer storage medium (ROM) 1123.

[0146] The computer storage medium 1102 may also include a program / utility 1125 having a set (at least one) of program modules 1124, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0147] Electronic device 1100 can also communicate with one or more external devices 1104 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with electronic device 1100, and / or with any device that enables electronic device 1100 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1105. Furthermore, electronic device 1100 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1106. As shown, network adapter 1106 communicates with other modules used in electronic device 1100 via bus 1103. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1100, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0148] In some possible implementations, various aspects of the VR device display method provided by the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in the VR device display method according to various exemplary embodiments of the present invention described above.

[0149] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A display method for a VR device, characterized in that, The method includes: When the VR device activates the screen material acquisition function, it acquires the user's eye data using a preset eye-tracking algorithm at specified intervals. The eye data includes the position coordinates of the pupil center point and the left and right eye identifiers. The screen material acquisition function includes a screenshot function or a screen recording function. The eye data also includes interpupillary distance. Once screen footage is acquired, for any given screen footage, an offset parameter is obtained based on the coordinates of the pupil center point in the eye data corresponding to the screen footage and the coordinates of the reference point in the preset standard eye data of the VR device. The offset parameter characterizes whether an offset occurs between the screen footage and the reference point. The screen footage includes left-eye and / or right-eye screen footage, and the left and right eye identifiers in the standard eye data are the same as those in the eye data. The reference point is the midpoint of the line connecting the center points of the left and right eyes of the VR device. If the offset parameter of the screen material is determined to be within a specified range, then for any eyeball data in the screen material, a first intermediate adjustment parameter is obtained based on the matrix corresponding to the eyeball data and the matrix corresponding to the standard eyeball data. The matrix corresponding to the eyeball data is composed of the interpupillary distance, the position coordinates of the pupil center point, and the left and right eye identifiers. Then, based on the first intermediate adjustment parameter and a preset adjustment parameter, a second intermediate adjustment parameter is obtained. If the number of eyeball data in the screen material is one, then the second intermediate adjustment parameter is determined as the adjustment parameter of the screen material. If the number of eyeball data in the screen material is multiple, the second intermediate adjustment parameters corresponding to each eyeball data are weighted and summed to obtain the adjustment parameter of the screen material. The position and size of the screen material are adjusted using the aforementioned adjustment parameters to obtain the target screen material; The target screen material is rendered using a preset rendering method, and the rendered screen material is then displayed.

2. The method according to claim 1, characterized in that, The offset parameters of the screen material are obtained based on the position coordinates of the pupil center point in the eye data corresponding to the screen material and the position coordinates of the reference point in the preset standard eye data in the VR device, including: For any eyeball data in the screen material, the distance between the pupil center point and the reference point is obtained based on the position coordinates of the pupil center point and the position coordinates of the reference point in the eyeball data; If the number of eyeball data in the screen material is one, then the distance is used to determine the offset parameter of the screen material; If there are multiple eyeball data points in the screen material, then the multiple distances corresponding to each eyeball data point are weighted and summed to obtain the offset parameters of the screen material.

3. The method according to claim 1, characterized in that, The first intermediate adjustment parameter is obtained based on the matrix corresponding to the eyeball data and the matrix corresponding to the standard eyeball data, including: Subtract the matrix corresponding to the eyeball data from the matrix corresponding to the standard eyeball data to obtain the first intermediate adjustment parameter; The step of obtaining the second intermediate adjustment parameter based on the first intermediate adjustment parameter and the preset adjustment parameter includes: The second intermediate adjustment parameter is obtained by multiplying the first intermediate adjustment parameter by the preset adjustment parameter.

4. The method according to claim 1 or 3, characterized in that, The preset adjustment parameters include a first preset adjustment parameter and a second preset adjustment parameter; The second intermediate adjustment parameter is obtained in the following manner: ； in, This is the second intermediate adjustment parameter corresponding to eyeball data i. Let i be the matrix corresponding to the eyeball data. This is the matrix corresponding to the standard eyeball data. The first preset adjustment parameter, This is the second preset adjustment parameter.

5. The method according to claim 1, characterized in that, The adjustment parameters include scaling parameters and displacement parameters, and the displacement parameters include moving distance and moving direction; The step of adjusting the position and size of the screen material using the adjustment parameters to obtain the target screen material includes: The screen material is adjusted in size using the scaling parameters and moved by the displacement parameters in the moving direction by the moving distance to obtain the target screen material.

6. The method according to claim 1, characterized in that, The method further includes: If the offset parameter is not within the specified range, the screen material will not be adjusted.

7. The method according to any one of claims 1 to 3, 5 and 6, characterized in that, The screen materials include screenshots or screen recordings.

8. An electronic device, characterized in that, It includes a processor and a memory, which are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: When the VR device activates the screen material acquisition function, it acquires the user's eye data using a preset eye-tracking algorithm at specified intervals. The eye data includes the position coordinates of the pupil center point and the left and right eye identifiers. The screen material acquisition function includes a screenshot function or a screen recording function. The eye data also includes interpupillary distance. Once screen footage is acquired, for any given screen footage, an offset parameter is obtained based on the coordinates of the pupil center point in the eye data corresponding to the screen footage and the coordinates of the reference point in the preset standard eye data of the VR device. The offset parameter characterizes whether an offset occurs between the screen footage and the reference point. The screen footage includes left-eye and / or right-eye screen footage, and the left and right eye identifiers in the standard eye data are the same as those in the eye data. The reference point is the midpoint of the line connecting the center points of the left and right eyes of the VR device. If the offset parameter of the screen material is determined to be within a specified range, then for any eyeball data in the screen material, a first intermediate adjustment parameter is obtained based on the matrix corresponding to the eyeball data and the matrix corresponding to the standard eyeball data. The matrix corresponding to the eyeball data is composed of the interpupillary distance, the position coordinates of the pupil center point, and the left and right eye identifiers. Then, based on the first intermediate adjustment parameter and a preset adjustment parameter, a second intermediate adjustment parameter is obtained. If the number of eyeball data in the screen material is one, then the second intermediate adjustment parameter is determined as the adjustment parameter of the screen material. If the number of eyeball data in the screen material is multiple, the second intermediate adjustment parameters corresponding to each eyeball data are weighted and summed to obtain the adjustment parameter of the screen material. The position and size of the screen material are adjusted using the aforementioned adjustment parameters to obtain the target screen material; The target screen material is rendered using a preset rendering method, and the rendered screen material is then displayed.

9. The electronic device according to claim 8, characterized in that, The processor executes the position coordinates of the pupil center point in the eye data corresponding to the screen material and the position coordinates of the reference point in the preset standard eye data in the VR device to obtain the offset parameters of the screen material, which are specifically configured as follows: For any eyeball data in the screen material, the distance between the pupil center point and the reference point is obtained based on the position coordinates of the pupil center point and the position coordinates of the reference point in the eyeball data; If the number of eyeball data in the screen material is one, then the distance is used to determine the offset parameter of the screen material; If there are multiple eyeball data points in the screen material, then the multiple distances corresponding to each eyeball data point are weighted and summed to obtain the offset parameters of the screen material.

Images