VR display method of 2D application and VR device

Abstract

The invention relates to the technical field of VR screen projection, and provides a VR display method of a 2D application and VR equipment, which are used for realizing immersive viewing of the 2D application. According to the method, the left eye image and the right eye image are respectively created according to the width and the height of the eye buffer area, and data sharing is performed on the image buffer data of the to-be-displayed 2D application, so that the left eye image and the right eye image are rendered by a synthesizer in a cross-process mode, the data copying process is reduced, the rendering display efficiency is improved, and the display effect is improved. Meanwhile, the left eye image and the right eye image are ensured to be complete 2D application images; moreover, the display positions of the left-eye image and the right-eye image relative to the viewer are respectively calculated through the six-degree-of-freedom head pose and the eye interpupillary distance, so that the left-eye screen and the right-eye screen are rendered and displayed by using a single buffer area according to the corresponding display positions, and compared with double buffer areas, the rendering display efficiency is further improved, and the rendering display time delay is reduced.

Description

Background Art

[0002] 2D applications are usually designed for traditional flat-panel displays, where the user's two eyes see the same content. However, a virtual reality (VR) scene is a relatively closed three-dimensional space, where the user's left eye can only see the image on the left eye screen, and the right eye can only see the image on the right eye screen. Therefore, if a 2D application is directly displayed on the left eye screen and the right eye screen of a VR device, the user's left eye and right eye will see different content. Figure 1 As shown in the figure, the dotted part of the 2D application is displayed on the left eye screen, and the solid part is displayed on the right eye screen. Since the two screens display different images, the user cannot use the VR device to watch the 2D application normally. Summary of the invention

[0003] The embodiments of the present application provide a VR display method and a VR device for a 2D application, which are used to reduce the rendering display delay of the 2D application while ensuring that the left and right eye screens display the complete 2D application screen.

[0004] On the one hand, an embodiment of the present application provides a VR display method for a 2D application, which is applied to a VR device, including:

[0005] Create an interface object according to the width and height of the screen on which the 2D application is to be displayed, wherein the interface object is used to indicate the width and height of the display window;

[0006] Creating a left-eye image and a right-eye image respectively according to the width and height of the eye buffer, sharing the image buffer data of the 2D application to be displayed, and calling a synthesizer to render the left-eye image and the right-eye image in a cross-process manner;

[0007] Calculating a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer according to the 6-DOF head posture and the pupil distance;

[0008] Using a single buffer, the left eye image is drawn and displayed on the left eye screen of the display window according to the first display position, and the right eye image is drawn and displayed on the right eye screen of the display window according to the second display position.

[0009] On the other hand, an embodiment of the present application provides a VR device, including a processor, a memory, and a display, wherein the display, the memory, and the processor are connected via a bus:

[0010] The memory stores a computer program, and the processor performs the following operations according to the computer program:

[0011] Create an interface object according to the width and height of the screen on which the 2D application is to be displayed, wherein the interface object is used to indicate the width and height of the display window;

[0012] Creating a left-eye image and a right-eye image respectively according to the width and height of the eye buffer, sharing the image buffer data of the 2D application to be displayed, and calling a synthesizer to render the left-eye image and the right-eye image in a cross-process manner;

[0013] Calculating a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer according to the 6-DOF head posture and the pupil distance;

[0014] Using a single buffer, the left eye image is drawn on the left eye screen of the display window according to the first display position and displayed through the display, and the right eye image is drawn on the right eye screen of the display window according to the second display position and displayed through the display.

[0015] Optionally, the processor shares data of the image buffer data of the 2D application to be displayed, and calls the synthesizer to render the left-eye image and the right-eye image in a cross-process manner, and the specific operations are:

[0016] Calling the synthesizer to convert the image buffer data of the 2D application to be displayed into a data type supported by a hardware buffer object; wherein the hardware buffer object is an abstract image buffer object that allows the synthesizer and the 2D application process to directly share and access data;

[0017] Create an image object and assign the converted data to the image object;

[0018] Creating a two-dimensional texture object, and binding the image object to the two-dimensional texture object;

[0019] Through a graphics rendering API, the data of the image object is directly used as the texture of a two-dimensional texture object to obtain the left-eye image and the right-eye image.

[0020] Optionally, the processor calculates a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer according to the 6-DOF head posture and the pupil distance, and the specific operation is:

[0021] Generate an observation matrix based on a coordinate conversion matrix between a preset VR coordinate system and an OpenGL coordinate system and the 6-DOF head posture;

[0022] Generate a first conversion matrix and a second conversion matrix according to the eye pupil distance, wherein the first conversion matrix is ​​used to convert the left eye perspective into the central perspective, and the second conversion matrix is ​​used to convert the right eye perspective into the central perspective;

[0023] Generate a posture transformation matrix from the central view to the IMU coordinate system according to the offset from the IMU to the center of the device;

[0024] A first display position of the left-eye image relative to a viewer is calculated based on the first transformation matrix, the posture transformation matrix and the observation matrix, and a second display position of the right-eye image relative to the viewer is calculated based on the second transformation matrix, the posture transformation matrix and the observation matrix.

[0025] Optionally, when the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is horizontal screen display, the processor draws the left eye image on the left eye screen of the display window according to the first display position and displays it through the display, and draws the right eye image on the right eye screen of the display window according to the second display position and displays it through the display, and the specific operations are:

[0026] Determining first texture coordinates corresponding to each vertex on the application image displayed in the horizontal screen according to the width and height of the eye buffer;

[0027] According to the obtained first texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye horizontal screen image, and, according to the obtained first texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye horizontal screen image;

[0028] The left-eye horizontal screen image is drawn on the left-eye screen of the display window according to the first display position and displayed through the display, and the right-eye horizontal screen image is drawn on the right-eye screen of the display window according to the second display position and displayed through the display.

[0029] Optionally, the processor calculates first texture coordinates corresponding to each vertex on the application image displayed in the horizontal screen according to the width and height of the eye buffer, and the specific operation is:

[0030] Calculating first vertex offsets of four vertices on the horizontally displayed application image according to the width and height of the eye buffer;

[0031] Acquire the image size of the application image, and determine the positions of four vertices on the application image displayed in the horizontal screen in combination with the first vertex screen offset; wherein the image width in the image size is greater than the image height;

[0032] According to a preset texture range, first texture coordinates corresponding to vertices at different positions on the application image displayed in the horizontal screen are determined.

[0033] Optionally, when the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is vertical screen display, the processor draws the left eye image on the left eye screen of the display window according to the first display position and displays it through the display, and draws the right eye image on the right eye screen of the display window according to the second display position and displays it through the display, and the specific operations are:

[0034] Determining second texture coordinates corresponding to each vertex on the application image displayed in the vertical screen according to the width and height of the eye buffer;

[0035] According to the obtained second texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye vertical screen image, and, according to the obtained second texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye vertical screen image;

[0036] The left-eye vertical screen image is drawn on the left-eye screen of the display window according to the first display position and displayed through the display, and the right-eye vertical screen image is drawn on the right-eye screen of the display window according to the second display position and displayed through the display.

[0037] Optionally, the processor determines, according to the width and height of the eye buffer, the second texture coordinates corresponding to each vertex on the application image displayed in the vertical screen, and the specific operation is:

[0038] Calculating second vertex offsets of four vertices on the application image displayed in the vertical screen according to the width and height of the eye buffer;

[0039] Acquire the image size of the application image, and determine the positions of four vertices on the vertically displayed application image in combination with the second vertex screen offset; wherein the image height in the image size is greater than the image width;

[0040] According to a preset texture range, second texture coordinates corresponding to vertices at different positions on the application image displayed in the vertical screen are determined.

[0041] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer device to execute any VR display method for 2D applications provided in the embodiment of the present application.

[0042] The beneficial effects of a VR display method and VR device for 2D applications provided in the embodiments of the present application are as follows:

[0043] According to the width and height of the screen of the 2D application to be displayed, an interface object for indicating the width and height of the display window is created, and, according to the width and height of the eye buffer, a left-eye image and a right-eye image are created respectively, and by sharing data of the image buffer data of the 2D application to be displayed, the synthesizer is enabled to render the left-eye image and the right-eye image in a cross-process manner, thereby reducing the data copy process, improving the rendering and display efficiency, and ensuring that both the left-eye image and the right-eye image are complete 2D application images; and, through the 6-DOF head posture and the eye pupil distance, a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer are calculated respectively, so that the left-eye image is drawn on the left-eye screen of the display window according to the first display position and displayed, and the right-eye image is drawn on the right-eye screen of the display window according to the second display position and displayed, so as to ensure that the left and right eyes see images with consistent content and realize immersive viewing of 2D applications. Among them, a single buffer is used in the rendering and display process of the left eye screen and the right eye screen. Compared with the double buffer, it further improves the rendering and display efficiency, reduces the rendering and display delay, and thus reduces the dizziness of 2D applications displayed on VR devices.

[0044] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 This is a diagram showing abnormal display effects of a 2D application on a VR device provided in an embodiment of the present application;

[0047] Figure 2A flow chart of a VR display method for a 2D application provided in an embodiment of the present application;

[0048] Figure 3 A flowchart of rendering left-eye images and right-eye images provided in an embodiment of the present application;

[0049] Figure 4 A flow chart for calculating the display position of left and right eye images relative to the viewer's viewing angle provided in an embodiment of the present application;

[0050] Figure 5 A display effect diagram of a 2D application provided in an embodiment of the present application on a VR device;

[0051] Figure 6 A schematic diagram of a horizontal and vertical screen provided in an embodiment of the present application;

[0052] Figure 7 A flowchart of rendering and displaying a 2D application in horizontal screen display provided in an embodiment of the present application;

[0053] Figure 8 A horizontal screen display effect diagram of a 2D application provided in an embodiment of the present application on a VR device;

[0054] Fig. 9 A flowchart of rendering and displaying a 2D application in a vertical screen display provided in an embodiment of the present application;

[0055] Fig.10 A vertical screen display effect diagram of a 2D application provided in an embodiment of the present application on a VR device;

[0056] Fig.11 A structural diagram of a VR device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0058] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0059] It should be understood that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0060] In addition, the terms "including" and "having" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0061] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0062] The terms involved in the embodiments of the present application are explained below.

[0063] Interface object: An object that can display and draw interfaces, used for graphics rendering and user interface presentation.

[0064] Eye buffer: A buffer used to render image data that the user directly sees visually.

[0065] Compositor: Also known as a display management service, it is responsible for merging image content from multiple sources into a single output.

[0066] Hardware Buffer Object: Used to provide direct access interface to the underlying graphics hardware.

[0067] Image objects: Objects used to wrap different types of graphics buffers for use in graphics rendering APIs.

[0068] Two-dimensional texture object: used to represent and sample image data in graphics rendering.

[0069] Graphics rendering API for 2D and 3D graphics processing with high performance.

[0070] It should be noted that VR devices use different systems (such as Android, Hongmeng, IOS, etc.), and the expressions of the above terms may be different, but the meanings are basically the same.

[0071] The following is an overview of the design concept of the embodiments of the present application in conjunction with application scenarios.

[0072] When using a VR device to watch a 2D application for an immersive experience, in order to ensure the normal display of the 2D application, the complete picture of the 2D application needs to be rendered on the left eye screen and the right eye screen respectively.

[0073] At present, when related technologies display 2D applications in VR, most of them display 2D applications directly in Unity launcher. Unity application creates a virtual screen in virtual space and draws the texture of 2D application on the virtual screen. However, due to the limitation of space size, the image size of 2D application needs to change with the size of virtual screen, which cannot be displayed flexibly. When the image size of 2D application is inconsistent with the size of virtual screen, the screen of 2D application needs to be cropped or scaled, which affects the visibility and usability of 2D application.

[0074] In view of this, an embodiment of the present application provides a VR display method and VR device for a 2D application, which obtains image buffer data of the 2D application through a synthesizer to achieve data sharing with the 2D application process, thereby drawing the obtained image buffer data to the left eye image and the right eye image newly created according to the width and height of the eye buffer, reducing the data copying process, improving the rendering display efficiency, and ensuring that the left eye image and the right eye image are complete 2D application images; on the other hand, the first display position of the left eye image relative to the viewer and the second display position of the right eye image relative to the viewer are calculated respectively through the 6-degree-of-freedom head posture and the eye pupil distance, and a single buffer is used to draw the left eye image on the left eye screen of the display window according to the first display position and display it, and the right eye image is drawn on the right eye screen of the display window according to the second display position and display it, compared with a double buffer used for display and rendering, the rendering display efficiency is further improved, the rendering display delay is reduced, and the dizziness of the 2D application displayed on the VR device is reduced.

[0075] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0076] See also Figure 2 , is a flow chart of a VR display method for a 2D application provided in an embodiment of the present application. The process is executed by a VR device and mainly includes the following steps:

[0077] S201: creating an interface object according to the width and height of the screen on which the 2D application is to be displayed, where the interface object is used to indicate the width and height of the display window.

[0078] Taking the VR device using Android system as an example, the interface object can be a Surface object, and its creation process is as follows: create a Surface object through the SurfaceControl class, and set the width of the Surface object according to the width of the screen to display the 2D application, and set the height of the Surface object according to the height of the screen to display the 2D application. The Surface object can be used as a display window to display the content of the 2D application.

[0079] It should be noted that the width and height of the interface object may be the same as the width and height of the screen on which the 2D application is to be displayed, or may be greater than the width and height of the screen on which the 2D application is to be displayed.

[0080] S202: creating a left-eye image and a right-eye image respectively according to the width and height of the eye buffer, sharing the image buffer data of the 2D application to be displayed, and calling the synthesizer to render the left-eye image and the right-eye image in a cross-process manner.

[0081] Still taking the Android system as an example, the system renders the standard view of each eye into the eye buffer. Therefore, the left eye image and the right eye image can be created according to the width and height of the eye buffer. The width of the left eye image and the right eye image is consistent with the width of the EyeBuffer, and the height of the left eye image and the right eye image is consistent with the height of the EyeBuffer.

[0082] After creating the left-eye image and the right-eye image, you can render the left-eye image and the right-eye image. The rendering process is as follows: Figure 3 As shown, it mainly includes the following steps:

[0083] S2021: Call the synthesizer to convert the image buffer data of the 2D application to be displayed into a data type supported by the hardware buffer object.

[0084] Taking the Android system as an example, the synthesizer is the SurfaceFlinger process, the image buffer data is the buffer data, and the hardware buffer object is the AHardwareBuffer object.

[0085] Among them, the AHardwareBuffer object is a low-level primitive provided by the Android system for sharing image data across processes. It is an abstract image buffer object that can realize direct sharing and access to data between the SurfaceFlinger process and the 2D application process without duplicating or copying the data, thereby effectively improving the rendering and display efficiency; and the AHardwareBuffer object has GPU memory independent of the CPU, and can directly process data on the GPU and other hardware accelerators without the mediation of the CPU, with high rendering performance and efficiency.

[0086] It should be noted that AHardwareBuffer is usually used by upper-level graphics libraries, rendering engines, or other graphics-related frameworks. Ordinary application developers may need to use graphics libraries or other encapsulation layers for advanced graphics programming before directly using AHardwareBuffer.

[0087] S2022: Create an image object and assign the type-converted data to the image object.

[0088] Taking the Android system as an example, the image object may be an EGLImage object, which is created by: creating an EGLImage object through the eglCreateImageKHR function.

[0089] Among them, eglCreateImageKHR is a function in the Embedded Graphics Library (EGL). EGLImage objects can be shared between different processes to achieve more flexible graphics resource management and collaboration. In addition, by encapsulating existing image resources into EGLImage objects, these graphics resources can be reused in different rendering processes to improve performance and efficiency.

[0090] S2023: Create a two-dimensional texture object and bind the image object to the two-dimensional texture object.

[0091] Taking the Android system as an example, the two-dimensional texture object can be a Tuxture2D object. After the object is created, the EGLImage object is bound to the Tuxture2D object through glEGLImageTargetTexture2DOES.

[0092] S2024: Through a graphics rendering API, directly use the data of the image object as the texture of a two-dimensional texture object to obtain a left-eye image and a right-eye image.

[0093] Taking the Android system as an example, the graphics rendering API may be OpenGL ES. By binding the EGLImage object to the Tuxture2D object, the data in the EGLImage object can be directly used as texture data, reducing data copying and conversion, so that it can be used in OpenGL ES rendering, improving rendering performance and efficiency.

[0094] S203: Calculate a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer according to the 6-DOF head posture and the pupil distance.

[0095] In one example, the calculation process of the display position is as follows Figure 4 As shown, it mainly includes the following steps:

[0096] S2031: Generate an observation matrix based on the coordinate conversion matrix between the preset VR coordinate system and the OpenGL coordinate system and the 6-DOF head posture.

[0097] Assume that the coordinate transformation matrix from VR coordinate system to OpenGL coordinate system is recorded as VR_GL, and the coordinate transformation matrix from OpenGL coordinate system to VR coordinate system is recorded as GL_VR, where: The 6-DOF head pose includes the rotation angle of the head relative to the three axes and the movement position of the three axes, which can be represented by a 4*4 pose matrix head.

[0098] Furthermore, by multiplying the coordinate transformation matrices VR_GL and GL_VR and the inverse matrix of the 6-DOF head pose matrix head, we can get the observation matrix view, which is expressed as follows:

[0099] view=GL_VR*inverse(head)*VR_GL Formula 1

[0100] Among them, inverse represents the inverse transformation of the matrix, and the observation matrix view defines the position and direction of the camera (used to represent the human eye) and is used to convert the scene to the viewer's perspective.

[0101] In the embodiment of the present application, an inverse transformation is performed through a 6-DOF head posture matrix to correct the rotation of the head, so that the observation matrix takes the rotation of the head into account, which helps to improve the accuracy of the display position calculation.

[0102] It should be noted that the embodiment of the present application does not impose any restrictive requirements on the method of acquiring the 6-DOF head posture. For example, it can be calculated through a SLAM algorithm or collected through sensors of a VR device.

[0103] S2032: Generate a first conversion matrix and a second conversion matrix respectively according to the pupil distance of the eyes.

[0104] The first conversion matrix is ​​used to convert the left eye perspective into the central perspective, and the second conversion matrix is ​​used to convert the right eye perspective into the central perspective.

[0105] Assume that the pupil distance is denoted as ipd, the first conversion matrix is ​​denoted as l_cen, and the second conversion matrix is ​​denoted as r_cen, then

[0106] S2033: Generate a posture transformation matrix from the central view to the IMU coordinate system according to the offset from the IMU to the center of the device.

[0107] Among them, the attitude transformation matrix represents the transformation from the central view to the IMU coordinate system, which is used to correct the rotation of the central view to take into account the attitude of the IMU.

[0108] Assuming that the three-axis offset from the IMU to the center of the device is x, y, and z, the attitude transformation matrix from the central view to the IMU coordinate system is

[0109] S2034: Calculate a first display position of the left eye image relative to the viewer based on the first transformation matrix, the posture transformation matrix and the observation matrix, and calculate a second display position of the right eye image relative to the viewer based on the second transformation matrix, the posture transformation matrix and the observation matrix.

[0110] The first display position is used to convert the scene from the left eye's perspective to the final observation perspective, and the second display position is used to convert the scene from the right eye's perspective to the final viewer's perspective.

[0111] Specifically, the formula for the first display position is expressed as:

[0112] view_l=l_cen*cen_imu*view Formula 2

[0113] The formula for the second display position is:

[0114] view_r=r_cen*cen_imu*view Formula 3

[0115] In an embodiment of the present application, a first display position of a left-eye image relative to a viewer and a second display position of a right-eye image relative to a viewer are calculated by using a 6-degree-of-freedom head posture and an eye pupil distance, thereby ensuring that the viewer can view the left-eye image and the right-eye image containing a complete 2D application screen through a VR device, thereby achieving immersive viewing of 2D applications.

[0116] S204: using a single buffer, drawing and displaying a left-eye image on a left-eye screen of a display window according to a first display position, and drawing and displaying a right-eye image on a right-eye screen of a display window according to a second display position.

[0117] Taking the Android system as an example, usually, the Android system uses double buffering for rendering and display, wherein one buffer is used for rendering and the other buffer is used for display, and the rendering and display efficiency is low.

[0118] In order to solve this problem, an embodiment of the present application uses a single buffer, and through the glViewport function, draws the left eye image on the left eye screen of the display window according to the first display position and displays it, and through the glViewport function, draws the right eye image on the right eye screen of the display window according to the second display position and displays it, that is, the buffer is used for both rendering and display, thereby improving the rendering and display efficiency, reducing the rendering and display delay, and further reducing the dizziness of 2D applications displayed on VR devices.

[0119] like Figure 5 As shown, the display effect of the 2D application provided in the embodiment of the present application on the VR device can display the complete 2D application screen on both the left eye screen and the right eye screen, thereby ensuring immersive viewing of the 2D application.

[0120] In an embodiment of the present application, a left-eye image and a right-eye image are created respectively according to the width and height of the eye buffer, and the image buffer data to be displayed for the 2D application is converted into a type, thereby realizing data sharing between the synthesizer and the 2D application process. In this way, the synthesizer can draw the left-eye image and the right-eye image according to the image buffer data in a cross-process manner, thereby ensuring that the left-eye image and the right-eye image are complete 2D application screens. Furthermore, after determining the display position of the left-eye image and the right-eye image relative to the viewer according to the 6-DOF head posture and the pupil distance of the eyes, a single buffer is used to render and display the left-eye image and the right-eye image on the left-eye screen and the right-eye screen of the display window, so that the viewer can see the same picture with the content of the left-eye screen and the right-eye screen, realizing immersive viewing. Compared with the double buffer, the rendering and display delay of the single buffer is lower, which effectively improves the viewing experience of 2D applications.

[0121] In some examples, the width and height of the eye buffer in the VR device are not necessarily equal, and it is necessary to display the screen in horizontal and vertical directions, such as Figure 6 As shown, at this time, when creating the interface object, the width of the display window can be set to be larger than the width of the screen to display the 2D application, and the height of the display window can be set to be larger than the height of the screen to display the 2D application, such as: the width of the display window is twice the width of the screen to display the 2D application, and the height of the display window is twice the height of the screen to display the 2D application, thereby ensuring that the content of the 2D application can be fully displayed in both horizontal and vertical states without distortion.

[0122] When the display type of the 2D application to be displayed is horizontal screen display, the rendering and display process of the left eye screen and the right eye screen is as follows: Figure 7 As shown, it mainly includes the following steps:

[0123] S2041: Determine, according to the width and height of the eye buffer, first texture coordinates corresponding to each vertex on the application image displayed in the horizontal screen.

[0124] Assume that the width and height of the eye buffer are eye_W and eye_H respectively, the size is eye_W=width*2, eye_H=height, and width and height are preset values.

[0125] In fact, the vertices on the horizontally displayed application image include the upper vertex, the lower vertex, the left vertex and the right vertex. Assume that the left vertex is defined as left = 0, the right vertex is positioned as right = r_width / eye_W, the upper vertex is defined as top = 1, and the lower vertex is defined as bottom = 1-r_height / eye_H. Among them, r_width and r_height are the real image width and the real image height respectively.

[0126] Specifically, r_width=width*2, r_height=height.

[0127] When displayed in landscape mode, the calculation process of the first texture coordinates of the four vertices is as follows:

[0128] S2041_1: Calculate the first vertex offset of four vertices on the application image displayed in the horizontal screen according to the width and height of the eye buffer.

[0129] First, calculate the initial positions of the four vertices, top, bottom, left and right. The formula is as follows:

[0130] left = left * 2 - 1;

[0131] right = right * 2 - 1;

[0132] top = (top*2-1) / 2;

[0133] bottom=(bottom*2-1) / 2.

[0134] Then, the first vertex offset is calculated based on the initial position, and the formula is expressed as:

[0135] x_w=(left+right) / 2;

[0136] y_w=(top+bottom) / 2.

[0137] S2041_2: Obtain the image size of the application image, and determine the positions of four vertices of the application image displayed in horizontal screen in combination with the first vertex offset.

[0138] First, based on the width and height of the eye buffer, define the adjustment variables corresponding to the four vertices. The formulas for each variable are as follows:

[0139] oriLeft = L / eye_W;

[0140] oriRight=(L+eye_W) / eye_W;

[0141] oriTop = (eye_H-T) / eye_H;

[0142] oriBottom=(eye_H-T-eye_H) / eye_H.

[0143] Among them, the value of L is 0 and the value of T is 0.

[0144] Then, the positions of the four vertices are adjusted based on the image size of the application image and four adjustment variables. The adjustment formula is expressed as follows:

[0145] left=(oriRight-oriLeft)*r_left / eye_W+oriLeft;

[0146] right=(oriRight-oriLeft)*r_W / eye_W+oriLeft;

[0147] top=(oriBottom-oriTop)*r_top / eye_H+oriTop;

[0148] bottom=(oriBottom-oriTop)*r_H / eye_H+oriTop;

[0149] left = left * 2 - 1;

[0150] right = right * 2 - 1;

[0151] top = (top*2-1) / 2;

[0152] bottom=(bottom*2-1) / 2.

[0153] Among them, r_leftWie is the actual position of the left vertex in the image, r_W is the width of the image, r_top is the actual position of the top vertex in the image, and r_H is the width of the image.

[0154] Finally, after adjusting the positions of the four vertices, combined with the first vertex offset, the final positions of the four vertices on the application image are calculated. The formula is as follows:

[0155] left-=x_w;

[0156] right-=x_w;

[0157] top-=y_w;

[0158] bottom-=y_w.

[0159] S2041_3: Determine first texture coordinates corresponding to vertices at different positions on the application image displayed in the horizontal screen according to a preset texture range.

[0160] Typically, texture coordinates are represented by (u, v), where u and v represent the position of the texture image on the U axis and V axis respectively. The range of texture coordinates is usually between 0 and 1, where (0, 0) represents the lower left corner of the texture image and (1, 1) represents the upper right corner of the texture image.

[0161] Assuming that uvLeft=0, uvRight=1, uvTop=0, uvBottom=1, the first texture coordinates of the four vertices can be expressed as (uvLeft, uvTop), (uvLeft, uvBottom), (uvRight, uvTop), and (uvRight, uvBottom), respectively.

[0162] S2042: According to the obtained first texture coordinates, texture sampling is performed from the rendered left eye image to obtain a left eye horizontal screen image; and, according to the obtained first texture coordinates, texture sampling is performed from the rendered right eye image to obtain a right eye horizontal screen image.

[0163] By specifying the texture coordinates of the vertices, OpenGL can map the pixels of the texture image to these vertices. The texture image includes the rendered left eye image and the rendered right eye image.

[0164] S2043: drawing and displaying the left-eye horizontal screen image on the left-eye screen of the display window according to the first display position, and drawing and displaying the right-eye horizontal screen image on the right-eye screen of the display window according to the second display position.

[0165] like Figure 8 As shown, the display effect of a 2D application displayed in horizontal screen on a VR device, wherein the width of the left-eye horizontal screen image is equal to the width of the left-eye screen, while the height is less than the height of the left-eye screen. Similarly, the width of the right-eye horizontal screen image is equal to the width of the left-eye screen, while the height is less than the height of the right-eye screen.

[0166] When the display type of the 2D application to be displayed is vertical screen display, the rendering and display process of the left eye screen and the right eye screen is as follows: Fig. 9 As shown, it mainly includes the following steps:

[0167] S2044: Determine, according to the width and height of the eye buffer, second texture coordinates corresponding to each vertex on the application image displayed in portrait mode.

[0168] Assume that the width and height of the eye buffer are eye_W and eye_H respectively, the size is eye_W=width*2, eye_H=height, and width and height are preset values.

[0169] In fact, the vertices on the application image displayed in the vertical screen include the upper vertex, the lower vertex, the left vertex and the right vertex. Assume that the left vertex is defined as left = 0, the right vertex is positioned as right = r_width / eye_W, the upper vertex is defined as top = 1, and the lower vertex is defined as bottom = 1-r_height / eye_H. Among them, r_width and r_height are the real image width and real image height respectively.

[0170] Specifically, r_width=height, r_height=width*2.

[0171] When the screen is displayed in portrait mode, the calculation process of the second texture coordinates of the four vertices is as follows:

[0172] S2044_1: Calculate second vertex offsets of four vertices on the application image displayed in portrait mode according to the width and height of the eye buffer.

[0173] First, calculate the initial positions of the four vertices, top, bottom, left and right. The formula is as follows:

[0174] left = left * 2 - 1;

[0175] right = right * 2 - 1;

[0176] top = (top*2-1) / 2;

[0177] bottom=(bottom*2-1) / 2.

[0178] Then, the second vertex offset is calculated based on the initial position, and the formula is expressed as:

[0179] x_h=(left+right) / 2;

[0180] y_h=(top+bottom) / 2.

[0181] S2044_2: Obtain the image size of the application image, and determine the positions of four vertices of the application image displayed in portrait mode in combination with the second vertex offset.

[0182] First, based on the width and height of the eye buffer, define the adjustment variables corresponding to the four vertices. The formulas for each variable are as follows:

[0183] oriLeft = L / eye_W;

[0184] oriRight=(L+eye_W) / eye_W;

[0185] oriTop = (eye_H-T) / eye_H;

[0186] oriBottom=(eye_H-Top-eye_H) / eye_H.

[0187] Among them, the value of L is 0 and the value of T is 0.

[0188] Then, the positions of the four vertices are adjusted based on the image size of the application image and four adjustment variables. The adjustment formula is expressed as follows:

[0189] left=(oriRight-oriLeft)*r_left / eye_W+oriLeft;

[0190] right=(oriRight-oriLeft)*r_width / eye_W+oriLeft;

[0191] top=(oriBottom-oriTop)*r_top / eye_H+oriTop;

[0192] bottom=(oriBottom-oriTop)*r_height / eye_H+oriTop;

[0193] left = left * 2 - 1;

[0194] right = right * 2 - 1;

[0195] top = (top*2-1) / 2;

[0196] bottom=(bottom*2-1) / 2.

[0197] Among them, r_leftWie is the actual position of the left vertex in the image, r_W is the width of the image, r_top is the actual position of the top vertex in the image, and r_H is the width of the image.

[0198] Finally, after adjusting the positions of the four vertices, combined with the second vertex offset, the final positions of the four vertices on the application image are calculated. The formula is as follows:

[0199] left-=x_h;

[0200] right-=x_h;

[0201] top-=y_h;

[0202] bottom-=y—h.

[0203] S2044_3: Determine second texture coordinates corresponding to vertices at different positions on the application image displayed in the vertical screen according to a preset texture range.

[0204] Typically, texture coordinates are represented by (u, v), where u and v represent the position of the texture image on the U axis and V axis respectively. The range of texture coordinates is usually between 0 and 1, where (0, 0) represents the lower left corner of the texture image and (1, 1) represents the upper right corner of the texture image.

[0205] Assuming that uvLeft=0, uvRight=1, uvTop=0, uvBottom=1, the texture coordinates of the four vertices can be expressed as (uvLeft, uvBottom), (uvRight, uvBottom), (uvLeft, uvTop), (uvRight, uvTop) respectively.

[0206] S2045: According to the obtained second texture coordinates, texture sampling is performed from the rendered left eye image to obtain a left eye vertical screen image; and, according to the obtained second texture coordinates, texture sampling is performed from the rendered right eye image to obtain a right eye vertical screen image.

[0207] By specifying the texture coordinates of the vertices, OpenGL can map the pixels of the texture image to these vertices. The texture image includes the rendered left eye image and the rendered right eye image.

[0208] S2046: drawing and displaying the left-eye vertical screen image on the left-eye screen of the display window according to the first display position, and drawing and displaying the right-eye vertical screen image on the right-eye screen of the display window according to the second display position.

[0209] like Fig.10 As shown, the display effect of a 2D application displayed in portrait mode on a VR device, wherein the height of the left-eye vertical screen image is equal to the height of the left-eye screen, while the width is smaller than the width of the left-eye screen. Similarly, the height of the right-eye vertical screen image is equal to the height of the left-eye screen, while the width is smaller than the width of the right-eye screen.

[0210] In the embodiment of the present application, by creating an interface object that is larger than the screen of the 2D application to be displayed, the horizontal screen display and the vertical screen display of the 2D application can be adaptively matched to ensure the display effect.

[0211] In some embodiments, the 2D application to be displayed is independently displayed in a sky box, and the menu bar, handle model, ray and other functions can be customized in the sky box, so as to achieve immersive viewing while being able to interact with the 2D application to be displayed to enhance viewing pleasure.

[0212] Based on the same technical concept, an embodiment of the present application provides a VR device that can implement the steps of any of the above-mentioned VR display methods for 2D applications and achieve the same technical effects.

[0213] See also Fig.11 , the VR device includes a processor 1101, a memory 1102 and a display 1103, and the display 1103, the memory 1102 and the processor 1101 are connected via a bus 1104:

[0214] The memory 1102 stores a computer program, and the processor 1101 performs the following operations according to the computer program:

[0215] Create an interface object according to the width and height of the screen on which the 2D application is to be displayed, where the interface object is used to indicate the width and height of the display window;

[0216] Create left-eye images and right-eye images according to the width and height of the eye buffer, share the image buffer data of the 2D application to be displayed, and call the compositor to render the left-eye images and right-eye images in a cross-process manner;

[0217] Calculate a first display position of a left-eye image relative to a viewer and a second display position of a right-eye image relative to a viewer according to a 6-DOF head posture and an eye pupil distance;

[0218] Using a single buffer, a left eye image is drawn on a left eye screen of a display window according to a first display position and displayed through display 1103, and a right eye image is drawn on a right eye screen of a display window according to a second display position and displayed through display 1103.

[0219] Optionally, the processor 1101 shares the image buffer data of the 2D application to be displayed, and calls the synthesizer to render the left-eye image and the right-eye image in a cross-process manner. The specific operations are:

[0220] Call the compositor to convert the image buffer data of the 2D application to be displayed into the data type supported by the hardware buffer object; the hardware buffer object is an abstract image buffer object that allows the compositor and the 2D application process to directly share and access data;

[0221] Create an image object and assign the converted data to it;

[0222] Create a 2D texture object and bind the image object to the 2D texture object;

[0223] Through the graphics rendering API, the data of the image object is directly used as the texture of the two-dimensional texture object to obtain the left eye image and the right eye image.

[0224] Optionally, the processor 1101 calculates a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer according to the 6-DOF head posture and the pupil distance of the eyes, and the specific operation is:

[0225] Generate an observation matrix based on the coordinate transformation matrix between the preset VR coordinate system and the OpenGL coordinate system combined with the 6-DOF head pose;

[0226] Generate a first conversion matrix and a second conversion matrix according to the pupil distance of the eyes, respectively; wherein the first conversion matrix is ​​used to convert the left eye perspective into the central perspective, and the second conversion matrix is ​​used to convert the right eye perspective into the central perspective;

[0227] Generate the attitude transformation matrix from the central view to the IMU coordinate system based on the offset from the IMU to the center of the device;

[0228] A first display position of a left-eye image relative to a viewer is calculated based on the first transformation matrix, the posture transformation matrix and the observation matrix, and a second display position of a right-eye image relative to a viewer is calculated based on the second transformation matrix, the posture transformation matrix and the observation matrix.

[0229] Optionally, when the width of the display window is greater than the width of the screen on which the 2D application is to be displayed, the height of the display window is greater than the height of the screen on which the 2D application is to be displayed, and the display type of the 2D application to be displayed is horizontal screen display, the processor 1101 draws the left-eye image on the left-eye screen of the display window according to the first display position and displays it through the display 1103, and draws the right-eye image on the right-eye screen of the display window according to the second display position and displays it through the display 1103. The specific operations are:

[0230] Determine, according to the width and height of the eye buffer, first texture coordinates corresponding to each vertex on the application image displayed in the horizontal screen;

[0231] According to the obtained first texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye horizontal screen image, and, according to the obtained first texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye horizontal screen image;

[0232] The left-eye horizontal screen image is drawn on the left-eye screen of the display window according to the first display position and displayed through the display 1103 , and the right-eye horizontal screen image is drawn on the right-eye screen of the display window according to the second display position and displayed through the display 1103 .

[0233] Optionally, the processor 1101 calculates the first texture coordinates corresponding to each vertex on the application image displayed in the horizontal screen according to the width and height of the eye buffer, and the specific operation is:

[0234] Calculate the first vertex offsets of four vertices on the application image displayed in the horizontal screen according to the width and height of the eye buffer;

[0235] The image size of the application image is obtained, and combined with the first vertex screen offset, the positions of the four vertices of the application image displayed in the horizontal screen are obtained;

[0236] According to a preset texture range, first texture coordinates corresponding to vertices at different positions on the application image displayed in the horizontal screen are determined.

[0237] Optionally, when the width of the display window is greater than the width of the screen on which the 2D application is to be displayed, the height of the display window is greater than the height of the screen on which the 2D application is to be displayed, and the display type of the 2D application to be displayed is vertical screen display, the processor 1101 draws the left-eye image on the left-eye screen of the display window according to the first display position and displays it through the display 1103, and draws the right-eye image on the right-eye screen of the display window according to the second display position and displays it through the display 1103. The specific operations are:

[0238] Determine, according to the width and height of the eye buffer, second texture coordinates corresponding to each vertex on the application image displayed in the vertical screen;

[0239] According to the obtained second texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye vertical screen image, and, according to the obtained texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye vertical screen image;

[0240] The left-eye vertical screen image is drawn on the left-eye screen of the display window according to the first display position and displayed through the display 1103 , and the right-eye vertical screen image is drawn on the right-eye screen of the display window according to the second display position and displayed through the display 1103 .

[0241] Optionally, the processor determines the second texture coordinates corresponding to each vertex on the application image displayed in the vertical screen according to the width and height of the eye buffer, and the specific operation is:

[0242] Calculate second vertex offsets of four vertices on the application image displayed in the vertical screen according to the width and height of the eye buffer;

[0243] The image size of the application image is obtained, and combined with the second vertex screen offset, the positions of the four vertices on the application image displayed in the horizontal screen are obtained;

[0244] According to a preset texture range, second texture coordinates corresponding to vertices at different positions on the application image displayed in the vertical screen are determined.

[0245] In an embodiment of the present application, the memory 1102 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, and programs required to run the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc. The memory 1102 may be a volatile memory (volatile memory), such as a random-access memory (RAM); the memory 1102 may also be a non-volatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1102 may be any other medium that can be used to carry or store a desired computer program in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1102 may be a combination of the above memories.

[0246] The processor 1101 may include one or more central processing units (CPU), GPU or digital processing unit, etc. The processor 1101 is used to implement the steps of any of the above-mentioned VR display methods for 2D applications when calling the computer program stored in the memory 1102.

[0247] Bus 1104 is Fig.11 The connections between the other components are described with bold lines, which are only for illustration and are not intended to be limiting. The bus 1104 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.11 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0248] It should be noted that Fig.11 This is just an example, and provides the necessary hardware for the VR device to execute the steps of the VR display method for 2D applications provided in the embodiment of the present application. If not shown, the VR device may also include conventional hardware such as a speaker, a microphone, a communication interface, and a power supply.

[0249] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0250] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0251] An embodiment of the present application also provides a computer-readable storage medium for storing some instructions, which, when executed, can complete the steps of any 2D application VR display method in the aforementioned embodiments.

[0252] An embodiment of the present application also provides a computer program product for storing a computer program, which is used to execute the steps of any one of the VR display methods for 2D applications in the aforementioned embodiments.

[0253] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0254] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0255] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0256] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0257] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A VR display method for 2D applications, characterized in that: Applied to VR equipment, the method comprises: Create an interface object according to the width and height of the screen on which the 2D application is to be displayed, wherein the interface object is used to indicate the width and height of the display window; Creating a left-eye image and a right-eye image respectively according to the width and height of the eye buffer, sharing the image buffer data of the 2D application to be displayed, and calling a synthesizer to render the left-eye image and the right-eye image in a cross-process manner; Calculating a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer according to the 6-DOF head posture and the pupil distance; Using a single buffer, the left eye image is drawn and displayed on the left eye screen of the display window according to the first display position, and the right eye image is drawn and displayed on the right eye screen of the display window according to the second display position.

2. The method according to claim 1, characterized in that The sharing of the image buffer data of the 2D application to be displayed and calling the synthesizer to render the left-eye image and the right-eye image in a cross-process manner include: Calling the synthesizer to convert the image buffer data of the 2D application to be displayed into a data type supported by a hardware buffer object; wherein the hardware buffer object is an abstract image buffer object that allows the synthesizer and the 2D application process to directly share and access data; Create an image object and assign the type-converted data to the image object; Creating a two-dimensional texture object, and binding the image object to the two-dimensional texture object; Through a graphics rendering API, the data of the image object is directly used as the texture of the two-dimensional texture object to obtain the left-eye image and the right-eye image.

3. The method according to claim 1, characterized in that The calculating, according to the 6-DOF head posture and the pupil distance, a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer comprises: Generate an observation matrix based on a coordinate conversion matrix between a preset VR coordinate system and an OpenGL coordinate system and the 6-DOF head posture; Generate a first conversion matrix and a second conversion matrix according to the eye pupil distance, wherein the first conversion matrix is ​​used to convert the left eye perspective into the central perspective, and the second conversion matrix is ​​used to convert the right eye perspective into the central perspective; Generate a posture transformation matrix from the central view to the IMU coordinate system according to the offset from the IMU to the center of the device; A first display position of the left-eye image relative to a viewer is calculated based on the first transformation matrix, the posture transformation matrix and the observation matrix, and a second display position of the right-eye image relative to the viewer is calculated based on the second transformation matrix, the posture transformation matrix and the observation matrix.

4. The method according to any one of claims 1 to 3, characterized in that When the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is horizontal screen display, drawing and displaying the left-eye image on the left-eye screen of the display window according to the first display position, and drawing and displaying the right-eye image on the right-eye screen of the display window according to the second display position, include: Determining first texture coordinates corresponding to each vertex on the application image displayed in the horizontal screen according to the width and height of the eye buffer; According to the obtained first texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye horizontal screen image, and, according to the obtained first texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye horizontal screen image; The left-eye horizontal screen image is drawn and displayed on the left-eye screen of the display window according to the first display position, and the right-eye horizontal screen image is drawn and displayed on the right-eye screen of the display window according to the second display position.

5. The method according to claim 4, characterized in that The calculating, according to the width and height of the eye buffer, first texture coordinates corresponding to each vertex on the application image displayed in the horizontal screen includes: Calculating first vertex offsets of four vertices on the horizontally displayed application image according to the width and height of the eye buffer; Acquire the image size of the application image, and determine the positions of four vertices on the application image displayed in the horizontal screen in combination with the first vertex screen offset; wherein the image width in the image size is greater than the image height; According to a preset texture range, first texture coordinates corresponding to vertices at different positions on the application image displayed in the horizontal screen are determined.

6. The method according to any one of claims 1 to 3, characterized in that When the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is vertical screen display, drawing and displaying the left-eye image on the left-eye screen of the display window according to the first display position, and drawing and displaying the right-eye image on the right-eye screen of the display window according to the second display position, include: Determining second texture coordinates corresponding to each vertex on the application image displayed in the vertical screen according to the width and height of the eye buffer; According to the obtained second texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye vertical screen image, and, according to the obtained second texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye vertical screen image; The left-eye vertical screen image is drawn and displayed on the left-eye screen of the display window according to the first display position, and the right-eye vertical screen image is drawn and displayed on the right-eye screen of the display window according to the second display position.

7. The method according to claim 6, characterized in that Determining, according to the width and height of the eye buffer, second texture coordinates corresponding to each vertex on the application image displayed in the vertical screen, includes: Calculating second vertex offsets of four vertices on the application image displayed in the vertical screen according to the width and height of the eye buffer; Acquire the image size of the application image, and determine the positions of four vertices on the vertically displayed application image in combination with the second vertex screen offset; wherein the image height in the image size is greater than the image width; According to a preset texture range, second texture coordinates corresponding to vertices at different positions on the application image displayed in the vertical screen are determined.

8. A VR device, characterized in that: The device comprises a processor, a memory and a display, wherein the display, the memory and the processor are connected via a bus: The memory stores a computer program, and the processor performs the following operations according to the computer program: Create an interface object according to the width and height of the screen on which the 2D application is to be displayed, wherein the interface object is used to indicate the width and height of the display window; Creating a left-eye image and a right-eye image respectively according to the width and height of the eye buffer, sharing the image buffer data of the 2D application to be displayed, and calling a synthesizer to render the left-eye image and the right-eye image in a cross-process manner; Calculating a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer according to the 6-DOF head posture and the pupil distance of the eyes; Using a single buffer, the left eye image is drawn on the left eye screen of the display window according to the first display position and displayed through the display, and the right eye image is drawn on the right eye screen of the display window according to the second display position and displayed through the display.

9. The VR device according to claim 8, characterized in that: The processor shares the image buffer data of the 2D application to be displayed, and calls the synthesizer to render the left-eye image and the right-eye image in a cross-process manner, and the specific operations are: Calling the synthesizer to convert the image buffer data of the 2D application to be displayed into a data type supported by a hardware buffer object; wherein the hardware buffer object is an abstract image buffer object that allows the synthesizer and the 2D application process to directly share and access data; Create an image object and assign the type-converted data to the image object; Creating a two-dimensional texture object, and binding the image object to the two-dimensional texture object; Through a graphics rendering API, the data of the image object is directly used as the texture of the two-dimensional texture object to obtain the left-eye image and the right-eye image.

10. The VR device according to claim 8, wherein: The processor calculates a first display position of the left-eye image relative to the viewer and a second display position of the right-eye image relative to the viewer according to the 6-DOF head posture and the pupil distance of the eyes, and the specific operation is: Generate an observation matrix based on a coordinate conversion matrix between a preset VR coordinate system and an OpenGL coordinate system and the 6-DOF head posture; Generate a first conversion matrix and a second conversion matrix according to the eye pupil distance, wherein the first conversion matrix is ​​used to convert the left eye perspective into the central perspective, and the second conversion matrix is ​​used to convert the right eye perspective into the central perspective; Generate a posture transformation matrix from the central view to the IMU coordinate system according to the offset from the IMU to the center of the device; A first display position of the left-eye image relative to a viewer is calculated based on the first transformation matrix, the posture transformation matrix and the observation matrix, and a second display position of the right-eye image relative to the viewer is calculated based on the second transformation matrix, the posture transformation matrix and the observation matrix.

Images