A vr display method and vr device for 2d applications

Abstract

The application relates to the technical field of VR screen projection, and provides a VR display method for 2D applications and VR equipment, which are used for realizing immersive viewing of 2D applications. According to the width and height of an eye buffer, left-eye images and right-eye images are respectively created, and through data sharing on image buffer data of a 2D application to be displayed, a synthesizer is used to render the left-eye images and the right-eye images in a cross-process mode, so that the data copying process is reduced, the rendering and display efficiency is improved, and it is ensured that the left-eye images and the right-eye images are complete 2D application images. Furthermore, the display positions of the left-eye images and the right-eye images relative to a viewer are respectively calculated through a 6-degree-of-freedom head pose and an eye pupil distance, so that the left-eye screen and the right-eye screen are rendered and displayed according to the corresponding display positions by using a single buffer, and compared with double buffering, the rendering and display efficiency is further improved, and the rendering and display delay is reduced.

Description

Technical Field

[0001] This application relates to the field of VR projection technology, and provides a VR display method and VR device for 2D applications. Background Technology

[0002] 2D applications are typically designed for traditional flat-panel displays, where the content seen by both eyes is the same. However, a virtual reality (VR) scene is a relatively enclosed three-dimensional space where the user's left eye only sees the image on the left screen, and the right eye only sees the image on the right screen. Therefore, if a 2D application is displayed directly on the left and right screens of a VR device, the content seen by the user's left and right eyes will be different. Figure 1 As shown, the dashed portion of the 2D application is displayed on the left-eye screen, while the solid portion is displayed on the right-eye screen. Because the two screens display different content, users cannot properly view 2D applications using VR devices. Summary of the Invention

[0003] This application provides a VR display method and VR device for 2D applications, which can reduce the rendering and display latency of 2D applications while ensuring that the left and right eye screens display the complete 2D application screen.

[0004] On one hand, embodiments of this application provide a VR display method for 2D applications, applied to VR devices, including:

[0005] Create an interface object based on the width and height of the screen from which the 2D application is to be displayed. The interface object is used to indicate the width and height of the display window.

[0006] Based on the width and height of the eye buffer, create left-eye and right-eye images respectively, share the image buffer data of the 2D application to be displayed, and call the compositor to render the left-eye and right-eye images in a cross-process manner;

[0007] Based on the 6-DOF head position and interpupillary distance, calculate the first display position of the left eye image relative to the viewer, and the second display position of the right eye image relative to the viewer;

[0008] Using a single buffer, the left-eye image is drawn and displayed on the left-eye screen of the display window according to the first display position, and the right-eye image is drawn and displayed on the right-eye screen of the display window according to the second display position.

[0009] On the other hand, this application provides a VR device including a processor, a memory, and a display, wherein the display, the memory, and the processor are connected via a bus.

[0010] The memory stores a computer program, and the processor performs the following operations according to the computer program:

[0011] Create an interface object based on the width and height of the screen from which the 2D application is to be displayed. The interface object is used to indicate the width and height of the display window.

[0012] Based on the width and height of the eye buffer, create left-eye and right-eye images respectively, share the image buffer data of the 2D application to be displayed, and call the compositor to render the left-eye and right-eye images in a cross-process manner;

[0013] Based on the 6-DOF head position and interpupillary distance, calculate the first display position of the left eye image relative to the viewer, and the second display position of the right eye image relative to the viewer;

[0014] Using a single buffer, the left-eye image is drawn on the left-eye screen of the display window and displayed on the monitor according to the first display position, and the right-eye image is drawn on the right-eye screen of the display window and displayed on the monitor according to the second display position.

[0015] Optionally, the processor shares image buffer data for the 2D application to be displayed, and calls the compositor to render the left-eye image and the right-eye image in a cross-process manner. Specifically, the operation is as follows:

[0016] The compositor is invoked to convert the image buffer data of the 2D application to be displayed into a data type supported by the hardware buffer object; wherein, the hardware buffer object is an abstract image buffer object that allows the compositor and the 2D application process to directly share and access data.

[0017] Create an image object and assign the type-converted data to the image object;

[0018] Create a two-dimensional texture object and bind the image object to the two-dimensional texture object;

[0019] By using the graphics rendering API, the data of the image object is directly used as the texture of a two-dimensional texture object to obtain the left eye image and the right eye image.

[0020] Optionally, the processor calculates the first display position of the left eye image relative to the viewer and the second display position of the right eye image relative to the viewer based on the 6-DOF head pose and interpupillary distance. Specifically, the processor performs the following operations:

[0021] Based on the preset coordinate transformation matrix between the VR coordinate system and the OpenGL coordinate system, and combined with the 6-DOF head pose, an observation matrix is ​​generated;

[0022] Based on the interpupillary distance, a first transformation matrix and a second transformation matrix are generated respectively; wherein, the first transformation matrix is ​​used to convert the left eye view to the central view, and the second transformation matrix is ​​used to convert the right eye view to the central view.

[0023] Based on the offset from the IMU to the device center, generate the attitude transformation matrix from the central viewpoint to the IMU coordinate system;

[0024] Based on the first transformation matrix, the pose transformation matrix, and the observation matrix, the first display position of the left-eye image relative to the viewer is calculated, and based on the second transformation matrix, the pose transformation matrix, and the observation matrix, the second display position of the right-eye image relative to the viewer is calculated.

[0025] Optionally, when the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is landscape display, the processor draws the left-eye image on the left-eye screen of the display window according to the first display position and displays it through the display; and draws the right-eye image on the right-eye screen of the display window according to the second display position and displays it through the display. Specifically, the operation is as follows:

[0026] Based on the width and height of the eye buffer, determine the first texture coordinates corresponding to each vertex on the landscape-displayed application image;

[0027] Based on the obtained first texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye landscape image; and based on the obtained first texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye landscape image.

[0028] The left-eye landscape image is drawn on the left-eye screen of the display window and displayed on the monitor according to the first display position, and the right-eye landscape image is drawn on the right-eye screen of the display window and displayed on the monitor according to the second display position.

[0029] Optionally, the processor calculates the first texture coordinates corresponding to each vertex on the landscape-displayed application image based on the width and height of the eye buffer. Specifically, the operation is as follows:

[0030] Based on the width and height of the eye buffer, calculate the first vertex offset of the four vertices on the landscape-displayed application image;

[0031] The image size of the application image is obtained, and the positions of the four vertices on the landscape-displayed application image are determined by combining the first vertex screen offset; wherein, the image width is greater than the image height in the image size;

[0032] Based on the preset texture range, the first texture coordinates corresponding to the vertices at different positions on the landscape application image are determined.

[0033] Optionally, when the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is portrait display, the processor draws the left-eye image on the left-eye screen of the display window according to the first display position and displays it through the display; and draws the right-eye image on the right-eye screen of the display window according to the second display position and displays it through the display. Specifically, the operation is as follows:

[0034] Based on the width and height of the eye buffer, determine the second texture coordinates corresponding to each vertex on the application image displayed in portrait mode;

[0035] Based on the obtained second texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye vertical screen image; and based on the obtained second texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye vertical screen image.

[0036] The left-eye vertical screen image is drawn on the left-eye screen of the display window and displayed on the monitor according to the first display position, and the right-eye vertical screen image is drawn on the right-eye screen of the display window and displayed on the monitor according to the second display position.

[0037] Optionally, the processor determines the second texture coordinates corresponding to each vertex on the vertically displayed application image based on the width and height of the eye buffer. Specifically, the operation is as follows:

[0038] Based on the width and height of the eye buffer, calculate the second vertex offset of the four vertices on the vertically displayed application image;

[0039] The image size of the application image is obtained, and the positions of the four vertices on the vertically displayed application image are determined by combining the second vertex screen offset; wherein, the image height is greater than the image width in the image size;

[0040] Based on the preset texture range, the second texture coordinates corresponding to vertices at different positions on the vertically displayed application image are determined.

[0041] On the other hand, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions for causing a computer device to execute a VR display method for any 2D application provided in embodiments of this application.

[0042] The beneficial effects of the VR display method and VR device for 2D applications provided in this application embodiment are as follows:

[0043] Based on the width and height of the screen of the 2D application to be displayed, an interface object is created to indicate the width and height of the display window. Left-eye and right-eye images are created based on the width and height of the eye buffer, respectively. By sharing the image buffer data of the 2D application to be displayed, the compositor renders the left-eye and right-eye images in a cross-process manner, reducing data copying and improving rendering efficiency, while ensuring that both the left-eye and right-eye images are complete 2D application images. Furthermore, the first display position of the left-eye image relative to the viewer and the second display position of the right-eye image relative to the viewer are calculated using 6-DOF head pose and interpupillary distance, respectively. The left-eye image is then drawn on the left-eye screen of the display window according to the first display position and displayed, and the right-eye image is drawn on the right-eye screen of the display window according to the second display position and displayed, ensuring that the left and right eyes see the same image, achieving immersive viewing of the 2D application. In the rendering and display process of the left and right eye screens, a single buffer is used, which further improves the rendering and display efficiency and reduces the rendering and display latency compared to a double buffer, thereby reducing the dizziness of 2D applications displayed on VR devices.

[0044] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1An abnormal display effect diagram of a 2D application on a VR device provided in the embodiments of this application;

[0047] Figure 2 A flowchart of a VR display method for a 2D application provided in this application embodiment;

[0048] Figure 3 A rendering flowchart of the left-eye and right-eye images provided in the embodiments of this application;

[0049] Figure 4 A flowchart illustrating the calculation of the display positions of the left and right eye images relative to the viewer's perspective, as provided in an embodiment of this application.

[0050] Figure 5 The image shows the display effect of the 2D application provided in this application on a VR device.

[0051] Figure 6 This is a schematic diagram of a landscape or portrait screen provided in an embodiment of this application;

[0052] Figure 7 A rendering and display flowchart for a landscape-oriented 2D application provided in this embodiment of the application;

[0053] Figure 8 A landscape display effect diagram of a 2D application provided in this application on a VR device;

[0054] Figure 9 A rendering and display flowchart of a portrait-oriented 2D application provided in an embodiment of this application;

[0055] Figure 10 A vertical screen display effect diagram of a 2D application provided in this application embodiment on a VR device;

[0056] Figure 11 A structural diagram of a VR device provided in an embodiment of this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some embodiments of the technical solutions of this application, and not all embodiments. Based on the embodiments recorded in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the technical solutions of this application.

[0058] Based on the exemplary embodiments shown in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can constitute a complete technical solution on its own.

[0059] It should be understood that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate, for example, to allow implementation in orders other than those given in the embodiments illustrated or described in this application.

[0060] Furthermore, the terms “including” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0061] As used in this application, the term "module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0062] The following is an explanation of the terms used in the embodiments of this application.

[0063] Interface objects: Objects that can display and draw interfaces, used for graphics rendering and user interface presentation.

[0064] Eye buffer: A buffer used to present image data that the user sees directly.

[0065] Composer: Also known as the display management service, it is responsible for merging image content from multiple sources into a single output.

[0066] Hardware buffer objects: used to provide a direct access interface to the underlying graphics hardware.

[0067] Image objects: Objects used to wrap different types of graphics buffers for use in the graphics rendering API.

[0068] Two-dimensional texture objects: used to represent and sample image data in graphics rendering.

[0069] The graphics rendering API is used for 2D and 3D graphics processing and has high performance.

[0070] It should be noted that VR devices use different operating systems (such as Android, HarmonyOS, iOS, etc.), so the above terms may be expressed differently, but their meanings are basically the same.

[0071] The design concept of the embodiments of this application will be summarized below in conjunction with application scenarios.

[0072] When using VR devices to watch 2D applications for an immersive experience, in order to ensure the normal display of the 2D application, the complete screen of the 2D application needs to be rendered on the left eye screen and the right eye screen respectively.

[0073] Currently, when displaying 2D applications in VR, most technologies directly display the 2D application in the Unity Launcher. The Unity application creates a virtual screen in virtual space, and then renders the textures of the 2D application on that virtual screen. However, due to space limitations, the image size of the 2D application needs to change with the size of the virtual screen, making it inflexible in display. Furthermore, when the image size of the 2D application does not match the size of the virtual screen, the 2D application's image needs to be cropped or scaled, thus affecting the visibility and usability of the 2D application.

[0074] In view of this, embodiments of this application provide a VR display method and VR device for 2D applications. The method acquires image buffer data of the 2D application through a synthesizer, enabling data sharing with the 2D application process. This allows the acquired image buffer data to be drawn onto newly created left-eye and right-eye images based on the width and height of the eye buffer, reducing data copying and improving rendering efficiency. Simultaneously, it ensures that both the left-eye and right-eye images are complete 2D application images. Furthermore, by calculating the first display position of the left-eye image relative to the viewer and the second display position of the right-eye image relative to the viewer using 6-DOF head pose and interpupillary distance, a single buffer is used. The left-eye image is drawn and displayed on the left-eye screen of the display window according to the first display position, and the right-eye image is drawn and displayed on the right-eye screen of the display window according to the second display position. Compared to a dual buffer used for display and one for rendering, this further improves rendering efficiency, reduces rendering latency, and consequently reduces dizziness associated with displaying 2D applications on VR devices.

[0075] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0076] See Figure 2 This is a flowchart of a VR display method for a 2D application provided in an embodiment of this application. The process is executed by a VR device and mainly includes the following steps:

[0077] S201: Create an interface object based on the width and height of the screen of the 2D application to be displayed. The interface object is used to indicate the width and height of the display window.

[0078] Taking a VR device using the Android system as an example, the interface object can be a Surface object. Its creation process is as follows: create a Surface object using the SurfaceControl class, set the width of the Surface object according to the width of the screen displaying the 2D application, and set the height of the Surface object according to the height of the screen displaying the 2D application. This Surface object can then be used as a display window to show the content of the 2D application.

[0079] It should be noted that the width and height of the interface object can be the same as or greater than the width and height of the screen of the 2D application to be displayed.

[0080] S202: Based on the width and height of the eye buffer, create the left eye image and the right eye image respectively, share the image buffer data of the 2D application to be displayed, and call the compositor to render the left eye image and the right eye image in a cross-process manner.

[0081] Taking the Android system as an example, the system renders the standard view of each eye into an eye buffer. Therefore, left and right eye images can be created separately based on the width and height of the eye buffer. The width of the left and right eye images is the same as the width of the eye buffer, and the height of the left and right eye images is the same as the height of the eye buffer.

[0082] After creating the left and right eye images, they can be rendered. The rendering process is as follows: Figure 3 As shown, it mainly includes the following steps:

[0083] S2021: Call the compositor to convert the image buffer data of the 2D application to be displayed into a data type supported by the hardware buffer object.

[0084] Taking the Android system as an example, the compositor is the SurfaceFlinger process, the image buffer data is buffer data, and the hardware buffer object is an AHardwareBuffer object.

[0085] The AHardwareBuffer object is a low-level primitive provided by the Android system for sharing image data across processes. It is an abstract image buffer object that enables direct sharing and access of data between the SurfaceFlinger process and 2D application processes without the need for data copying or duplication, thereby effectively improving rendering efficiency. Furthermore, the AHardwareBuffer object has GPU memory independent of the CPU, allowing data processing to be performed directly on the GPU and other hardware accelerators without the need for CPU intermediaries, resulting in high rendering performance and efficiency.

[0086] It should be noted that AHardwareBuffer is typically used by upper-level graphics libraries, rendering engines, or other graphics-related frameworks. Before using AHardwareBuffer directly, ordinary application developers may need to use graphics libraries or other encapsulation layers for advanced graphics programming.

[0087] S2022: Create an image object and assign the type-converted data to the image object.

[0088] Taking the Android system as an example, the image object can be an EGLImage object, which is created by using the eglCreateImageKHR function.

[0089] Among them, eglCreateImageKHR is a function in the Embedded Graphics Library (EGL). EGLImage objects can be shared between different processes, thereby enabling more flexible graphics resource management and collaboration. Furthermore, by encapsulating existing image resources into EGLImage objects, these graphics resources can be reused in different rendering processes, improving performance and efficiency.

[0090] S2023: Create a 2D texture object and bind the image object to the 2D texture object.

[0091] Taking the Android system as an example, a two-dimensional texture object can be a Tuxture2D object. After the object is created, the EGLImage object is bound to the Tuxture2D object using glEGLImageTargetTexture2DOES.

[0092] S2024: Using the graphics rendering API, the data of the image object is directly used as the texture of the two-dimensional texture object to obtain the left eye image and the right eye image.

[0093] Taking the Android system as an example, the graphics rendering API can be OpenGL ES. By binding EGLImage objects with Tuxture2D objects, the data in the EGLImage object can be directly used as texture data, reducing data copying and conversion, thus improving rendering performance and efficiency in OpenGL ES rendering.

[0094] S203: Based on the 6-DOF head pose and interpupillary distance, calculate the first display position of the left eye image relative to the viewer, and the second display position of the right eye image relative to the viewer.

[0095] In one example, the calculation process for the displayed position is as follows: Figure 4 As shown, it mainly includes the following steps:

[0096] S2031: Generate the observation matrix based on the preset coordinate transformation matrix between the VR coordinate system and the OpenGL coordinate system, combined with the 6-DOF head pose.

[0097] Assume the coordinate transformation matrix from VR coordinate system to OpenGL coordinate system is denoted as VR_GL, and the coordinate transformation matrix from OpenGL coordinate system to VR coordinate system is denoted as GL_VR, where, The 6-DOF head pose includes the rotation angle of the head relative to the three axes and the position of the head relative to the three axes, which can be represented by a 4*4 pose matrix head.

[0098] Furthermore, by multiplying the coordinate transformation matrices VR_GL and GL_VR with the inverse of the 6-DOF head pose matrix head, we can obtain the view matrix, expressed by the following formula:

[0099] view=GL_VR*inverse(head)*VR_GL Formula 1

[0100] Here, inverse represents the inverse transformation of the matrix, and the view matrix defines the position and orientation of the camera (used to represent the human eye) to transform the scene to the viewer's perspective.

[0101] In the embodiments of this application, the head rotation is corrected by inverse transformation using a 6-DOF head pose matrix, so that the observation matrix takes into account the head rotation, which helps to improve the accuracy of the display position calculation.

[0102] It should be noted that the embodiments of this application do not impose restrictive requirements on the method of obtaining the 6-DOF head pose. For example, it can be calculated by the SLAM algorithm or collected by the sensors of the VR device.

[0103] S2032: Generate the first transformation matrix and the second transformation matrix according to the interpupillary distance.

[0104] The first transformation matrix is ​​used to convert the left eye view to the central view, and the second transformation matrix is ​​used to convert the right eye view to the central view.

[0105] Assuming the interpupillary distance is denoted as ipd, the first transformation matrix is ​​denoted as l_cen, and the second transformation matrix is ​​denoted as r_cen, then

[0106] S2033: Generate the attitude transformation matrix from the central viewpoint to the IMU coordinate system based on the offset of the IMU from the device center.

[0107] The attitude transformation matrix represents the transformation from the central viewpoint to the IMU coordinate system, and is used to correct the rotation of the central viewpoint to take into account the attitude of the IMU.

[0108] Assuming the three-axis offsets from the IMU to the device center are x, y, and z, then the attitude transformation matrix from the central viewpoint to the IMU coordinate system is...

[0109] S2034: Calculate the first display position of the left-eye image relative to the viewer based on the first transformation matrix, the pose transformation matrix, and the observation matrix; and calculate the second display position of the right-eye image relative to the viewer based on the second transformation matrix, the pose transformation matrix, and the observation matrix.

[0110] The first display position is used to convert the scene from the left eye's perspective to the final observation perspective, and the second display position is used to convert the scene from the right eye's perspective to the final viewer's perspective.

[0111] Specifically, the formula for the first display position is expressed as:

[0112] view_l=l_cen*cen_imu*view Formula 2

[0113] The formula for the second display position is expressed as:

[0114] view_r=r_cen*cen_imu*view Formula 3

[0115] In the embodiments of this application, the first display position of the left eye image relative to the viewer and the second display position of the right eye image relative to the viewer are calculated by using 6-DOF head pose and interpupillary distance, thereby ensuring that the viewer can watch the left eye image and right eye image containing the complete 2D application screen through the VR device, and realize immersive viewing of 2D applications.

[0116] S204: Using a single buffer, draw and display the left-eye image on the left-eye screen of the display window according to the first display position, and draw and display the right-eye image on the right-eye screen of the display window according to the second display position.

[0117] Taking the Android system as an example, the Android system typically uses double buffering for rendering and display, where one buffer is used for rendering and the other is used for display, resulting in relatively low rendering and display efficiency.

[0118] To address this issue, this application embodiment uses a single buffer. The left-eye image is drawn and displayed on the left-eye screen of the display window according to the first display position using the glViewport function, and the right-eye image is drawn and displayed on the right-eye screen of the display window according to the second display position using the glViewport function. That is, the buffer is used for both rendering and display, thereby improving rendering and display efficiency, reducing rendering and display latency, and thus reducing the dizziness of 2D applications displayed on VR devices.

[0119] like Figure 5 As shown, this application demonstrates the display effect of a 2D application on a VR device. The complete 2D application can be displayed on both the left and right eye screens, thus ensuring an immersive viewing experience for the 2D application.

[0120] In the embodiments of this application, left-eye and right-eye images are created according to the width and height of the eye buffer, respectively. By converting the image buffer data of the 2D application to be displayed, data sharing between the compositor and the 2D application process is achieved. This allows the compositor to draw the left-eye and right-eye images across processes based on the image buffer data, ensuring that both the left-eye and right-eye images are complete 2D application images. Furthermore, after determining the display positions of the left-eye and right-eye images relative to the viewer based on the 6-DOF head pose and interpupillary distance, a single buffer is used to render and display the left-eye and right-eye images on the left-eye and right-eye screens of the display window. This ensures that the viewer sees a consistent image on both the left-eye and right-eye screens, achieving an immersive viewing experience. Compared to a dual-buffer setup, a single-buffer setup has lower rendering latency, effectively improving the viewing experience of 2D applications.

[0121] In some examples, the width and height of the eye buffer in VR devices are not necessarily equal, requiring separate landscape and portrait displays, such as... Figure 6As shown, when creating the interface object, the width of the display window can be set to be greater than the width of the screen of the 2D application to be displayed, and the height of the display window can be greater than the height of the screen of the 2D application to be displayed. For example, the width of the display window can be twice the width of the screen of the 2D application to be displayed, and the height of the display window can be twice the height of the screen of the 2D application to be displayed. This ensures that the content of the 2D application can be fully displayed in both landscape and portrait modes without distortion.

[0122] When the 2D application to be displayed is in landscape mode, the rendering process for the left and right eye screens is as follows: Figure 7 As shown, it mainly includes the following steps:

[0123] S2041: Determine the first texture coordinates corresponding to each vertex on the application image displayed in landscape mode based on the width and height of the eye buffer.

[0124] Assume the width and height of the eye buffer are eye_W and eye_H respectively, and the size is eye_W = width * 2, eye_H = height, where width and height are preset values.

[0125] In reality, the vertices of an application image displayed in landscape mode include a top vertex, a bottom vertex, a left vertex, and a right vertex. Let's assume the left vertex is defined as left = 0, the right vertex as right = r_width / eye_W, the top vertex as top = 1, and the bottom vertex as bottom = 1 - r_height / eye_H. Here, r_width and r_height represent the actual image width and height, respectively.

[0126] Specifically, r_width = width * 2, r_height = height.

[0127] When displaying in landscape mode, the calculation process for the first texture coordinates of the four vertices is as follows:

[0128] S2041_1: Calculate the first vertex offset of the four vertices on the landscape-displayed application image based on the width and height of the eye buffer.

[0129] First, calculate the initial positions of the four vertices (top, bottom, left, and right), as shown by the following formula:

[0130] left = left * 2 - 1;

[0131] right = right * 2 - 1;

[0132] top = (top * 2 - 1) / 2;

[0133] bottom = (bottom * 2 - 1) / 2.

[0134] Then, the offset of the first vertex is calculated based on the initial position, expressed by the formula:

[0135] x_w = (left + right) / 2;

[0136] y_w=(top+bottom) / 2.

[0137] S2041_2: Obtain the image size of the application image, and combine it with the first vertex offset to determine the positions of the four vertices of the application image displayed in landscape mode.

[0138] First, based on the width and height of the eye buffer, adjustment variables are defined for each of the four vertices, and the formulas for each variable are as follows:

[0139] oriLeft = L / eye_W;

[0140] oriRight=(L+eye_W) / eye_W;

[0141] oriTop = (eye_H - T) / eye_H;

[0142] oriBottom=(eye_H-T-eye_H) / eye_H.

[0143] Where L takes the value of 0, and T takes the value of 0.

[0144] Then, based on the image size of the applied image and four adjustment variables, the positions of the four vertices are adjusted. The adjustment formula is as follows:

[0145] left=(oriRight-oriLeft)*r_left / eye_W+oriLeft;

[0146] right=(oriRight-oriLeft)*r_W / eye_W+oriLeft;

[0147] top=(oriBottom-oriTop)*r_top / eye_H+oriTop;

[0148] bottom=(oriBottom-oriTop)*r_H / eye_H+oriTop;

[0149] left = left * 2 - 1;

[0150] right = right * 2 - 1;

[0151] top = (top * 2 - 1) / 2;

[0152] bottom = (bottom * 2 - 1) / 2.

[0153] Where r_leftWie is the actual position of the left vertex in the image, r_W is the width of the image, r_top is the actual position of the top vertex in the image, and r_H is the width of the image.

[0154] Finally, after adjusting the positions of the four vertices, and combining this with the offset of the first vertex, the final positions of the four vertices on the applied image are calculated, as shown in the following formula:

[0155] left- = x_w;

[0156] right-=x_w;

[0157] top- = y_w;

[0158] bottom-=y_w.

[0159] S2041_3: Determine the first texture coordinates corresponding to vertices at different positions on the landscape application image based on the preset texture range.

[0160] Typically, texture coordinates are represented by (u, v), where u and v represent the positions of the texture image on the U-axis and V-axis, respectively. Texture coordinates are usually between 0 and 1, where (0, 0) represents the lower left corner of the texture image and (1, 1) represents the upper right corner of the texture image.

[0161] Assuming uvLeft=0, uvRight=1, uvTop=0, uvBottom=1, then the first texture coordinates of the four vertices can be represented as (uvLeft, uvTop), (uvLeft, uvBottom), (uvRight, uvTop), and (uvRight, uvBottom), respectively.

[0162] S2042: Based on the obtained first texture coordinates, perform texture sampling from the rendered left-eye image to obtain a left-eye landscape image; and based on the obtained first texture coordinates, perform texture sampling from the rendered right-eye image to obtain a right-eye landscape image.

[0163] By specifying the texture coordinates of the vertices, OpenGL can map the pixels of a texture image onto those vertices. The texture image includes the rendered left-eye image and the rendered right-eye image.

[0164] S2043: Draw and display the left-eye landscape image on the left-eye screen of the display window according to the first display position, and draw and display the right-eye landscape image on the right-eye screen of the display window according to the second display position.

[0165] like Figure 8 The image shows the display effect of a landscape 2D application on a VR device. The width of the landscape image for the left eye is equal to the width of the left eye screen, while the height is less than the height of the left eye screen. Similarly, the width of the landscape image for the right eye is equal to the width of the left eye screen, while the height is less than the height of the right eye screen.

[0166] When the 2D application to be displayed is in portrait mode, the rendering process for the left and right eye screens is as follows: Figure 9 As shown, it mainly includes the following steps:

[0167] S2044: Determine the second texture coordinates corresponding to each vertex on the application image displayed in portrait mode based on the width and height of the eye buffer.

[0168] Assume the width and height of the eye buffer are eye_W and eye_H respectively, and the size is eye_W = width * 2, eye_H = height, where width and height are preset values.

[0169] In reality, the vertices of an application image displayed in portrait mode include a top vertex, a bottom vertex, a left vertex, and a right vertex. Let's assume the left vertex is defined as left = 0, the right vertex as right = r_width / eye_W, the top vertex as top = 1, and the bottom vertex as bottom = 1 - r_height / eye_H. Here, r_width and r_height represent the actual image width and height, respectively.

[0170] Specifically, r_width = height, r_height = width * 2.

[0171] When displayed in portrait mode, the calculation process for the second texture coordinates of the four vertices is as follows:

[0172] S2044_1: Calculate the second vertex offset of the four vertices on the application image displayed in portrait mode, based on the width and height of the eye buffer.

[0173] First, calculate the initial positions of the four vertices (top, bottom, left, and right), as shown by the following formula:

[0174] left = left * 2 - 1;

[0175] right = right * 2 - 1;

[0176] top = (top * 2 - 1) / 2;

[0177] bottom = (bottom * 2 - 1) / 2.

[0178] Then, the offset of the second vertex is calculated based on the initial position, expressed by the formula:

[0179] x_h = (left + right) / 2;

[0180] y_h = (top + bottom) / 2.

[0181] S2044_2: Obtain the image size of the application image, and combine it with the second vertex offset to determine the positions of the four vertices of the application image displayed in portrait mode.

[0182] First, based on the width and height of the eye buffer, adjustment variables are defined for each of the four vertices, and the formulas for each variable are as follows:

[0183] oriLeft = L / eye_W;

[0184] oriRight=(L+eye_W) / eye_W;

[0185] oriTop = (eye_H - T) / eye_H;

[0186] oriBottom=(eye_H-Top-eye_H) / eye_H.

[0187] Where L takes the value of 0, and T takes the value of 0.

[0188] Then, based on the image size of the applied image and four adjustment variables, the positions of the four vertices are adjusted. The adjustment formula is as follows:

[0189] left=(oriRight-oriLeft)*r_left / eye_W+oriLeft;

[0190] right=(oriRight-oriLeft)*r_width / eye_W+oriLeft;

[0191] top=(oriBottom-oriTop)*r_top / eye_H+oriTop;

[0192] bottom=(oriBottom-oriTop)*r_height / eye_H+oriTop;

[0193] left = left * 2 - 1;

[0194] right = right * 2 - 1;

[0195] top = (top * 2 - 1) / 2;

[0196] bottom = (bottom * 2 - 1) / 2.

[0197] Where r_leftWie is the actual position of the left vertex in the image, r_W is the width of the image, r_top is the actual position of the top vertex in the image, and r_H is the width of the image.

[0198] Finally, after adjusting the positions of the four vertices, and combining this with the offset of the second vertex, the final positions of the four vertices on the applied image are calculated, as shown in the following formula:

[0199] left-=x_h;

[0200] right-=x_h;

[0201] top- = y_h;

[0202] bottom-=y—h.

[0203] S2044_3: Determine the second texture coordinates corresponding to vertices at different positions on the application image displayed in portrait mode, based on the preset texture range.

[0204] Typically, texture coordinates are represented by (u, v), where u and v represent the positions of the texture image on the U-axis and V-axis, respectively. Texture coordinates are usually between 0 and 1, where (0, 0) represents the lower left corner of the texture image and (1, 1) represents the upper right corner of the texture image.

[0205] Assuming uvLeft=0, uvRight=1, uvTop=0, uvBottom=1, the texture coordinates of the four vertices can be represented as (uvLeft, uvBottom), (uvRight, uvBottom), (uvLeft, uvTop), and (uvRight, uvTop), respectively.

[0206] S2045: Based on the obtained second texture coordinates, perform texture sampling from the rendered left-eye image to obtain a left-eye vertical screen image; and based on the obtained second texture coordinates, perform texture sampling from the rendered right-eye image to obtain a right-eye vertical screen image.

[0207] By specifying the texture coordinates of the vertices, OpenGL can map the pixels of a texture image onto those vertices. The texture image includes the rendered left-eye image and the rendered right-eye image.

[0208] S2046: Draw and display the left-eye vertical screen image on the left-eye screen of the display window according to the first display position, and draw and display the right-eye vertical screen image on the right-eye screen of the display window according to the second display position.

[0209] like Figure 10 The image shows the display effect of a 2D application in portrait mode on a VR device. The height of the portrait image for the left eye is equal to the height of the left eye screen, while the width is less than the width of the left eye screen. Similarly, the height of the portrait image for the right eye is equal to the height of the left eye screen, while the width is less than the width of the right eye screen.

[0210] In the embodiments of this application, by creating an interface object larger than the screen of the 2D application to be displayed, the landscape and portrait displays of the 2D application can be adapted to ensure the display effect.

[0211] In some embodiments, the 2D application to be displayed is displayed independently in a skybox. The skybox can be customized with functions such as menu bars, handle models, and rays, so as to achieve immersive viewing while allowing interaction with the 2D application to be displayed, thus enhancing the viewing experience.

[0212] Based on the same technical concept, this application provides a VR device that can implement the steps of any of the above-mentioned 2D application VR display methods and achieve the same technical effect.

[0213] See Figure 11 The VR device includes a processor 1101, a memory 1102, and a display 1103, which are connected via a bus 1104.

[0214] The memory 1102 stores a computer program, and the processor 1101 performs the following operations according to the computer program:

[0215] Create a UI object based on the width and height of the screen from which the 2D application is to be displayed. The UI object is used to indicate the width and height of the display window.

[0216] Based on the width and height of the eye buffer, create the left eye image and the right eye image respectively, share the image buffer data to be displayed in the 2D application, and call the compositor to render the left eye image and the right eye image in a cross-process manner;

[0217] Based on the 6-DOF head position and interpupillary distance, calculate the first display position of the left eye image relative to the viewer, and the second display position of the right eye image relative to the viewer;

[0218] Using a single buffer, the left-eye image is drawn on the left-eye screen of the display window and displayed on the display 1103 according to the first display position, and the right-eye image is drawn on the right-eye screen of the display window and displayed on the display 1103 according to the second display position.

[0219] Optionally, the processor 1101 shares the image buffer data of the 2D application to be displayed, and calls the compositor to render the left-eye and right-eye images in a cross-process manner. Specifically, the operation is as follows:

[0220] The compositor is invoked to convert the image buffer data of the 2D application to be displayed into a data type supported by the hardware buffer object; where the hardware buffer object is an abstract image buffer object that allows direct sharing and access to data between the compositor and the 2D application process;

[0221] Create an image object and assign the type-converted data to the image object;

[0222] Create a 2D texture object and bind the image object to the 2D texture object;

[0223] By using the graphics rendering API, the data of the image object is directly used as the texture of the two-dimensional texture object to obtain the left-eye image and the right-eye image.

[0224] Optionally, the processor 1101 calculates the first display position of the left eye image relative to the viewer and the second display position of the right eye image relative to the viewer based on the 6-DOF head pose and interpupillary distance. The specific operation is as follows:

[0225] Based on the coordinate transformation matrix between the preset VR coordinate system and the OpenGL coordinate system, combined with the 6-DOF head pose, an observation matrix is ​​generated;

[0226] Based on the interpupillary distance, a first transformation matrix and a second transformation matrix are generated respectively; wherein, the first transformation matrix is ​​used to convert the left eye view to the central view, and the second transformation matrix is ​​used to convert the right eye view to the central view.

[0227] Based on the offset from the IMU to the device center, generate the attitude transformation matrix from the central viewpoint to the IMU coordinate system;

[0228] Based on the first transformation matrix, the pose transformation matrix, and the observation matrix, the first display position of the left-eye image relative to the viewer is calculated, and based on the second transformation matrix, the pose transformation matrix, and the observation matrix, the second display position of the right-eye image relative to the viewer is calculated.

[0229] Optionally, when the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is landscape display, the processor 1101 draws the left-eye image on the left-eye screen of the display window according to the first display position and displays it through the display 1103; and draws the right-eye image on the right-eye screen of the display window according to the second display position and displays it through the display 1103. Specifically, the processor 1101 draws the left-eye image on the right-eye screen of the display window according to the second display position and displays it through the display 1103.

[0230] Based on the width and height of the eye buffer, determine the first texture coordinates corresponding to each vertex on the application image displayed in landscape mode;

[0231] Based on the obtained first texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye landscape image; and based on the obtained first texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye landscape image.

[0232] The left-eye landscape image is drawn on the left-eye screen of the display window and displayed on the monitor 1103 according to the first display position, and the right-eye landscape image is drawn on the right-eye screen of the display window and displayed on the monitor 1103 according to the second display position.

[0233] Optionally, the processor 1101 calculates the first texture coordinates corresponding to each vertex on the landscape-displayed application image based on the width and height of the eye buffer. Specifically, the operation is as follows:

[0234] Based on the width and height of the eye buffer, calculate the offset of the first vertex of the four vertices on the landscape-viewed application image;

[0235] Obtain the image size of the application image, and combine it with the screen offset of the first vertex to obtain the positions of the four vertices on the application image displayed in landscape mode;

[0236] Based on the preset texture range, determine the first texture coordinates corresponding to vertices at different positions on the application image displayed in landscape mode.

[0237] Optionally, when the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is portrait display, the processor 1101 draws the left-eye image on the left-eye screen of the display window according to the first display position and displays it through the display 1103; and draws the right-eye image on the right-eye screen of the display window according to the second display position and displays it through the display 1103. Specifically, the processor 1101 draws the left-eye image on the right-eye screen of the display window according to the second display position and displays it through the display 1103.

[0238] Based on the width and height of the eye buffer, determine the second texture coordinates corresponding to each vertex on the application image displayed in portrait mode;

[0239] Based on the obtained second texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye vertical screen image; and based on the obtained texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye vertical screen image.

[0240] The left-eye vertical screen image is drawn on the left-eye screen of the display window and displayed on the monitor 1103 according to the first display position, and the right-eye vertical screen image is drawn on the right-eye screen of the display window and displayed on the monitor 1103 according to the second display position.

[0241] Optionally, the processor determines the second texture coordinates corresponding to each vertex on the application image displayed in portrait mode based on the width and height of the eye buffer. Specifically:

[0242] Based on the width and height of the eye buffer, calculate the offset of the second vertex of the four vertices on the application image displayed in portrait mode;

[0243] Obtain the image size of the application image, and combine it with the second vertex screen offset to obtain the positions of the four vertices on the landscape-displayed application image;

[0244] Based on the preset texture range, determine the second texture coordinates corresponding to vertices at different positions on the application image displayed in portrait mode.

[0245] In embodiments of this application, memory 1102 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and programs required for running instant messaging functions; the data storage area may store various instant messaging information and operation instruction sets. Memory 1102 may be volatile memory, such as random-access memory (RAM); memory 1102 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1102 may be any other medium capable of carrying or storing a desired computer program having an instruction or data structure form and accessible by a computer, but is not limited thereto. Memory 1102 may be a combination of the above-described memories.

[0246] The processor 1101 may include one or more central processing units (CPUs), GPUs, or digital processing units, etc. The processor 1101 is used to implement the steps of any of the above-described 2D application VR display methods when invoking a computer program stored in memory 1102.

[0247] Bus 1104 in Figure 11The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. Bus 1104 can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 11 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0248] It should be noted that, Figure 11 This is merely an example illustrating the hardware necessary for a VR device to execute the VR display method steps of the 2D application provided in this application embodiment. Not shown, the VR device may also include conventional hardware such as speakers, microphones, communication interfaces, and power supplies.

[0249] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

[0250] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0251] This application also provides a computer-readable storage medium for storing instructions that, when executed, can complete the steps of any of the 2D application VR display methods described in the foregoing embodiments.

[0252] This application also provides a computer program product for storing a computer program that performs the steps of the VR display method for any of the 2D applications described in the foregoing embodiments.

[0253] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0254] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0255] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0256] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0257] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A VR display method for 2D applications, characterized in that, Applied to VR devices, the method includes: Create an interface object based on the width and height of the screen from which the 2D application is to be displayed. The interface object is used to indicate the width and height of the display window. Based on the width and height of the eye buffer, create left-eye and right-eye images respectively, share the image buffer data of the 2D application to be displayed, and call the compositor to render the left-eye and right-eye images in a cross-process manner; Based on the 6-DOF head position and interpupillary distance, calculate the first display position of the left eye image relative to the viewer, and the second display position of the right eye image relative to the viewer; Using a single buffer, the left-eye image is drawn and displayed on the left-eye screen of the display window according to the first display position, and the right-eye image is drawn and displayed on the right-eye screen of the display window according to the second display position.

2. The method as described in claim 1, characterized in that, The process of sharing image buffer data for the 2D application to be displayed, and calling the compositor to render the left-eye and right-eye images in a cross-process manner, includes: The compositor is invoked to convert the image buffer data of the 2D application to be displayed into a data type supported by the hardware buffer object; wherein, the hardware buffer object is an abstract image buffer object that allows direct sharing and access to data between the compositor and the 2D application process; Create an image object and assign the type-converted data to the image object; Create a two-dimensional texture object and bind the image object to the two-dimensional texture object; By using the graphics rendering API, the data of the image object is directly used as the texture of the two-dimensional texture object to obtain the left eye image and the right eye image.

3. The method as described in claim 1, characterized in that, The step of calculating the first display position of the left eye image relative to the viewer and the second display position of the right eye image relative to the viewer based on the 6-DOF head pose and interpupillary distance includes: Based on the preset coordinate transformation matrix between the VR coordinate system and the OpenGL coordinate system, and combined with the 6-DOF head pose, an observation matrix is ​​generated; Based on the interpupillary distance, a first transformation matrix and a second transformation matrix are generated respectively; wherein, the first transformation matrix is ​​used to convert the left eye view to the central view, and the second transformation matrix is ​​used to convert the right eye view to the central view. Based on the offset from the IMU to the device center, generate the attitude transformation matrix from the central viewpoint to the IMU coordinate system; Based on the first transformation matrix, the pose transformation matrix, and the observation matrix, the first display position of the left-eye image relative to the viewer is calculated, and based on the second transformation matrix, the pose transformation matrix, and the observation matrix, the second display position of the right-eye image relative to the viewer is calculated.

4. The method according to any one of claims 1-3, characterized in that, When the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is landscape display, the step of drawing and displaying the left-eye image on the left-eye screen of the display window according to the first display position, and drawing and displaying the right-eye image on the right-eye screen of the display window according to the second display position, includes: Based on the width and height of the eye buffer, determine the first texture coordinates corresponding to each vertex on the landscape-displayed application image; Based on the obtained first texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye landscape image; and based on the obtained first texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye landscape image. The left-eye landscape image is drawn and displayed on the left-eye screen of the display window according to the first display position, and the right-eye landscape image is drawn and displayed on the right-eye screen of the display window according to the second display position.

5. The method as described in claim 4, characterized in that, The step of determining the first texture coordinates corresponding to each vertex on the landscape-displayed application image based on the width and height of the eye buffer includes: Based on the width and height of the eye buffer, calculate the first vertex offset of the four vertices on the landscape-displayed application image; The image size of the application image is obtained, and the positions of the four vertices on the landscape-displayed application image are determined by combining the first vertex screen offset; wherein, the image width is greater than the image height in the image size; Based on the preset texture range, the first texture coordinates corresponding to the vertices at different positions on the landscape application image are determined.

6. The method according to any one of claims 1-3, characterized in that, When the width of the display window is greater than the width of the screen of the 2D application to be displayed, the height of the display window is greater than the height of the screen of the 2D application to be displayed, and the display type of the 2D application to be displayed is portrait display, the step of drawing and displaying the left-eye image on the left-eye screen of the display window according to the first display position, and drawing and displaying the right-eye image on the right-eye screen of the display window according to the second display position, includes: Based on the width and height of the eye buffer, determine the second texture coordinates corresponding to each vertex on the application image displayed in portrait mode; Based on the obtained second texture coordinates, texture sampling is performed from the rendered left-eye image to obtain a left-eye vertical screen image; and based on the obtained second texture coordinates, texture sampling is performed from the rendered right-eye image to obtain a right-eye vertical screen image. The left-eye vertical screen image is drawn and displayed on the left-eye screen of the display window according to the first display position, and the right-eye vertical screen image is drawn and displayed on the right-eye screen of the display window according to the second display position.

7. The method as described in claim 6, characterized in that, The step of determining the second texture coordinates corresponding to each vertex on the vertically displayed application image based on the width and height of the eye buffer includes: Based on the width and height of the eye buffer, calculate the second vertex offset of the four vertices on the vertically displayed application image; The image size of the application image is obtained, and the positions of the four vertices on the vertically displayed application image are determined by combining the second vertex screen offset; wherein, the image height is greater than the image width in the image size; Based on the preset texture range, the second texture coordinates corresponding to vertices at different positions on the vertically displayed application image are determined.

8. A VR device, characterized in that, It includes a processor, a memory, and a display, wherein the display, the memory, and the processor are connected via a bus: The memory stores a computer program, and the processor performs the following operations according to the computer program: Create an interface object based on the width and height of the screen from which the 2D application is to be displayed. The interface object is used to indicate the width and height of the display window. Based on the width and height of the eye buffer, create left-eye and right-eye images respectively, share the image buffer data of the 2D application to be displayed, and call the compositor to render the left-eye and right-eye images in a cross-process manner; Based on the 6-DOF head position and interpupillary distance, calculate the first display position of the left eye image relative to the viewer, and the second display position of the right eye image relative to the viewer; Using a single buffer, the left-eye image is drawn on the left-eye screen of the display window and displayed on the monitor according to the first display position, and the right-eye image is drawn on the right-eye screen of the display window and displayed on the monitor according to the second display position.

9. The VR device as described in claim 8, characterized in that, The processor shares image buffer data for the 2D application to be displayed, and calls the compositor to render the left-eye and right-eye images in a cross-process manner. Specifically, the operation is as follows: The compositor is invoked to convert the image buffer data of the 2D application to be displayed into a data type supported by the hardware buffer object; wherein, the hardware buffer object is an abstract image buffer object that allows the compositor and the 2D application process to directly share and access data. Create an image object and assign the type-converted data to the image object; Create a two-dimensional texture object and bind the image object to the two-dimensional texture object; By using the graphics rendering API, the data of the image object is directly used as the texture of the two-dimensional texture object to obtain the left eye image and the right eye image.

10. The VR device as described in claim 8, characterized in that, The processor calculates the first display position of the left eye image relative to the viewer and the second display position of the right eye image relative to the viewer based on the 6-DOF head pose and interpupillary distance. Specifically, the operation is as follows: Based on the preset coordinate transformation matrix between the VR coordinate system and the OpenGL coordinate system, and combined with the 6-DOF head pose, an observation matrix is ​​generated; Based on the interpupillary distance, a first transformation matrix and a second transformation matrix are generated respectively; wherein, the first transformation matrix is ​​used to convert the left eye view to the central view, and the second transformation matrix is ​​used to convert the right eye view to the central view. Based on the offset from the IMU to the device center, generate the attitude transformation matrix from the central viewpoint to the IMU coordinate system; Based on the first transformation matrix, the pose transformation matrix, and the observation matrix, the first display position of the left-eye image relative to the viewer is calculated, and based on the second transformation matrix, the pose transformation matrix, and the observation matrix, the second display position of the right-eye image relative to the viewer is calculated.

Images