VR rendering method and related device

Abstract

The invention provides a VR rendering method and a related device, relates to the technical field of VR, is applied to a server, and aims to render a static scene of a target virtual space by moving different moving positions in a two-dimensional plane to obtain cube maps corresponding to the different moving positions. A rendering request sent by the VR terminal is received, the rendering request comprises a target moving position and a target visual angle of a moving object in a moving two-dimensional plane when the moving object in the target virtual space is rendered by the VR terminal, and a rendering result of the VR terminal on the moving object comprises a first two-dimensional map; determining a target cube map corresponding to the target moving position, and calculating a second two-dimensional map of the target cube map under the target visual angle; and sending the second two-dimensional chartlet to the VR terminal, so that the VR terminal synthesizes the picture content according to the first two-dimensional chartlet and the second two-dimensional chartlet. According to the method, less server resources are occupied, the rendering effect can be ensured, and particularly, the VR rendering cost of a large space can be reduced.

Description

Technical Field

[0001] This application relates to the field of VR (Virtual Reality) technology, and in particular, to a VR rendering method and related device. Background Art

[0002] VR technology is a computer simulation technology that can create and experience virtual worlds. Users can enter an interactive three-dimensional dynamic space by wearing VR devices.

[0003] Traditional VR rendering mainly includes server-side real-time rendering and pure terminal rendering. Among them, server-side real-time rendering is streaming rendering. Although the rendering effect is good, it will occupy too many software and hardware resources, especially at a high cost when rendering large spaces; pure terminal rendering is limited by terminal performance, especially with poor effects when rendering large spaces. Summary of the Invention

[0004] In view of the above problems, this application provides a VR rendering method and related device to achieve the purpose of reducing the occupancy of server-side resources while ensuring the rendering effect. The specific solutions are as follows:

[0005] In a first aspect of this application, a VR rendering method is provided. The method is applied to a server and includes:

[0006] Render the static scene of the target virtual space at different moving positions in a moving two-dimensional plane to obtain cube maps corresponding to different moving positions;

[0007] Receive a rendering request sent by a VR terminal. The rendering request includes the target moving position and target viewing angle of a moving object in the target virtual space when it is rendered by the VR terminal. The rendering result of the moving object by the VR terminal includes a first two-dimensional map;

[0008] Determine the target cube map corresponding to the target moving position, and calculate a second two-dimensional map of the target cube map at the target viewing angle;

[0009] Send the second two-dimensional map to the VR terminal so that the VR terminal synthesizes the picture content according to the first two-dimensional map and the second two-dimensional map.

[0010] In a possible implementation, the step of rendering the static scene of the target virtual space at different moving positions in a moving two-dimensional plane to obtain cube maps corresponding to different moving positions includes:

[0011] Divide the moving two-dimensional plane into multiple grid regions;

[0012] Render the texture cube map of each grid area under the static scene at a preset height, where the texture cube map covers a 360° view;

[0013] During the rendering process of the texture cube map of each grid area, obtain the depth value of each grid area and write it into an image to synthesize the depth cube map of each grid area;

[0014] Establish the mapping relationship between each grid area and its texture cube map and depth cube map;

[0015] Correspondingly, determining the target cube map corresponding to the target movement position includes:

[0016] Determine the target grid area where the target movement position is located, and based on the mapping relationship, determine the target texture cube map and target depth cube map corresponding to the target grid area.

[0017] In a possible implementation, the first two-dimensional map includes a first texture map, a first depth map, and a transparency map;

[0018] Calculating the second two-dimensional map of the target cube map at the target view includes:

[0019] Generate a first rendering background layer by covering the target texture cube map on the inner surface of the first spherical grid body, and calculate the second texture map of the first rendering background layer at the target view;

[0020] Generate a second rendering background layer by covering the target depth cube map on the inner surface of the second spherical grid body, and calculate the second depth map of the second rendering background layer at the target view.

[0021] In a possible implementation, sending the second two-dimensional map to the VR terminal so that the VR terminal synthesizes the picture content according to the first two-dimensional map and the second two-dimensional map includes:

[0022] Send the second texture map and the second depth map to the VR terminal to achieve:

[0023] The VR terminal determines the opaque area and the permeable area according to the transparency map; for each pixel point in the opaque area, determine the first depth value of the pixel point in the first depth map and the second depth value of the pixel point in the second depth map; if the first depth value is less than the second depth value, determine the first texture value of the pixel point in the first texture map and output it for display; if the first depth value is greater than or equal to the second depth value, determine the second texture value of the pixel point in the second texture map and output it for display; for each pixel point in the permeable area, determine the third texture value of the pixel point in the first texture map and the fourth texture value in the second texture map; determine the transparency value of the pixel point in the transparency map; calculate the actual texture value of the pixel point according to the transparency value, the third texture value and the fourth texture value, and output it for display.

[0024] The second aspect of the present application provides a VR rendering device, which is applied to a server, and the device includes:

[0025] A preprocessing module for rendering a static scene of a target virtual space at different moving positions in a moving two-dimensional plane to obtain cube maps corresponding to different moving positions;

[0026] A static rendering module for receiving a rendering request sent by the VR terminal, where the rendering request includes the target moving position and the target viewing angle of a moving object in the target virtual space when the moving object is rendered by the VR terminal, and the rendering result of the moving object by the VR terminal includes a first two-dimensional map; determine the target cube map corresponding to the target moving position, and calculate a second two-dimensional map of the target cube map at the target viewing angle; send the second two-dimensional map to the VR terminal, so that the VR terminal synthesizes the picture content according to the first two-dimensional map and the second two-dimensional map.

[0027] The third aspect of the present application provides a VR rendering method, which is applied to a VR terminal and includes:

[0028] Render a moving object in a target virtual space and send a rendering request to the server. The rendering request includes the target moving position and the target viewing angle of the moving object in the moving two-dimensional plane when the moving object is rendered, and the rendering result of the moving object includes a first two-dimensional map;

[0029] Receive the second two-dimensional map sent by the server. The server pre-renders the static scene in the target virtual space at different moving positions in the moving two-dimensional plane to obtain cube maps corresponding to different moving positions. The second two-dimensional map is calculated by the server based on the target cube map corresponding to the target moving position and the target viewing angle;

[0030] Synthesize the screen content according to the first two-dimensional map and the second two-dimensional map.

[0031] In a possible implementation, the first two-dimensional map includes a first texture map, a first depth map, and a transparency map. The cube map includes a texture cube map and a depth cube map. The target cube map includes a target texture cube map and a target depth cube map. The second two-dimensional map includes a second texture map and a second depth map;

[0032] The synthesizing the screen content according to the first two-dimensional map and the second two-dimensional map includes:

[0033] Determine the opaque area and the permeable area according to the transparency map;

[0034] For each pixel point in the opaque area, determine the first depth value of the pixel point in the first depth map and the second depth value of the pixel point in the second depth map; if the first depth value is less than the second depth value, determine the first texture value of the pixel point in the first texture map and output for display; if the first depth value is greater than or equal to the second depth value, determine the second texture value of the pixel point in the second texture map and output for display;

[0035] For each pixel point in the permeable area, determine the third texture value of the pixel point in the first texture map and the fourth texture value of the pixel point in the second texture map; determine the transparency value of the pixel point in the transparency map; calculate the actual texture value of the pixel point according to the transparency value, the third texture value, and the fourth texture value, and output for display.

[0036] The fourth aspect of this application provides a VR rendering device. The device is applied to a VR terminal. The device includes:

[0037] A dynamic rendering module, configured to render a moving object in the target virtual space and send a rendering request to the server. The rendering request includes the target moving position and the target viewing angle of the moving object in the moving two-dimensional plane when the moving object is rendered. The rendering result of the moving object includes a first two-dimensional map;

[0038] The screen composition module is configured to receive the second two-dimensional map sent by the server. The server pre-renders the static scene in the target virtual space at different moving positions in the moving two-dimensional plane to obtain cube maps corresponding to different moving positions. The second two-dimensional map is calculated by the server based on the target cube map corresponding to the target moving position and the target viewing angle; and composes the screen content according to the first two-dimensional map and the second two-dimensional map.

[0039] A fifth aspect of the present application provides a computer program product, including computer-readable instructions, which, when running on an electronic device, enable the electronic device to implement the VR rendering method according to the first aspect or any implementation manner of the first aspect or the third aspect or any implementation manner of the third aspect.

[0040] A sixth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, they can enable the electronic device to implement the VR rendering method according to the first aspect or any implementation manner of the first aspect or the third aspect or any implementation manner of the third aspect.

[0041] By means of the above technical solutions, a VR rendering method and related device provided by the present application are applied to the server, pre-render the static scene in the target virtual space at different moving positions in the moving two-dimensional plane to obtain cube maps corresponding to different moving positions; receive a rendering request sent by the VR terminal, where the rendering request includes the target moving position and the target viewing angle of the moving object in the target virtual space when it is rendered by the VR terminal, and the rendering result of the moving object by the VR terminal includes the first two-dimensional map; determine the target cube map corresponding to the target moving position, and calculate the second two-dimensional map of the target cube map at the target viewing angle; send the second two-dimensional map to the VR terminal, so that the VR terminal composes the screen content according to the first two-dimensional map and the second two-dimensional map. In the present application, the server pre-renders the static scene as cube maps at different moving positions, and when the VR terminal renders the two-dimensional map of the moving object in real time, returns the two-dimensional map of the static scene at its position and viewing angle to the VR terminal, so that the VR terminal generates a complete VR screen by superimposing the two-dimensional maps, which can implement the VR rendering task through the cooperation of the server and the VR terminal, occupy less server resources, and can ensure the rendering effect, especially can reduce the VR rendering cost of a large space. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and that the original elements and elements are not necessarily drawn to scale.

[0043] Figure 1 A schematic diagram of a system architecture provided by an embodiment of the present application;

[0044] Figure 2 A schematic diagram of the structure of a VR terminal provided by an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the structure of a server provided by an embodiment of the present application;

[0046] Figure 4 A schematic flow chart of a VR rendering method provided by an embodiment of the present application;

[0047] Figure 5 A partial schematic flow chart of a VR rendering method provided by an embodiment of the present application;

[0048] Figure 6 A schematic diagram of the structure of a VR rendering device provided by an embodiment of the present application;

[0049] Figure 7 Another schematic flow chart of a VR rendering method provided by an embodiment of the present application;

[0050] Figure 8 Another schematic diagram of the structure of a VR rendering device provided by an embodiment of the present application. Specific embodiments

[0051] The following describes the embodiments of the present application in combination with the accompanying drawings in the embodiments of the present application. The terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0052] The following describes the embodiments of the present application in combination with the accompanying drawings. Those of ordinary skill in the art will know that with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0053] In the description of this application and the above-mentioned drawings, terms such as "first" and "second" are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances, which is only a way of distinguishing objects with the same attributes when describing embodiments of this application. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device comprising a series of units does not have to be limited to those units, but may include other units not clearly listed or inherent to these processes, methods, products or devices.

[0054] See Figure 1 , Figure 1 which is a schematic diagram of a system architecture provided for an embodiment of this application. The system may include a VR terminal 100 and a server 200. Among them, the server 200 may include one or more servers ( Figure 1 illustrated by including one server as an example), and the server 200 may provide the VR rendering method provided for the embodiment of this application for one or more VR terminals 100.

[0055] Among them, an application program for executing the VR rendering method may be installed on the VR terminal 100. The above application program and web page may provide an interface. The VR terminal 100 may receive relevant parameters of a moving object input by a user on the VR application interface, and send the target moving position and target viewing angle of the moving object in the moving two-dimensional plane when it is rendered by the VR terminal to the server 200. The server 200 obtains a two-dimensional texture map at the target moving position and target viewing angle based on the received parameters and returns it to the VR terminal 100.

[0056] See Figure 2 , Figure 2 which is a schematic diagram of the structure of a VR terminal provided for an embodiment of this application. As Figure 2 shown, the VR terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a headphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190, etc. Those skilled in the art can understand that Figure 2 this is only an example of a terminal or a multifunctional device, and does not constitute a limitation on the terminal or multifunctional device. It may include more or fewer components than shown, or combine certain components, or different components.

[0057] The input unit 130 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the portable multifunctional device. The input unit 130 may also include other input devices. Specifically, the other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, joysticks, etc.

[0058] Among them, the input device 132 can receive input data and the like.

[0059] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the VR terminal 100, interactive interfaces, file displays, and / or the playback of any multimedia file. In the embodiments of the present application, the display unit 140 can be used to display a complete VR picture generated by overlaying two-dimensional texture maps, etc.

[0060] The memory 120 can be used to store instructions and data. The memory 120 mainly includes a storage instruction area and a storage data area. The storage data area can store various data, such as multimedia files, texts, etc.; the storage instruction area can store software units such as an operating system, applications, instructions required for at least one function, or their subsets and extended sets. It can also include a non-volatile random access memory; it provides for managing the hardware, software, and data resources in the computing processing device, supporting control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0061] The processor 170 is the control center of the VR terminal 100, connecting various parts of the entire VR terminal 100 through various interfaces and lines. By running or executing the instructions stored in the memory 120 and calling the data stored in the memory 120, it executes various functions of the VR terminal 100 and processes data, thereby performing overall control of the terminal device. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory can be implemented on a single chip. In some embodiments, they can also be separately implemented on independent chips. The processor 170 can also be used to generate corresponding operation control signals, send them to the corresponding components of the computing processing device, read and process the data in the software, especially read and process the data and programs in the memory 120, so that each functional module therein executes the corresponding function, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0062] Among them, the memory 120 can be used to store software codes related to the VR rendering method, and the processor 170 can execute the steps of the VR rendering method and can also schedule other units (such as the above input unit 130 and display unit 140) to implement corresponding functions.

[0063] The radio frequency unit 110 (optional) can be used to receive and send information or signals during a call. For example, after receiving the downlink information of the base station, it is given to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Usually, the RF circuit includes but is not limited to antennas, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. This wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0064] Among them, in the embodiment of this application, the radio frequency unit 110 can send data to the server 200 and receive the processing result sent by the server 200.

[0065] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0066] The VR terminal 100 also includes a power supply 190 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system.

[0067] The VR terminal 100 also includes an external interface 180. This external interface can be a standard Micro USB interface or a multi-pin connector, and can be used to connect the VR terminal 100 to other devices for communication, and can also be used to connect a charger to charge the VR terminal 100.

[0068] Although not shown, the VR terminal 100 may further include a flash, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be elaborated here. Some or all of the methods described below can be applied to the VR terminal 100 as shown in Figure 2 .

[0069] Refer to Figure 3 , Figure 3 which is a schematic structural diagram of a server provided by an embodiment of the present application. As shown in Figure 3 , the server 200 includes a bus 210, a processing device 220, a communication interface 230, and a storage device 240. The processing device 220, the storage device 240, and the communication interface 230 communicate with each other through the bus 210.

[0070] The bus 210 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, Figure 3 only a thick line is used to represent it in

[0071] , but it does not mean that there is only one bus or one type of bus.

[0072] The processing device 220 may be any one or more of processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0073] Among them, the storage device 240 can be used to store software codes related to the VR rendering method, and the processing device 220 can execute the steps of the VR rendering method of the chip, and can also schedule other units to implement corresponding functions.

[0074] It should be understood that the above VR terminal 100 and server 200 can be centralized or distributed devices.

[0075] See Figure 4 , Figure 4 is a schematic flowchart of a VR rendering method provided by an embodiment of the present application. As Figure 4 shown, a VR rendering method provided by an embodiment of the present application is applied to the server 200. The VR rendering method may include steps S401 to S403, and the following will describe these steps in detail.

[0076] S401, rendering the static scene of the target virtual space at different moving positions in the moving two-dimensional plane to obtain cube maps corresponding to different moving positions.

[0077] In the embodiment of the present application, the target virtual space is a VR virtual space to be rendered, which includes a static scene and a moving object. The moving two-dimensional plane is the two-dimensional plane where the moving position is located when the moving object moves in the target virtual space.

[0078] The mobile terminal 200 determines multiple moving positions that the moving object may reach in the moving two-dimensional plane, and then renders the static scene at each moving position to obtain the cube map corresponding to each moving position. This cube map is used as the background for calculating the two-dimensional map of the moving object at a certain position (i.e., the subsequent target moving position) and a certain viewing angle (i.e., the subsequent target viewing angle).

[0079] In a possible implementation, the moving two-dimensional plane can be divided into multiple grid regions, and then the static scene is rendered under each grid region to obtain the texture cube map and depth cube map corresponding to each grid region. See Figure 5 , Figure 5 is a partial flowchart of a VR rendering method provided by an embodiment of the present application. As Figure 5 shown, in a VR rendering method provided by an embodiment of the present application, step S401, "rendering the static scene of the target virtual space at different moving positions in the moving two-dimensional plane to obtain cube maps corresponding to different moving positions", may include steps S4011 to S4014, and the following will describe these steps in detail.

[0080] S4011, dividing the moving two-dimensional plane into multiple grid regions.

[0081] In the embodiments of the present application, the size of a single grid region can be determined according to the positioning accuracy of the target virtual space , assuming that the minimum value of the moving two-dimensional plane in the direction is 0 and the maximum value , and continuing to assume that the minimum value of the moving two-dimensional plane in the direction is 0 and the maximum value is , then the size of a single grid region is divided according to the moving two-dimensional plane to obtain on the grid regions, on the grid regions, then it can be defined that the th in the direction, the th in the direction grid regions are , where ranges from , ranges from .

[0082] S4012, Render the texture cube map of each grid region in the static scene at a preset height, and the texture cube map covers a 360° view angle.

[0083] In the embodiments of the present application, the height of the virtual camera when rendering the static scene is set to the preset height. For each grid region in the moving two-dimensional plane, place the virtual camera at the center position of the grid region and the height of the virtual camera is the preset height, and use the engine rendering method to obtain the texture cube map corresponding to the grid region, and the texture cube map covers a 360° view angle.

[0084] S4013, During the rendering process of the texture cube map of each grid region, obtain the depth value of each grid region and write it into the image to synthesize the depth cube map of each grid region.

[0085] In the embodiments of the present application, for each grid region in the moving two-dimensional plane, during the rendering process of the texture cube map corresponding to the grid region, the depth value of the depth buffer can be synchronously written into the image to synthesize the depth cube map.

[0086] S4014, Establish the mapping relationship between each grid region and its texture cube map and depth cube map.

[0087] In the embodiments of the present application, for each grid region in the mobile two-dimensional plane, a mapping relationship is established between the grid region and its corresponding texture cube map and depth cube map. Specifically, the mapping relationship can be expressed as , where represents the texture cube map corresponding to the grid region , represents the depth cube map corresponding to the grid region .

[0088] Subsequently, based on the above mapping relationship, the texture cube map and depth cube map corresponding to the specified grid region can be indexed.

[0089] In a possible implementation, the texture cube map and depth cube map corresponding to each grid region can also be compressed and stored to reduce the occupancy of the storage resources of the server 200. In this regard, a VR rendering method provided by the embodiments of the present application, wherein step S401 "render the static scene of the target virtual space at different moving positions in the mobile two-dimensional plane to obtain cube maps corresponding to different moving positions" can further include the following steps:

[0090] Based on the mapping relationship, compress and store the texture cube map and depth cube map corresponding to each grid region.

[0091] In the embodiments of the present application, based on the above mapping relationship, the texture cube map and depth cube map corresponding to each grid region can be compressed using ASTC or H.265 compression and stored in the storage system.

[0092] S402, receive a rendering request sent by the VR terminal. The rendering request includes the target moving position and target viewing angle of the moving object in the target virtual space when it is rendered by the VR terminal. The rendering result of the moving object by the VR terminal includes a first two-dimensional map.

[0093] In the embodiments of the present application, during the user usage stage, the VR terminal 100 initializes the dynamic scene configuration (including the configuration of virtual FOV resolution, etc.), and synchronizes the position and viewing angle between virtual and reality (i.e., the orientation of the VR glasses) in the positioning solution. The VR terminal obtains the rendering result of the moving object in the current frame through real-time front-end rendering. The rendering result includes a two-dimensional map (i.e., the first two-dimensional map), and puts the position (i.e., the target moving position) and viewing angle (i.e., the target viewing angle) of the moving object corresponding to the current frame in the mobile two-dimensional plane into the rendering request and sends it to the server 200.

[0094] In response to this, after the server 200 receives the rendering request sent by the VR terminal 100, it can parse and obtain the target movement position and target viewing angle corresponding to the current frame from it.

[0095] S403. Determine the target cube map corresponding to the target movement position, and calculate the second two-dimensional map of the target cube map at the target viewing angle.

[0096] In the embodiment of the present application, the server 200 pre-obtains cube maps at different movement positions. In response to this, it can determine the target cube map corresponding to the target movement position from them, synchronize the target viewing angle to the virtual camera, generate a rendering background layer by covering the target cube map on the inner surface of a spherical grid body, and use a three-dimensional rendering process (a currently mature transformation and rasterization calculation process can be used) to calculate the two-dimensional map (i.e., the second two-dimensional map) of the rendering background layer recorded by the virtual camera.

[0097] In a possible implementation, in the case where the moving two-dimensional plane is divided into multiple grid regions, in step S403 above, "determine the target cube map corresponding to the target movement position" can be performed by the following steps:

[0098] Determine the target grid region where the target movement position is located, and determine the target texture cube map and target depth cube map corresponding to the target grid region based on the mapping relationship.

[0099] In the embodiment of the present application, according to the size of a single grid region the grid region (i.e., the target grid region) where the target movement position is located can be calculated, and then the texture cube map (i.e., the target texture cube map) and depth cube map (i.e., the target depth cube map) corresponding to the target grid region can be determined based on the above mapping relationship.

[0100] S404. Send the second two-dimensional map to the VR terminal so that the VR terminal synthesizes the picture content according to the first two-dimensional map and the second two-dimensional map.

[0101] In the embodiment of the present application, after the server 200 obtains the second two-dimensional map, it can send the second two-dimensional map to the VR terminal 100, and the VR terminal 100 synthesizes the first two-dimensional map and the second two-dimensional map to obtain the picture content of the current frame.

[0102] In a possible implementation, the rendering result of the current frame obtained by the VR terminal 100 includes a texture map (i.e., the first texture map), a depth map (i.e., the first depth map), and an alpha map. Among them, the first texture map records the texture values of each pixel when the moving object is rendered; the first depth map records the depth values of each pixel when the moving object is rendered; the alpha map records the alpha values of each pixel when the moving object is rendered, and the value range of the alpha value is , when the alpha value is 1, it means opaque, and when the alpha value is 0, it means fully transparent.

[0103] The cube map includes a texture cube map and a depth cube map. The target cube map corresponding to the target moving position includes a target texture cube map and a target depth cube map. For this, the server 200 can sequentially use the target texture cube map and the target depth cube map as the rendering background layer, and the obtained second two-dimensional map includes a second texture map and a second depth map. Among them, the second texture map records the texture values of each pixel in the static scene at the target moving position and the target viewing angle; the second depth map records the depth values of each pixel in the static scene at the target moving position and the target viewing angle.

[0104] For this, a VR rendering method provided by an embodiment of the present application, wherein, in step S403, "calculating the second two-dimensional map of the target cube map at the target viewing angle" can be performed by the following steps:

[0105] Generate a first rendering background layer by covering the target texture cube map on the inner surface of the first spherical grid, and calculate the second texture map of the first rendering background layer at the target viewing angle; generate a second rendering background layer by covering the target depth cube map on the inner surface of the second spherical grid, and calculate the second depth map of the second rendering background layer at the target viewing angle.

[0106] In the embodiment of the present application, after obtaining the target texture cube map and the target depth cube map, the target viewing angle is synchronized to the virtual camera. On the one hand, generate a rendering background layer (i.e., the first rendering background layer) by covering the target texture cube map on the inner surface of a spherical grid (i.e., the first spherical grid), and use a three-dimensional rendering process to calculate the texture map (i.e., the second texture map) of the first rendering background layer recorded by the virtual camera; on the other hand, generate a rendering background layer (i.e., the second rendering background layer) by covering the target depth cube map on the inner surface of a spherical grid (i.e., the second spherical grid), and use a three-dimensional rendering process to calculate the depth map (i.e., the second depth map) of the second rendering background layer recorded by the virtual camera.

[0107] Correspondingly, for step S404, "sending the second 2D map to the VR terminal so that the VR terminal synthesizes the screen content according to the first 2D map and the second 2D map", the following steps can be adopted:

[0108] Send the second texture map and the second depth map to the VR terminal to achieve:

[0109] The VR terminal determines the opaque area and the permeable area according to the transparency map; for each pixel point in the opaque area, determine the first depth value of the pixel point in the first depth map and the second depth value of the pixel point in the second depth map; if the first depth value is less than the second depth value, determine the first texture value of the pixel point in the first texture map and output it for display; if the first depth value is greater than or equal to the second depth value, determine the second texture value of the pixel point in the second texture map and output it for display; for each pixel point in the permeable area, determine the third texture value of the pixel point in the first texture map and the fourth texture value in the second texture map; determine the transparency value of the pixel point in the transparency map; calculate the actual texture value of the pixel point according to the transparency value, the third texture value and the fourth texture value, and output it for display.

[0110] In the embodiment of the present application, the server 200 can compress and transmit the second texture map and the second depth map to the VR terminal 100, and the VR terminal 100 can obtain the second texture map and the second depth map after decompression.

[0111] During the process of the VR terminal 100 synthesizing the screen content of the current frame, first determine the opaque area and the permeable area according to the transparency map. Among them, the transparency value of the pixel points in the opaque area is greater than the transparency value of the pixel points in the permeable area. For example, the transparency value of the pixel points in the opaque area is 1, and the transparency value of the pixel points in the permeable area is less than 1.

[0112] For each pixel point in the opaque area, determine the depth value of the pixel point in the first depth map (i.e., the first depth value) and the depth value of the pixel point in the second depth map (i.e., the second depth value), and compare the first depth value with the second depth value; if the first depth value is less than the second depth value, it means that the moving object is closer to the VR glasses. At this time, determine the texture value of the pixel point in the first texture map (i.e., the first texture value) and output and display the first texture value; if the first depth value is greater than or equal to the second depth value, it means that the static scene is closer to the VR glasses (i.e., the moving object is blocked by the static scene). At this time, determine the second texture value of the pixel point in the second texture map and output and display the second texture value.

[0113] For each pixel point in the transmissive region, the texture value of the pixel point in the first texture map (i.e., the third texture value) and the texture value of the pixel point in the second texture map (i.e., the fourth texture value) can be determined. Furthermore, the transparency value of the pixel point in the transparency map can be determined, and the actual texture value of the pixel point can be calculated based on the transparency value, the third texture value, and the fourth texture value, and the actual texture value can be output and displayed. Specifically, assume that the third texture value corresponding to the pixel point is , the fourth texture value is , the transparency value is , then the actual texture value of the pixel point is .

[0114] Based on this, the texture values of each pixel point in the current frame can be output to form the picture content.

[0115] Through the above description, a VR rendering method provided by an embodiment of the present application is different from traditional full-scene real-time rendering or per-perspective pre-rendering. Through two-dimensional grid sampling and cube map mapping, only less rendering resources are required to support the continuous immersive experience of large-space perspective rotation, occupying less server resources and ensuring the rendering effect. In particular, the VR rendering cost of large spaces can be reduced.

[0116] Refer to Figure 6 , Figure 6 which is a schematic structural diagram of a VR rendering device provided by an embodiment of the present application. As Figure 6 shown, a VR rendering device provided by an embodiment of the present application is applied to the server 200, and the device includes:

[0117] A preprocessing module 601, configured to render the static scene of the target virtual space at different moving positions in the moving two-dimensional plane to obtain cube maps corresponding to different moving positions.

[0118] A static rendering module 602, configured to receive a rendering request sent by the VR terminal. The rendering request includes the target moving position and the target perspective of the moving object in the target virtual space in the moving two-dimensional plane when the VR terminal renders the moving object. The rendering result of the moving object by the VR terminal includes a first two-dimensional map; determine the target cube map corresponding to the target moving position, and calculate a second two-dimensional map of the target cube map at the target perspective; send the second two-dimensional map to the VR terminal so that the VR terminal synthesizes the picture content according to the first two-dimensional map and the second two-dimensional map.

[0119] In a possible implementation, the preprocessing module 601 is specifically configured to:

[0120] Divide the moving two-dimensional plane into multiple grid regions; render the texture cube map of each grid region in the static scene at a preset height, and the texture cube map covers a 360° view; during the rendering process of the texture cube map of each grid region, obtain the depth value of each grid region and write it into the image to synthesize the depth cube map of each grid region; establish the mapping relationship between each grid region and its texture cube map and depth cube map.

[0121] Correspondingly, the static rendering module 602 for determining the target cube map corresponding to the target moving position is specifically configured to:

[0122] Determine the target grid region where the target moving position is located, and determine the target texture cube map and target depth cube map corresponding to the target grid region based on the mapping relationship.

[0123] In a possible implementation, the first two-dimensional map includes a first texture map, a first depth map, and a transparency map;

[0124] The static rendering module 602 for calculating the second two-dimensional map of the target cube map at the target view angle is specifically configured to:

[0125] Generate a first rendering background layer by covering the target texture cube map on the inner surface of the first spherical grid body, and calculate the second texture map of the first rendering background layer at the target view angle; generate a second rendering background layer by covering the target depth cube map on the inner surface of the second spherical grid body, and calculate the second depth map of the second rendering background layer at the target view angle.

[0126] In a possible implementation, the static rendering module 602 for sending the second two-dimensional map to the VR terminal so that the VR terminal synthesizes the picture content according to the first two-dimensional map and the second two-dimensional map is specifically configured to:

[0127] Send the second texture map and the second depth map to the VR terminal to achieve:

[0128] The VR terminal determines the opaque areas and the permeable areas according to the transparency map; for each pixel point in the opaque areas, it determines the first depth value of the pixel point in the first depth map and the second depth value of the pixel point in the second depth map; if the first depth value is less than the second depth value, it determines the first texture value of the pixel point in the first texture map and outputs it for display; if the first depth value is greater than or equal to the second depth value, it determines the second texture value of the pixel point in the second texture map and outputs it for display; for each pixel point in the permeable areas, it determines the third texture value of the pixel point in the first texture map and the fourth texture value in the second texture map; it determines the transparency value of the pixel point in the transparency map; it calculates the actual texture value of the pixel point according to the transparency value, the third texture value and the fourth texture value, and outputs it for display.

[0129] It should be noted that for the refined functions of each module provided in the embodiments of the present application, reference can be made to the corresponding disclosed parts in the embodiments of the above VR rendering method, which will not be elaborated here.

[0130] See Figure 7 , Figure 7 is another process schematic diagram of a VR rendering method provided by the embodiments of the present application. As Figure 7 shown, a VR rendering method provided by the embodiments of the present application is applied to the VR terminal 100. The VR rendering method may include steps S701 to S703, and the following will describe these steps in detail.

[0131] S701, renders a moving object in the target virtual space and sends a rendering request to the server. The rendering request includes the target moving position and the target viewing angle of the moving object in the moving two-dimensional plane when the moving object is rendered. The rendering result of the moving object includes the first two-dimensional map.

[0132] S702, receives the second two-dimensional map sent by the server. The server pre-renders the static scene in the target virtual space at different moving positions in the moving two-dimensional plane to obtain cube maps corresponding to different moving positions. The second two-dimensional map is calculated by the server according to the target cube map and the target viewing angle corresponding to the target moving position.

[0133] S703, synthesizes the picture content according to the first two-dimensional map and the second two-dimensional map.

[0134] In a possible implementation, the first two-dimensional map includes the first texture map, the first depth map and the transparency map. The cube map includes the texture cube map and the depth cube map. The target cube map includes the target texture cube map and the target depth cube map. The second two-dimensional map includes the second texture map and the second depth map;

[0135] The above-mentioned step S703, "synthesizing the screen content according to the first two-dimensional texture map and the second two-dimensional texture map", includes:

[0136] Determining the opaque area and the permeable area according to the transparency texture map;

[0137] For each pixel point in the opaque area, determining the first depth value of the pixel point in the first depth texture map and the second depth value of the pixel point in the second depth texture map; if the first depth value is less than the second depth value, determining the first texture value of the pixel point in the first texture map and outputting for display; if the first depth value is greater than or equal to the second depth value, determining the second texture value of the pixel point in the second texture map and outputting for display;

[0138] For each pixel point in the permeable area, determining the third texture value of the pixel point in the first texture map and the fourth texture value in the second texture map; determining the transparency value of the pixel point in the transparency texture map; calculating the actual texture value of the pixel point according to the transparency value, the third texture value and the fourth texture value, and outputting for display.

[0139] It should be noted that for the specific implementation of each step in the embodiments of the present application, reference can be made to the corresponding disclosed parts in the embodiments of the above VR rendering method, which will not be elaborated here.

[0140] See Figure 8 , Figure 8 which is another structural schematic diagram of a VR rendering device provided by the embodiments of the present application. As Figure 8 shown, a VR rendering device provided by the embodiments of the present application is applied to a VR terminal 100, and the device includes:

[0141] A dynamic rendering module 801, configured to render a moving object in a target virtual space and send a rendering request to the server. The rendering request includes the target moving position and the target viewing angle of the moving object in the moving two-dimensional plane when the moving object is rendered. The rendering result of the moving object includes a first two-dimensional texture map;

[0142] A screen synthesis module 802, configured to receive the second two-dimensional texture map sent by the server. The server pre-renders the static scene in the target virtual space at different moving positions in the moving two-dimensional plane to obtain cube texture maps corresponding to different moving positions. The second two-dimensional texture map is calculated by the server according to the target cube texture map and the target viewing angle corresponding to the target moving position; synthesizing the screen content according to the first two-dimensional texture map and the second two-dimensional texture map.

[0143] In a possible implementation, the first two-dimensional texture map includes a first texture map, a first depth map, and a transparency map; the cube map includes a texture cube map and a depth cube map; the target cube map includes a target texture cube map and a target depth cube map; and the second two-dimensional texture map includes a second texture map and a second depth map.

[0144] The frame synthesis module 802 for synthesizing frame content according to the first two-dimensional texture map and the second two-dimensional texture map is specifically configured to:

[0145] Determine the opaque area and the permeable area according to the transparency map; for each pixel point in the opaque area, determine the first depth value of the pixel point in the first depth map and the second depth value of the pixel point in the second depth map; if the first depth value is less than the second depth value, determine the first texture value of the pixel point in the first texture map and output it for display; if the first depth value is greater than or equal to the second depth value, determine the second texture value of the pixel point in the second texture map and output it for display; for each pixel point in the permeable area, determine the third texture value of the pixel point in the first texture map and the fourth texture value in the second texture map; determine the transparency value of the pixel point in the transparency map; calculate the actual texture value of the pixel point according to the transparency value, the third texture value, and the fourth texture value, and output it for display.

[0146] It should be noted that for the refined functions of each module in the embodiments of the present application, reference may be made to the corresponding disclosed parts in the embodiments of the above VR rendering method, which will not be elaborated here.

[0147] The embodiments of the present application also provide a computer program product including computer-readable instructions. When the computer-readable instructions run on an electronic device, the electronic device is enabled to implement any one of the VR rendering methods provided by the embodiments of the present application.

[0148] The embodiments of the present application also provide a computer-readable storage medium. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device is enabled to implement any one of the VR rendering methods provided by the embodiments of the present application.

[0149] In addition, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines.

[0150] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be various, such as analog circuits, digital circuits or dedicated circuits, etc. However, for the present application, in more cases, software program implementation is a better embodiment. Based on such an understanding, the technical solution of the present application, in essence or the part that makes contributions to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disc of a computer, etc., and includes several instructions to enable a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0151] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0152] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training device or data center to another website, computer, training device or data center in a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. A VR rendering method, characterized in that, The method is applied to a server and includes: Rendering a static scene of a target virtual space at different moving positions in a moving two-dimensional plane to obtain cubemaps corresponding to the different moving positions; Receiving a rendering request sent by a VR terminal, where the rendering request includes a target moving position and a target viewing angle of a moving object in the target virtual space when the moving object is rendered by the VR terminal, and the rendering result of the moving object by the VR terminal includes a first two-dimensional map; Determining a target cubemap corresponding to the target moving position, and calculating a second two-dimensional map of the target cubemap at the target viewing angle; Sending the second two-dimensional map to the VR terminal so that the VR terminal synthesizes picture content according to the first two-dimensional map and the second two-dimensional map.

2. The VR rendering method according to claim 1, wherein The rendering of the static scene of the target virtual space at different moving positions in the moving two-dimensional plane to obtain cubemaps corresponding to the different moving positions includes: Dividing the moving two-dimensional plane into a plurality of grid regions; Rendering a texture cubemap of each grid region in the static scene at a preset height, where the texture cubemap covers a 360° viewing angle; During the rendering process of the texture cubemap of each grid region, obtaining the depth value of each grid region and writing it into an image to synthesize a depth cubemap of each grid region; Establishing a mapping relationship between each grid region and its texture cubemap and depth cubemap; Correspondingly, the determining of the target cubemap corresponding to the target moving position includes: Determining a target grid region where the target moving position is located, and determining a target texture cubemap and a target depth cubemap corresponding to the target grid region based on the mapping relationship.

3. The VR rendering method according to claim 2, wherein The first two-dimensional map includes a first texture map, a first depth map, and a transparency map; The calculating of the second two-dimensional map of the target cubemap at the target viewing angle includes: Generating a first rendering background layer by covering the target texture cubemap on the inner surface of a first spherical grid body, and calculating a second texture map of the first rendering background layer at the target viewing angle; Generating a second rendering background layer by covering the target depth cubemap on the inner surface of a second spherical grid body, and calculating a second depth map of the second rendering background layer at the target viewing angle.

4. The VR rendering method according to claim 3, wherein The sending of the second two-dimensional map to the VR terminal so that the VR terminal synthesizes picture content according to the first two-dimensional map and the second two-dimensional map includes: Sending the second texture map and the second depth map to the VR terminal to achieve: The VR terminal determines an opaque area and a transmissive area according to the transparency map; for each pixel point in the opaque area, determine a first depth value of the pixel point in the first depth map and a second depth value of the pixel point in the second depth map; if the first depth value is less than the second depth value, determine a first texture value of the pixel point in the first texture map and output it for display; if the first depth value is greater than or equal to the second depth value, determine a second texture value of the pixel point in the second texture map and output it for display; for each pixel point in the transmissive area, determine a third texture value of the pixel point in the first texture map and a fourth texture value of the pixel point in the second texture map; determine a transparency value of the pixel point in the transparency map; calculate an actual texture value of the pixel point according to the transparency value, the third texture value and the fourth texture value, and output it for display.

5. A VR rendering device, characterized in that, The device is applied to a server, and the device includes: A preprocessing module for rendering a static scene of a target virtual space at different moving positions in a moving two-dimensional plane to obtain cube maps corresponding to different moving positions; A static rendering module for receiving a rendering request sent by the VR terminal, where the rendering request includes a target moving position and a target viewing angle of a moving object in the target virtual space when the moving object is rendered by the VR terminal, and the rendering result of the moving object by the VR terminal includes a first two-dimensional map; determine a target cube map corresponding to the target moving position, and calculate a second two-dimensional map of the target cube map at the target viewing angle; send the second two-dimensional map to the VR terminal so that the VR terminal synthesizes the picture content according to the first two-dimensional map and the second two-dimensional map.

6. A VR rendering method, characterized in that, The method is applied to a VR terminal and includes: Render a moving object in a target virtual space and send a rendering request to the server, where the rendering request includes a target moving position and a target viewing angle of the moving object in a moving two-dimensional plane when the moving object is rendered, and the rendering result of the moving object includes a first two-dimensional map; Receive a second two-dimensional map sent by the server, where the server pre-renders a static scene in the target virtual space at different moving positions in the moving two-dimensional plane to obtain cube maps corresponding to different moving positions, and the second two-dimensional map is calculated by the server according to the target cube map corresponding to the target moving position and the target viewing angle; Synthesize the picture content according to the first two-dimensional map and the second two-dimensional map.

7. The VR rendering method according to claim 6, wherein The first two-dimensional map includes a first texture map, a first depth map and a transparency map, the cube map includes a texture cube map and a depth cube map, the target cube map includes a target texture cube map and a target depth cube map, and the second two-dimensional map includes a second texture map and a second depth map; The synthesizing the picture content according to the first two-dimensional map and the second two-dimensional map includes: Determine the opaque regions and the transmissive regions according to the transparency map; For each pixel point in the opaque regions, determine the first depth value of the pixel point in the first depth map and the second depth value of the pixel point in the second depth map; if the first depth value is less than the second depth value, determine the first texture value of the pixel point in the first texture map and output it for display; if the first depth value is greater than or equal to the second depth value, determine the second texture value of the pixel point in the second texture map and output it for display; For each pixel point in the transmissive regions, determine the third texture value of the pixel point in the first texture map and the fourth texture value of the pixel point in the second texture map; determine the transparency value of the pixel point in the transparency map; calculate the actual texture value of the pixel point according to the transparency value, the third texture value and the fourth texture value, and output it for display.

8. A VR rendering device, characterized in that, The device is applied to a VR terminal, and the device includes: A dynamic rendering module, configured to render a moving object in a target virtual space and send a rendering request to a server, where the rendering request includes the target moving position and the target viewing angle of the moving object in a moving two-dimensional plane when the moving object is rendered, and the rendering result of the moving object includes a first two-dimensional map; A frame synthesis module, configured to receive the second two-dimensional map sent by the server, where the server pre-renders a static scene in the target virtual space at different moving positions in the moving two-dimensional plane to obtain a cube map corresponding to each different moving position, and the second two-dimensional map is calculated by the server according to the target cube map corresponding to the target moving position and the target viewing angle; synthesize the frame content according to the first two-dimensional map and the second two-dimensional map.

9. A computer program product, characterized in that, Includes computer-readable instructions, which when running on an electronic device, cause the electronic device to implement the VR rendering method according to any one of claims 1, 2, 3, 4, 6, 7.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs, which when executed by an electronic device, can cause the electronic device to implement the VR rendering method according to any one of claims 1, 2, 3, 4, 6, 7.

Images