Virtual reality binocular rendering acceleration method and system

Abstract

The invention discloses a virtual reality binocular rendering acceleration method, which comprises the following steps of: 1, acquiring low-resolution motion vectors of a view angle needing to be rendered and a view angle without being rendered, and a projection vector between the two view angles; 2, generating a historical high-resolution picture of each view angle in the current frame; step 3, collecting a low-resolution picture of a view angle needing to be rendered of the current frame; 4, reconstructing a high-resolution picture of which the view angle needs to be rendered in the current frame; step 5, aligning the high-resolution picture obtained in the step 4 to a non-rendering visual angle through the projection vector of the current frame so as to generate a reference picture, and reconstructing a high-resolution picture without a rendering visual angle in the current frame by using the reference picture; and step 6, exchanging the viewing angle needing to be rendered and the viewing angle not needing to be rendered in the current frame so as to reconstruct a high-resolution picture with two viewing angles in the next frame. The invention also provides a virtual reality binocular rendering acceleration system. According to the method provided by the invention, the binocular image rendering frame rate is effectively optimized.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image processing, and in particular relates to a virtual reality binocular rendering acceleration method and system. Background Art

[0002] In the field of virtual reality (VR) and mixed reality (MR), high-quality, low-latency binocular rendering results can provide users with a highly immersive experience. However, this often means high hardware performance requirements and high power consumption. Most mobile VR devices are limited by cost and weight and often cannot meet these requirements. Therefore, some acceleration algorithms designed specifically for binocular rendering application scenarios are very necessary.

[0003] Existing binocular rendering acceleration methods usually use forward deformation technology to reuse the redundant spatial information inherent in the binocular perspective and generate most of the picture from one perspective from another perspective.

[0004] Patent document CN107330846A discloses a binocular rendering pipeline process and method based on screen block pairs, including completing space division according to the spatial relationship between two perspectives in stereo rendering, and generating a corresponding input primitive list for the divided space; searching for non-empty primitive lists of spatial divisions and obtaining surfaces with spatial consistency; scheduling all generated spatial divisions, and performing rasterization and drawing of two perspectives for the primitives in each spatial division at the same time. These new measures take into account the spatial correlation of the two perspectives in stereo rendering at the same time. During the drawing process, the two perspectives are rasterized and drawn at the same time, reducing the bandwidth required to read triangle data multiple times during the drawing process.

[0005] Patent document CN110060351A discloses a dynamic three-dimensional character reconstruction and live broadcast method based on an RGBD camera, including step S1: performing real-time matting on the collected RGBD image to cut out the character part of the RGBD image; step S2: compressing the matted RGBD image and using a video streaming protocol for real-time streaming; step S3: receiving the matting result of the RGBD image on a mobile terminal, decompressing it, generating a three-dimensional point cloud, connecting the point cloud into a surface, and obtaining a three-dimensional model of the human body; step S4: performing binocular rendering on the three-dimensional model on the mobile terminal, and the image with stereoscopic and AR effects can be seen by wearing a three-dimensional display device.

[0006] The existing binocular rendering acceleration methods mainly have the following problems: First, rendering data is not fully reused for binocular rendering application scenarios. Most of the current algorithms only reuse historical frames or another perspective, but there is no method to reuse historical frames and another perspective at the same time. This results in the generated picture still having a large area of ​​unblocked area, which needs to be redrawn through an additional high-cost rendering link, or filled using heuristic methods. However, the heuristic filling method will introduce obvious visual artifacts when filling large areas.

[0007] Secondly, these methods cannot correctly handle illumination changes that are not reflected in geometry because they reuse a single data source. The method that reuses historical data cannot correctly handle changes in shadows, and the method that reuses another perspective cannot correctly handle reflections and highlights observed from different perspectives. Summary of the invention

[0008] The purpose of the present invention is to provide a virtual reality binocular rendering acceleration method and system, which optimizes the image rendering process to achieve the task of rendering binocular images by only rendering a single-eye image at a low resolution, thereby improving the rendering frame rate.

[0009] In order to achieve the first object of the present invention, the following technical solution is provided: a virtual reality binocular rendering acceleration method, comprising the following steps: Step 1: Based on the real-time deferred rendering pipeline, a first low-resolution motion vector of the perspective to be rendered and a second low-resolution motion vector of the perspective not to be rendered in the current frame are collected, as well as a projection vector correspondingly mapped between the picture in the perspective to be rendered and the picture in the perspective not to be rendered; Step 2: Obtain a previous frame of historical high-resolution images corresponding to the viewing angles that need to be rendered and the viewing angles that do not need to be rendered in the previous frame, and reversely deform the previous frame of high-resolution images of the corresponding viewing angles based on the first low-resolution motion vector and the second low-resolution motion vector in the current frame and align them to the corresponding viewing angles, so as to generate a first historical high-resolution image of the viewing angle that needs to be rendered and a second historical high-resolution image of the viewing angle that does not need to be rendered in the current frame; Step 3: Based on the real-time deferred rendering pipeline, collect the low-resolution image of the viewing angle to be rendered in the current frame; Step 4: using the first historical high-resolution picture and the low-resolution picture to reconstruct the first high-resolution picture of the viewing angle to be rendered in the current frame; Step 5: Align the reconstructed first high-resolution picture to a viewing angle that does not require rendering by using the projection vector in the current frame to generate a reference picture; Using the reference picture and the second historical high-resolution picture, to reconstruct a second high-resolution picture in the current frame without rendering perspective; Step 6: For the next frame, swap the perspective that needs to be rendered with the perspective that does not need to be rendered in the current frame, and repeat steps 1 to 5 to reconstruct a high-resolution image of the two perspectives for the next frame.

[0010] The present invention aligns the synthesized high-resolution picture of the current frame that needs rendering to the perspective that does not require rendering through a projection vector, and combines the historical perspective pictures that do not require rendering that are aligned to the current frame to generate a high-resolution picture of the perspective that does not require rendering, thereby improving the rendering frame rate while ensuring the quality of the binocular picture.

[0011] Specifically, the process of obtaining the projection vector is as follows: Obtaining homogeneous coordinates of a pixel point in the eye screen space corresponding to the viewing angle without rendering, wherein the homogeneous coordinates include the pixel coordinates and corresponding depth information; Performing a reverse projection transformation on the homogeneous coordinates using the eyeball cameras corresponding to the viewing angle that does not need to be rendered and the viewing angle that needs to be rendered, and outputting first coordinates and second coordinates corresponding to the two viewing angles respectively; Subtract the first coordinate from the second coordinate and scale to [-1, 1] to get the projection vector.

[0012] Specifically, in step 2, the low-resolution motion vector of the current frame's high-resolution image is upsampled by the texture bilinear acceleration sampling method to obtain a high-resolution motion vector, and the previous frame's historical high-resolution image is reversely deformed based on the high-resolution motion vector to obtain an aligned historical reference image.

[0013] Specifically, in step 4, when the first high-resolution image is reconstructed, the first historical high-resolution image needs to be corrected, and the process is as follows: For each pixel in the first historical high-resolution picture, its color is confined to an axis-aligned bounding box composed of the corresponding pixel color neighborhood in the low-resolution picture to obtain a corrected first historical high-resolution picture, wherein the correction weight is controlled by the modulus of the motion vector; According to the distance between the historical sampling point and the new sampling point in the low-resolution picture, upsampling the low-resolution picture by bilinear interpolation to obtain a directly upsampled high-resolution reference picture; Based on the obtained distance and the high-resolution reference picture, a corresponding high-resolution picture is reconstructed.

[0014] Specifically, the correction expression is as follows: in, Indicates the motion vector of the viewing angle to be rendered relative to the previous frame. Indicates adjustable parameters. Indicates that The value is limited to and Within the range, It is used to correct the weight. It shows the first historical high-resolution picture after correction, Shows the first historical high-resolution picture, Indicates the low-resolution view that needs to be rendered.

[0015] Specifically, based on the obtained distance and the high-resolution reference picture, the corresponding high-resolution picture is reconstructed, and its expression is as follows: in, Represents the high-resolution image of the synthetic perspective to be rendered. Learned through data-driven methods, This is the first historical high-resolution picture after correction. It is a low-resolution picture that needs to be rendered from a perspective High-resolution reference image obtained directly through bilinear upsampling.

[0016] Specifically, in step 5, the reconstruction expression of the second high-resolution picture is as follows: in, Represents a high-resolution image that is synthesized without rendering the viewpoint. The coordinates for the second auxiliary screen are Pixels on The third auxiliary screen is filled in at coordinates Pixels on represents the brightness difference between pixels at corresponding positions in the second auxiliary picture and the third auxiliary picture, For the coordinates The projection vector corresponding to the mapping between the viewing angle that needs to be rendered and the viewing angle that does not need to be rendered, For coordinates The second lowest resolution motion vector for the perspective that does not need to be rendered.

[0017] Specifically, using the projection vector correspondingly mapped between the picture in the perspective to be rendered and the picture in the perspective not to be rendered obtained in step 1, the first high-resolution picture in the perspective to be rendered obtained in step 4 is reversely deformed to obtain a reference picture, and the depth of the reference picture is compared with the depth information of the perspective not to be rendered collected in the deferred rendering pipeline, and pixels with a large depth difference are marked to obtain a first auxiliary picture, and the marked pixels are the unblocked areas; In the same way, the second historical high-resolution picture without rendering perspective is compared and marked with the depth information of the rendering-free perspective to obtain a second auxiliary picture. The marked unblocked area in the first auxiliary picture is filled with pixels in the second auxiliary picture at the corresponding position, and the depth-based multi-level filtering method is continued to be used to fill the unblocked area still existing in the first auxiliary picture for a second time to obtain a third auxiliary picture.

[0018] In order to achieve the second object of the present invention, the following technical solution is provided: a virtual reality binocular rendering acceleration system, used to execute the steps of the above virtual reality binocular rendering acceleration method, comprising a projection vector calculation unit, a picture deformation unit, a low-resolution rendering frame super-resolution unit and a target perspective picture generation unit; The vector calculation unit is used to obtain the low-resolution motion vector of the perspective that does not need to be rendered and the perspective that does not need to be rendered in the current frame, and the projection vector between the two perspectives; The picture deformation unit reversely deforms the historical frame pictures of each perspective based on the low-resolution motion vectors of each perspective to obtain the historical high-resolution picture of the current frame at the corresponding perspective; The low-resolution rendering frame super-resolution unit aligns the historical high-resolution picture of the current frame under the viewing angle to be rendered and the low-resolution picture of the current frame under the viewing angle to be rendered based on the low-resolution motion vector of the viewing angle to be rendered, so as to generate a high-resolution picture of the current frame under the viewing angle to be rendered; The target perspective picture generation unit converts the high-resolution picture of the perspective that needs to be rendered into a corresponding reference picture based on the projection vector, and aligns the historical high-resolution picture of the perspective that does not need to be rendered with the reference picture through the motion vector of the perspective that does not need to be rendered, so as to generate a high-resolution picture of the current frame of the perspective that does not need to be rendered.

[0019] Compared with the prior art, the present invention has the following beneficial effects: An algorithm is proposed to comprehensively reuse historical frames and another perspective to generate the target perspective image. It only needs to render the image in the single camera, and can be used to quickly align the rendered perspective image to the non-rendered perspective using reverse deformation based on the projection vector. This can properly handle geometry-independent lighting details such as shadows, reflections, and highlights, and effectively avoid screen flickering and ghosting problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of the virtual reality binocular rendering acceleration method provided in this embodiment; Figure 2 A schematic diagram of projection vector calculation provided in this embodiment; Figure 3 A schematic diagram of alternate rendering perspectives and no rendering perspectives provided in this embodiment. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] The present invention performs virtual reality binocular rendering acceleration based on image reprojection and spatiotemporal supersampling. In this embodiment, the left-eye image and the right-eye image in the VR device are rendered to specifically illustrate the technical solution of the present invention.

[0023] like Figure 1 As shown in FIG. 1 , a virtual reality binocular rendering acceleration method provided by this embodiment is provided, and the steps are as follows: Step 1: Based on the real-time deferred rendering pipeline, collect the low-resolution motion vector of the binocular perspective of the tth frame and the projection vector of the conversion between the binocular perspective images; More specifically, in this round of rendering processing, the left eye perspective is the perspective that needs to be rendered, and the right eye perspective is the perspective that does not need to be rendered.

[0024] The method of collecting the projection vector in this round is as follows Figure 2 As shown, the details are as follows: For a pixel in the right eye screen space , whose x and y components are in the range [0, 1], and the corresponding depth information is , which also ranges from [0,1], and its homogeneous coordinates are expressed in normalized device coordinates (NDC) as: ; According to the parameters of the right eye camera, including the projection matrix and the view matrix, Perform an inverse projection transformation to restore the pixels in the right eye screen space to the coordinates in the three-dimensional world space. Project the world coordinates to the left eye camera to obtain the coordinates of the pixel in the left eye screen space. Subtract the two coordinates and scale them to [-1, 1] to get the projection vector of a single pixel: The superscript n represents the normalized device coordinates (NDC), represents the projection matrix, Represents the view matrix.

[0025] Step 2, obtain the previous frame historical high-resolution picture corresponding to the left eye perspective and the right eye perspective in the previous frame, and reversely deform the previous frame high-resolution picture of the corresponding perspective based on the first low-resolution motion vector and the second low-resolution motion vector in the current frame and align it to the corresponding perspective, so as to generate the first historical high-resolution picture of the left eye perspective and the second historical high-resolution picture of the right eye perspective in the current frame; more specifically, since the historical frame is high-resolution and the collected motion vector is low-resolution, the motion vector needs to be upsampled before using reverse deformation. To ensure the efficiency and high quality of upsampling, this embodiment only introduces the texture bilinear acceleration sampling method supported by modern GPUs, but similar methods, such as the nearest neighbor and bicubic sampling schemes, are all protected by this patent. According to the upsampled high-resolution motion vectors, the historical frames of both eyes are reversely deformed respectively to obtain pictures aligned to the current frame.

[0026] Step 3: Based on the real-time deferred rendering pipeline, render the low-resolution image from the left eye's perspective. Since the computational overhead of directly rendering a high-resolution image is too high, especially when the scene is complex, in order to minimize the rendering overhead, only the low-resolution image of the left eye is rendered here.

[0027] Step 4: Reconstructing the first high-resolution picture of the left eye perspective in the current frame by using the first historical high-resolution picture and the low-resolution picture; Since not all pixels can find references in the historical frames, the historical high-resolution images aligned by reverse deformation have some perspective artifacts. Directly using them to merge with the rendered low-resolution images will result in obvious perspective noise. Before fusion, the aligned historical images are corrected. The correction process uses the axis-aligned bounding box method to limit the historical pixel colors to the low-resolution rendering pixel color neighborhood, and linearly interpolate with the original aligned historical pixels. Considering that pixels with large motion between the previous and next frames are not very reliable in the historical frames, the modulus of the motion vector is used as the parameter for linear interpolation: The corrected historical pixels are further combined with the bilinear interpolation to upsample the low-resolution image based on the distance between the historical sampling points and the new sampling points in the low-resolution image. What you get Perform linear interpolation to obtain the final target left eye high-resolution pixel: in Learned through data-driven methods.

[0028] Step 5: aligning the reconstructed first high-resolution picture to the right eye perspective through the projection vector in the current frame to generate a reference picture; Using the reference picture and the second historical high-resolution picture, reconstructing the second high-resolution picture of the right eye perspective in the current frame; More specifically, the image is reversely deformed using the projection vector upsampled by bilinear interpolation to increase the resolution of the left eye. Align to the right eye of the current frame, use depth as the criterion for whether the aligned pixel can be reused, and if there is a large difference between the depth value of the pixel and the target pixel, mark it as a deoccluded area, and get , also marked the historical right eye high resolution after alignment .

[0029] By merging the non-unoccluded regions of the two aligned frames, that is, The non-unblocked area is filled to By aligning the unoccluded areas of the two frames, most of the unoccluded areas of the generated image can be eliminated, leaving only small holes that are difficult to detect. These holes can be further filled with low-frequency background information through a depth-based multi-level filtering method to obtain a high-resolution image for the right eye without holes. .

[0030] Since the corresponding pixels of the left and right viewing angles may have color differences due to the existence of non-Lambertian surface reflection, the generated results need to be further corrected. The reliability of the multiplexed pixels is judged according to the relative size relationship between the projection vector and the motion vector. and Brightness difference A larger area is color corrected to obtain the final right eye image: in, Represents a high-resolution image that is synthesized without rendering the viewpoint. The coordinates for the second auxiliary screen are Pixels on The third auxiliary screen is filled in at coordinates Pixels on represents the brightness difference between pixels at corresponding positions in the second auxiliary picture and the third auxiliary picture, For the coordinates The projection vector corresponding to the mapping between the viewing angle that needs to be rendered and the viewing angle that does not need to be rendered, For coordinates The second lowest resolution motion vector for the perspective that does not need to be rendered.

[0031] The second auxiliary picture and the third auxiliary picture are obtained as follows: using the projection vector correspondingly mapped between the picture in the perspective to be rendered and the picture in the perspective not to be rendered obtained in step 1, the first high-resolution picture of the perspective to be rendered obtained in step 4 is reversely deformed to obtain a reference picture, and the depth of the reference picture is compared with the depth information of the perspective not to be rendered collected in the deferred rendering pipeline, and pixels with large depth differences are marked to obtain a first auxiliary picture, and the marked pixels are the unblocked areas; In the same way, the second historical high-resolution picture without rendering perspective is compared and marked with the depth information of the rendering-free perspective to obtain a second auxiliary picture. The marked unblocked area in the first auxiliary picture is filled with pixels in the second auxiliary picture at the corresponding position, and the depth-based multi-level filtering method is continued to be used to fill the unblocked area still existing in the first auxiliary picture for a second time to obtain a third auxiliary picture.

[0032] Step 6: For the next frame, Figure 3 As shown, the perspective that needs to be rendered and the perspective that does not need to be rendered in the current frame are swapped, and steps 1 to 5 are repeated to reconstruct a high-resolution image of the two perspectives for the next frame. By alternating the perspective that needs to be rendered and the perspective that does not need to be rendered, the historical frames are updated in a timely manner, avoiding the accumulation of unblocked areas over time.

[0033] The present embodiment also provides a virtual reality binocular rendering acceleration system, which is used to execute the steps of the virtual reality binocular rendering acceleration method provided in the above embodiment, and includes a projection vector calculation unit, a picture deformation unit, a low-resolution rendering frame super-resolution unit and a target perspective picture generation unit.

[0034] The vector calculation unit is used to obtain the motion vectors of the perspective that does not need to be rendered and the perspective that does not need to be rendered in the current frame, and the projection vector between the two perspectives; The picture deformation unit reversely deforms the historical frame pictures of each perspective based on the motion vector of each perspective to obtain the historical high-resolution picture of the current frame at the corresponding perspective; The low-resolution rendering frame super-resolution unit aligns the historical high-resolution picture of the current frame under the viewing angle to be rendered and the low-resolution picture of the current frame under the viewing angle to be rendered based on the motion vector of the viewing angle to be rendered, so as to generate a high-resolution picture of the current frame under the viewing angle to be rendered; The target perspective picture generation unit converts the high-resolution picture of the perspective that needs to be rendered into a corresponding reference picture based on the projection vector, and aligns the historical high-resolution picture of the perspective that does not need to be rendered with the reference picture through the motion vector of the perspective that does not need to be rendered, so as to generate a high-resolution picture of the current frame of the perspective that does not need to be rendered.

[0035] In summary, the algorithm proposed in the present invention mainly includes six key steps: the first is the calculation of motion vector and projection vector, which ensures the accuracy of reverse deformation through precise calculation; the second is to render only the low-resolution picture of the monocular eye to reduce the rendering cost; the third is the deformation of the binocular historical rendering frame, and align the historical rendering frame to the current frame; the fourth is to use the low-resolution rendering frame and the high-resolution historical frame to synthesize the high-resolution picture of the current frame that needs to be rendered; the fifth is to align the synthesized high-resolution picture of the current frame that needs to be rendered to the perspective that does not need to be rendered through the projection vector, and combine the historical perspective that does not need to be rendered to the current frame to generate a high-resolution picture of the perspective that does not need to be rendered; the sixth is to exchange the rendering perspective and the perspective that does not need to be rendered in the next frame to reduce the accumulation of the unblocked area over time. These six steps are interconnected to ensure the high quality and efficient generation of binocular pictures.

[0036] In addition, the terms "upper", "lower", "inner", "outer", "front", "rear" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps described in these embodiments do not limit the scope of the present invention.

[0037] Of course, the above description is only a specific embodiment of the present invention and is not intended to limit the scope of implementation of the present invention. All equivalent changes or modifications made according to the structure, characteristics and principles described in the patent application scope of the present invention should be included in the patent application scope of the present invention.

[0038] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A virtual reality binocular rendering acceleration method, characterized in that: The following steps are involved: Step 1: Based on the real-time deferred rendering pipeline, a first low-resolution motion vector of the perspective to be rendered and a second low-resolution motion vector of the perspective not to be rendered in the current frame are collected, as well as a projection vector correspondingly mapped between the picture in the perspective to be rendered and the picture in the perspective not to be rendered; Step 2: Obtain a previous frame of historical high-resolution images corresponding to the viewing angles that need to be rendered and the viewing angles that do not need to be rendered in the previous frame, and reversely deform the previous frame of high-resolution images of the corresponding viewing angles based on the first low-resolution motion vector and the second low-resolution motion vector in the current frame and align them to the corresponding viewing angles, so as to generate a first historical high-resolution image of the viewing angle that needs to be rendered and a second historical high-resolution image of the viewing angle that does not need to be rendered in the current frame; Step 3: Based on the real-time deferred rendering pipeline, collect the low-resolution image of the viewing angle to be rendered in the current frame; Step 4: Using the first historical high-resolution picture and the low-resolution picture, reconstruct a first high-resolution picture of the viewing angle to be rendered in the current frame; Step 5: Align the reconstructed first high-resolution picture to a viewing angle that does not require rendering by using the projection vector in the current frame to generate a reference picture; Using the reference picture and the second historical high-resolution picture, to reconstruct a second high-resolution picture in the current frame without rendering perspective; Step 6: For the next frame, swap the perspective that needs to be rendered with the perspective that does not need to be rendered in the current frame, and repeat steps 1 to 5 to reconstruct a high-resolution image of the two perspectives for the next frame.

2. The virtual reality binocular rendering acceleration method according to claim 1, characterized in that: The process of obtaining the projection vector is as follows: Obtaining homogeneous coordinates of a pixel point in the eye screen space corresponding to the viewing angle without rendering, wherein the homogeneous coordinates include the pixel coordinates and corresponding depth information; Performing a reverse projection transformation on the homogeneous coordinates using the eyeball cameras corresponding to the viewing angle that does not need to be rendered and the viewing angle that needs to be rendered, and outputting first coordinates and second coordinates corresponding to the two viewing angles respectively; Subtract the first coordinate from the second coordinate and scale to [-1, 1] to get the projection vector.

3. The virtual reality binocular rendering acceleration method according to claim 1, characterized in that: In step 2, the low-resolution motion vector of the current frame's high-resolution picture is upsampled by the texture bilinear acceleration sampling method to obtain a high-resolution motion vector, and the previous frame's historical high-resolution picture is reversely deformed based on the high-resolution motion vector to obtain an aligned historical reference image.

4. The virtual reality binocular rendering acceleration method according to claim 1, characterized in that: In step 4, when the first high-resolution image is reconstructed, the first historical high-resolution image needs to be corrected, and the process is as follows: For each pixel in the first historical high-resolution picture, its color is confined to an axis-aligned bounding box composed of the corresponding pixel color neighborhood in the low-resolution picture to obtain a corrected first historical high-resolution picture, wherein the correction weight is controlled by the modulus of the motion vector; According to the distance between the historical sampling point and the new sampling point in the low-resolution picture, upsampling the low-resolution picture by bilinear interpolation to obtain a directly upsampled high-resolution reference picture; Based on the obtained distance and the high-resolution reference picture, a corresponding high-resolution picture is reconstructed.

5. The virtual reality binocular rendering acceleration method according to claim 4, characterized in that: The correction expression is as follows: in, Indicates the motion vector of the viewing angle to be rendered relative to the previous frame. Indicates adjustable parameters. Indicates that The value is limited to and Within the range, It is used to correct the weight. It shows the first historical high-resolution picture after correction, Shows the first historical high-resolution picture, Indicates the low-resolution image that needs to be rendered.

6. The virtual reality binocular rendering acceleration method according to claim 4, characterized in that: Based on the obtained distance and high-resolution reference picture, the corresponding high-resolution picture is reconstructed, and its expression is as follows: in, Represents the high-resolution image of the synthetic perspective to be rendered. Learned through data-driven methods, This is the first historical high-resolution picture after correction. It is a low-resolution picture that needs to be rendered from a perspective High-resolution reference image obtained directly through bilinear upsampling.

7. The virtual reality binocular rendering acceleration method according to claim 1, characterized in that: In step 5, the reconstruction expression of the second high-resolution picture is as follows: in, Represents a high-resolution image that is synthesized without rendering the viewpoint. The coordinates for the second auxiliary screen are Pixels on The third auxiliary screen is filled in at coordinates Pixels on represents the brightness difference between pixels at corresponding positions in the second auxiliary picture and the third auxiliary picture, For the coordinates The projection vector corresponding to the mapping between the viewing angle that needs to be rendered and the viewing angle that does not need to be rendered, For coordinates The second lowest resolution motion vector for the perspective that does not need to be rendered.

8. A virtual reality binocular rendering acceleration system, characterized in that: The method for accelerating the virtual reality binocular rendering according to any one of claims 1 to 7 comprises a projection vector calculation unit, a picture deformation unit, a low-resolution rendering frame super-resolution unit, and a target perspective picture generation unit; The vector calculation unit is used to obtain the low-resolution motion vector of the perspective that does not need to be rendered and the perspective that does not need to be rendered in the current frame, and the projection vector between the two perspectives; The picture deformation unit reversely deforms the historical frame pictures of each perspective based on the low-resolution motion vectors of each perspective to obtain the historical high-resolution picture of the current frame at the corresponding perspective; The low-resolution rendering frame super-resolution unit aligns the historical high-resolution picture of the current frame under the viewing angle to be rendered and the low-resolution picture of the current frame under the viewing angle to be rendered based on the low-resolution motion vector of the viewing angle to be rendered, so as to generate a high-resolution picture of the current frame under the viewing angle to be rendered; The target perspective picture generation unit converts the high-resolution picture of the perspective that needs to be rendered into a corresponding reference picture based on the projection vector, and aligns the historical high-resolution picture of the perspective that does not need to be rendered with the reference picture through the motion vector of the perspective that does not need to be rendered, so as to generate a high-resolution picture of the current frame of the perspective that does not need to be rendered.

Images