A virtual reality distortion correction method, device, and related equipment

Abstract

An embodiment of the present invention provides a virtual reality anti-distortion method, the method comprising: reading vertex data of an image to be rendered; performing image rendering on the vertex data, performing anti-distortion processing on the vertex data using an anti-distortion algorithm during the image rendering process to obtain image frame data, and outputting an image based on the image frame data. The method proposes integrating anti-distortion processing into an image rendering, so that the image frame data obtained after the image rendering is already anti-distorted image frame data, and the image outputted based on the image frame data is an image with an anti-distortion effect. Therefore, the method can save the computational effort of the graphics processor and improve the display frame rate of the virtual reality device. In addition, the virtual reality anti-distortion device and related equipment corresponding to the embodiment of the present application can achieve the same technical effect.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to virtual reality anti-distortion technology. Background Art

[0002] Virtual Reality (VR) technology refers to a technology that uses computers or other intelligent computing devices to simulate a three-dimensional virtual world, which can provide users with a simulated environment for senses such as vision, hearing, and touch, making users feel as if they are in the scene.

[0003] In order to give users a real sense of immersion, VR devices must cover the visual range of the human eye as much as possible, so a specific spherical curvature lens, namely a VR lens, needs to be installed in the VR device. However, when VR lenses project traditional images into human eyes, the images are distorted, and the human eye cannot obtain positioning in the virtual space. In other words, in virtual reality, you are surrounded by a funhouse mirror space with distorted images all around. This phenomenon is called VR lens distortion.

[0004] However, for VR devices, VR lens distortion is inevitable, and as the field of view increases, the edge image distortion will become more obvious. Due to the existence of distortion, the picture effect of VR devices cannot be viewed normally. Summary of the invention

[0005] Based on this, the present invention provides a virtual reality anti-distortion method, which integrates anti-distortion processing into the image rendering process, that is, interspersed with anti-distortion processing when rendering the image, so that the image obtained after one rendering is already an image that has been anti-distortion processed. The image can be directly rendered to the screen, thereby solving the problem of poor picture effects caused by VR lens distortion.

[0006] In view of this, a first aspect of the present invention provides a method for VR anti-distortion, the method comprising:

[0007] Read the vertex data of the image to be rendered;

[0008] Performing image rendering on the vertex data, and in the process of the image rendering, performing anti-distortion processing on the vertex data using an anti-distortion algorithm to obtain image frame data;

[0009] An image is output according to the image frame data.

[0010] A second aspect of the present invention provides a VR anti-distortion device, the device comprising:

[0011] A reading module, used for reading vertex data of an image to be rendered;

[0012] A rendering and anti-distortion module, used for performing image rendering on the vertex data, and in the process of the image rendering, using an anti-distortion algorithm to perform anti-distortion processing on the vertex data to obtain image frame data;

[0013] An output module is used to output an image according to the image frame data.

[0014] A third aspect of the present invention provides an intelligent terminal, the intelligent terminal comprising:

[0015] Memory and processor;

[0016] The memory is used to store programs and virtual reality applications;

[0017] The processor is used to read the program in the memory when running the virtual reality application, and the program is used to:

[0018] Read the vertex data of the image to be rendered;

[0019] Performing image rendering on the vertex data, and in the process of the image rendering, performing anti-distortion processing on the vertex data using an anti-distortion algorithm to obtain image frame data;

[0020] An image is output according to the image frame data.

[0021] A fourth aspect of the present invention provides a VR display system, the VR display system comprising:

[0022] A smart terminal and virtual reality glasses; the virtual reality glasses and the smart phone communicate with each other using near field communication technology;

[0023] The smart terminal is loaded with a virtual reality application. When the smart terminal runs the virtual reality application, the following procedures are executed:

[0024] Read the vertex data of the image to be rendered;

[0025] Performing image rendering on the vertex data, and in the process of the image rendering, performing anti-distortion processing on the vertex data using an anti-distortion algorithm to obtain image frame data;

[0026] An image is output according to the image frame data.

[0027] A fifth aspect of the present invention provides a VR device, wherein the VR device is an all-in-one VR device, comprising:

[0028] Memory, processor and display unit;

[0029] The memory is used to store programs and virtual reality applications;

[0030] The processor is used to read the program in the memory when running the virtual reality application, and the program is used to:

[0031] Read the vertex data of the image to be rendered;

[0032] Performing image rendering on the vertex data, and in the process of the image rendering, performing anti-distortion processing on the vertex data using an anti-distortion algorithm to obtain image frame data;

[0033] An image is output according to the image frame data.

[0034] A sixth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method described in the first aspect above.

[0035] A seventh aspect of the present invention provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0036] It can be seen from the above technical solution that the embodiment of the present invention has the following advantages:

[0037] By using the virtual reality anti-distortion method provided by the embodiment of the present invention, the anti-distortion processing of the image is interspersed in the image rendering process, that is, the anti-distortion processing of the image is integrated into the traditional image rendering process, so that the image frame data obtained after one image rendering is already the image frame data after the anti-distortion processing, and the image output according to the image frame data is an image with anti-distortion effect; specifically, the vertex data of the image to be rendered is read first; then the image rendering is performed on the vertex data, and the anti-distortion algorithm is used to perform anti-distortion processing on the vertex data during the image rendering process to obtain the image frame data, and the image is output according to the image frame data. Based on this, the image viewed by the user through the VR lens is a normal image, thereby solving the problem of image distortion caused by the distortion phenomenon of the VR lens; in addition, the method integrates the anti-distortion processing into the image rendering process, so that no additional cache is required, no secondary rendering is required, the calculation amount of the graphics processor is reduced, and the display frame rate of the virtual reality device is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0039] Figure 1Schematic diagram of VR lens distortion phenomenon in the application scenario of the present invention;

[0040] Figure 2 A schematic diagram of an implementation framework of a VR anti-distortion method in an application scenario of the present invention;

[0041] Figure 3 This is an example diagram of a practical scene of the VR anti-distortion processing method in an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of a flow chart of a VR anti-distortion method in an embodiment of the present invention;

[0043] Figure 5 For Figure 4 A flowchart of an implementation method of step 402;

[0044] Figure 6 Schematic diagram of the structure of a VR anti-distortion device in an embodiment of the present invention;

[0045] Figure 7 Schematic diagram of the structure of another VR anti-distortion device in an embodiment of the present invention;

[0046] Figure 8 is a schematic diagram of the architecture of an intelligent terminal in an embodiment of the present invention;

[0047] Fig. 9 Schematic diagram of the architecture of a VR display system in an embodiment of the present invention;

[0048] Fig.10 This is a schematic diagram of the architecture of a VR device in an embodiment of the present invention;

[0049] Fig.11 Schematic diagram of the hardware structure of the intelligent terminal in the embodiment of the present invention;

[0050] Fig.12 Schematic diagram of the hardware structure of a VR device in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] The terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0053] In order to allow users to have a better sense of visual immersion through VR devices, VR devices must cover the visual range of the human eye as much as possible and increase the field of view. Based on this, VR devices need to be equipped with VR lenses to give users a good sense of immersion, but this comes at a price. Since VR lenses are spherical curvature lenses, the optical properties of the lenses will cause distortion, causing the image projected to the human eye to be distorted. The larger the field of view, the greater the image distortion, and this phenomenon is called VR lens distortion. It is precisely because of the existence of VR lens distortion that when users use VR devices to watch normal images, the actual images they see are distorted, especially when the VR lens distortion is severe, users cannot watch normally at all.

[0054] Combine the following Figure 1 The distortion phenomenon of the above-mentioned VR lenses is illustrated with examples. Figure 1 The image 101 in the figure is a normal grid image displayed on the display screen. When the user uses the VR device to view the image 101, the spherical curvature lens configured on the VR device, namely the VR lens, will magnify the parts of the viewed image 101 at different magnifications. Specifically, the VR lens magnifies the middle part of the image 101 at a larger magnification and the edge part of the image at a smaller magnification. Therefore, the image actually viewed by the user through the VR lens will be distorted in a pincushion shape, such as Figure 1 As shown in image 102. Figure 1 It can be seen that when viewing images through VR devices, the images will be distorted due to the distortion of VR lenses, and the visual effect is very poor.

[0055] In order to enable users to view normal images when using VR devices to ensure picture quality, the image of the VR resource is actively reverse distorted before it is displayed on the screen. Through this active reverse distortion processing, the distortion caused by the VR lens is offset. This active reverse distortion processing is called anti-distortion processing. It can be understood that anti-distortion processing is the distortion processing opposite to the distortion of VR lenses. Specifically, when an image of a VR resource needs to be displayed, the image is first anti-distorted using an anti-distortion processing method, and then the anti-distorted image is displayed on the display screen. When the user uses the VR device to watch, the distortion effect of the VR lens just offsets the anti-distortion effect, so the user can watch the normal image.

[0056] Therefore, the process of anti-distortion processing of the image of VR resources is an important link in determining the high-quality image display of VR devices. The present invention proposes a corresponding anti-distortion solution for this important link. Figure 2 The core technical idea of ​​the VR anti-distortion method provided by the embodiment of the present invention is introduced. Figure 2 This is the core technical framework diagram of the VR anti-distortion method.

[0057] like Figure 2 As shown, the core technical idea of ​​the VR anti-distortion method is to intersperse the anti-distortion processing operation in the image rendering process, that is, to integrate the anti-distortion processing operation into the traditional image rendering process. When the image to be processed is rendered, the anti-distortion processing is performed at the same time, so that the image frame data obtained after one image rendering is data that has undergone both image rendering and anti-distortion processing, and no secondary rendering is required. In this way, the image output by the VR device based on the image frame data is an image with an anti-distortion effect. When the user views the image through the VR device, the distortion effect of the VR lens can offset the effect of the anti-distortion processing, so that the user can see a normal image.

[0058] The VR anti-distortion method provided by the present invention performs anti-distortion processing interspersed in the image rendering process, so that the image frame data after one image rendering is the image frame data that has undergone anti-distortion, and the image output according to the image frame data has an anti-distortion effect accordingly, which can reduce one rendering operation, thereby reducing the calculation amount of the GPU and improving the overall working performance of the VR device.

[0059] It should be understood that the VR anti-distortion method provided by the present invention can be applied to VR display systems, see Figure 3 , Figure 3 This is a VR display system architecture diagram provided in an embodiment of the present application. Figure 3The smart terminal 301 and the VR device 302 are included. The smart terminal 301 is capable of playing VR resources, and is used to implement the VR anti-distortion method provided in the embodiment of the present invention; from the perspective of hardware implementation, the smart terminal 301 can be a smart phone, a tablet computer and other devices; the VR device 302 at least includes a VR lens, and a user can use the VR device 302 to watch the VR resources played by the smart terminal 301 through the VR lens, for example, the VR device can be a VR glasses, a VR helmet and other devices.

[0060] Figure 3 The image 303 in the figure is a normal grid map of the VR resource. In order to ensure that the user can still view the normal grid map as shown in the image 303 using the VR device, the smart terminal 301 needs to perform VR anti-distortion processing on the image 303 during the rendering process before displaying it to obtain the image 304. Then, the image 304 is displayed on the display screen of the smart terminal 301. When the user uses the VR device 302 to view the image 304, the VR lens configured on the VR device 302 undergoes lens distortion, which causes the image 304 to undergo a pincushion distortion. The pincushion distortion just offsets the pre-dedistortion effect. That is, the image 304 is transformed back to the image 303 through the pincushion distortion. In this way, the image actually viewed by the user using the VR device is a normal image. Figure 3 As shown in the image 305 in FIG. 1 , it can be understood that the image 305 is substantially the same as the image 303 .

[0061] It should be understood that the above Figure 3 The scenario shown is only an example. The VR anti-distortion method provided in the embodiment of the present invention can also be applied to a hardware scenario including a VR all-in-one machine, or can also be applied to a hardware scenario including other hardware that requires VR anti-distortion processing.

[0062] Next, combine Figure 4 The VR anti-distortion method provided by an embodiment of the present invention is introduced.

[0063] See also Figure 4 , Figure 4 A flowchart of a VR anti-distortion method provided by an embodiment of the present invention. It should be noted that, for the sake of ease of description, the following embodiments are described with a smart terminal as the execution subject. It should be understood that the execution subject of the VR anti-distortion method is not limited to a smart terminal. The VR anti-distortion method is applicable to other VR devices that require anti-distortion processing, such as an all-in-one VR machine, etc. Figure 4 As shown, the VR anti-distortion method includes the following steps:

[0064] Step 401: Read vertex data of the image to be rendered.

[0065] The smart terminal needs to determine the image to be rendered, and then read the vertex data of the image to be rendered; in specific implementation, the smart terminal can respond to the VR resource viewing operation triggered by the user, determine the image to be rendered, and then read the corresponding vertex data; for example, the user uses the smart terminal to enable the VR application, selects the VR resource of interest in the VR application, such as clicking to view a VR movie, voice control to enter a VR game, etc.; for another example, the user uses the smart terminal to view VR resources through a browser, selects the VR resource of interest, such as clicking to view a VR video, etc. Then the smart terminal responds to the relevant viewing operation triggered by the user, determines the image of interest to the user as the image to be rendered, and then the smart terminal can read the relevant image to be rendered locally; if the relevant image to be rendered is not stored locally, the smart terminal can also send a read request to the server storing the VR resource through network communication to request the image to be rendered, receive the data corresponding to the image to be rendered fed back by the server, and then read the relevant vertex data. Among them, the server with VR resources can be a server that provides data support for VR applications, an application server, or a Web server.

[0066] In specific implementation, the smart terminal may not need to execute step 401 after responding to the user's operation, but may spontaneously execute step 401. In this way, the smart terminal can pre-dedistort the image of the VR resource to prepare data for subsequent user viewing. Specifically, when the smart terminal runs the VR application, it can actively obtain the image to be rendered from the local smart terminal according to the preset acquisition cycle, or communicate with the server through the network to actively request to obtain the image to be rendered. The image to be rendered here can be an image corresponding to any resource supported by the VR application. The smart terminal can determine the image to be rendered according to the priority of the VR resource. For example, it can give priority to popular VR resources for dedistortion processing.

[0067] Of course, the smart terminal may also use other methods to implement step 401, which is not specifically limited in the embodiment of the present invention.

[0068] It should be noted that, in the embodiment of the present invention, the image to be rendered specifically refers to the image contained in the VR resource to be rendered; wherein, the VR resource, as described above, may be a data resource in VR form such as a VR game, a VR video, a VR movie, etc.

[0069] After the smart terminal obtains the image to be rendered, it needs to render the image, but the rendering process is performed in units of vertex data in the image. Therefore, it is necessary to read the vertex data from the image to be rendered, where vertex data refers to the relevant data of the pixel points on the image to be rendered. The vertex data includes the attribute information of the vertex, specifically including: vertex position (position in space), vertex color, vertex texture coordinates, normal, etc.

[0070] Step 402: performing image rendering on the vertex data. During the image rendering process, an anti-distortion algorithm is used to perform anti-distortion processing on the vertex data to obtain image frame data.

[0071] Step 403: Output an image according to the above image frame data.

[0072] After the intelligent terminal obtains the vertex data of the image to be rendered, it starts to perform image rendering processing on the vertex data. In the process of image rendering processing, the anti-distortion algorithm is inserted to perform anti-distortion processing on the vertex data, and the data finally obtained after anti-distortion and image rendering processing is used as image frame data. The image is output according to the image frame data, and the output image is an image with anti-distortion effect.

[0073] In this embodiment, the intelligent terminal implements image rendering processing on the image to be rendered based on the graphics image program development interface technology, but anti-distortion processing operations are specifically interspersed in the image rendering process, so that image frame data with anti-distortion effect can be obtained after one rendering process, and the image frame data can be directly rendered to the screen.

[0074] In the specific implementation of this embodiment, the smart terminal can implement image rendering processing on the image to be rendered based on the OpenGL (Open Graphics Library) graphics image program development interface technology. Of course, the smart terminal can also implement image rendering processing on the image to be rendered based on other types of graphics image program development interface technologies, such as D3D (Direct 3D), Vulkan or Metal and other interface technologies. Specifically, when the smart terminal is implemented based on the graphics image program development interface technology, it performs rendering processing on the vertex data and performs anti-distortion operations on the vertex data during the rendering process. Therefore, this embodiment does not limit the graphics image program development interface technology used.

[0075] Through the VR anti-distortion method provided by the embodiment of the present invention, the image is rendered and anti-distortion processing is performed at the same time to obtain image frame data, and the image output according to the image frame data has an anti-distortion effect. Based on this, the smart terminal can directly use the image frame data and play the image frame data to display the image screen; in this way, when the user uses the VR device to view the image screen, the distortion effect of the VR lens can be offset by the anti-distortion effect of the image frame data, so that the user can view a normal picture.

[0076] In specific implementation, the distortion produced by VR lenses is generally pincushion distortion, and the characteristic of the pincushion distortion of VR lenses is that the magnification of the middle part of the image is smaller than that of the edge part, which causes the image to be distorted. Therefore, the anti-distortion algorithm used in the VR anti-distortion method provided in the embodiment of the present invention is a barrel distortion algorithm, and the characteristic of the barrel distortion is that the magnification of the middle part of the image is larger than that of the edge part; therefore, the distortion effect of the barrel distortion is exactly opposite to the distortion effect of the pincushion distortion. Based on this, the anti-distortion processing of the image using the barrel distortion can just offset the distortion effect of the pincushion distortion of the VR lenses, so that when the user views the image after the barrel distortion processing through the VR lenses, the real image seen is a normal image.

[0077] In specific implementation, in order to provide users with a stronger immersive virtual reality experience, when rendering an image, the smart terminal specifically renders the left and right eye perspectives separately to obtain the images corresponding to the left and right eye perspectives, that is, the image frame data mentioned above specifically includes left screen display data and right screen display data. The left screen display data is the image frame data obtained by performing image rendering processing on the vertex data of the image from the left eye perspective, and the right screen display data is the image frame data obtained by performing image rendering processing on the vertex data of the image from the right eye perspective, wherein the left screen display data and the right screen display data have a certain image difference. Based on this, the smart terminal outputs the image from the left eye perspective according to the left screen display data, and outputs the image from the right eye perspective according to the right screen display data. Then the user can use the VR device to view the image from the left eye perspective and the image from the right eye perspective, and these two images have a certain image difference, so that a three-dimensional stereoscopic effect can be presented.

[0078] The VR anti-distortion method provided by the embodiment of the present invention intersperses the anti-distortion processing in the image rendering process, that is, the image rendering and the anti-distortion processing are performed synchronously, so as to complete the anti-distortion processing of the vertex data of the image to be rendered in the image rendering process, so that the image frame data obtained after the image rendering is completed has the anti-distortion effect, and the image output according to the image frame data also has the anti-distortion effect accordingly. The VR anti-distortion method provided by the embodiment of the present invention does not require secondary rendering, reduces the consumption of GPU resources, thereby reducing power consumption and improving the working performance of VR equipment.

[0079] As mentioned above, the intelligent terminal is based on the graphic image program development interface technology, and anti-distortion processing is interspersed in the image rendering process, so that the image frame data after anti-distortion processing can be obtained through a single rendering process, and the image output according to the image frame data is an image with anti-distortion effect. The following will take the OpenGL technology as an example to exemplify the method provided in the present application. It should be understood that the intelligent terminal can also perform image rendering based on other development interfaces such as D3D (Direct), Vulkan and Metal. When performing image rendering based on other development interfaces, the specific implementation process of image rendering is similar to the specific implementation process of image rendering based on the OpenGL development interface.

[0080] When image rendering is performed based on the OpenGL development interface technology, the specific implementation of step 402 can be found in Figure 5 , Figure 5 The present invention is a flow chart of performing image rendering and anti-distortion processing on vertex data of an image to be rendered based on OpenGL development interface technology. The image rendering method comprises the following steps:

[0081] Step 501: The read vertex data is passed into the programmable rendering pipeline.

[0082] First, let's briefly introduce the OpenGL development interface technology. OpenGL is the most widely used 2D / 3D graphics API in graphics technology, which refers to a professional graphics and image program development interface that defines cross-programming language and cross-platform programming interface specifications. It is a hardware-independent software interface. It is an open 3D graphics software package that can be independent of the window system and operating system. Applications based on it can be easily ported between various platforms, such as Windows, Unix, Linux, and OS. Therefore, software that supports OpenGL has good portability and can be widely used.

[0083] Most OpenGL implementations have a similar order of operations. This series of related processing nodes is called the OpenGL graphics rendering pipeline, which is considered the core of real-time graphics rendering. Its main function is to generate or render a two-dimensional image from a given virtual camera, three-dimensional object, light source, lighting model, texture map or others.

[0084] In order to implement the anti-distortion method of the present invention, the present invention adopts a programmable rendering pipeline, and programs certain links in the programmable rendering pipeline, so as to control the anti-distortion processing during the rendering process. In a specific implementation, the rendering process of the programmable rendering pipeline may include the following links: vertex data> vertex shader> primitive assembly and rasterization> fragment shader> raster operation.

[0085] The VR anti-distortion method provided by the embodiment of the present invention specifically involves the following links in the implementation process: vertex data > vertex shader. This link involves a programmable vertex processor component (Programmable Vertex Processor) on the GPU, also known as a vertex shader. The vertex shader is a hardware unit of the GPU, which is used to run a vertex shading program to realize the processing of vertex data.

[0086] The above is an introduction to the OpenGL development interface adopted by the present invention.

[0087] Next, the specific implementation step 501 of the embodiment of the present invention is described.

[0088] After the smart terminal reads the vertex data of the image, where the vertex data can be understood as the primitive information of the image, the vertex data is input into the functional components of the programmable rendering pipeline. The programmable rendering pipeline implements each function through each functional component in a pipeline manner to complete the image rendering.

[0089] Of course, it should be noted that the embodiments of the present invention do not limit the specific implementation form of the programmable rendering pipeline, and the forms listed above or other forms may be adopted. For example, in certain implementation scenarios, the links included in the programmable rendering pipeline may also be adaptively deleted or modified. As long as the vertex shader in the programmable rendering pipeline adopts the above implementation method, it can have the vertex transformation and anti-distortion processing functions.

[0090] Step 502: calling the programmable rendering pipeline to render the vertex data to obtain image frame data; wherein the programmable rendering pipeline includes a pre-programmed vertex shader; the vertex shader is used to use an anti-distortion algorithm to perform anti-distortion processing on the vertex data during the process of performing vertex transformation on the vertex data.

[0091] As described above, in the embodiment of the present invention, the programmable rendering pipeline may include other functional components in addition to vertex processing, and each functional component implements its own corresponding function to ultimately achieve rendering of the image.

[0092] In the specific implementation, a VR application is loaded in the smart terminal, and the programmable rendering pipeline in the GPU is driven by the application program of the VR application. Alternatively, a browser is loaded in the smart terminal, and the programmable rendering pipeline in the GPU is driven by the search program of the browser. Specifically, when driving the programmable rendering pipeline in the GPU, it is necessary to control the parameters and rendering state of the shader. The corresponding vertex shading program is called by the vertex shader, and the vertex shading program is run to realize its function. Among them, the vertex shading program is used to realize the process of vertex transformation of vertex data, and the anti-distortion algorithm is used to perform anti-distortion processing on the vertex data. That is, the vertex shading program is generated based on the combination of the traditional vertex transformation processing code and the anti-distortion algorithm processing code.

[0093] Furthermore, the vertex shader is used to perform model transformation, view transformation, projection transformation (Model-View-Projection, MVP) and anti-distortion transformation based on the anti-distortion algorithm on the vertex data. Specifically, when performing model transformation, view transformation and projection transformation on the vertex data, the vertex coordinates of the image to be rendered can be converted into two-dimensional pixel coordinates by the mathematical method of left multiplication of the transformation matrix. More importantly, anti-distortion processing of the vertex data is inserted in the transformation process, and the anti-distortion processing can also be achieved by the mathematical method of left multiplication of the anti-distortion matrix.

[0094] Furthermore, regarding how to perform vertex transformation and anti-distortion processing on vertex data in a vertex shader, the embodiment of the present invention provides several optional implementation methods.

[0095] The first optional implementation method is to first call the vertex shader to perform model transformation and view transformation on the vertex data to obtain model-view transformation data; then call the vertex shader to use the anti-distortion algorithm to perform anti-distortion transformation on the model-view transformation data to obtain anti-distorted model-view transformation data; finally, call the vertex shader to perform projection transformation on the anti-distorted model-view transformation data to obtain anti-distorted vertex transformation data.

[0096] The VR anti-distortion method provided by this implementation performs anti-distortion interspersed between view transformation and projection transformation, so that the vertex shader in the programmable rendering pipeline can achieve the dual functions of traditional vertex transformation and anti-distortion processing when processing vertex data.

[0097] The second optional implementation method is to first call the vertex shader to perform model transformation on the vertex data to obtain model transformation data; then call the vertex shader to use the anti-distortion algorithm to perform anti-distortion transformation on the model transformation data to obtain anti-distorted model transformation data; finally, call the vertex shader to perform view transformation and projection transformation on the anti-distorted model transformation data to obtain anti-distorted vertex transformation data.

[0098] The VR anti-distortion method provided by this implementation intersperses the anti-distortion operation between the model transformation and the view transformation, so that the vertex shader in the programmable rendering pipeline can achieve the dual functions of traditional vertex transformation and anti-distortion processing when processing vertex data.

[0099] A third optional implementation method is to first call the vertex shader to perform model transformation, view transformation and projection transformation on the vertex data to obtain vertex transformation data; then call the vertex shader to use an anti-distortion algorithm to perform anti-distortion transformation on the vertex transformation data to obtain anti-distorted vertex transformation data.

[0100] The VR anti-distortion method provided by this implementation sets the anti-distortion operation to be performed after the projection transformation, and can also enable the vertex shader in the programmable rendering pipeline to achieve the dual effects of traditional vertex transformation and anti-distortion processing when processing vertex data.

[0101] It is understandable that, in actual implementation, the code logic of the above three implementations is simple, thereby saving the amount of GPU calculation. In addition, it should be noted that, through experiments, it is proved that the code implementation of the above first optional implementation is simpler and has the best effect.

[0102] It can be seen from the above-mentioned several optional implementation methods that in an embodiment of the present invention, in addition to the traditional coordinate transformation of the vertex data, the vertex shader code is called by the pre-edited vertex shader, and the anti-distortion processing operation is also interspersed, so that the processed vertex data is the vertex data that has been distorted. The distortion is precisely to offset the distortion effect of the VR lens.

[0103] As can be seen from the working principle of the programmable rendering pipeline described above, after the vertex data is processed by the vertex shader to obtain the anti-distorted vertex transformation data, it is necessary to continue to call other components in the programmable rendering pipeline to process the anti-distorted vertex transformation data. Specifically, the anti-distorted vertex transformation data needs to be input into the primitive assembly and rasterization components, and the connectivity relationship between each vertex data is changed accordingly according to the change of the vertex data position; then, the data processed by the primitive assembly and rasterization components is input into the fragment shader, and the fragment shader changes the texture and color of each vertex data according to the change of the vertex data position to ensure that the texture and color of each vertex data can change accordingly with the change of the vertex data position; finally, it is necessary to use the raster operation component to perform a series of tests on the obtained data, such as clipping test, transparency test, template test and depth test, to obtain the final image frame data.

[0104] It should be noted that due to the differences in VR lens materials, refractive index, etc., different VR lenses produce different degrees of distortion, and VR anti-distortion processing is to eliminate the distortion of VR lenses. Therefore, when using the VR anti-distortion algorithm to perform VR anti-distortion processing on the rendered image, it is necessary to adaptively configure the relevant parameters involved in the anti-distortion operation process, such as the anti-distortion coefficient, according to the attribute parameters of the VR lens that can determine the degree of distortion. Specifically, when the user uses a VR device to watch VR resources, he can first select the attribute parameters of the VR lens that needs to be adapted, and then the smart terminal configures the relevant parameters required in the anti-distortion operation process according to the attribute parameters.

[0105] The VR anti-distortion method provided in the embodiment of the present invention performs image rendering on the vertex data in the image to be rendered based on the OpenGL development interface, and inserts anti-distortion processing in the image rendering process to obtain image frame data. In this way, it can be ensured that the image frame data has an anti-distortion effect, thereby solving the problem of poor picture caused by the distortion effect of VR lenses when using VR equipment; in addition, since OpenGL is relatively mature in the field of image processing and is relatively simple to implement, this makes the VR anti-distortion method provided in the embodiment of the present invention very applicable.

[0106] Regarding the above Figure 6 and Figure 7 The VR anti-distortion method described herein, the present invention also provides a corresponding VR anti-distortion device, so as to facilitate the application and implementation of these methods in practice.

[0107] See also Figure 6 , Figure 6 It is the same as above Figure 4 A structural diagram of a VR anti-distortion device 600 corresponding to the method shown, the device 600 comprises:

[0108] The reading module 601 is used to read the vertex data of the image to be rendered.

[0109] The rendering and anti-distortion module 602 is used to perform image rendering on the vertex data. In the process of image rendering, an anti-distortion algorithm is used to perform anti-distortion processing on the vertex data to obtain image frame data.

[0110] The output module 603 is used to output an image according to the image frame data.

[0111] The VR anti-distortion device provided by the embodiment of the present invention also intersperses anti-distortion processing on the image during the image rendering process, so that the image frame data obtained after the image rendering is already data that has undergone anti-distortion processing, and the image output according to the image frame data can have an anti-distortion effect, and the user can see a normal picture through the VR device. This method of interspersing anti-distortion processing during the image rendering process does not require the use of additional cache, does not require secondary rendering, reduces the fill rate of the GPU, and improves the display frame rate of the VR device.

[0112] Optional, in the above Figure 6 Based on the device shown, see Figure 7 , Figure 7 It is the same as above Figure 5 A structural diagram of a VR anti-distortion device 700 corresponding to the method shown, wherein the rendering and anti-distortion module 602 in the device includes:

[0113] The transmission unit 701 is used to transmit vertex data to the programmable rendering pipeline.

[0114] The calling unit 702 is used to call the programmable rendering pipeline to render the vertex data to obtain image frame data; wherein the programmable rendering pipeline includes a pre-programmed vertex shader; the vertex shader is used to use an anti-distortion algorithm to perform anti-distortion processing during the vertex transformation of the vertex data.

[0115] Optionally, the calling unit 702 is specifically used for:

[0116] Passing vertex data into the vertex shader;

[0117] Calling the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on the anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data;

[0118] The vertex transformation data is passed into a programmable rendering pipeline for rendering to obtain the image frame data.

[0119] Optionally, the calling unit 702 is specifically used for:

[0120] Call the vertex shader to perform model transformation and view transformation on the vertex data to obtain model view transformation data;

[0121] Calling the vertex shader to use an anti-distortion algorithm to perform an anti-distortion transformation on the model view transformation data to obtain anti-distorted model view transformation data;

[0122] The vertex shader is called to perform projection transformation on the anti-distorted model view transformation data to obtain anti-distorted vertex transformation data.

[0123] Optionally, the calling unit 702 is specifically used for:

[0124] Call the vertex shader to perform model transformation on the vertex data to obtain model transformation data;

[0125] Calling the vertex shader to use an anti-distortion algorithm to perform anti-distortion transformation on the model transformation data to obtain anti-distorted model transformation data;

[0126] The vertex shader is called to perform view transformation and projection transformation on the anti-distorted model transformation data to obtain anti-distorted vertex transformation data.

[0127] Optionally, the calling unit 702 is specifically used for:

[0128] Call the vertex shader to perform model transformation, view transformation, and projection transformation on the vertex data to obtain vertex transformation data;

[0129] The vertex shader is called to use an anti-distortion algorithm to perform an anti-distortion transformation on the vertex transformation data to obtain anti-distorted vertex transformation data.

[0130] The VR anti-distortion device provided by the embodiment of the present invention integrates anti-distortion processing into image rendering based on the graphic image program development interface technology, performs image rendering on the vertex data in the image to be rendered, and inserts anti-distortion processing in the image rendering process to obtain image frame data. In this way, it can be ensured that the image frame data has an anti-distortion effect, thereby solving the problem of poor picture caused by the distortion effect of VR lenses when using VR equipment; in addition, since the graphic image program development interface technology such as OpenGL technology is relatively mature in the field of image processing and is relatively simple to implement, this makes the VR anti-distortion method provided by the embodiment of the present invention very applicable and can be promoted and used on a large scale.

[0131] In addition, in order to ensure the application and implementation of the VR anti-distortion method provided in the above-mentioned embodiment of the present invention in the VR resource playback scenario, the present invention also provides corresponding hardware equipment. Specifically, the present invention provides a smart terminal, a VR display system and a VR device. The hardware devices provided by the present invention are introduced below.

[0132] See also Figure 8 , Figure 8A schematic diagram of the architecture of a smart terminal provided in an embodiment of the present invention. The smart terminal 800 is a terminal device capable of playing VR resources. For example, the smart terminal may be a smart phone, a tablet computer, or a personal digital assistant (PDA). The smart terminal 800 includes a memory 801 and a processor 802, wherein the memory 801 is used to store programs and VR applications, and the processor 802 is used to read the programs in the memory 801 when running the VR application to execute the VR anti-distortion method provided in the present invention; the processor 802 specifically includes a central processing unit (CPU) and a GPU, and the CPU is used to control the GPU to execute the VR anti-distortion method provided in the embodiment of the present invention according to the needs of the actual situation, and to perform VR anti-distortion processing while performing image rendering on the processed image to obtain an image with VR anti-distortion effect, and the CPU is also used to control the playback of the image with VR anti-distortion effect obtained by GPU processing. Among them, the specific implementation of the VR anti-distortion method provided in the embodiment of the present invention can be found above. Figure 4 Description of the illustrated method embodiment.

[0133] See also Fig. 9 , Fig. 9 Schematic diagram of the architecture of the VR display system provided by the embodiment of the present invention. The VR display system 900 includes a smart terminal 901 and a VR device 902; wherein the smart terminal 901 can be Figure 8 The provided smart terminal 800, that is, the smart terminal 901 is a terminal device that can play VR resources and can execute the VR anti-distortion method provided in the embodiment of the present invention; the VR device 902 at least includes a VR lens, and the VR device 902 is a device that can watch the VR resources played by the smart terminal through the VR lens. For example, the VR device can be VR glasses, VR helmets and other devices.

[0134] In a specific implementation, the VR device 902 can fix the smart terminal 901 through a hardware component, for example, the VR device 902 fixes the smart terminal 901 through a card slot, so that the user can watch the VR resources played by the smart terminal 901 through VR lenses. The hardware component can be a component of the VR device itself or an external component.

[0135] In a specific implementation, the VR device 902 can communicate with the smart terminal 901 through near-field communication technology. For example, the VR device 902 transmits the user's operation signal to the smart terminal through near-field communication technology, and the smart terminal receives the operation signal to display the corresponding virtual reality scene, thereby giving the user a stronger immersive experience.

[0136] Since the smart terminal 901 and VR device 902 in the VR display system 900 are separate components, different users can wear different VR devices at the same time to watch VR resources played on the same smart terminal, that is, the VR resources played on the smart terminal can be shared by multiple users at the same time.

[0137] In addition, in order to facilitate users' daily carrying and use, the embodiment of the present invention also provides a VR device, which is an integrated device with image processing function, referred to as a VR all-in-one machine. Fig.10 , Fig.10 Schematic diagram of the architecture of a VR device provided in an embodiment of the present invention, the VR device 1000 includes: a memory 1001, a processor 1002 and a display unit 1003, wherein the memory 1001 is used to store programs and VR applications, and the processor 1002 is used to read the program in the memory 1001 when running the VR application to execute the VR anti-distortion method provided in the embodiment of the present invention; wherein the specific implementation of the VR anti-distortion method provided in the embodiment of the present invention can be found in the above Figure 4 Description of the illustrated method embodiment.

[0138] Specifically, the processor 1002 specifically includes a CPU and a GPU. The CPU is used to control the GPU to execute the VR anti-distortion method provided in the embodiment of the present invention to perform VR anti-distortion processing on the image to be processed according to the actual needs, so as to obtain an image with VR anti-distortion effect. The CPU is also used to control the playback of the image with VR anti-distortion effect obtained by GPU processing. The display unit 1003 in the VR device 1000 specifically may include a display screen and VR lenses, wherein the display screen is used to display the image obtained by the processor 1002 performing VR anti-distortion processing, and the user views the image displayed on the display screen through the VR lenses.

[0139] In addition, with respect to the smart terminal provided by the embodiment of the present invention mentioned above, the embodiment of the present invention also provides a specific implementation method, which is described below in conjunction with Fig.11 The specific implementation of the smart terminal is explained, but for the sake of convenience, Fig.11 Only the parts related to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present invention.

[0140] The intelligent terminal may be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking a mobile phone as an example:

[0141] Fig.11FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal provided in an embodiment of the present invention. Fig.11 The mobile phone includes: a radio frequency (RF) circuit 1110, a memory 1120, an input unit 1130, a display unit 1140, a sensor 1150, an audio circuit 1160, a wireless fidelity (WiFi) module 1170, a processor 1180, and a power supply 1190. Those skilled in the art will understand that Fig.11 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0142] Combine the following Fig.11 A detailed introduction to the various components of the mobile phone:

[0143] The RF circuit 1110 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1180 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 1110 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 1110 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0144] The memory 1120 can be used to store software programs and modules. The processor 1180 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1120. The memory 1120 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1120 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0145] The input unit 1130 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1130 may include a touch panel 1131 and other input devices 1132. The touch panel 1131, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1131 or near the touch panel 1131 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1131 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 1180, and can receive and execute the command sent by the processor 1180. In addition, the touch panel 1131 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1131, the input unit 1130 may also include other input devices 1132. Specifically, other input devices 1132 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0146] The display unit 1140 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1140 may include a display panel 1141. Optionally, the display panel 1141 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 1131 may cover the display panel 1141. When the touch panel 1131 detects a touch operation on or near it, it is transmitted to the processor 1180 to determine the type of touch event. Subsequently, the processor 1180 provides a corresponding visual output on the display panel 1141 according to the type of touch event. Although in Fig.11 In the embodiment, the touch panel 1131 and the display panel 1141 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1131 and the display panel 1141 can be integrated to realize the input and output functions of the mobile phone.

[0147] The mobile phone may also include at least one sensor 1150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1141 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1141 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0148] The audio circuit 1160, the speaker 1161, and the microphone 1162 can provide an audio interface between the user and the mobile phone. The audio circuit 1160 can transmit the received audio data to the speaker 1161 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1160 and converted into audio data, and then the audio data is output to the processor 1180 for processing, and then sent to another mobile phone through the RF circuit 1110, or the audio data is output to the memory 1120 for further processing.

[0149] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 1170. It provides users with wireless broadband Internet access. Fig.11 A WiFi module 1170 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0150] The processor 1180 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1120, and calling data stored in the memory 1120. Optionally, the processor 1180 may include one or more processing units; preferably, the processor 1180 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1180.

[0151] The mobile phone also includes a power supply 1190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1180 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0152] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0153] In the embodiment of the present invention, the processor 1180 included in the terminal further has the following functions:

[0154] Read the vertex data of the image to be rendered;

[0155] Performing image rendering on the vertex data, and in the process of the image rendering, performing anti-distortion processing on the vertex data using an anti-distortion algorithm to obtain image frame data;

[0156] An image is output according to the image frame data.

[0157] Of course, the processor 1180 can also be used to read the program in the memory to implement the above Figure 4 Steps in the illustrated embodiment.

[0158] In addition, for the VR device provided by the embodiment of the present invention mentioned above, the embodiment of the present invention also provides a specific implementation method, which is described below in conjunction with Fig.12 Explain the specific implementation of the VR device. Fig.12Only the parts related to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present invention. Fig.12 The hardware structure diagram of the VR device provided by the embodiment of the present invention is as follows: Fig.12 As shown, the VR device may include: a radio frequency (RF) circuit 1210, a memory 1220, an input unit 1230, a display unit 1240, a sensor 1250, an audio circuit 1260, a wireless fidelity (WiFi) module 1270, a processor 1280, and a power supply 1290. Those skilled in the art will appreciate that Fig.12 The structure shown in the figure does not constitute a limitation of the VR device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0159] The display unit 1240 in the VR device may include a display panel 1241 and a VR lens 1242, and the user views the image displayed on the display panel 1241 through the VR lens. Fig.11 The various components shown will not be described in detail here.

[0160] In addition, in order to ensure that the above method is applied and implemented in practice, an embodiment of the present invention further provides a computer-readable storage medium for storing a computer program, which is used to execute any implementation of a VR anti-distortion method described in the above embodiments.

[0161] An embodiment of the present invention further provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute any one of the implementations of the VR anti-distortion method described in the aforementioned embodiments.

[0162] The above is an exemplary description of the method, device and equipment for VR anti-distortion processing provided by the present invention. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0163] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0164] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0166] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store computer programs.

[0167] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A virtual reality anti-distortion method, characterized in that: include: Reading vertex data of an image to be rendered; the vertex data includes vertex positions, vertex colors, vertex texture coordinates and normals; Performing image rendering on the vertex data, and simultaneously using an anti-distortion algorithm to perform anti-distortion processing on the vertex data during the image rendering process to obtain image frame data; Outputting an image according to the image frame data; The step of performing image rendering on the vertex data and using an anti-distortion algorithm to perform anti-distortion processing on the vertex data to obtain image frame data during the image rendering process includes: The vertex data is transmitted to a programmable rendering pipeline; wherein the programmable rendering pipeline includes a pre-programmed vertex shader, a primitive assembly, a fragment shader, a rasterization component and a raster operation component; the vertex shader is used to perform anti-distortion processing on the vertex data by using an anti-distortion algorithm during the process of performing vertex transformation on the vertex data; Calling the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data; Inputting the vertex transformation data into the primitive assembly and the rasterization component, and changing the connectivity relationship between each vertex data according to the position information of the vertex data; Input the data processed by the primitive assembly and the rasterization component into a fragment shader, and change the texture and color of each vertex data according to the position information of the vertex data; Using the raster operation component to test the data processed by the fragment shader to obtain the image frame data; The calling of the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data includes: Calling the vertex shader to perform model transformation on the vertex data to obtain model transformation data; Calling the vertex shader to use an anti-distortion algorithm to perform an anti-distortion transformation on the model transformation data to obtain anti-distorted model transformation data; The vertex shader is called to perform view transformation and projection transformation on the anti-distorted model transformation data to obtain anti-distorted vertex transformation data.

2. The method according to claim 1, characterized in that: The calling of the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data also includes: Calling the vertex shader to perform model transformation and view transformation on the vertex data to obtain model view transformation data; Calling the vertex shader to use an anti-distortion algorithm to perform an anti-distortion transformation on the model view transformation data to obtain anti-distorted model view transformation data; The vertex shader is called to perform projection transformation on the anti-distorted model view transformation data to obtain anti-distorted vertex transformation data.

3. The method according to claim 1, characterized in that: The calling of the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data also includes: Calling the vertex shader to perform model transformation, view transformation, and projection transformation on the vertex data to obtain vertex transformation data; The vertex shader is called to adopt an anti-distortion algorithm to perform an anti-distortion transformation on the vertex transformation data to obtain anti-distorted vertex transformation data.

4. A virtual reality anti-distortion device, characterized in that: include: A reading module, used for reading vertex data of an image to be rendered; The vertex data includes vertex position, vertex color, vertex texture coordinates and normal; A rendering and anti-distortion module, used for performing image rendering on the vertex data, and in the process of the image rendering, an anti-distortion algorithm is simultaneously used to perform anti-distortion processing on the vertex data to obtain image frame data; An output module, used for outputting an image according to the image frame data; The rendering and anti-distortion module includes: A transmission unit, used for transmitting the vertex data into a programmable rendering pipeline; A calling unit, used for calling the programmable rendering pipeline to render the vertex data to obtain the image frame data; wherein the programmable rendering pipeline includes a pre-programmed vertex shader, a primitive assembly, a fragment shader, a rasterization component and a raster operation component; the vertex shader is used for performing anti-distortion processing using an anti-distortion algorithm during the process of performing vertex transformation on the vertex data; The calling unit is specifically used for: Passing the vertex data into the vertex shader; Calling the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data; Inputting the vertex transformation data into the primitive assembly and the rasterization component, and changing the connectivity relationship between each vertex data according to the position information of the vertex data; Input the data processed by the primitive assembly and the rasterization component into a fragment shader, and change the texture and color of each vertex data according to the position information of the vertex data; Using the raster operation component to test the data processed by the fragment shader to obtain the image frame data; The calling unit is further specifically used for: Call the vertex shader to perform model transformation on the vertex data to obtain model transformation data; Calling the vertex shader to use an anti-distortion algorithm to perform anti-distortion transformation on the model transformation data to obtain anti-distorted model transformation data; The vertex shader is called to perform view transformation and projection transformation on the anti-distorted model transformation data to obtain anti-distorted vertex transformation data.

5. The virtual reality anti-distortion device according to claim 4, characterized in that: The calling unit is further specifically used for: Calling the vertex shader to perform model transformation and view transformation on the vertex data to obtain model view transformation data; Calling the vertex shader to use an anti-distortion algorithm to perform an anti-distortion transformation on the model view transformation data to obtain anti-distorted model view transformation data; The vertex shader is called to perform projection transformation on the anti-distorted model view transformation data to obtain anti-distorted vertex transformation data.

6. The device according to claim 4, characterized in that The calling unit is further specifically used for: Call the vertex shader to perform model transformation, view transformation, and projection transformation on the vertex data to obtain vertex transformation data; The vertex shader is called to use an anti-distortion algorithm to perform an anti-distortion transformation on the vertex transformation data to obtain anti-distorted vertex transformation data.

7. An intelligent terminal, characterized in that: The intelligent terminal comprises: Memory and processor; The memory is used to store programs and virtual reality applications; The processor is used to read the program in the memory when running the virtual reality application, and the program is used to: Reading vertex data of an image to be rendered; the vertex data includes vertex positions, vertex colors, vertex texture coordinates and normals; Performing image rendering on the vertex data, and simultaneously using an anti-distortion algorithm to perform anti-distortion processing on the vertex data during the image rendering process to obtain image frame data; Outputting an image according to the image frame data; The step of performing image rendering on the vertex data and using an anti-distortion algorithm to perform anti-distortion processing on the vertex data to obtain image frame data during the image rendering process includes: The vertex data is transmitted to a programmable rendering pipeline; wherein the programmable rendering pipeline includes a pre-programmed vertex shader, a primitive assembly, a fragment shader, a rasterization component and a raster operation component; the vertex shader is used to perform anti-distortion processing on the vertex data by using an anti-distortion algorithm during the process of performing vertex transformation on the vertex data; Calling the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data; Inputting the vertex transformation data into the primitive assembly and the rasterization component, and changing the connectivity relationship between each vertex data according to the position information of the vertex data; Input the data processed by the primitive assembly and the rasterization component into a fragment shader, and change the texture and color of each vertex data according to the position information of the vertex data; Using the raster operation component to test the data processed by the fragment shader to obtain the image frame data; The calling of the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data includes: Calling the vertex shader to perform model transformation on the vertex data to obtain model transformation data; Calling the vertex shader to use an anti-distortion algorithm to perform an anti-distortion transformation on the model transformation data to obtain anti-distorted model transformation data; The vertex shader is called to perform view transformation and projection transformation on the anti-distorted model transformation data to obtain anti-distorted vertex transformation data.

8. A virtual reality display system, characterized in that: The virtual reality display system comprises: A smart terminal and virtual reality glasses; the virtual reality glasses and the smart terminal communicate with each other using near field communication technology; The smart terminal is loaded with a virtual reality application. When the smart terminal runs the virtual reality application, the following procedures are executed: Reading vertex data of an image to be rendered; the vertex data includes vertex positions, vertex colors, vertex texture coordinates and normals; Performing image rendering on the vertex data, and simultaneously using an anti-distortion algorithm to perform anti-distortion processing on the vertex data during the image rendering process to obtain image frame data; Outputting an image according to the image frame data; The step of performing image rendering on the vertex data and using an anti-distortion algorithm to perform anti-distortion processing on the vertex data to obtain image frame data during the image rendering process includes: The vertex data is transmitted to a programmable rendering pipeline; wherein the programmable rendering pipeline includes a pre-programmed vertex shader, a primitive assembly, a fragment shader, a rasterization component and a raster operation component; the vertex shader is used to perform anti-distortion processing on the vertex data by using an anti-distortion algorithm during the process of performing vertex transformation on the vertex data; Calling the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data; Inputting the vertex transformation data into the primitive assembly and the rasterization component, and changing the connectivity relationship between each vertex data according to the position information of the vertex data; Input the data processed by the primitive assembly and the rasterization component into a fragment shader, and change the texture and color of each vertex data according to the position information of the vertex data; Using the raster operation component to test the data processed by the fragment shader to obtain the image frame data; The calling of the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data includes: Calling the vertex shader to perform model transformation on the vertex data to obtain model transformation data; Calling the vertex shader to use an anti-distortion algorithm to perform an anti-distortion transformation on the model transformation data to obtain anti-distorted model transformation data; The vertex shader is called to perform view transformation and projection transformation on the anti-distorted model transformation data to obtain anti-distorted vertex transformation data.

9. A virtual reality device, characterized in that: The virtual reality device is a virtual reality all-in-one machine, comprising: Memory, processor and display unit; The memory is used to store programs and virtual reality applications; The processor is used to read the program in the memory when running the virtual reality application, and the program is used to: Reading vertex data of an image to be rendered; the vertex data includes vertex positions, vertex colors, vertex texture coordinates and normals; Performing image rendering on the vertex data, and simultaneously using an anti-distortion algorithm to perform anti-distortion processing on the vertex data during the image rendering process to obtain image frame data; Outputting an image according to the image frame data; The step of performing image rendering on the vertex data and using an anti-distortion algorithm to perform anti-distortion processing on the vertex data to obtain image frame data during the image rendering process includes: The vertex data is transmitted to a programmable rendering pipeline; wherein the programmable rendering pipeline includes a pre-programmed vertex shader, a primitive assembly, a fragment shader, a rasterization component and a raster operation component; the vertex shader is used to perform anti-distortion processing on the vertex data by using an anti-distortion algorithm during the process of performing vertex transformation on the vertex data; Calling the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data; Inputting the vertex transformation data into the primitive assembly and the rasterization component, and changing the connectivity relationship between each vertex data according to the position information of the vertex data; Input the data processed by the primitive assembly and the rasterization component into a fragment shader, and change the texture and color of each vertex data according to the position information of the vertex data; Using the raster operation component to test the data processed by the fragment shader to obtain the image frame data; The calling of the vertex shader to perform model transformation, view transformation, projection transformation and anti-distortion transformation based on an anti-distortion algorithm on the vertex data to obtain anti-distorted vertex transformation data includes: Calling the vertex shader to perform model transformation on the vertex data to obtain model transformation data; Calling the vertex shader to use an anti-distortion algorithm to perform an anti-distortion transformation on the model transformation data to obtain anti-distorted model transformation data; The vertex shader is called to perform view transformation and projection transformation on the anti-distorted model transformation data to obtain anti-distorted vertex transformation data.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the virtual reality anti-distortion method according to any one of claims 1 to 3.

11. A computer program product comprising instructions, characterized in that When the method is run on a computer, the computer is enabled to execute the virtual reality anti-distortion method according to any one of claims 1 to 3.

Images