Simplified rendering method and system for light field image

Abstract

The invention discloses a simplified rendering method and system for a light field image, and belongs to the technical field of image rendering. According to the simplified rendering method of the light field image, a topology reconstruction unit, a model simplification processing unit, a high polygon feature acquisition unit and a fusion rendering unit are constructed to simplify a high polygon model HPM of the light field image to obtain a low polygon model LPM; according to the method, a high polygon model HPM and a low polygon model LPM are combined, and coupling rendering is carried out on a normal map of the high polygon model HPM and the low polygon model LPM to realize rapid rendering of the light field image, so that the data volume of the model can be greatly reduced, the calculation amount in the light field image rendering process is remarkably reduced, the rendering rate is improved, meanwhile, the high-quality image effect is kept, and the sense of reality of the light field image is greatly improved; and meanwhile, the data volume of the model is further reduced by adopting a back rejection technology, so that the rendering of the light field image is accelerated.

Description

Technical Field

[0001] The present invention relates to a simplified rendering method and system for light field images, belonging to the technical field of image rendering. Background Art

[0002] Light field images have extensive applications in fields such as 3D display, virtual reality, augmented reality, and the metaverse. These fields require realistic light field images with large amounts of data. Current rendering technologies still face significant challenges in the generation of massive real light field image data. The model-based rendering technology (MBR) is an effective method for achieving high-quality image rendering. However, for light field rendering applications, the MBR technology still has problems such as a huge amount of data to be processed, a lot of redundant calculations, slow speed, and weak image realism and complex lighting rendering.

[0003] To improve the rendering efficiency and quality of light field images, in 1998, Michael Halle proposed a multiple viewpoint rendering (MVR) algorithm, which significantly improved the rendering speed by utilizing the perspective correlation between different viewpoint views. In 2017, Shu Li et al. proposed an algorithm based on a multi-orthogonal frustum array and utilized the parallel computing power of the GPU to improve the rendering efficiency. In the same year, Shujun Xing et al. proposed a reverse ray tracing algorithm and significantly shortened the rendering time by using a hierarchical bounding box structure. However, due to the large amount of computation, real-time rendering of complex scenes could still not be achieved. In 2019, Weiping Huo et al. proposed a real-time rendering algorithm for light field images based on lens position based on reverse ray tracing technology. This algorithm can correct the real-time rendered 3D image by collecting the offset information of the lens position in the display device. In 2020, Yanxin Guan et al. proposed a parallel multi-viewpoint polygon rasterization rendering algorithm, which constructed a fast rendering pipeline for super multi-viewpoint light field images by utilizing geometric correlation and parallel rasterization technology. In 2022, Shuo Chen et al. proposed a two-stage virtual view synthesis method based on NeRF and 3D voxel rendering to accelerate the generation of high-resolution 3D images. In 2023, Quanzhen Wan proposed a rendering algorithm based on voxel look-up table, which achieved fast rendering of light field images by using ray tracing and a pre-computed mapping table. In 2024, Ningchi Li proposed an optical flow prediction network based on texture enhancement and performed fast encoding of 3D light fields based on optical flow guidance, capable of generating 8K three-dimensional light field images in real time.

[0004] Although the above method has greatly improved the rendering speed and quality of light field images, there is a common problem: the rendering cost is still proportional to the complexity of the scene, and the fast rendering of high-quality light field images for large-scale complex scenes has not been achieved. Research shows that complex scenes have a more refined structure composed of more vertices and triangles, and the calculation and processing of vertices and triangles will occupy most of the rendering time. Therefore, how to reduce the number of vertices and triangles while maintaining a high rendering quality is an effective method to improve the rendering efficiency of light field images and simplify the rendering process.

[0005] The information disclosed in this background technology is only used to understand the background of the inventive concept, so it may include information that does not constitute prior art. Summary of the Invention

[0006] In view of the above problems or one of the above problems, the first object of the present invention is to provide a simplified rendering method for light field images, which can effectively reduce the amount of data processing, improve the rendering efficiency, while maintaining a high image quality, and achieve the fast rendering of light field images with large data volume and high quality.

[0007] In view of the above problems or one of the above problems, the second object of the present invention is to provide a simplified rendering method and system for light field images, which can effectively overcome the rendering bottleneck encountered by traditional methods in the rendering of large-scale complex scenes, greatly improve the rendering efficiency; and can give full play to the parallel computing power of the GPU, greatly reducing the communication between the CPU and the GPU required in traditional light field rendering methods, thereby improving the communication and computing efficiency.

[0008] In view of the above problems or one of the above problems, the third object of the present invention is to provide a simplified rendering method and system for light field images, which can quickly calculate the light field images of large-scale complex scenes with complex lighting, shadows and textures, etc., and greatly improve the realism of light field images.

[0009] In view of the above problems or one of the above problems, the fourth object of the present invention is to provide a simplified rendering method and system for light field images. By constructing a topology reconstruction unit, a model simplification processing unit, a high polygon feature acquisition unit, and a fusion rendering unit, the high polygon model HPM of the light field image is simplified to obtain a low polygon model LPM, and the normal map of the high polygon model HPM is coupled with the low polygon model LPM for rendering, realizing the fast rendering of the light field image, thereby greatly reducing the data volume of the model, significantly reducing the calculation amount in the light field image rendering process, improving the rendering rate, while maintaining a high-quality image effect, and greatly improving the realism of the light field image; at the same time, the back face culling technology is used to further reduce the data volume of the model, thereby accelerating the rendering of the light field image.

[0010] To achieve one of the above - mentioned purposes, the first technical solution of the present invention is as follows:

[0011] A simplified rendering method for light field images, including the following:

[0012] Through a pre - constructed topology reconstruction unit, based on the level - of - detail algorithm, reconstruct the high - polygon model HPM of the light field image to obtain a low - polygon model LPM;

[0013] Based on a pre - constructed model simplification processing unit, use the back - face culling technique to perform back - face culling on the low - polygon model LPM to obtain a simplified low - polygon model LPM;

[0014] Utilize a pre - constructed high - polygon feature acquisition unit to bake the normal information of the high - polygon model HPM into the texture coordinates of the low - polygon model LPM according to the model baking technique to obtain normal map data;

[0015] Adopt a pre - constructed fusion rendering unit to perform coupled rendering on the normal map data and the simplified low - polygon model LPM to achieve fast rendering of the light field image.

[0016] Through continuous exploration and experimentation, the present invention simplifies the high - polygon model HPM of the light field image by constructing a topology reconstruction unit, a model simplification processing unit, a high - polygon feature acquisition unit, and a fusion rendering unit to obtain a low - polygon model LPM, and performs coupled rendering on the normal map of the high - polygon model HPM and the low - polygon model LPM to achieve fast rendering of the light field image. Thereby, the data volume of the model can be significantly reduced, the computational amount during the rendering of the light field image can be significantly reduced, the rendering rate can be improved, while maintaining a high - quality image effect, and the realism of the light field image can be greatly improved; at the same time, the back - face culling technique is adopted to further reduce the data volume of the model, thereby accelerating the rendering of the light field image.

[0017] As a preferred technical measure:

[0018] The method of reconstructing the high - polygon model HPM of the light field image through a pre - constructed topology reconstruction unit based on the level - of - detail algorithm to obtain a low - polygon model LPM is as follows:

[0019] Obtain the high - polygon model HPM of the light field image, which is three - dimensional model data;

[0020] Utilize the level - of - detail algorithm to perform topology reconstruction on the three - dimensional model data to generate a set of low - polygon model objects with different levels of detail, and make the number of triangles of each low - polygon model object at a detail level be half of the number of triangles of the low - polygon model object in the previous - level model object;

[0021] Next, render the low - polygon model objects at different levels of detail to obtain a set of low - polygon rendered objects;

[0022] Test the image quality of each low - polygon rendered object to obtain the triangle information and rendering quality data of the low - polygon rendered object;

[0023] According to the triangle information and rendering quality data, select the low - polygon rendered object with the fewest number of triangles and the rendering quality closest to the high - polygon model HPM as the finally optimized low - polygon model LPM.

[0024] As an optimal technical measure:

[0025] Based on a pre - constructed model simplification processing unit, the method of using back - face culling technology to perform back - face culling on the low - polygon model LPM to obtain a simplified low - polygon model LPM is as follows:

[0026] Obtain the triangle information of the low - polygon model LPM and the camera array parameters;

[0027] According to the triangle information and camera array parameters, obtain the invisible triangles in the entire field - of - view range and the triangles whose visibility changes in the entire field - of - view range;

[0028] First - stage back - face culling: Based on the invisible triangles, cull the triangles that are always facing away from the camera array;

[0029] Second - stage back - face culling: Based on the triangles whose visibility changes, cull the triangles that become invisible as the camera viewpoint changes, thereby simplifying the low - polygon model LPM to obtain a simplified low - polygon model LPM.

[0030] As an optimal technical measure:

[0031] Based on a pre - constructed model simplification processing unit, the method of using back - face culling technology to perform back - face culling on the low - polygon model LPM to obtain a simplified low - polygon model LPM is as follows:

[0032] Obtain the low - polygon model LPM and send it into the vertex shader of the GPU rendering pipeline to obtain vertex data;

[0033] Then assemble the vertex data into triangles and pass them into the geometry shader. In the geometry shader, calculate the vertex positions of each triangle in the screen coordinate system according to the left and right camera parameters respectively;

[0034] Next, calculate the directed areas of the triangles in the left and right camera spaces respectively according to these vertex positions, and perform the first - stage back - face culling;

[0035] Slice the triangle using the scan-line algorithm and sort the slice list from bottom to top according to the y value of the scan line to obtain the slice scan-line data;

[0036] Next, pass the slice scan-line data to the GPU. In the vertex shader, convert the vertex data back to normalized device coordinates, then assemble it into line primitives and pass them to the geometry shader, and perform the second-stage back-face culling in the geometry shader to obtain the simplified low-polygon model LPM.

[0037] As a preferred technical measure:

[0038] The method for obtaining the invisible triangles in the entire field of view according to the triangle information and the camera array parameters is as follows:

[0039] According to the triangle information, obtain the vertex coordinates of each triangle;

[0040] According to the camera array parameters, determine the leftmost view-point position information and the rightmost view-point position information;

[0041] Use the shoelace formula, and based on the vertex coordinates of the triangle, as well as the leftmost view-point position information and the rightmost view-point position information, calculate the signed area one S of each triangle under the leftmost view-point L and the signed area two S under the rightmost view-point R ;

[0042] The shoelace formula is as follows:

[0043]

[0044] where x i is the screen X-axis coordinate value of the i-th vertex among the n vertices of the triangle, and y i is the screen Y-axis coordinate value of the i-th vertex among the n vertices of the triangle, and S is the signed area of the triangle;

[0045] When the S L and S R of a certain triangle are both less than 0 at the same time, then this triangle is always facing away from the camera, and this triangle is an invisible triangle, so the vertex data of this triangle will not be processed subsequently.

[0046] As a preferred technical measure:

[0047] The method for obtaining the triangles whose visibility changes in the entire field of view according to the triangle information and the camera array parameters is as follows:

[0048] According to the triangle information, cut the triangle to obtain several primitives PST;

[0049] Calculate the base lengths d of two triangles in the graphic primitive PST L and d R

[0050] Determine whether the graphic primitive intersects itself. If the product of d R and d L is less than 0, it indicates that the graphic primitive PST does not self-intersect. At this time, render a quadrilateral graphic primitive;

[0051] If the product of d R and d L is not less than 0, it indicates that the graphic primitive PST will self-intersect, and then the data of the transition point v T needs to be calculated;

[0052] According to the data of the transition point v T , calculate the directed area one S L and the directed area two S R ;

[0053] Then, respectively determine the positive and negative of the areas S L and S R . Based on the positive and negative of the S L and S R values, determine whether each graphic primitive PST is located on the back or the front;

[0054] If S L is less than 0, it means that all the triangles to which the graphic primitive belongs observed by the camera between the left view point and the view point corresponding to the transition point face the back. In this way, the triangles with changed visibility in the entire field of view range and the invisible parts are obtained;

[0055] The method for calculating the data of the transition point v T is as follows:

[0056] Obtain the endpoint data of the slice, which includes the X-axis coordinates x 0L , x 0R , x 1L , x 1R ;

[0057] According to the endpoint data, calculate the differences of x 0L , x 0R , x 1L , x 1R to obtain the base lengths d L and d R , and their calculation formula is as follows:

[0058]

[0059] Based on d L and d R, and through similar triangles, the proportional relationship k of the front part and the back part in the perspective direction is calculated T and k R , and its calculation formula is as follows:

[0060]

[0061] Among them, k T represents the position of the transition point v T in the perspective space, and k R is the length of the entire perspective space;

[0062] According to the proportional relationships k T and k R , through interpolation between geometric parameters, the position and its attribute data of the transition point v T are calculated, and its calculation formula is as follows:

[0063] v T = k T V 0R +(1 - k T )V 0L

[0064] a T = k T a0+(1 - k T )a1

[0065] Among them, V 0L , V 0R respectively represent the positions of the left endpoint of the slice in the extremely left and extremely right view point spaces; a T represents the attribute data of the transition point v T , including data such as the corresponding camera position, normal, tangent, and texture coordinates; a0 represents the attribute data of the left endpoint of the slice, and a1 represents the attribute data of the right endpoint of the slice.

[0066] As a preferred technical measure:

[0067] The method of realizing the fast rendering of the light field image by coupling and rendering the normal map data and the simplified low - polygon model LPM using a pre - constructed fusion rendering unit is as follows:

[0068] Input the normal map data and the simplified low - polygon model LPM;

[0069] Then, according to the camera array parameters, set the parameter matrix of the camera to be rendered;

[0070] Then, according to the parameter matrix, perform view model transformation, perspective projection transformation, and viewport transformation on the low - polygon model LPM;

[0071] Then, perform lighting calculations based on the normal map data and the Phong lighting model, and calculate the shadow parameters by judging the occlusion situation through the shadow map. The final color of each fragment is obtained through comprehensive calculation;

[0072] Next, rasterize and output a view captured by the camera, and determine whether there are still cameras not rendered. If so, update the camera parameter matrix, enter the next rendering loop until all cameras are rendered;

[0073] Finally, output all light field images to achieve fast rendering of light field images.

[0074] To achieve one of the above purposes, the second technical solution of the present invention is:

[0075] A simplified rendering method for light field images, including the following:

[0076] Read the high polygon model HPM and the camera array parameters corresponding to the light field acquisition;

[0077] Adopt model optimization technology to convert the high polygon model HPM into a low polygon model LPM; the model optimization technology includes using the level of detail algorithm LOD technology to perform topological reconstruction on the high polygon model HPM to generate a set of low polygon models LPM with different levels of detail;

[0078] Render the low polygon models LPM with different levels of detail; select the low polygon model LPM with the fewest triangles and the rendering quality closest to the high polygon model HPM as the optimized low polygon model LPM;

[0079] Use model baking technology to bake the normal information of the high polygon model HPM into the texture coordinates of the optimized low polygon model LPM and store it as a normal map;

[0080] Use back face culling technology to perform back face culling on the optimized low polygon model LPM to obtain a simplified low polygon model LPM;

[0081] Use the simplified low polygon model LPM and the normal map for rendering to obtain high-precision light field images.

[0082] The present invention uses model optimization technology to significantly reduce the data volume of the model, thereby significantly reducing the computational amount in the process of rendering light field images, improving the rendering rate, and at the same time maintaining high-quality image effects. Furthermore, the present invention uses back face culling technology to further reduce the data volume of the model, thereby accelerating the rendering of light field images.

[0083] To achieve one of the above purposes, the third technical solution of the present invention is:

[0084] A simplified rendering method for light field images. The simplified rendering method is a multi-viewpoint rendering method, which includes the following:

[0085] First, initialize and input an optimized low polygon model (LPM) through the CPU, and send it to the vertex shader in the GPU rendering pipeline for the first-stage processing;

[0086] The first-stage processing includes the following:

[0087] Assemble the vertex data into triangle primitives and pass them to the geometry shader; in the geometry shader, according to the left and right camera parameters, convert the vertex data into screen coordinates in the left and right viewpoints, that is, calculate the vertex positions of each triangle in the screen coordinate system;

[0088] Then, calculate the directed areas of the triangles in the left and right camera spaces respectively based on these vertex positions, perform the first-stage back-face culling, obtain the assembled primitive triangles, and perform stream output, and transfer them to the transform feedback buffer;

[0089] The transform feedback buffer is created based on the transform feedback technology, which can map the vertex positions, texture coordinates, and vertex normals in the primitive triangles back to the local end; then process the TFB data in units of triangles to complete the assembly of the TFB data; furthermore, slice each triangle along the scan line to obtain several slices; then create a slice list and store the slices in the slice list in the order of the y coordinates of the scan lines, and transfer them to the vertex shader;

[0090] At the same time, perform rasterization operations based on the primitive triangles, rasterize the scene in the light source view space to generate a shadow map. Furthermore, in the depth buffer, store the z values of the primitive triangles in the shadow map, and then transfer the shadow map to the fragment shader for the second-stage processing;

[0091] The second-stage processing includes the following:

[0092] Use the vertex shader to convert the vertices of the slices into normalized device coordinates; perform primitive assembly of line segments based on the normalized device coordinates (NDC) to obtain line segment primitives;

[0093] Based on the line segment primitives, use the geometry shader to perform the second-stage back-face culling and reconstruct the PST primitives;

[0094] According to the PST primitives, perform primitive assembly of triangle strips to obtain primitive strips;

[0095] Rasterize the primitive strips to obtain strip rasterization data and input them into the fragment shader;

[0096] Then, in the fragment shader, the lighting information is calculated using the normal map, and the occlusion situation is judged through the shadow map to calculate the shadow parameters, and then the final color of each fragment is comprehensively calculated;

[0097] In the frame buffer, the rendering result of the PST primitive is saved to a frame buffer to obtain the EPI; the EPI is converted into a light field image and saved in another frame buffer, thus completing the simplified rendering of the light field image.

[0098] The present invention uses model optimization technology to significantly reduce the data volume of the model, thereby significantly reducing the computational amount in the process of rendering the light field image, improving the rendering rate, and at the same time maintaining high-quality image effects.

[0099] Furthermore, the present invention adopts back-face culling technology to further reduce the data volume of the model, thereby accelerating the rendering of the light field image.

[0100] Even further, the present invention uses a parallel rendering method to quickly calculate a large number of light field images with complex lighting, shadow, and texture effects, significantly enhancing the realism of the light field images.

[0101] To achieve one of the above purposes, the fourth technical solution of the present invention is:

[0102] A simplified rendering system for light field images, provided with a rendering pipeline and rendering hardware;

[0103] The rendering pipeline is used to implement the simplified rendering of light field images, and includes an application program, an application program interface, a serial computing architecture, a general-purpose GPU computing architecture, and an operating system;

[0104] The rendering hardware is used to provide the hardware resources required to run the rendering pipeline, and includes a memory, a CPU, a GPU, and a video memory;

[0105] The operating system is used to control the application program, and call the application program interface, the serial computing architecture, and the general-purpose parallel computing architecture;

[0106] The application program, the application program interface, the serial computing architecture, and the general-purpose parallel computing architecture drive the CPU, the GPU, the memory, and the video memory, and when executed, implement the above-mentioned simplified rendering method of light field images, and complete the rapid simplified rendering from three-dimensional model data to light field images.

[0107] To achieve one of the above purposes, the fifth technical solution of the present invention is:

[0108] An electronic device, which includes:

[0109] One or more processors;

[0110] A storage device for storing one or more programs;

[0111] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned simplified rendering method of a light field image.

[0112] To achieve one of the above purposes, the sixth technical solution of the present invention is:

[0113] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the above-mentioned simplified rendering method of a light field image.

[0114] Compared with the prior art solutions, the present invention has the following beneficial effects:

[0115] The present invention uses model optimization technology to greatly reduce the data volume of the model, thereby significantly reducing the computational amount in the light field image rendering process, improving the rendering rate, and at the same time maintaining high-quality image effects.

[0116] Furthermore, the present invention adopts back-face culling technology to further reduce the data volume of the model, thereby accelerating the rendering of light field images.

[0117] Even further, the method of the present invention can quickly calculate a large number of light field images with complex lighting, shadow, and texture effects, greatly improving the realism of light field images.

[0118] Furthermore, through continuous exploration and experiments, the present invention simplifies the high-polygon model HPM of the light field image by constructing a topology reconstruction unit, a model simplification processing unit, a high-polygon feature acquisition unit, and a fusion rendering unit to obtain a low-polygon model LPM, and couples and renders the normal map of the high-polygon model HPM with the low-polygon model LPM to achieve fast rendering of light field images, thereby greatly reducing the data volume of the model, significantly reducing the computational amount in the light field image rendering process, improving the rendering rate, and at the same time maintaining high-quality image effects, greatly improving the realism of light field images; at the same time, back-face culling technology is adopted to further reduce the data volume of the model, thereby accelerating the rendering of light field images. BRIEF DESCRIPTION OF THE DRAWINGS

[0119] Figure 1 It is a schematic flowchart of a simplified rendering method of a light field image of the present invention;

[0120] Figure 2 It is a schematic flowchart of a light field image rendering method based on single-view loop of the present invention;

[0121] Figure 3 It is another schematic flowchart of a simplified rendering method of a light field image of the present invention;

[0122] Figure 4A schematic flowchart of a light field image rendering method based on the multi-viewpoint rendering (MVR) algorithm according to the present invention;

[0123] Figure 5 A schematic diagram of the change of the triangle orientation with the camera viewpoint according to the present invention;

[0124] Figure 6 A schematic diagram of performing the first-stage back-face culling according to the present invention;

[0125] Figure 7 A schematic diagram of performing the second-stage back-face culling according to the present invention;

[0126] Figure 8 A schematic rendering flowchart of performing the second-stage back-face culling according to the present invention;

[0127] Figure 9 A schematic structural diagram of a simplified rendering system for the light field image according to the present invention. Detailed implementation manners

[0128] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0129] On the contrary, the present invention covers any alternatives, modifications, equivalent methods and solutions made within the spirit and scope of the present invention defined by the claims. Further, in order to enable the public to have a better understanding of the present invention, some specific details are described in detail in the following detailed description of the present invention. Those skilled in the art can fully understand the present invention without the description of these details.

[0130] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0131] The first specific embodiment of the simplified rendering method for the light field image according to the present invention:

[0132] A simplified rendering method for a light field image, including the following:

[0133] Through a pre-constructed topology reconstruction unit, based on the level-of-detail algorithm, the high-polygon model HPM of the light field image is reconstructed to obtain a low-polygon model LPM;

[0134] Based on a pre-constructed model simplification processing unit, the back-face culling technology is used to perform back-face culling on the low-polygon model LPM to obtain a simplified low-polygon model LPM;

[0135] Using a pre - constructed high - polygon feature acquisition unit, bake the normal information of the high - polygon model HPM into the texture coordinates of the low - polygon model LPM according to the model baking technology to obtain normal map data;

[0136] Adopt a pre - constructed fusion rendering unit to couple - render the normal map data and the simplified low - polygon model LPM to achieve fast rendering of the light - field image.

[0137] The second specific embodiment of the simplified rendering method of the light - field image of the present invention:

[0138] A simplified rendering method of a light - field image, comprising the following steps:

[0139] In the first step, read the model and use next - generation model optimization technology to convert the complex high - polygon model HPM into a low - polygon model LPM, and obtain the optimized low - polygon model LPM.

[0140] In the second step, use the model baking technology to bake the normal information of the high - polygon model HPM into the texture coordinates of the low - polygon model LPM and store it as a normal map.

[0141] In the third step, use the back - face culling technology to perform back - face culling on the optimized low - polygon model LPM to obtain a simplified low - polygon model LPM.

[0142] In the fourth step, use the simplified low - polygon model LPM and the normal map of the high - polygon model HPM to render a high - precision light - field image.

[0143] The present invention can effectively reduce the number of vertices and triangles, improve the rendering efficiency, and at the same time maintain a high image quality, thus enabling fast rendering of large - data - volume and high - quality light - field images.

[0144] The third specific embodiment of the simplified rendering method of the light - field image of the present invention:

[0145] A simplified rendering method of a light - field image, including the following:

[0146] 1. Read the three - dimensional model data and the camera array parameters corresponding to the light - field acquisition;

[0147] 2. Use next - generation model optimization technology to convert the complex high - polygon model HPM into a low - polygon model LPM. The model optimization technology includes using the level - of - detail algorithm LOD technology to perform topology reconstruction on the high - polygon model HPM to generate a set of low - polygon models LPM with different levels of detail.

[0148] 3. Render the low - polygon models LPM at different levels of detail. Select the low - polygon model LPM with the fewest number of triangles and the rendering quality closest to the high - polygon model HPM as the optimized low - polygon model LPM.

[0149] 4. Use model baking technology to bake the normal information of the high - polygon model HPM into the texture coordinates of the optimized low - polygon model LPM and store it as a normal map.

[0150] 5. Use back - face culling technology to perform back - face culling on the optimized low - polygon model LPM to obtain a simplified low - polygon model LPM.

[0151] The back - face culling technology includes: In the first stage, cull the triangles that are always facing away from the camera array. In the second stage, find the triangles whose visibility changes as the camera viewpoint changes and cull those triangles that are facing away from the camera. This includes: triangles that change from facing the camera array to facing away from the camera; or triangles that change from facing away from the camera to facing the camera.

[0152] 6. Render a high - precision light - field image using the simplified low - polygon model LPM and the normal map of the high - polygon model HPM.

[0153] The rendering method of the high - precision light - field image includes, but is not limited to, single - view loop rendering method, multi - view rendering (MVR) algorithm, etc.

[0154] The single - view loop rendering method and the multi - view rendering (MVR) algorithm preferably adopt a parallel computing architecture based on GPU; or adopt a hybrid computing architecture based on GPU + CPU. The second - preferred option is to adopt a serial computing architecture based on CPU.

[0155] The rendering method of the high - precision light - field image includes the rendering of lighting effects.

[0156] The present invention significantly reduces the data volume of the model by using model optimization technology, thereby significantly reducing the computational amount during the rendering process of the light - field image, improving the rendering speed, and maintaining high - quality image effects at the same time.

[0157] The present invention further reduces the data volume of the model by using back - face culling technology, thereby accelerating the rendering of the light - field image.

[0158] The present invention can quickly calculate a large number of light - field images with complex lighting, shadow, and texture effects by using a parallel rendering method, significantly enhancing the realism of the light - field image.

[0159] The fourth specific embodiment of the simplified rendering method of the light - field image of the present invention:

[0160] A simplified rendering method for light field images is a simplified rendering method for light field images based on single-view loop rendering, including the following:

[0161] As Figure 1 shown, first read the 3D model data, then set the parameters of the camera array, including n rows and m columns of cameras. Then use the next-generation model optimization technology to convert the complex high-polygon model HPM into a low-polygon model LPM. The model optimization technology uses the level of detail (LOD) technology to perform topological reconstruction on the high-polygon model HPM, generating a set of low-polygon models LPM with different levels of detail. For example, the number of triangles in each level of detail of the low-polygon model LPM is half of the previous level.

[0162] Then render these low-polygon models LPM with different levels of detail and test the image quality. Select the low-polygon model LPM with the fewest triangles and the rendering quality closest to the high-polygon model HPM as the optimized low-polygon model LPM.

[0163] Next, use the model baking technology to bake the normal information of the high-polygon model HPM into the texture coordinates of the optimized low-polygon model LPM and store it as a normal map.

[0164] After that, use the two-stage back-face culling technology to obtain the simplified low-polygon model LPM. The two-stage back-face culling technology includes: in the first stage, cull the triangles that are always facing away from the camera array; in the second stage, cull the triangles that become invisible as the camera viewpoint changes. Then use the optimized low-polygon model LPM, the normal map, and the Phong lighting model to perform single-view loop rendering, and finally output the light field image.

[0165] As Figure 2 shown, the single-view loop rendering method first inputs 3D data such as the optimized low-polygon model LPM and the normal map. Then set the parameter matrix of the camera to be rendered according to the Figure 1 camera array parameters in. Then perform view model transformation, perspective projection transformation, and viewport transformation on the model according to the matrix parameters, then perform lighting calculation according to the Phong lighting model, and then rasterize to output a view captured by a camera. Then determine whether there are still cameras not rendered. If so, update the set camera parameter matrix and enter the next rendering loop until all cameras are rendered, and finally output the light field image.

[0166] The fifth specific embodiment of the simplified rendering method for light field images of the present invention:

[0167] A simplified rendering method for light field images is a simplified rendering method for light field images based on the multi-view rendering (MVR) algorithm, including the following:

[0168] As Figure 3 shown, first read the 3D model data, then set the parameters of the camera array, including n rows and m columns of cameras. Then use the next-generation model optimization technology to convert the complex high-polygon model HPM into a low-polygon model LPM. The model optimization technology uses the level of detail (LOD) technology to perform topological reconstruction on the high-polygon model HPM, generating a set of low-polygon models LPM with different levels of detail. For example, the number of triangles in each low-polygon model LPM of a detail level is half of the previous level.

[0169] Then render these low-polygon models LPM with different levels of detail and test the image quality. Select the low-polygon model LPM with the fewest triangles and the rendering quality closest to the high-polygon model HPM as the optimized low-polygon model LPM.

[0170] Next, use the model baking technology to bake the normal information of the high-polygon model HPM into the texture coordinates of the optimized low-polygon model LPM and store it as a normal map.

[0171] Then perform multi-view rendering (MVR) using the optimized low-polygon model LPM, the normal map, and the Blin-Phong lighting model, and finally output the light field image.

[0172] In this embodiment, an implementation process of the multi-view rendering (MVR) method is as follows:

[0173] First, initialize and input 3D data such as the optimized low-polygon model LPM and the normal map through the CPU, and send them into the vertex shader of the GPU rendering pipeline. Then assemble the vertex data into triangle (Triangles) primitives and pass them into the geometry shader. In the geometry shader, calculate the vertex positions of each triangle in the screen coordinate system according to the left and right camera parameters.

[0174] Next, calculate the directed areas of the triangles in the left and right camera spaces respectively based on these vertex positions, and perform the first-stage back-face culling.

[0175] Then, on the one hand, the transform feedback technology is used to map attributes such as vertex positions, texture coordinates, and vertex normals back to the local end; on the other hand, the scene in the light source view space is rasterized to generate a shadow map. In the second stage, on the CPU side, the triangles are sliced using the scan-line algorithm, and the slice list is sorted from bottom to top according to the y value of the scan line. Then, the data of the scan-line slices is passed to the GPU. In the vertex shader, the vertex data is converted to Normalized Device Coordinates (NDC), and then assembled into line primitives and passed into the geometry shader. In the geometry shader, the second-stage back-face culling is performed, and then the slice endpoint data is assembled into geometric primitives in the form of a triangle strip (Polygon Slice Track, PST). In the fragment shader, the lighting information is calculated using the normal map, and the occlusion situation is judged through the shadow map to calculate the shadow parameters. Finally, the final color of each fragment is comprehensively calculated and rendered to the FBO frame buffer to generate an EPI image. Finally, the EPI is converted into a light field image.

[0176] In this embodiment, as Figure 4 shown, another implementation process of the multi-view rendering (MVR) method is as follows:

[0177] The multi-view rendering (MVR) method first initializes and inputs three-dimensional data such as an optimized low-polygon model LPM and a normal map through the CPU, and sends them into the vertex shader of the GPU rendering pipeline for the first-stage processing. Then, the vertex data is assembled into triangle primitives and passed into the geometry shader. In the geometry shader, according to the left and right camera parameters, the vertex data is converted into screen coordinates in the left and right viewpoints, that is, the vertex positions of each triangle in the screen coordinate system are calculated.

[0178] Next, the directed areas of the triangles in the left and right camera spaces are calculated respectively based on these vertex positions, the first-stage back-face culling is performed to obtain the assembled primitive triangles, and stream output is performed, and they are transmitted to the transform feedback buffer.

[0179] The transform feedback buffer is created based on transform feedback technology and can map attributes such as vertex positions, texture coordinates, and vertex normals in primitive triangles back to the local end; then process the TFB data in units of triangles to complete the assembly of the TFB data; further slice each triangle along the scan line to obtain several slices; further create a slice list and store the slices in the slice list in the order of the y coordinates of the scan lines. Then use the vertex shader to convert the vertices of the slices into normalized device coordinates (NDC); perform primitive assembly of line segments based on the normalized device coordinates (NDC) to obtain line segment primitives; based on the line segment primitives, use the geometry shader to perform the second-stage back face culling and reconstruct the PST primitives. According to the PST primitives, perform primitive assembly of triangle strips to obtain primitive strips; rasterize the primitive strips to obtain strip rasterization data and input it into the fragment shader.

[0180] At the same time, perform a rasterization operation based on primitive triangles, rasterize the scene in the light source view space to generate a shadow map. Then, in the depth buffer, store the z values of the primitive triangles in the shadow map, and then transfer the shadow map to the fragment shader.

[0181] Finally, in the fragment shader, calculate the lighting information using the normal map and judge the occlusion situation through the shadow map to calculate the shadow parameters, and finally comprehensively calculate the final color of each fragment; then, in the frame buffer, save the rendering result of the PST primitives to a frame buffer to obtain the EPI; convert the EPI to a light field image and save it in another frame buffer. Finally, convert the EPI to a light field image.

[0182] A specific embodiment of the back face culling method of the present invention:

[0183] As Figures 5 - 8 shown, the back face culling method includes two stages, including the following content:

[0184] Since the perspective change of different viewpoints will affect the visibility of the same polygon under different perspectives, the back face culling can be divided into two stages: The first stage: judge and cull the invisible polygons within the entire field of view. The second stage: judge and cull the polygons whose visibility changes within the entire field of view. As Figure 5 shown is the change of the triangle orientation with the camera viewpoint. For a polygon relative to a camera on the same horizontal line, there are only three possible situations for its direction: 1) The triangle is always facing the back of the camera, such as face ①; 2) The triangle is facing the front for the left camera Camera right and is facing the front for the right camera Camera leftFor it, it is towards the back, and the visibility of the triangle transitions from one state to another, such as face ③; 3) The triangle is always facing the camera with its back, like face ②. The two-stage back-face culling method processes different triangles separately.

[0185] In this embodiment, the method for determining and culling invisible polygons within the entire field of view is as follows:

[0186] If a triangle is facing the camera with its back or facing the camera frontally with respect to both the leftmost and rightmost viewport cameras, then the polygon will maintain the same orientation and visibility with respect to other cameras between the leftmost and rightmost viewports. For this case, we can use the "Shoelace Formula" to determine the orientation of the polygon with respect to the camera. The formula is as follows:

[0187]

[0188] where x i and y i are respectively the screen coordinates x and y of the i-th vertex among the n vertices of the polygon. Through the Shoelace Formula, the directed area S of the polygon can be determined. When the directed area S > 0, the polygon is defined as visible; when the directed area S < 0, the polygon is defined as invisible. As Figure 6 shown, for the triangle data in the model, the directed areas S L and S R of the triangle under the leftmost and rightmost viewports are respectively calculated using the Shoelace Formula. If S L and S R are both less than 0, then the triangle is always facing the camera with its back, and the vertex data of this triangle will not be processed subsequently and will be directly culled.

[0189] In this embodiment, the method for determining and culling polygons whose visibility changes within the entire field of view is as follows:

[0190] If the orientation of a triangle changes with respect to the leftmost and rightmost viewport cameras, then the visibility of the triangle will also change with respect to other cameras between the leftmost and rightmost viewports. The PST formed by the slices on this triangle will intersect itself. To accurately distinguish the front and back of the PST, the principle of triangle similarity as Figure 7 shown can be used to determine the transition point of the PST. Using the positive and negative values of the triangle areas S L and S R calculated in the first stage, it can be further determined which parts of the PST divided by the transition point are the back. After culling these back parts, the correct PST primitive can be constructed.

[0191] Calculation Figure 7 For the two PST triangles shown in the right figure in L and d R The relationship can obtain the condition for determining the intersection of PST with itself:

[0192] d R ×d L < 0.

[0193] d L and d R can be calculated from the difference in the x - coordinates x 0L , x 0R , x 1L , x 1R of the slice endpoints. The formula is as follows:

[0194]

[0195] At the same time, for the self - intersecting PST, the proportional relationship k T and k R of the front and back parts in the perspective direction can be calculated through similar triangles. According to formulas (3) - (5), the position and its attribute data of the transition point v T can be calculated through interpolation between geometric parameters.

[0196]

[0197] v T = k T V OR +(1 - k T )V 0L (4)

[0198] a T = k T a0+(1 - k T )a1(5)

[0199] where k T represents the position of the transition point v L in the perspective space, and k R is the length of the entire perspective space. V 0L , V 0R represent the positions of the left endpoint of the slice in the extremely left and extremely right view - point spaces respectively. a T represents the attribute data of the transition point v T , including data such as the corresponding camera position, normal, tangent, and texture coordinates. a0 represents the attribute data of the left endpoint of the slice, and a1 represents the attribute data of the right endpoint of the slice.

[0200] Such as Figure 8Shown is the algorithm flow of back face culling in the second stage. When the slice data is passed into the geometry shader, we first need to determine whether the PST intersects with itself. If the product of d R and d L is less than 0, it indicates that the PST does not self-intersect. At this time, a quadrilateral primitive will be rendered. If the PST self-intersects, the data of the transition point v T needs to be calculated, and then the signs of the areas S L and S R are judged respectively. If S L is less than 0, it means that all the model triangles to which the slices observed by the camera between the left view point and the view point corresponding to the transition point face the back. Therefore, the slice vertex data V 0L , V 1L and the triangle formed by the transition point v T need to be culled on the PST. And the triangle primitive composed of V 0R , V 1R and v T is constructed. On the contrary, if S R is less than 0, then the triangle primitive composed of V 0L , V 1L and v T is constructed. By setting different primitives in this way, we can effectively reduce the number of pixels that need to be calculated in the rasterization stage and improve the rendering efficiency to a certain extent.

[0201] The first specific embodiment of the simplified rendering system of the light field image of the present invention:

[0202] A simplified rendering system for light field images, including a rendering pipeline and rendering hardware;

[0203] The rendering pipeline is used to implement the simplified rendering of the light field image, and includes an application program, an application program interface, a serial computing architecture, a general-purpose GPU computing architecture, and an operating system;

[0204] The rendering hardware is used to provide the hardware resources required to run the rendering pipeline, and includes a memory, a CPU, a GPU, and a video memory;

[0205] The operating system is used to control the application program and call the application program interface, the serial computing architecture, and the general-purpose parallel computing architecture;

[0206] The application program, the application program interface, the serial computing architecture, and the general-purpose parallel computing architecture drive the CPU, the GPU, the memory, and the video memory, and when executed, implement the above-mentioned simplified rendering method for light field images to complete the fast simplified rendering from three-dimensional model data to light field images.

[0207] As Figure 9As shown below, the second specific embodiment of the simplified rendering system for light field images of the present invention:

[0208] A simplified rendering system for light field images, which is embedded in a computer system. The computer system includes an operating system that provides a running environment for the rendering system. The rendering system includes an application program that calls the simplified rendering pipeline for light field images, and the rendering pipeline calls an application programming interface and a general-purpose GPU computing architecture. The application program, the application programming interface, and the general-purpose GPU computing architecture drive computing hardware such as the CPU, GPU, memory, and video memory to complete the rendering from 3D model data to light field images.

[0209] An apparatus embodiment applying the method of the present invention:

[0210] An electronic device, which includes:

[0211] One or more processors;

[0212] A storage device for storing one or more programs;

[0213] When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the above-mentioned simplified rendering method for light field images.

[0214] A computer medium embodiment applying the method of the present invention:

[0215] A computer-readable storage medium having stored thereon a computer program, which when executed by a processor implements the above-mentioned simplified rendering method for light field images.

[0216] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, and computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program code.

[0217] The present application is described according to the flowcharts and / or block diagrams of the methods, apparatuses (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate for implementation in the processFigure 1 one process or multiple processes or / and boxes Figure 1 a device for the functions specified in one box or multiple boxes.

[0218] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device that implements the functions in the process Figure 1 one process or multiple processes or / and boxes Figure 1 in one box or multiple boxes.

[0219] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operating steps are performed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions in the process Figure 1 one process or multiple processes or / and boxes Figure 1 in one box or multiple boxes.

[0220] The unit in this application is an object that constitutes an objective description of the morphological structure by means of an entity or a virtual representation. The object is not equal to an object and is not limited to an entity or a virtual one. It can be a data processing function, a software program, a processing mode, a usage method, an operation method, a work process, an application process, electronic hardware, a circuit module, a processing system, a system imitation, or a simulation object.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art can still modify or equivalently replace the specific implementation manners of the present invention. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A simplified rendering method for a light field image, characterized in that: Includes the following: Through the pre-built topology reconstruction unit, based on the detail level algorithm, the high polygon model HPM of the light field image is reconstructed to obtain the low polygon model LPM; Based on a pre-built model simplification processing unit, a back-face culling technology is used to perform back-face culling on the low-polygon model LPM to obtain a simplified low-polygon model LPM; Using the pre-built high polygon feature acquisition unit, the normal information of the high polygon model HPM is baked into the texture coordinates of the low polygon model LPM according to the model baking technology to obtain the normal map data; A pre-built fusion rendering unit is used to couple the normal map data with the simplified low polygon model LPM for rendering, thus achieving fast rendering of light field images.

2. The simplified rendering method of a light field image according to claim 1, characterized in that: The method of reconstructing the high polygon model HPM of the light field image by the pre-built topology reconstruction unit based on the detail level algorithm to obtain the low polygon model LPM is as follows: Obtain a high polygon model HPM about the light field image, which is three-dimensional model data; Using a detail level algorithm, topological reconstruction is performed on the three-dimensional model data to generate a group of low-polygon model objects with different detail levels, and the number of triangles of the low-polygon model object at each detail level is half the number of triangles of the low-polygon model object in the model object at the previous level; Then, the low-polygon model objects at different detail levels are rendered to obtain a group of low-polygon rendering objects; Testing the image quality of each low-polygon rendering object, and obtaining triangle information and rendering quality data of the low-polygon rendering object; According to the triangle information and the rendering quality data, a low-polygon rendering object with the least triangle number and a rendering quality closest to the high-polygon model HPM is selected as the final optimized low-polygon model LPM.

3. The simplified rendering method of a light field image according to claim 2, characterized in that: Based on the pre-built model simplification processing unit, the back face culling technology is used to perform back face culling on the low polygon model LPM, and the method for obtaining the simplified low polygon model LPM is as follows: Get the triangle information and camera array parameters of the low polygon model LPM; According to the triangle information and the camera array parameters, the invisible triangles in the entire field of view are obtained, and the triangles whose visibility has changed in the entire field of view are obtained; The first stage of backface culling: based on invisible triangles, culling triangles whose back faces always face the camera array; The second stage is back-face culling: based on the triangles whose visibility changes, the triangles that become invisible as the camera viewpoint changes are culled, thereby simplifying the low-polygon model LPM to obtain a simplified low-polygon model LPM.

4. The simplified rendering method of a light field image according to claim 2, characterized in that: Based on the pre-built model simplification processing unit, the back face culling technology is used to perform back face culling on the low polygon model LPM, and the method for obtaining the simplified low polygon model LPM is as follows: Get the low polygon model LPM and send it to the vertex shader of the GPU rendering pipeline to get vertex data; The vertex data is then assembled into triangles and passed to the geometry shader. In the geometry shader, the vertex position of each triangle in the screen coordinate system is calculated according to the left and right camera parameters. Next, the directed areas of the triangles in the left and right camera spaces are calculated based on the positions of these vertices, and the first stage of backface culling is performed; The triangle is sliced ​​using the scan line algorithm, and the slice list is sorted from bottom to top according to the y value of the scan line to obtain the slice scan line data; Next, the slice scan line data is passed to the GPU, and the vertex data is converted into normalized device coordinates in the vertex shader, assembled into line segment primitives and passed to the geometry shader, and the second stage of backface culling is performed in the geometry shader to obtain a simplified low polygon model LPM.

5. The simplified rendering method of a light field image according to claim 3, characterized in that: According to the triangle information and camera array parameters, the method to obtain the invisible triangles in the entire field of view is as follows: According to the triangle information, obtain the vertex coordinates of each triangle; According to the camera array parameters, determine the extreme left viewpoint position information and the extreme right viewpoint position information; Using the shoelace formula, and based on the vertex coordinates of the triangle, as well as the extreme left viewpoint position information and the extreme right viewpoint position information, calculate the directed area of ​​each triangle under the extreme left viewpoint - S L and the directed area S from the extreme right viewpoint R ; The shoelace formula is as follows: where x i is the screen X-axis coordinate value of the i-th vertex among the n vertices of the triangle, y i is the screen Y-axis coordinate value of the i-th vertex among the n vertices of the triangle, and S is the directed area of ​​the triangle; When the S of a triangle L and S R If it is less than 0 at the same time, the back of the triangle always faces the camera, and this triangle is an invisible triangle, so the vertex data of this triangle will not be processed subsequently.

6. The simplified rendering method of a light field image according to claim 5, characterized in that: According to the triangle information and the camera array parameters, the method of obtaining the triangle whose visibility changes in the entire field of view is as follows: According to the triangle information, the triangle is cut to obtain several primitives PST; Calculate the length d of the base of the two triangles in the primitive PST L and d R Determine whether the primitive will intersect with itself. If d R With d L If the product of is less than 0, it means that the primitive PST does not intersect itself, and a quadrilateral primitive is rendered; If d R With d L The product of is not less than 0, indicating that the primitive PST will intersect itself, so it is necessary to calculate the transition point v T data; According to the transition point v T Data, calculate the directed area of ​​the triangle S L and the directed area S R ; Then determine the area S L and S R The positive and negative of S L and S R The positive or negative value determines whether each primitive PST is located at the back or the front; If S L If it is less than 0, it means that the triangles to which the primitives are observed by the camera between the left viewpoint and the viewpoint corresponding to the transition point are all facing the back side, so as to obtain the triangles whose visibility changes in the entire field of view and the parts that become invisible; Calculate the transition point v T The data method is as follows: Get the endpoint data of the slice, which includes the X-axis coordinates x of the four endpoints 0L ,x 0R ,x 1L ,x 1R ; Based on the endpoint data, calculate x 0L ,x 0R ,x 1L ,x 1R The difference between the two triangle base lengths d L and d R , and its calculation formula is as follows: Based on d L and d R , and calculate the proportional relationship k between the front part and the back part in the perspective direction through similar triangles T and k R , and its calculation formula is as follows: Among them, k T Indicates the transition point v T The position in perspective space, k R is the length of the entire perspective space; According to the proportional relationship k T and k R , by interpolating between geometric parameters, the transition point v is calculated T The location and attribute data of the , the calculation formula is as follows: v T =k T V 0R +(1-k T )V 0L a T =k T a0+(1-k T )a1 Among them, V 0L 、V 0R Respectively represent the position of the left endpoint of the slice in the extreme left and extreme right viewpoint space; a T Indicates the transition point v T The attribute data of the slice includes the corresponding camera position, normal, tangent, and texture coordinates. a0 represents the attribute data of the left endpoint of the slice, and a1 represents the attribute data of the right endpoint of the slice.

7. The simplified rendering method of a light field image according to claim 1, characterized in that: The method of using a pre-built fusion rendering unit to couple the normal map data with the simplified low polygon model LPM to achieve fast rendering of light field images is as follows: Input normal map data and simplified low polygon model LPM; Then, according to the camera array parameters, the parameter matrix of the camera to be rendered is set; Then, according to the parameter matrix, the low polygon model LPM is transformed into a view model, a perspective projection, and a viewport. Then, the illumination is calculated based on the normal map data and Phong's illumination model, and the occlusion is determined by the shadow map to calculate the shadow parameters, and the final color of each fragment is obtained by comprehensive calculation; Then, the rasterization output is a view captured by a camera, and it is determined whether there are cameras that have not been rendered. If there are, the camera parameter matrix is ​​updated and set, and the next rendering cycle is entered until all cameras have been rendered. Finally, all light field images are output to achieve fast rendering of light field images.

8. A simplified rendering method for light field images, characterized in that: Includes the following: Read the camera array parameters corresponding to the high polygon model HPM and light field acquisition; The high polygon model HPM is converted into a low polygon model LPM by using a model optimization technology; the model optimization technology includes topologically reconstructing the high polygon model HPM by using a detail level algorithm LOD technology to generate a group of low polygon models LPM with different detail levels; Rendering the low polygonal models LPM of different detail levels; selecting the low polygonal model LPM with the least number of triangles and the rendering quality closest to the high polygonal model HPM as the optimized low polygonal model LPM; Using a model baking technique, the normal information of the high polygon model HPM is baked into the texture coordinates of the optimized low polygon model LPM, and stored as a normal map; Using a back face culling technique to perform back face culling on the optimized low polygon model LPM to obtain a simplified low polygon model LPM; Use simplified low polygon model LPM and normal map for rendering to obtain high-precision light field images.

9. A simplified rendering method for a light field image, characterized in that: The simplified rendering method is a multi-view rendering method, which Includes the following: First, the optimized low polygon model LPM is initialized and input through the CPU, and then sent to the vertex shader of the GPU rendering pipeline for the first stage of processing; The first stage of processing includes the following: Assemble the vertex data into triangle primitives and pass them into the geometry shader; in the geometry shader, according to the left and right camera parameters, convert the vertex data into screen coordinates in the left and right viewpoints, that is, calculate the vertex position of each triangle in the screen coordinate system; Next, the directed areas of the triangles in the left and right camera spaces are calculated based on the positions of these vertices, and the first stage of backface culling is performed to obtain assembled primitive triangles, which are then streamed out and transmitted to the transform feedback buffer. The transform feedback buffer is created based on the transform feedback technology, which can map the vertex positions, texture coordinates, and vertex normals in the primitive triangle back to the local end; Then the TFB data is processed in units of triangles to complete the assembly of the TFB data; each triangle is then sliced ​​along the scan line to obtain a number of slices; Then create a slice list and store the slices in the slice list in the order of the y coordinates of the scan line and pass it to the vertex shader; At the same time, rasterization is performed based on primitive triangles to rasterize the scene in the light source viewpoint space to generate a shadow map. Then, in the depth buffer, the z value of the primitive triangle is stored in the shadow map, and then the shadow map is transmitted to the fragment shader for the second stage of processing; The second stage of processing includes the following: The vertex shader is used to convert the vertices of the slice into normalized device coordinates; the primitives of the line segment are assembled based on the normalized device coordinates (NDC) to obtain the line segment primitives; Based on the line segment primitives, the geometry shader is used to perform the second stage backface culling and rebuild the PST primitives; According to the PST primitives, the primitives of the triangle strips are assembled to obtain the primitive strips; Rasterize the primitive strips, obtain the strip rasterization data, and input it into the fragment shader; Then, in the fragment shader, the normal map is used to calculate the lighting information, and the shadow map is used to determine the occlusion situation to calculate the shadow parameters, and then the final color of each fragment is obtained by comprehensive calculation; In the frame buffer, the rendering result of the PST primitive is saved to a frame buffer to obtain the EPI; the EPI is converted into a light field image and saved in another frame buffer, thereby completing the simplified rendering of the light field image.

10. A simplified rendering system for light field images, characterized in that: It has a rendering pipeline and rendering hardware; The rendering pipeline is used to implement simplified rendering of light field images, which includes an application program, an application program interface, a serial computing architecture, a general GPU computing architecture, and an operating system; The rendering hardware is used to provide the hardware resources required to run the rendering pipeline, including memory, CPU, GPU and video memory; The operating system is used to control applications, call application program interfaces and serial computing architectures, and general parallel computing architectures; The application, application program interface, serial computing architecture and general parallel computing architecture drive the CPU, GPU, memory and video memory, and when executed, implement a simplified rendering method for a light field image as described in any one of claims 1 to 9, completing fast simplified rendering from three-dimensional model data to light field images.

Images