A three-dimensional model rendering method and apparatus

Abstract

The application provides a three-dimensional model rendering method, comprising the following steps: S1, controlling a virtual camera to emit a rendering light and determining a plurality of intersection point coordinates; S2, determining a direction vector of a rendering light emitted by the virtual camera corresponding to one of the intersection point coordinates based on parameters of the virtual camera and the one of the intersection point coordinates; S3, determining a camera direction vector group based on a coordinate of each RGB camera and the one of the intersection point coordinates; S4, determining a plurality of RGB cameras used for texture calculation based on an included angle relationship between the direction vector of the rendering light and the camera direction vector group; S5, determining a texture value corresponding to the one of the intersection point coordinates based on a plurality of images captured by the plurality of RGB cameras; S6, continuing to perform the steps of S2-S5 until a plurality of texture values are determined respectively; and S7, determining a viewpoint image corresponding to the virtual camera based on the plurality of texture values, so as to realize fast rendering of the three-dimensional model.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging technology, and in particular to a three-dimensional model rendering method and apparatus. Background Technology

[0002] In recent years, with the development of computer vision and computer graphics, 3D model reconstruction and texture mapping based on multi-view images for multi-view rendering and 3D display have been increasingly widely used in education, healthcare, and augmented reality. Texture mapping based on multi-view images contains essential lighting and object texture information of the scene. Compared with traditional 2D texture mapping, it can more realistically and vividly reproduce real-world scenes, achieving richly detailed multi-view rendering effects. Therefore, 3D model texture mapping and multi-view rendering methods suitable for real-world scenes have received increasing attention.

[0003] However, texture mapping and multi-view rendering methods for 3D models still suffer from issues such as texture aliasing, texture indexing errors, and slow texture mapping and rendering speeds. In practical applications, RGB camera arrays in 3D space are often arbitrarily arranged, their positions do not follow a strictly uniform distribution, and the directions of the camera optical centers are rarely converged to the same point in the 3D scene. Furthermore, the positions of the 3D models captured by the RGB camera arrays are also arbitrary; that is, the 3D models are often not located at the center of the captured 3D scene. Texture mapping methods based on depth information can calculate the texture of each 3D point of the 3D model in real time based on the depth map captured by each camera, but they are not suitable for applications based on RGB camera arrays. RGB camera arrays struggle to obtain the depth map of every point of the 3D model through matching or depth estimation. Secondly, even if a depth map is obtained, due to the arbitrary distribution of RGB cameras and the 3D model, the distribution and occlusion relationships of the entire 3D scene cannot be obtained from the depth map of a single camera, leading to texture aliasing.

[0004] Therefore, in practical applications, how to quickly and accurately perform texture mapping and multi-view rendering of 3D models using multi-view information collected by RGB camera arrays in complex scenes is an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a three-dimensional model rendering method and apparatus to solve the problems of texture aliasing and slow rendering speed in the mapping and rendering of three-dimensional models in the prior art.

[0006] This invention provides a three-dimensional model rendering method and apparatus, comprising: S1, controlling a virtual camera to emit rendering rays, wherein the rendering rays intersect with a three-dimensional model whose texture is to be calculated, and determining multiple intersection point coordinates; S2, based on the parameters of the virtual camera and one of the multiple intersection point coordinates, determining a rendering ray emitted by the virtual camera corresponding to the one of the intersection point coordinates and the direction vector of the rendering ray; S3, based on the coordinates of each RGB camera in the RGB camera array and the one of the intersection point coordinates, determining a camera direction vector group, wherein the camera direction vector group includes the camera direction vector from each RGB camera to the one of the intersection point coordinates; S4, based on the angle relationship between the direction vector of the rendering ray and the camera direction vector group, determining multiple RGB cameras for texture calculation; S5, based on multiple images captured by the multiple RGB cameras for texture calculation, determining the texture value corresponding to the one of the intersection point coordinates; S6, continuing to execute steps S2-S5 until multiple texture values ​​corresponding to the multiple intersection point coordinates are determined respectively; S7, determining the viewpoint image corresponding to the virtual camera based on the multiple texture values ​​corresponding to the multiple intersection point coordinates.

[0007] In some embodiments, determining multiple RGB cameras for texture calculation based on the angle relationship between the direction vector of a rendering ray and the camera direction vector group includes: determining the minimum angle RGB camera corresponding to the camera direction vector with the smallest angle to the direction vector of a rendering ray in the camera direction vector group, and determining multiple RGB cameras for texture calculation based on the minimum angle RGB camera.

[0008] In some embodiments, determining a plurality of RGB cameras for texture calculation based on the minimum angle RGB camera includes: determining a RGB cameras located on one side of the minimum angle RGB camera that are close to the minimum angle RGB camera, determining a RGB cameras located on the other side of the minimum angle RGB camera that are close to the minimum angle RGB camera, and using the minimum angle RGB camera, the a RGB cameras on one side, and the a RGB cameras on the other side as a plurality of RGB cameras for texture calculation.

[0009] In some embodiments, determining the minimum angle RGB camera corresponding to the camera direction vector with the smallest angle to the direction vector of the rendering ray in the camera direction vector group includes: based on the formula The minimum included angle RGB camera is determined, where n s P0 represents the index of the RGB camera with the smallest included angle, and P0 represents the coordinates of one of the intersection points. This represents the direction vector pointing towards P0 of a rendering ray emitted by the virtual camera, corresponding to the coordinates of one of the intersection points. This represents the position of the nth real camera in 3D coordinates, where N represents the number of real cameras. This represents a function that calculates a unit vector.

[0010] In some embodiments, determining the texture value corresponding to the coordinates of one of the intersection points based on the multiple images captured by the multiple RGB cameras used for texture calculation includes: projecting the one of the intersection points into the multiple images captured by the multiple RGB cameras used for texture calculation to obtain multiple initial texture values ​​obtained by the multiple RGB cameras used for texture calculation observing the one of the intersection points, and obtaining the texture value corresponding to the coordinates of the one of the intersection points based on the multiple initial texture values.

[0011] In some embodiments, obtaining the texture value corresponding to one of the intersection coordinates based on the plurality of initial texture values ​​includes: assigning different weights to the plurality of initial texture values ​​respectively, wherein the initial texture value corresponding to the RGB camera with the smallest included angle has the highest weight, the sum of the different weights of the plurality of initial texture values ​​is 1, and obtaining the texture value corresponding to one of the intersection coordinates by weighted summation based on the plurality of initial texture values ​​and the weights corresponding to the plurality of initial texture values.

[0012] A three-dimensional light field rendering device includes: a control module for controlling a virtual camera to emit rendering rays, the rendering rays intersecting with a three-dimensional model of a texture to be calculated, and determining multiple intersection point coordinates; a determination module for determining a rendering ray emitted by the virtual camera corresponding to one of the multiple intersection point coordinates and a direction vector of the rendering ray, based on parameters of the virtual camera and one of the multiple intersection point coordinates; further for determining a camera direction vector group based on the coordinates of each RGB camera in an RGB camera array and the one of the intersection point coordinates, the camera direction vector group including camera direction vectors from each RGB camera to the one of the intersection point coordinates; further for determining multiple RGB cameras for texture calculation based on the angle relationship between the direction vector of the rendering ray and the camera direction vector group; further for determining a texture value corresponding to the one of the intersection point coordinates based on multiple images captured by the multiple RGB cameras for texture calculation; and further for continuing to execute the above steps of the determination module until multiple texture values ​​corresponding to the multiple intersection point coordinates are determined respectively; and an image module for determining a viewpoint image corresponding to the virtual camera based on the multiple texture values ​​corresponding to the multiple intersection point coordinates.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the three-dimensional model rendering methods described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the three-dimensional model rendering methods described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the three-dimensional model rendering methods described above.

[0016] This invention provides a three-dimensional model rendering method and apparatus. By using multiple selected RGB cameras for texture calculation, the texture value corresponding to the intersection coordinates of the rendering light rays and the three-dimensional model is calculated, instead of calculating the texture value based on all cameras. This allows for fast or real-time rendering of the target scene, enabling the generation of multi-view images, saving time and computational costs. Furthermore, since the selected multiple RGB cameras for texture calculation have a similar viewing angle to the virtual camera, the acquired texture is close to the texture from the virtual camera's perspective, enabling correct texture selection and calculation, and avoiding texture aliasing. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the OpenGL coordinate system and the OpenCV coordinate system in this invention;

[0019] Figure 2 This is a flowchart illustrating a three-dimensional model rendering method provided by the present invention;

[0020] Figure 3 This is a schematic diagram of the virtual camera, RGB camera array, and 3D model arrangement of the present invention;

[0021] Figure 4 This is a schematic diagram of the GPU multi-threaded computing process provided by the present invention;

[0022] Figure 5 This is a schematic diagram of the structure of a three-dimensional light field rendering device provided by the present invention;

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] Before implementing the method of this invention, it is necessary to unify the three-dimensional coordinate system. For example, in the field of three-dimensional reconstruction technology, since the OpenGL (Open Graphics Library) coordinate system is used for rendering three-dimensional models and the OpenCV (Open Source Computer Vision Library) coordinate system is used for reconstructing three-dimensional models, unifying the three-dimensional coordinate system is to avoid situations such as the rendering result being mirror-symmetric with the three-dimensional model or the screen being flipped without rendering due to different coordinate systems. Figure 1 This is a schematic diagram of the OpenGL coordinate system and the OpenCV coordinate system in this invention. Because the OpenGL coordinate system and the OpenCV coordinate system are defined differently, both coordinate systems need to be considered together. The OpenGL coordinate system is transformed into the OpenCV coordinate system using a transformation matrix, where the transformation matrix is:

[0026]

[0027] In the subsequent rendering methods of this invention, the virtual camera coordinates, RGB camera array coordinates, etc., are all based on the unified OpenCV coordinate system.

[0028] The following is combined Figures 2-4 This invention describes a three-dimensional model rendering method.

[0029] Figure 2 This is a flowchart illustrating a three-dimensional model rendering method provided by the present invention, including steps S1 to S7:

[0030] Step S1: Control the virtual camera to emit rendering rays, and determine the coordinates of multiple intersection points between the rendering rays and the 3D model of the texture to be calculated.

[0031] A virtual camera can emit multiple rendering rays, and collision detection methods can be used to determine the coordinates of multiple intersection points between these rays and the 3D model of the texture to be calculated. Collision detection methods can utilize BVTree (Bounding Volume Tree) and its derivatives. BVTree can be implemented based on collision detection methods such as AABB (Axis Aligned Bounding Box), OBB (Oriented Bounding Box), and bounding spheres. For example, dynamic collision detection can be implemented using a Dynamic Bounding Volume Hierarchy Based On AABB Tree.

[0032] Step S2: Based on the parameters of the virtual camera and the coordinates of one of the multiple intersection points, determine a rendering ray emitted by the virtual camera corresponding to the coordinates of the one intersection point and the direction vector of the rendering ray.

[0033] The direction vector of a rendering ray represents the direction vector of a rendering ray emitted from the virtual camera position coordinates to one of the intersection point coordinates, with its direction being from the virtual camera position to one of the intersection points.

[0034] Step S3: Based on the coordinates of each RGB camera in the RGB camera array and the coordinates of one of the intersection points, determine the camera direction vector group, which includes the camera direction vector from each RGB camera to the coordinates of one of the intersection points.

[0035] An RGB camera is an industrial camera that uses the three primary colors as its system and has digital imaging capabilities. The position and orientation of an RGB camera array are arbitrarily distributed in three-dimensional space. In some embodiments, multiple cameras in an RGB camera array have different intrinsic and extrinsic parameters. Intrinsic parameters are parameters related to the camera's own characteristics, such as its focal length and optical center. Extrinsic parameters are external parameters of the camera in the world coordinate system, such as its position, rotation direction, and translation direction.

[0036] In some embodiments, the coordinates of the nth RGB camera in the three-dimensional coordinate system can be determined by the following formula (1). The nth T is the rotation matrix of the camera's extrinsic parameters. n Let be the translation matrix of the extrinsic parameters of the nth camera, where N represents the number of RGB cameras.

[0037]

[0038] The camera orientation vector set comprises a group of orientation vectors consisting of camera orientation vectors from the coordinates of each RGB camera to one of the intersection points. The actual camera orientation vectors in the camera orientation vector set are orientation vectors from each RGB camera coordinate to one of the intersection points, and the direction of each ray contained in the camera orientation vector set points from each actual camera position to one of the intersection points.

[0039] Step S4: Based on the angle relationship between the direction vector of the rendering ray and the camera direction vector group, determine multiple RGB cameras for texture calculation.

[0040] Because RGB cameras are arbitrarily distributed in 3D space, and irregular 3D models have a certain thickness, they exhibit self-occlusion when observed from a certain position, and their position is not necessarily at the center of 3D space. In complex scenes, the intersection points formed by the rendering rays emitted by the virtual camera and the 3D model are often obscured by the 3D model itself when observed by the RGB cameras, due to the different observation positions of the RGB cameras. Therefore, in the above situations, the color values ​​of the intersection points observed by the RGB cameras that are obscured by the 3D model will be incorrect, and not all intersection point color values ​​observed by all RGB cameras can be used for solving the problem. Therefore, for one observation intersection point of the 3D model, this invention uses the angle relationship between the direction vector of a rendering ray and the camera direction vector group to select multiple RGB cameras for texture calculation to calculate the texture of one observation point of the 3D model. Figure 3 This is a schematic diagram of the virtual camera, RGB camera array, and 3D model arrangement of the present invention.

[0041] In some embodiments, the minimum angle RGB camera corresponding to the camera direction vector with the smallest angle to the direction vector of the rendering ray in the camera direction vector group can be determined, and multiple RGB cameras for texture calculation can be determined based on the minimum angle RGB camera.

[0042] In some embodiments, the minimum included angle RGB camera can be determined based on the following formula (2), where n s P0 represents the index of the RGB camera with the smallest included angle, and P0 represents the coordinates of one of the intersection points. This represents the direction vector pointing towards P0 of a rendering ray emitted by the virtual camera, corresponding to the coordinates of one of the intersection points. This represents the position of the nth real camera in 3D coordinates, where N represents the number of real cameras. This represents a function to calculate a unit vector. It can be understood that the smaller the angle between the vector of a rendered ray and a certain RGB camera direction vector, the closer the angle between the RGB camera and the virtual camera corresponding to the camera direction vector. Therefore, at a model intersection point observed by this rendered ray, the observation result of the corresponding RGB camera will not be obscured by the 3D model itself.

[0043]

[0044] In some embodiments, *a* RGB cameras located on one side of the minimum-angle RGB camera and adjacent to it can be identified, and *a* RGB cameras located on the other side of the minimum-angle RGB camera and adjacent to it can be identified. The minimum-angle RGB camera, the *a* RGB cameras on one side, and the *a* RGB cameras on the other side are then used as a plurality of RGB cameras for texture calculation. For example, after identifying the minimum-angle RGB camera, the minimum-angle RGB camera, the *a* cameras near the left side of the minimum-angle RGB camera, and the *a* cameras near the right side of the minimum-angle RGB camera are all used as the plurality of RGB cameras for texture calculation. It is understood that the closer the RGB cameras are to the minimum-angle RGB camera, the less likely their viewing angle will be obstructed by the 3D model itself.

[0045] It is understandable that using multiple real-viewpoint images captured by different RGB cameras can reduce color jumps at the same model point when the virtual viewpoint moves continuously in 3D space. Determining the texture value corresponding to one of the intersection points using multiple RGB cameras can reduce the errors caused by a single RGB camera. The texture value corresponding to one intersection point represents the color value of the pixel corresponding to that intersection point.

[0046] Step S5: Based on the multiple images captured by the multiple RGB cameras used for texture calculation, determine the texture value corresponding to one of the intersection points.

[0047] In some embodiments, one of the intersection points can be perspective-projected onto multiple images captured by the plurality of RGB cameras used for texture calculation to obtain multiple initial texture values ​​of the intersection point observed by the plurality of RGB cameras used for texture calculation, and the texture value corresponding to the coordinates of the intersection point can be obtained based on the plurality of initial texture values.

[0048] In some embodiments, different weights can be assigned to the plurality of initial texture values, wherein the initial texture value corresponding to the minimum angle RGB camera has the highest weight, and the sum of the different weights of the plurality of initial texture values ​​is 1. Based on the plurality of initial texture values ​​and the weights corresponding to the plurality of initial texture values, a weighted sum is obtained to obtain the texture value corresponding to one of the intersection coordinates. For example, the texture value corresponding to one of the intersection coordinates can be determined by the following formula (3), where a is the number of RGB cameras on the left and right sides of the minimum angle RGB camera, and n s From the central viewpoint, A q For the weights of the q-th viewpoint, the function Indicates the nth captured s +q pictures And the two-dimensional projection coordinates of P0 in this image Extract the corresponding color value, where i, j, and k represent the unit vectors of the three coordinate axes in the OpenCV coordinate system.

[0049]

[0050] Under the observation of a virtual camera, the texture of the intersection coordinates of the 3D model and the color of the image rendered by the virtual camera are the same. This is because the RGB camera has already recorded factors such as lighting, model materials, and shadows in the real world during the acquisition process, so there is no need to further set virtual light sources and materials to adjust the texture mapping results. Since the lighting, materials, and shadows of the environment are realistically mapped onto the 3D model, the final displayed content is closer to reality, and the calculation process based on perspective projection and weight superposition has strong robustness. Therefore, the texture value of the intersection coordinates of the 3D model acquired by the RGB camera can be directly assigned to the sub-pixel corresponding to the intersection coordinates of the 3D model, and the texture value is used as the color value corresponding to this pixel in the rendering result.

[0051] Step S6: Continue executing steps S2-S5 until the multiple texture values ​​corresponding to the multiple intersection point coordinates are determined.

[0052] Steps S2-S5 generate a texture value for one intersection point coordinate. In some embodiments, steps S2-S5 can be executed using GPU multithreading to generate multiple texture values ​​corresponding to multiple intersection point coordinates. For example, steps S2-S5 can be executed in parallel using GPU multithreading to generate multiple texture values ​​corresponding to multiple intersection point coordinates. Each GPU thread is responsible for calculating one intersection point coordinate, and the calculation process is accelerated in parallel using CUDA (Compute Unified Device Architecture) to achieve real-time rendering of the 3D model. Figure 4This is a schematic diagram of the GPU multi-threaded continuous computing process provided by the present invention. Figure 4 As shown, after the calculation begins, parameter initialization and virtual camera parameter updates are performed first. Each virtual camera parameter update can be triggered by keyboard and mouse events. The number of virtual cameras, their observation matrix, position, orientation, and all other camera-related parameters can be updated according to actual needs. Then, the GPU kernel function is started to perform parallel computation to generate the virtual camera's rendered image (viewpoint image). If rendering is not yet complete, the virtual camera parameters are updated again, and the GPU kernel function is started again for parallel computation. The parallel computation includes: setting up a corresponding virtual camera array, emitting rendering rays through the virtual cameras, obtaining texture values ​​at multiple intersection points based on the rendering rays, and generating a rendered image based on the texture values ​​at multiple intersection points.

[0053] Step S7: Determine the viewpoint image corresponding to the virtual camera based on the texture values ​​corresponding to the coordinates of the multiple intersection points.

[0054] Once the texture values ​​of all intersections between the rendering ray and the 3D model are determined, the viewpoint image corresponding to the virtual camera can be obtained based on the texture values ​​of all intersections.

[0055] Figure 5 The three-dimensional light field rendering device provided by the present invention can be referred to in correspondence with the three-dimensional light field rendering method described above.

[0056] A three-dimensional light field rendering device includes: a control module 51, configured to control a virtual camera to emit rendering rays, the rendering rays intersecting with a three-dimensional model of a texture to be calculated, and determining multiple intersection point coordinates; a determination module 52, configured to determine a rendering ray emitted by the virtual camera corresponding to one of the multiple intersection point coordinates and a direction vector of the rendering ray, based on the coordinates of the virtual camera and one of the multiple intersection point coordinates; further configured to determine a camera direction vector group based on the coordinates of each RGB camera in an RGB camera array and the one of the intersection point coordinates, the camera direction vector group including camera direction vectors from each RGB camera to the one of the intersection point coordinates; further configured to determine multiple RGB cameras for texture calculation based on the angle relationship between the direction vector of the rendering ray and the camera direction vector group; further configured to determine the texture value corresponding to the one of the intersection point coordinates based on multiple images captured by the multiple RGB cameras for texture calculation; and further configured to continue executing the above steps of the determination module until multiple texture values ​​corresponding to the multiple intersection point coordinates are determined respectively.

[0057] Image module 53 is used to determine the viewpoint image corresponding to the virtual camera based on the texture values ​​corresponding to the coordinates of the plurality of intersection points.

[0058] Furthermore, in a three-dimensional light field rendering device provided by the present invention, the determining module 52 is further configured to: determine the minimum angle RGB camera corresponding to the camera direction vector with the smallest angle to the direction vector of the rendering ray in the camera direction vector group, and determine a plurality of RGB cameras for texture calculation based on the minimum angle RGB camera.

[0059] Furthermore, in a three-dimensional light field rendering apparatus provided by the present invention, the determining module 52 is further configured to: determine a RGB cameras located on one side of the minimum angle RGB camera that are adjacent to the minimum angle RGB camera, determine a RGB cameras located on the other side of the minimum angle RGB camera that are adjacent to the minimum angle RGB camera, and use the minimum angle RGB camera, the a RGB cameras adjacent on one side, and the a RGB cameras adjacent on the other side as a plurality of RGB cameras for texture calculation:

[0060] Furthermore, in a three-dimensional light field rendering apparatus provided by the present invention, the determining module 52 is further configured to: based on a formula The minimum included angle RGB camera is determined, where n s P0 represents the index of the RGB camera with the smallest included angle, and P0 represents the coordinates of one of the intersection points. This represents the direction vector pointing towards P0 of a rendering ray emitted by the virtual camera, corresponding to the coordinates of one of the intersection points. This represents the position of the nth real camera in 3D coordinates, where N represents the number of real cameras. This represents a function that calculates a unit vector.

[0061] Furthermore, in a three-dimensional light field rendering apparatus provided by the present invention, the determining module 52 is further configured to: project one of the intersection points into multiple images captured by the multiple RGB cameras used for texture calculation, obtain multiple initial texture values ​​obtained by the multiple RGB cameras used for texture calculation observing one of the intersection points, and obtain the texture value corresponding to the coordinates of one of the intersection points based on the multiple initial texture values.

[0062] Furthermore, in a three-dimensional light field rendering device provided by the present invention, the determining module 52 is further configured to: assign different weights to the plurality of initial texture values ​​respectively, wherein the initial texture value corresponding to the minimum angle RGB camera has the highest weight, the sum of the different weights of the plurality of initial texture values ​​is 1, and obtain the texture value corresponding to one of the intersection coordinates by weighted summation based on the plurality of initial texture values ​​and the weights corresponding to the plurality of initial texture values.

[0063] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a 3D model rendering method, which includes: S1, controlling a virtual camera to emit rendering rays, the rendering rays intersecting with the 3D model of the texture to be calculated, and determining multiple intersection point coordinates; S2, based on the parameters of the virtual camera and one of the multiple intersection point coordinates, determining a rendering ray emitted by the virtual camera corresponding to the one intersection point coordinate and the direction vector of the rendering ray; S3, based on the coordinates of each RGB camera in the RGB camera array and the one intersection point coordinate, determining a camera direction vector group, the... The camera direction vector group includes the camera direction vector from each RGB camera to one of the intersection point coordinates; S4, based on the angle relationship between the direction vector of the rendering ray and the camera direction vector group, determine multiple RGB cameras for texture calculation; S5, based on multiple images captured by the multiple RGB cameras for texture calculation, determine the texture value corresponding to one of the intersection point coordinates; S6, continue to execute steps S2-S5 until multiple texture values ​​corresponding to the multiple intersection point coordinates are determined respectively; S7, based on the multiple texture values ​​corresponding to the multiple intersection point coordinates, determine the viewpoint image corresponding to the virtual camera.

[0064] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0065] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the three-dimensional model rendering method provided by the above methods. The method includes: S1, controlling a virtual camera to emit rendering rays, the rendering rays intersecting with the three-dimensional model of the texture to be calculated, and determining multiple intersection point coordinates; S2, based on the parameters of the virtual camera and one of the multiple intersection point coordinates, determining a rendering ray emitted by the virtual camera corresponding to the one of the intersection point coordinates and the direction vector of the rendering ray; S3, based on the coordinates of each RGB camera in the RGB camera array and the one of the intersection point coordinates, determining a camera direction vector group, the camera direction vector group including the camera direction vector from each RGB camera to the one of the intersection point coordinates; S4, based on the angle relationship between the direction vector of the rendering ray and the camera direction vector group, determining multiple RGB cameras for texture calculation.

[0066] S5. Based on the multiple images captured by the multiple RGB cameras used for texture calculation, determine the texture value corresponding to one of the intersection coordinates; S6. Continue to execute steps S2-S5 until the multiple texture values ​​corresponding to the multiple intersection coordinates are determined respectively; S7. Based on the multiple texture values ​​corresponding to the multiple intersection coordinates, determine the viewpoint image corresponding to the virtual camera.

[0067] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a three-dimensional model rendering method provided by the above methods. The method includes: S1, controlling a virtual camera to emit rendering rays, the rendering rays intersecting with a three-dimensional model of a texture to be calculated, and determining multiple intersection point coordinates; S2, based on the parameters of the virtual camera and one of the multiple intersection point coordinates, determining a rendering ray emitted by the virtual camera corresponding to the one of the intersection point coordinates and the direction vector of the rendering ray; S3, based on the coordinates of each RGB camera in the RGB camera array and the one of the intersection point coordinates... S2-S5: Determine the camera direction vector group based on the coordinates of the points, wherein the camera direction vector group includes the camera direction vector from each RGB camera to one of the intersection point coordinates; S4: Determine multiple RGB cameras for texture calculation based on the angle relationship between the direction vector of the rendering ray and the camera direction vector group; S5: Determine the texture value corresponding to one of the intersection point coordinates based on the multiple images captured by the multiple RGB cameras used for texture calculation; S6: Continue to execute steps S2-S5 until multiple texture values ​​corresponding to the multiple intersection point coordinates are determined; S7: Determine the viewpoint image corresponding to the virtual camera based on the multiple texture values ​​corresponding to the multiple intersection point coordinates.

[0068] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0069] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of rendering a three-dimensional model, the method comprising: include: S1. Control the virtual camera to emit rendering rays, the rendering rays intersect with the 3D model of the texture to be calculated, and determine the coordinates of multiple intersection points; S2. Based on the parameters of the virtual camera and the coordinates of one of the multiple intersection points, determine a rendering ray emitted by the virtual camera corresponding to the coordinates of the one intersection point and the direction vector of the rendering ray. S3. Based on the coordinates of each RGB camera in the RGB camera array and the coordinates of one of the intersection points, determine the camera direction vector group, which includes the camera direction vector from each RGB camera to the coordinates of one of the intersection points; S4. Based on the angle relationship between the direction vector of the rendering ray and the camera direction vector group, determine multiple RGB cameras for texture calculation; S5. Based on the multiple images captured by the multiple RGB cameras used for texture calculation, determine the texture value corresponding to one of the intersection point coordinates; S6. Continue executing steps S2-S5 until the multiple texture values ​​corresponding to the multiple intersection point coordinates are determined respectively; S7. Determine the viewpoint image corresponding to the virtual camera based on the multiple texture values ​​corresponding to the multiple intersection point coordinates; The method of determining multiple RGB cameras for texture calculation based on the angle relationship between the direction vector of a rendering ray and the camera direction vector group includes: determining the minimum angle RGB camera corresponding to the camera direction vector with the smallest angle to the direction vector of the rendering ray in the camera direction vector group, and determining multiple RGB cameras for texture calculation based on the minimum angle RGB camera. The step of determining the minimum angle RGB camera corresponding to the camera direction vector with the smallest angle to the direction vector of the rendering ray in the camera direction vector group includes: based on the formula The minimum included angle RGB camera was determined, where This represents the index of the RGB camera with the smallest included angle. This represents the coordinates of one of the intersection points. This refers to the direction of a rendered ray emitted by the virtual camera corresponding to the coordinates of the intersection point. directional vector, This represents the position of the nth real camera in 3D coordinates, where N represents the number of real cameras. This represents a function that calculates a unit vector.

2. The three-dimensional model rendering method of claim 1, wherein, The step of determining multiple RGB cameras for texture calculation based on the minimum angle RGB camera includes: determining a RGB cameras located on one side of the minimum angle RGB camera that are close to the minimum angle RGB camera, determining a RGB cameras located on the other side of the minimum angle RGB camera that are close to the minimum angle RGB camera, and using the minimum angle RGB camera, the a RGB cameras on one side, and the a RGB cameras on the other side as multiple RGB cameras for texture calculation.

3. The three-dimensional model rendering method of claim 1, wherein, The step of determining the texture value corresponding to the coordinates of one of the intersection points based on the multiple images captured by the multiple RGB cameras used for texture calculation includes: projecting the one of the intersection points into the multiple images captured by the multiple RGB cameras used for texture calculation to obtain multiple initial texture values ​​obtained by the multiple RGB cameras used for texture calculation observing the one of the intersection points, and obtaining the texture value corresponding to the coordinates of the one of the intersection points based on the multiple initial texture values.

4. The three-dimensional model rendering method of claim 3, wherein, The step of obtaining the texture value corresponding to one of the intersection coordinates based on the plurality of initial texture values ​​includes: assigning different weights to the plurality of initial texture values ​​respectively, wherein the initial texture value corresponding to the RGB camera with the smallest included angle has the highest weight, the sum of the different weights of the plurality of initial texture values ​​is 1, and obtaining the texture value corresponding to one of the intersection coordinates by weighted summation based on the plurality of initial texture values ​​and the weights corresponding to the plurality of initial texture values.

5. A three-dimensional light field rendering apparatus, characterized by, The method described in any one of claims 1-4 comprises: The control module is used to control the virtual camera to emit rendering rays, which intersect with the 3D model of the texture to be calculated and determine the coordinates of multiple intersection points. The determining module is configured to: determine a rendering ray emitted by the virtual camera corresponding to one of the multiple intersection point coordinates, and the direction vector of the rendering ray, based on the parameters of the virtual camera and one of the multiple intersection point coordinates; determine a camera direction vector group, including the camera direction vectors from each RGB camera to the one of the intersection point coordinates, based on the coordinates of each RGB camera in the RGB camera array and the one of the intersection point coordinates; determine multiple RGB cameras for texture calculation based on the angle relationship between the direction vector of the rendering ray and the camera direction vector group; determine the texture value corresponding to the one of the intersection point coordinates based on multiple images captured by the multiple RGB cameras for texture calculation; and continue to execute the above steps of the determining module until multiple texture values ​​corresponding to the multiple intersection point coordinates are determined respectively. The image module is used to determine the viewpoint image corresponding to the virtual camera based on multiple texture values ​​corresponding to the multiple intersection point coordinates.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the three-dimensional model rendering method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the three-dimensional model rendering method as described in any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the three-dimensional model rendering method as described in any one of claims 1 to 4.

Images