Image rendering method, apparatus, device, and storage medium

Abstract

The present disclosure provides an image rendering method, device and equipment, and a storage medium. It relates to the technical field of computer, and particularly relates to the technical field of image rendering. The specific implementation scheme is as follows: obtaining model data in a target three-dimensional scene, the model data including a triangular patch and a two-dimensional Gaussian point; constructing a plurality of emission lines under a current view angle for a virtual camera of the target three-dimensional scene; based on the emission lines, detecting a to-be-rendered point satisfying a target condition in the target three-dimensional scene; in a case that the to-be-rendered point belongs to the target two-dimensional Gaussian point, rendering a target pixel corresponding to the emission line to a target canvas based on a color value of the target two-dimensional Gaussian point; in a case that the to-be-rendered point belongs to the target triangular patch, rendering the target pixel to the target canvas by using a color value of the target triangular patch; and outputting the target pixels corresponding to the plurality of emission lines on the target canvas, so as to obtain a target image rendered by the two-dimensional Gaussian point and the triangular patch.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of image rendering and ray tracing. Background Technology

[0002] With the continuous maturation of image rendering technology, it has been widely used in many industries. Image rendering technology can transform three-dimensional geometric models in a three-dimensional scene into two-dimensional images with visual effects that can be displayed on display devices. Summary of the Invention

[0003] This disclosure provides an image rendering method, apparatus, device, and storage medium to solve or alleviate one or more technical problems in the prior art.

[0004] According to one aspect of this disclosure, an image rendering method is provided, comprising:

[0005] Acquire model data from the target 3D scene, the model data including triangular facets and 2D Gaussian points;

[0006] Construct multiple emission lines from the current viewpoint for the virtual camera of the target 3D scene;

[0007] Perform the following separately for each transmission line:

[0008] Based on the emission line, in the target 3D scene, a point to be rendered that meets the target conditions is detected; the target conditions include: the point to be rendered is the intersection of the emission line and the 3D model expressed by the model data in the target 3D scene, and the point to be rendered is closest to the virtual camera among all the intersections of the emission line and the 3D model.

[0009] If the point to be rendered is a target two-dimensional Gaussian point, the target pixel corresponding to the emission line is rendered to the target canvas based on the color value of the target two-dimensional Gaussian point.

[0010] If the point to be rendered belongs to the target triangle, the target pixel is rendered onto the target canvas using the color value of the target triangle.

[0011] Output the target pixels corresponding to the multiple emission lines on the target canvas to obtain a target image rendered by a mixture of two-dimensional Gaussian points and triangular facets.

[0012] According to another aspect of this disclosure, an image rendering apparatus is provided, comprising:

[0013] The acquisition module is used to acquire model data in the target 3D scene, the model data including triangular facets and 2D Gaussian points;

[0014] The construction module is used to construct multiple emission lines from the current viewpoint for the virtual camera of the target 3D scene;

[0015] The build module is specifically used to execute the following for each transmit line:

[0016] Based on the emission line, in the target 3D scene, a point to be rendered that meets the target conditions is detected; the target conditions include: the point to be rendered is the intersection of the emission line and the 3D model expressed by the model data in the target 3D scene, and the point to be rendered is closest to the virtual camera among all the intersections of the emission line and the 3D model.

[0017] If the point to be rendered is a target two-dimensional Gaussian point, the target pixel corresponding to the emission line is rendered to the target canvas based on the color value of the target two-dimensional Gaussian point.

[0018] If the point to be rendered belongs to the target triangle, the target pixel is rendered onto the target canvas using the color value of the target triangle.

[0019] The rendering module is used to output the target pixels corresponding to the multiple emission lines on the target canvas, so as to obtain a target image rendered by a mixture of two-dimensional Gaussian points and triangular patches.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments provided according to this disclosure and should not be construed as limiting the scope of this disclosure.

[0022] Figure 1 This is a schematic flowchart of an image rendering method according to a first embodiment of the present disclosure;

[0023] Figure 2 This is a flowchart illustrating the process of detecting points to be rendered that satisfy the target conditions according to the second embodiment of this disclosure;

[0024] Figure 3 This is a flowchart illustrating the process of finding the target two-dimensional Gaussian point in the second set that intersects the emission line and is closest to the virtual camera, according to the third embodiment of this disclosure.

[0025] Figure 4This is a schematic diagram of a fictional patch created for a two-dimensional Gaussian point according to the fourth embodiment of this disclosure;

[0026] Figure 5 This is a flowchart illustrating the cyclic operation according to the fifth embodiment of this disclosure when the transparency of the target two-dimensional Gaussian point is higher than a preset threshold.

[0027] Figure 6 This is a schematic diagram of the process of rendering the target pixel corresponding to the emission line to the target canvas based on the color value of the target two-dimensional Gaussian point according to the sixth embodiment of this disclosure;

[0028] Figure 7 This is a schematic diagram of the overall process of the image rendering method according to the seventh embodiment of this disclosure;

[0029] Figure 8 This is a schematic diagram of the structure of an image rendering apparatus according to the eighth embodiment of the present disclosure;

[0030] Figure 9 This is a schematic diagram of an electronic device structure for an image rendering method according to the ninth embodiment of this disclosure. Detailed Implementation

[0031] The present disclosure will now be described in further detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0032] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0033] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "multiple" means two or more, unless otherwise explicitly specified.

[0034] It should be noted that, unless it is explicitly stated that there is a sequential order of execution between different operations, or that there is a sequential order of execution between different operations in terms of technical implementation, the execution order between multiple operations may not be significant, and multiple operations may be executed simultaneously.

[0035] 3D model data in a 3D scene can be expressed in various ways. For example, triangular facets are a mainstream method for representing model data in 3D scenes, and the vast majority of 3D scene models on the market currently use triangular facets as their basic representation. A triangular facet is the basic renderable unit of 3D model data in a 3D scene. A triangle can be defined by three vertices, and this triangle is a triangular facet. Combining multiple triangular facets can represent complex model surfaces of arbitrary shapes in a 3D scene. For example, a cube can be composed of 12 triangular facets.

[0036] For example, Gaussian Splatting (GS) is a novel data representation method for models in 3D scenes. Two-Dimensional Gaussian Splatting (2DGS) is increasingly favored in the market due to its high rendering efficiency and simple generation conditions, and more and more products are choosing to use the 2DGS data structure as the data representation method for models in 3D scenes. 2DGS points are graphic primitives defined by a two-dimensional elliptical Gaussian distribution and can carry attributes such as color and transparency. In image rendering, they can be used to represent special effects such as particle effects and highlights.

[0037] When rendering 3D models, triangular faces are primarily rendered using two methods: rasterization and ray tracing. Gaussian data mainly uses rasterization. Rasterization is a fundamental rendering technique in computer graphics, its main task being to convert the geometry in a 3D scene into pixels on a 2D screen. The specific process includes transforming and projecting the vertex information of objects, then determining which pixels are covered by the objects based on this information, and calculating the color values ​​of these pixels. Global illumination is a lighting model that considers the propagation and interaction of all light rays in a scene. It considers not only direct lighting, i.e., light rays directly illuminating the surface of objects, but also indirect lighting, such as reflection and refraction of light between objects. Global illumination can make rendered scenes more realistic and natural.

[0038] In related technologies, rendering pipelines have solved the problem of rendering triangular facets and Gaussian data separately, but they lack pipelines that can render both types of 3D data in the same scene. A rendering pipeline is a complete processing flow that progressively transforms a 3D scene description into a 2D pixel image.

[0039] In addition, current Gaussian rendering methods use rasterization, which primarily focuses on converting geometry into pixels. Therefore, when using Gaussian data to provide global illumination in hybrid rendering, it may be insufficient in handling the complex propagation and interaction of light rays, making it impossible for Gaussian rendering methods relying solely on rasterization to achieve the desired results.

[0040] In view of this, this disclosure proposes an image rendering method that supports Gaussian blending rendering in triangular facet rendering, in order to solve at least one of the above-mentioned problems.

[0041] like Figure 1 The diagram shown is a flowchart of the image rendering method provided in this disclosure, including the following:

[0042] S101, acquire model data from the target 3D scene. The model data includes triangular facets and 2D Gaussian points.

[0043] A target 3D scene refers to a specific 3D spatial environment specified by the user that needs to be processed or analyzed. This target 3D scene can be virtually constructed or obtained from the real world through technologies such as 3D scanning. A target 3D scene can contain one or more 3D models represented using 3D model data. For example, in a 3D scene of home design, the 3D models can include furniture such as coffee tables and sofas. In a target 3D scene, each model can be independently represented using triangular facets or two-dimensional Gaussian points, or a model can have multiple components, each component represented using triangular facets or two-dimensional Gaussian points.

[0044] For example, in a 3D scene of home design, a coffee table might be composed of triangular facets, while a sofa might be represented using two-dimensional Gaussian points. The representation of different models can be flexibly set according to rendering requirements and data sources; this disclosure does not impose any limitations on this.

[0045] S102 constructs multiple emission lines from the current viewpoint for the virtual camera of the target 3D scene.

[0046] A virtual camera is a digital tool used to create a viewing perspective in a 3D virtual scene. During implementation, parameters such as the virtual camera's position, orientation, and field of view can be set to control the viewing angle of the final rendered image.

[0047] A radiation line is a ray emitted from the lens position of a virtual camera into a 3D scene, and each radiation line corresponds to a pixel in the final generated target image.

[0048] When implementing, such as Figure 1 As shown, perform the following operations for each transmission line:

[0049] S1021, based on the emission line, detect the points to be rendered in the target 3D scene that meet the target conditions. The target conditions include: the point to be rendered is the intersection of the emission line and the 3D model represented by the model data in the target 3D scene, and among all the intersections of the emission line and the 3D model, the point to be rendered is closest to the virtual camera.

[0050] Based on the preceding explanation, the model data in the target 3D scene can contain pre-defined model information, such as triangular facets and two-dimensional Gaussian points. Therefore, when a ray enters the target 3D scene, it may intersect with multiple triangular facets and pass through multiple two-dimensional Gaussian points. In implementation, all intersection points can be determined by first calculating the intersections of the ray with the triangular facets and determining whether the ray passes through two-dimensional Gaussian points. Then, from all intersection points, the point closest to the virtual camera is selected as the rendering point.

[0051] S1022, if the point to be rendered belongs to the target 2D Gaussian point, render the target pixel corresponding to the emission line to the target canvas based on the color value of the target 2D Gaussian point.

[0052] The target canvas can be understood as a two-dimensional image buffer used to store the rendered pixel values.

[0053] If the point to be rendered belongs to the target 2D Gaussian point, then the color value of that 2D Gaussian point is directly used to render the target pixel corresponding to the emission line. For example, if a 2D Gaussian point represents a highlight point with a color value of white, then the corresponding target pixel will be rendered as white.

[0054] S1023, if the point to be rendered belongs to the target triangle, the target pixel is rendered to the target canvas using the color value of the target triangle.

[0055] If the point to be rendered belongs to the target triangle, then the color value of the triangle is used to render the target pixel.

[0056] Understandably, the observation results are obtained by viewing the target 3D scene along the emission line from the perspective of a virtual camera. These observation results can be understood as follows: a single emission line can detect one or more points to be rendered. For example, observing a completely opaque object point detects one point to be rendered. If the closest virtual object observed along the emission line exhibits some transparency, the state of objects beyond that object can be observed, thus obtaining a second point to be rendered, and so on. Multiple points to be rendered can be observed along a single emission line.

[0057] S103 outputs the target pixels corresponding to multiple emission lines on the target canvas to obtain a target image rendered by a mixture of two-dimensional Gaussian points and triangular patches.

[0058] The target image is a two-dimensional image obtained by combining the target pixels corresponding to all emission lines on the target canvas. This two-dimensional image is rendered by mixing two-dimensional Gaussian points and triangular patches.

[0059] In this embodiment, the use of mainstream triangular facets and two-dimensional Gaussian points to hybridize the description of a target 3D scene is supported. During rendering, by acquiring model data containing triangular facets and two-dimensional Gaussian points, a comprehensive understanding of the target 3D scene's model data information can be obtained, providing the necessary data foundation for subsequent hybrid rendering within the same scene. Depending on whether the point to be rendered belongs to a two-dimensional Gaussian point or a triangular facet, corresponding color values ​​are used for pixel rendering, supporting the rendering of triangular facets and two-dimensional Gaussian data within the same scene. A rendering pipeline capable of hybrid rendering of triangular facets and two-dimensional Gaussian points is provided. This rendering pipeline can process multiple emission lines separately, improving the accuracy of hybrid rendering and enhancing the image quality of the final rendered target image.

[0060] In this embodiment of the disclosure, points to be rendered that meet the target conditions are detected in the target 3D scene, such as... Figure 2 As shown, it includes the following:

[0061] S201, Obtain the first set, which includes multiple triangular faces in the target 3D scene.

[0062] In practice, all triangular faces in the target 3D scene can be collected and grouped into a set, denoted as the first set. Alternatively, based on the virtual camera's field of view, all triangular faces within that field of view in the target 3D scene can be acquired to construct the first set.

[0063] S202, determine the triangular facet in the first set that intersects the emission line and is closest to the virtual camera, and obtain the target triangular facet.

[0064] In implementation, a ray-plane intersection algorithm is used to identify the triangular facets intersecting the emission line in the first set. Next, for each triangular facet intersecting the emission line, the distance between each facet and the virtual camera is calculated. This distance can be obtained by calculating the Euclidean distance between the virtual camera position and the intersection point of the triangular facets. Finally, from all triangular facets intersecting the emission line, the facet closest to the virtual camera is selected as the target facet.

[0065] S203, within the target distance range, find the target two-dimensional Gaussian point in the second set that intersects the emission line and is closest to the virtual camera; the distance between the virtual camera and the target triangular facet is the target distance range; the second set includes multiple two-dimensional Gaussian points in the target three-dimensional scene.

[0066] A second set can be constructed by acquiring all two-dimensional Gaussian points within the field of view of the virtual camera in the target 3D scene, and constructing such a set. Alternatively, the second set can be constructed by filtering out two-dimensional Gaussian points within the field of view that are also within the target's distance range.

[0067] In practice, a preliminary selection of two-dimensional Gaussian points can be obtained by identifying the two-dimensional Gaussian points intersecting the emission line in the second set. Similarly, for each intersecting two-dimensional Gaussian point, the distance between that point and the virtual camera is calculated. From all two-dimensional Gaussian points within the target distance range that intersect the emission line, the two-dimensional Gaussian point closest to the virtual camera is selected as the target two-dimensional Gaussian point.

[0068] S204: If the target 2D Gaussian point is found, use the target 2D Gaussian point as the point to be rendered.

[0069] If a target 2D Gaussian point is found within the target distance range that intersects the emission line and is closest to the virtual camera, then this target 2D Gaussian point is used as the rendering point and subsequently used for rendering the target pixels.

[0070] S205: If no target 2D Gaussian point is found, the target triangular facet will be used as the point to be rendered.

[0071] If no suitable 2D Gaussian point is found within the target distance range, the previously determined target triangle will be used as the rendering point for subsequent rendering of the target pixels.

[0072] In this embodiment, a target triangular facet that intersects the emission line and is closest to the virtual camera is selected from a first set of multiple triangular facets in the target 3D scene. Then, using the distance between the virtual camera and the target triangular facet as a range, a target 2D Gaussian point that intersects the emission line and is closest to the camera is searched in a second set containing multiple 2D Gaussian points. If a target 2D Gaussian point is found, it is used as the rendering point; otherwise, the target triangular facet is used as the rendering point for subsequent rendering of the target pixels. This allows for the rendering of triangular facets and 2D Gaussian data within the same scene, thereby improving rendering efficiency and the quality of the target image. Prioritizing the search for target triangular facets and determining the target distance range through them effectively defines the range of 2D Gaussian points to be processed, improving rendering efficiency.

[0073] In this embodiment of the disclosure, finding the target two-dimensional Gaussian point in the second set that intersects the emission line and is closest to the virtual camera within the target distance range can be implemented as follows: for each two-dimensional Gaussian point, obtain its corresponding elliptical range. If the emission line intersects with the elliptical range, then it is determined that the emission line intersects with the two-dimensional Gaussian point.

[0074] In other embodiments, to improve the efficiency of filtering out target two-dimensional Gaussian points, such as Figure 3 As shown, finding the target 2D Gaussian point in the second set that intersects the emission line and is closest to the virtual camera can also be implemented as follows:

[0075] S301, find the virtual facet that intersects the emission line and is closest to the virtual camera within the target distance range; the virtual facet is a triangular facet that is pre-created for each two-dimensional Gaussian point in the target three-dimensional scene. Each two-dimensional Gaussian point in the second set has a corresponding virtual facet created, and each two-dimensional Gaussian point is represented by two corresponding virtual facets.

[0076] To facilitate the detection of the intersection between the emission line and the two-dimensional Gaussian point, a corresponding triangular patch can be pre-created for each two-dimensional Gaussian point. These triangular patches are not the actual triangular patches used in the target three-dimensional scene, but rather fictitious patches used to assist in the calculation. In implementation, each two-dimensional Gaussian point is represented by two fictitious patches, so that the two fictitious patches can cover the elliptical range of the two-dimensional Gaussian point.

[0077] In this embodiment of the disclosure, for any two-dimensional Gaussian point, the virtual patch created for the two-dimensional Gaussian point satisfies the following requirements:

[0078] (1) Two imaginary faces of a two-dimensional Gaussian point form the minimum bounding rectangle of the ellipse of the two-dimensional Gaussian point;

[0079] Based on the foregoing description, a two-dimensional Gaussian point is a graphical unit defined by a two-dimensional elliptical Gaussian distribution. In this embodiment of the disclosure, the shape of a two-dimensional Gaussian point can be considered as an ellipse, which is composed of the following attributes: pos, Tu, Tv, Su, Sv. Figure 4 As shown, pos is the center position of the two-dimensional Gaussian point; Tu and Tv are the two axes of the ellipse of the two-dimensional Gaussian point, respectively; Su is the semi-axis length of the Tu axis; and Sv is the semi-axis length of the Tv axis.

[0080] The minimum bounding rectangle is the bounding rectangle with the smallest area that completely contains the ellipse, such as... Figure 4As shown, the two imaginary faces of the two-dimensional Gaussian point form the minimum bounding rectangle of the ellipse of the two-dimensional Gaussian point. The parameters of the minimum bounding rectangle include: the center is the center position of the Gaussian point; the length is 2Su; and the width is 2Sv. The four vertices V0, V1, V2, and V3 of the minimum bounding rectangle can be represented by the following expressions (1):

[0081] V0=pos-Su Tu-Sv TV

[0082] V1=pos+Su Tu-Sv TV (1)

[0083] V2=pos+Su Tu+Sv TV

[0084] V3=pos-Su Tu+Sv TV

[0085] (2) The vertices of the virtual facet of the two-dimensional Gaussian point are expressed using the two axis parameters of the two-dimensional Gaussian point;

[0086] like Figure 4 V0, V1, V2, and V3 are the vertices of two imaginary faces of a two-dimensional Gaussian point.

[0087] (3) The ellipse of the two-dimensional Gaussian point is represented by the centroid of the fictitious surface; the centroid of the fictitious surface is provided by the built-in parameters of Vulkan.

[0088] Vulkan is a cross-platform, high-performance graphics and computing API (Application Programming Interface) that provides built-in parameters and functions to assist in graphics computing.

[0089] In this embodiment of the disclosure, the centroid of the virtual facet provided by Vulkan is obtained by utilizing the built-in parameters provided by Vulkan, and then the ellipse of the two-dimensional Gaussian point is represented based on the centroid of the virtual facet, thereby accelerating the rendering process.

[0090] In practice, if the three vertices of the triangular facet are V0, V1, and V2, then any point O in the triangular facet can be represented by the following expression (2):

[0091] O = V0 + (V1 - V0) barycentric.x+(V2-V0) barycentric.y (2)

[0092] In expression (2), O represents the centroid coordinates; V0, V1, and V2 are the three vertices of the virtual face; V1-V0 represents the vector from V0 to V1; V2-V0 represents the vector from V0 to V2; barycentric represents the centroid of the obtained virtual face; barycentric.x and barycentric.y are the first two components of the centroid coordinates, respectively.

[0093] Based on this, it can be further determined whether the intersection point is within the ellipse range of the two-dimensional Gaussian point. In practice, it can be determined whether the emission line intersects with the two-dimensional Gaussian point by judging whether the following expression (3) is true. Wherein, if expression (3) is true, then they intersect; otherwise, they do not intersect.

[0094] (barycentric.x 2.0-1.0) 2 +(barycentric.y 2.0-1.0) 2 ≤1 (3)

[0095] In this embodiment of the disclosure, by using the centroid of a fictitious patch to represent the ellipse of a two-dimensional Gaussian point, the abstract Gaussian distribution information can be transformed into structured patch data that can be processed efficiently. This enables fast intersection detection of two-dimensional Gaussian points in three-dimensional scene rendering, thereby improving the efficiency of image rendering.

[0096] S302, if a fictitious surface is found and the intersection of the emission line and the fictitious surface is within the ellipse range of the candidate two-dimensional Gaussian point corresponding to the fictitious surface, the candidate two-dimensional Gaussian point is determined as the found target two-dimensional Gaussian point.

[0097] When the emission line intersects with the virtual surface and the intersection point falls within the elliptical range of the candidate two-dimensional Gaussian point corresponding to the virtual surface, it indicates that the emission line does indeed have a valid interaction with the two-dimensional Gaussian point. At this time, the candidate two-dimensional Gaussian point is determined as the target two-dimensional Gaussian point found.

[0098] If a virtual patch is found within the target distance range that intersects the emission line and is closest to the virtual camera, and the intersection of the emission line and the virtual patch is not within the ellipse range of the candidate 2D Gaussian point corresponding to the virtual patch, it indicates that the emission line and the 2D Gaussian point corresponding to the virtual patch do not intersect. This 2D Gaussian point is then removed from the target distance range, and step S301 is returned to execution until a target 2D Gaussian point satisfying condition S302 is found, or until all 2D Gaussian points within the target distance range have been traversed. If no target 2D Gaussian point is found after traversing all 2D Gaussian points within the target distance range, it indicates that no target 2D Gaussian point exists within the target distance range.

[0099] In this embodiment of the disclosure, within a given target distance range, a virtual patch that intersects with the emission line and is closest to the virtual camera is found. When such a virtual patch is found, and the intersection of the emission line and the virtual patch is within the elliptical range of the candidate two-dimensional Gaussian point corresponding to the virtual patch, the candidate two-dimensional Gaussian point is determined as the finally found target two-dimensional Gaussian point. This can improve the efficiency of filtering two-dimensional Gaussian points and ultimately improve the efficiency of hybrid rendering.

[0100] In this embodiment of the disclosure, if any one of the following conditions is met, it is determined that no target two-dimensional Gaussian point was found within the target distance range between the virtual camera and the target triangular facet; the condition set includes:

[0101] Condition 1) No fictitious face pieces were found;

[0102] Based on the above description, in order to facilitate the detection of the intersection between the emission line and the two-dimensional Gaussian point, a virtual patch was created in advance for each two-dimensional Gaussian point in the target three-dimensional scene, with each two-dimensional Gaussian point corresponding to two virtual patches.

[0103] When searching within the target distance range, since the fictitious patch is closely related to the two-dimensional Gaussian point, if no fictitious patch intersects with the emission line, it is impossible to further determine the target two-dimensional Gaussian point based on the fictitious patch. Therefore, it can be determined that the target two-dimensional Gaussian point was not found within the target distance range.

[0104] Condition 2) A fictitious patch is found, and the intersection of the emission line and the fictitious patch is not within the ellipse range of the candidate two-dimensional Gaussian point corresponding to the fictitious patch.

[0105] Based on the preceding explanation, a two-dimensional Gaussian point can be considered as an ellipse, and the ellipse defines its effective area of ​​influence. Only when the intersection of the emission line and the illusory surface falls within this ellipse is the emission line considered to have a valid interaction with the two-dimensional Gaussian point. Therefore, if an illusory surface intersects with the emission line, but the intersection point is not within the ellipse of its corresponding candidate two-dimensional Gaussian point, it indicates that although the emission line intersects the illusory surface, this intersection does not occur within the effective area of ​​influence of the two-dimensional Gaussian point, and thus, an actual interaction between the emission line and the two-dimensional Gaussian point cannot be considered. Therefore, if no two-dimensional Gaussian point meeting the requirements is found within the target distance range, it can be determined that no target two-dimensional Gaussian point was found within the target distance range.

[0106] In this embodiment of the disclosure, by using any of the condition determination mechanisms in the condition set, it is determined whether a target two-dimensional Gaussian point is found within the target distance range. This can avoid invalid rendering based on a non-existent or non-compliant target two-dimensional Gaussian point during subsequent rendering, thereby improving rendering accuracy.

[0107] In this embodiment, when searching for a two-dimensional Gaussian point between the virtual camera and the target triangular facet, a two-dimensional Gaussian point with high transparency may be encountered. Transparency can be understood as the proportion of light that can pass through. Because these highly transparent two-dimensional Gaussian points have a weaker blocking effect on light, light can pass through them and continue to reach the target triangular facet. Therefore, if the transparency of the target two-dimensional Gaussian point is higher than a preset threshold, the distance from the target two-dimensional Gaussian point to the target triangular facet along the emission line direction is determined as a new target distance range, and the position of the target two-dimensional Gaussian point is determined as the new position of the virtual camera. The process then returns to the step of searching for the target two-dimensional Gaussian point that intersects the emission line and is closest to the virtual camera within the target distance range, until any of the following termination conditions are met: Condition 1, the loop operation is executed n times, where n is a preset positive integer; Condition 2, no triangular facet or new target two-dimensional Gaussian point is found within the new target distance range.

[0108] In implementation, when the transparency of the target 2D Gaussian point is higher than a preset threshold, firstly, the distance from the target 2D Gaussian point to the target triangular facet along the emission line direction is determined as the new target distance range. Next, the position of the target 2D Gaussian point is determined as the new position of the virtual camera. It can be understood that during the search process, the virtual camera can be considered as the source of light; after updating its position, the subsequent search will continue along the emission line from the new position, simulating the process of the emission line acting as light propagating after passing through the current target 2D Gaussian point. Finally, after updating the target distance range and the virtual camera position, the process returns to finding the target 2D Gaussian point within the target distance range that intersects the emission line and is closest to the virtual camera in the second set. That is, within the new target distance range, the search again finds the target 2D Gaussian point that intersects the emission line and is closest to the virtual camera at the new position. This process is repeated, and each time a new target 2D Gaussian point is found, its transparency is considered to determine whether to continue updating the range and position and searching again.

[0109] It should be noted that the loop operation will not continue indefinitely; the loop will stop when any of the following termination conditions are met:

[0110] (1) The loop operation is executed n times, where n is a preset positive integer;

[0111] This is achieved by setting up a loop operation to avoid infinite loops. For example, after the loop operation is executed n times, even if the transparency of the target 2D Gaussian point found each time is higher than a preset threshold, the search will stop after n executions.

[0112] (2) No triangular facets or new two-dimensional Gaussian points were found within the new target distance range;

[0113] If no triangular facet or a new two-dimensional Gaussian point intersecting the emission line is found within the new target distance range, it means that there are no more interactive objects within this range, and the loop will stop at this point.

[0114] To further understand the loop operation described above, such as Figure 5 As shown, for each emission line, after finding the first intersecting target 2D Gaussian point, the following can be executed repeatedly:

[0115] S501, determine whether the transparency of the target 2D Gaussian point is higher than a preset threshold. If yes, continue to S503; if no, proceed to S502.

[0116] S502, take the two-dimensional Gaussian point of the target as the point to be rendered.

[0117] S503 defines the new target distance range as the distance between the target's two-dimensional Gaussian point and the target triangular facet along the emission line direction. The position of the target's two-dimensional Gaussian point is the new position of the virtual camera.

[0118] S504, within the new target distance range, find the target 2D Gaussian point in the second set that intersects the emission line and is closest to the virtual camera.

[0119] S505, determine if either of the following termination conditions is met: Condition 1, the loop operation is executed n times, where n is a preset positive integer; Condition 2, no triangular facets or new target 2D Gaussian points are found within the new target distance range. If not, return to execute S501; if yes, execute S506.

[0120] S506, terminate the loop, and use each of the found target 2D Gaussian points as points to be rendered.

[0121] In this embodiment of the disclosure, by adjusting the target distance range and the virtual camera position based on the transparency of the target two-dimensional Gaussian point, it is possible to further search for the two-dimensional Gaussian point in the target three-dimensional scene, thereby improving the rendering quality of the generated target image.

[0122] In this embodiment of the disclosure, the target pixels corresponding to the emission lines are rendered to the target canvas based on the color values ​​of the target two-dimensional Gaussian points, such as... Figure 6As shown, it may include the following:

[0123] S601, through loop operation, when multiple target two-dimensional Gaussian points are found, determines the rendering value of each target two-dimensional Gaussian point based on its transparency and color values.

[0124] In other words, after the above iterative operations, when multiple target 2D Gaussian points are found in the 3D target scene, the rendering value of each target 2D Gaussian point is determined based on its transparency and color values.

[0125] In practice, the transparency and color values ​​of each target 2D Gaussian point can be multiplied together, and the result can be used as the rendering value of that target 2D Gaussian point. For example, if the transparency of the first target 2D Gaussian point is p and the color value is q, then the rendering value of the first target 2D Gaussian point is p × q.

[0126] S602 accumulates the rendering values ​​of each target 2D Gaussian point to obtain the color value of the target pixel.

[0127] Since the same ray passes through multiple target 2D Gaussian points, when accumulating the rendering values ​​of each target 2D Gaussian point, the influence of the portion of the first target 2D Gaussian point that was not transmitted must be considered. For example, if the transparency of the second target 2D Gaussian point is still p, and its color value is m, then when superimposing the rendering value of the first target 2D Gaussian point, the rendering value of the second target 2D Gaussian point is (1-p) × p × m. And so on, by adding the rendering values ​​of multiple target 2D Gaussian points, the color value of the target pixel can be obtained.

[0128] S603 fills the color value into the target pixel position on the target canvas.

[0129] In this embodiment of the disclosure, when multiple target two-dimensional Gaussian points are found through cyclic operation, the color value of the target pixel is obtained by accumulating the rendering values ​​of each two-dimensional Gaussian point. This can comprehensively consider the common influence of multiple two-dimensional Gaussian points on the color of the target pixel, thereby improving the rendering effect of the generated target image.

[0130] In summary, the overall flowchart of the image rendering method provided in this embodiment is as follows: Figure 7 As shown, it includes:

[0131] S701, acquire model data from the target 3D scene. The model data includes triangular facets and 2D Gaussian points.

[0132] S702 initializes the rendering state of each pixel in the target canvas, including the loop count, rendered, and unrendered, and then starts the rendering loop.

[0133] Execute S703-S707 for each emission line corresponding to each pixel on the target canvas.

[0134] S703: Determine if rendering in the direction of this emission line has terminated. If not terminated, continue with S704. If terminated, proceed with S708.

[0135] S704, find the intersection of the emission line with the triangular facet in the first set.

[0136] First, a first set of triangles is obtained, comprising multiple triangular faces in the target 3D scene. A ray-plane intersection algorithm is used to determine whether the ray passes through the plane containing the triangle and whether the intersection point is within the triangle's area. Next, for each triangle intersecting the ray, the distance between that triangle and the virtual camera is calculated. In practice, this can be achieved by calculating the Euclidean distance between the virtual camera position and the intersection point of the triangles. Finally, from all triangles intersecting the ray, the triangle closest to the virtual camera is selected as the target triangle.

[0137] S705, based on the target triangular patch and the current position of the virtual camera, determines the target distance range, and searches for the target's two-dimensional Gaussian intersection point in the second set within the target distance range.

[0138] Within the target distance range, the target two-dimensional Gaussian point is selected by checking whether the emission line intersects with the two-dimensional Gaussian point in the second set.

[0139] S706, pixel illumination calculation.

[0140] If a target 2D Gaussian point is found within the target distance range that intersects the emission line and is closest to the virtual camera, then this target 2D Gaussian point is used as the rendering point and will be used for subsequent rendering of the target pixels. If no 2D Gaussian point that meets the conditions is found within the target distance range, then the previously determined target triangle is used as the rendering point and will be used for subsequent rendering of the target pixels.

[0141] S707, update rendering state, return to execute S703.

[0142] S708 outputs the rendering result after the color values ​​of the target pixels in all emission line directions are determined.

[0143] This means outputting the target pixels corresponding to multiple emission lines on the target canvas to obtain a target image rendered by a mixture of two-dimensional Gaussian points and triangular facets.

[0144] In summary, the image rendering method provided in this embodiment can support rendering of triangular patches and two-dimensional Gaussian data in the same scene, increasing the flexibility and diversity of rendering, and making the final output target image rendered by the mixture of two-dimensional Gaussian points and triangular patches have a high-quality rendering effect.

[0145] Based on the same technical concept, this disclosure also provides an image rendering apparatus 800, such as... Figure 8 As shown, it includes:

[0146] The acquisition module 801 is used to acquire model data in the target 3D scene. The model data includes triangular facets and 2D Gaussian points.

[0147] Module 802 is used to construct multiple emission lines from the current viewpoint for the virtual camera of the target scene;

[0148] Module 802 is specifically used to execute the following for each transmit line:

[0149] Based on the emission line, detect the points to be rendered in the target 3D scene that meet the target conditions; the target conditions include: the point to be rendered is the intersection of the emission line and the 3D model expressed by the model data in the target 3D scene, and the point to be rendered is closest to the virtual camera among all the intersections of the emission line and the 3D model.

[0150] If the point to be rendered is a target 2D Gaussian point, render the target pixel corresponding to the emission line to the target canvas based on the color value of the target 2D Gaussian point.

[0151] If the point to be rendered belongs to the target triangle, the target pixel is rendered to the target canvas using the color value of the target triangle.

[0152] The rendering module 803 is used to output the target pixels corresponding to multiple emission lines on the target canvas, so as to obtain the target image rendered by a mixture of two-dimensional Gaussian points and triangular facets.

[0153] In some embodiments, the building module includes:

[0154] The first acquisition unit is used to acquire a first set, which includes multiple triangular facets in the target 3D scene;

[0155] The first determining unit is used to determine the triangular facet in the first set that intersects the emission line and is closest to the virtual camera, so as to obtain the target triangular facet.

[0156] The search unit is used to find the target two-dimensional Gaussian point in the second set that intersects the emission line and is closest to the virtual camera within the target distance range; the distance between the virtual camera and the target triangular facet is the target distance range; the second set includes multiple two-dimensional Gaussian points in the target three-dimensional scene;

[0157] The second determining unit is used to determine the target two-dimensional Gaussian point as the rendering point if the target two-dimensional Gaussian point is found, and to determine the target triangular facet as the rendering point if the target two-dimensional Gaussian point is not found.

[0158] In some embodiments, the lookup unit is specifically used for:

[0159] Within the target distance range, find the virtual patch that intersects the emission line and is closest to the virtual camera. The virtual patch is a triangular patch pre-created for each two-dimensional Gaussian point in the target three-dimensional scene. Each two-dimensional Gaussian point in the second set has a corresponding virtual patch created. Each two-dimensional Gaussian point is represented by two corresponding virtual patches.

[0160] If a fictitious surface is found, and the intersection of the emission line and the fictitious surface is within the ellipse range of the candidate two-dimensional Gaussian point corresponding to the fictitious surface, then the candidate two-dimensional Gaussian point is determined as the found target two-dimensional Gaussian point.

[0161] In some embodiments, a determining module is further included, configured to:

[0162] If any of the following conditions are met, it is determined that no target 2D Gaussian point was found within the target distance range between the virtual camera and the target triangular facet.

[0163] The condition set includes:

[0164] Condition 1) No fictitious face pieces were found;

[0165] Condition 2) A fictitious patch is found, and the intersection of the emission line and the fictitious patch is not within the ellipse range of the candidate two-dimensional Gaussian point corresponding to the fictitious patch.

[0166] In some embodiments, the fictitious patch created for any two-dimensional Gaussian point satisfies the following requirements:

[0167] Two imaginary faces of a two-dimensional Gaussian point form the minimum bounding rectangle of the ellipse of the two-dimensional Gaussian point;

[0168] The vertices of the fictional facet of a two-dimensional Gaussian point are expressed using the two axis parameters of the two-dimensional Gaussian point;

[0169] An ellipse of a two-dimensional Gaussian point is represented by the centroid of a fictitious facet; the centroid of the fictitious facet is provided by a parameter built into Vulkan.

[0170] In some embodiments, a loop module is also included for:

[0171] If the transparency of the target 2D Gaussian point is higher than the preset threshold, the distance between the target 2D Gaussian point and the target triangular facet along the emission line direction is determined as the new target distance range, and the position of the target 2D Gaussian point is determined as the new position of the virtual camera. The search unit is then triggered to return to the step of searching for the target 2D Gaussian point in the second set that intersects the emission line and is closest to the virtual camera within the target distance range, until any of the following termination conditions are met.

[0172] The loop operation is executed n times, where n is a preset positive integer;

[0173] No triangular facets or new target 2D Gaussian points were found within the new target distance range.

[0174] In some embodiments, the building module includes:

[0175] The third determining unit is used to determine the rendering value of each target two-dimensional Gaussian point based on the transparency and color values ​​of each target two-dimensional Gaussian point when multiple target two-dimensional Gaussian points are found through loop operation.

[0176] The combination unit is used to accumulate the rendering values ​​of each two-dimensional Gaussian point to obtain the color value of the target pixel;

[0177] Fill unit, used to fill the target pixel position of the target canvas with color values.

[0178] Figure 9 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Figure 9 As shown, the electronic device includes a memory 910 and a processor 920. The memory 910 stores a computer program that can run on the processor 920. The number of memories 910 and processors 920 can be one or more. The memory 910 can store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the methods provided in the above-described method embodiments. The electronic device may also include a communication interface 930 for communicating with external devices and performing data exchange and transmission.

[0179] If the memory 910, processor 920, and communication interface 930 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0180] Optionally, in a specific implementation, if the memory 910, processor 920, and communication interface 930 are integrated on a single chip, the memory 910, processor 920, and communication interface 930 can communicate with each other through an internal interface.

[0181] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0182] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAMBUS RAM (DR RAM).

[0183] In the description of the embodiments of this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0184] In the description of the embodiments disclosed herein, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0185] In the description of embodiments of this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0186] The above description is merely an exemplary embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. An image rendering method, comprising: Acquire model data from the target 3D scene, the model data including triangular facets and 2D Gaussian points; The triangular facet is the basic renderable unit of the 3D model data; Construct multiple emission lines from the current viewpoint for the virtual camera of the target 3D scene; Perform the following separately for each transmission line: Based on the emission line, in the target 3D scene, a point to be rendered that meets the target conditions is detected; the target conditions include: the point to be rendered is the intersection of the emission line and the 3D model expressed by the model data in the target 3D scene, and the point to be rendered is closest to the virtual camera among all the intersections of the emission line and the 3D model. For the two-dimensional Gaussian point in the model data, the intersection point is the intersection point of the emission line and the virtual facet; the virtual facet is a triangular facet pre-created for the two-dimensional Gaussian point; the two-dimensional Gaussian point is a graphic primitive defined by a two-dimensional elliptical Gaussian distribution; the virtual facet is not the actual triangular facet used in the target three-dimensional scene, but a virtual facet used to assist in calculation. If the point to be rendered is a target two-dimensional Gaussian point, the target pixel corresponding to the emission line is rendered to the target canvas based on the color value of the target two-dimensional Gaussian point. If the point to be rendered belongs to the target triangle, the target pixel is rendered onto the target canvas using the color value of the target triangle. Output the target pixels corresponding to the multiple emission lines on the target canvas to obtain a target image rendered by a mixture of two-dimensional Gaussian points and triangular facets.

2. The method according to claim 1, wherein, The step of detecting rendering points that satisfy the target conditions in the target 3D scene includes: Obtain a first set, which includes multiple triangular facets in the target 3D scene; The target triangular facet is obtained by identifying the triangular facet in the first set that intersects the emission line and is closest to the virtual camera; Within the target distance range, find the target two-dimensional Gaussian point in the second set that intersects the emission line and is closest to the virtual camera; the distance between the virtual camera and the target triangular facet is the target distance range; the second set includes multiple two-dimensional Gaussian points of the target three-dimensional scene; If the target two-dimensional Gaussian point is found, the target two-dimensional Gaussian point is used as the point to be rendered; If the target 2D Gaussian point is not found, the target triangular facet is used as the point to be rendered.

3. The method according to claim 2, wherein, The step of finding the target two-dimensional Gaussian point in the second set that intersects the emission line and is closest to the virtual camera within the target distance range includes: Within the target distance range, find the virtual patch that intersects the emission line and is closest to the virtual camera; the virtual patch is a triangular patch pre-created for each two-dimensional Gaussian point in the target three-dimensional scene, and each two-dimensional Gaussian point in the second set has a corresponding virtual patch created, and each two-dimensional Gaussian point is represented by two corresponding virtual patches. If the fictitious patch is found and the intersection of the emission line and the fictitious patch is within the ellipse range of the candidate two-dimensional Gaussian point corresponding to the fictitious patch, then the candidate two-dimensional Gaussian point is determined to be the found target two-dimensional Gaussian point.

4. The method according to claim 3, further comprising: If any one of the following conditions is met, it is determined that no two-dimensional Gaussian point of the target is found within the target distance range between the virtual camera and the target triangular facet. The set of conditions includes: Condition 1) The fictitious facet was not found; Condition 2) The fictitious patch is found, and the intersection of the emission line and the fictitious patch is not within the ellipse range of the candidate two-dimensional Gaussian point corresponding to the fictitious patch.

5. The method according to claim 3, wherein, For any two-dimensional Gaussian point, the virtual patch created for the two-dimensional Gaussian point satisfies the following requirements: The two imaginary faces of the two-dimensional Gaussian point form the minimum bounding rectangle of the ellipse of the two-dimensional Gaussian point; The vertices of the fictional facet of the two-dimensional Gaussian point are expressed using the two axis parameters of the two-dimensional Gaussian point; The ellipse of the two-dimensional Gaussian point is represented by the centroid of the fictional facet; the centroid of the fictional facet is provided by a parameter built into Vulkan.

6. The method according to any one of claims 2-5, further comprising: If the transparency of the target two-dimensional Gaussian point is higher than a preset threshold, the distance from the target two-dimensional Gaussian point to the target triangular facet along the direction of the emission line is determined as the new target distance range, and the position of the target two-dimensional Gaussian point is determined as the new position of the virtual camera. Then, the step of finding the target two-dimensional Gaussian point in the second set that intersects the emission line and is closest to the virtual camera within the target distance range is returned until any of the following termination conditions are met. The loop operation is executed n times, where n is a preset positive integer; No triangular facets or new target 2D Gaussian points were found within the new target distance range.

7. The method according to claim 6, wherein, The step of rendering the target pixel corresponding to the emission line to the target canvas based on the color value of the target two-dimensional Gaussian point includes: By looping through multiple target 2D Gaussian points, the rendering value of each target 2D Gaussian point is determined based on its transparency and color values. The color value of the target pixel is obtained by accumulating the rendering values ​​of each target 2D Gaussian point; The color value is filled into the target pixel position of the target canvas.

8. An image rendering apparatus, comprising: The acquisition module is used to acquire model data in the target 3D scene, the model data including triangular facets and 2D Gaussian points; The triangular facet is the basic renderable unit of the 3D model data; A construction module is used to construct multiple emission lines from the current viewpoint for the virtual camera of the target 3D scene; The build module is specifically used to execute the following for each transmit line: Based on the emission line, in the target 3D scene, a point to be rendered that meets the target conditions is detected; the target conditions include: the point to be rendered is the intersection of the emission line and the 3D model expressed by the model data in the target 3D scene, and the point to be rendered is closest to the virtual camera among all the intersections of the emission line and the 3D model. For the two-dimensional Gaussian point in the model data, the intersection point is the intersection point of the emission line and the virtual facet; the virtual facet is a triangular facet pre-created for the two-dimensional Gaussian point; the two-dimensional Gaussian point is a graphic primitive defined by a two-dimensional elliptical Gaussian distribution; the virtual facet is not the actual triangular facet used in the target three-dimensional scene, but a virtual facet used to assist in calculation. If the point to be rendered is a target two-dimensional Gaussian point, the target pixel corresponding to the emission line is rendered to the target canvas based on the color value of the target two-dimensional Gaussian point. If the point to be rendered belongs to the target triangle, the target pixel is rendered onto the target canvas using the color value of the target triangle. The rendering module is used to output the target pixels corresponding to the multiple emission lines on the target canvas, so as to obtain a target image rendered by a mixture of two-dimensional Gaussian points and triangular patches.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images