Image rendering method and device, electronic equipment and storage medium

Abstract

The invention provides an image rendering method, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models, augmented reality and the like, and can be applied to scenes such as three-dimensional reconstruction and the like. According to the specific implementation scheme, a target grid model is generated based on a three-dimensional Gaussian sputtering model, the three-dimensional Gaussian sputtering model comprises a plurality of Gaussian points used for describing an object to be rendered, and the target grid model comprises respective target patches of the Gaussian points; the transparency of the middle area of the target patch in the target mesh model is lower than that of the peripheral area of the target patch; taking the target grid model as a mask to project a shadow, and obtaining a shadow map of the to-be-rendered object under a predetermined resolution; rendering the to-be-rendered object based on the three-dimensional Gaussian sputtering model and the shadow map to obtain a rendered target image; and displaying the rendered target image. The invention further provides an image rendering device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technologies, particularly to technical fields such as computer vision, deep learning, large models, and augmented reality, and can be applied to scenarios such as three-dimensional reconstruction. More specifically, the present disclosure provides an image rendering method, apparatus, electronic device, and storage medium. Background Art

[0002] With the development of artificial intelligence technologies, the application of three-dimensional rendering technologies has become increasingly widespread. For example, 3D Gaussian Splatting (3DGS), as a technology for object reconstruction, visualization, and rendering, can efficiently process complex volume data, improve the quality of rendering results, and is easily integrated into the rendering pipeline of modern engines, having broad application prospects in fields such as medicine, science, games, and industrial design. Among them, the shadow rendering of 3DGS is crucial. A good shadow rendering effect can enhance the realism of the reconstructed scene and improve the details of the rendered image. Summary of the Invention

[0003] The present disclosure provides an image rendering method, apparatus, device, and storage medium.

[0004] According to one aspect of the present disclosure, there is provided an image rendering method, which includes: generating a target mesh model based on a 3D Gaussian splatting model, where the 3D Gaussian splatting model includes a plurality of Gaussian points for describing the object to be rendered, the target mesh model includes target patches for each Gaussian point, and the transparency of the middle region of the target patches in the target mesh model is lower than that of the surrounding regions of the target patches; projecting a shadow using the target mesh model as a mask to obtain a shadow map of the object to be rendered at a predetermined resolution; rendering the object to be rendered based on the 3D Gaussian splatting model and the shadow map to obtain a rendered target image; and displaying the rendered target image.

[0005] According to another aspect of the present disclosure, there is provided an image rendering apparatus, which includes: a generation module for generating a target mesh model based on a 3D Gaussian splatting model, where the 3D Gaussian splatting model includes a plurality of Gaussian points for describing the object to be rendered, the target mesh model includes target patches for each Gaussian point, and the transparency of the middle region of the target patches in the target mesh model is lower than that of the surrounding regions of the target patches; a projection module for projecting a shadow using the target mesh model as a mask to obtain a shadow map of the object to be rendered at a predetermined resolution; a rendering module for rendering the object to be rendered based on the 3D Gaussian splatting model and the shadow map to obtain a rendered target image; and a display module for displaying the rendered target image.

[0006] According to another aspect of the present disclosure, there is provided an electronic device, including: at least one display device; and a processor communicatively connected to the at least one display device; wherein, the processor executes to obtain a rendered target image according to the present disclosure, and outputs the rendered target image to the at least one display device for display.

[0007] According to another aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to the present disclosure.

[0008] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method according to the present disclosure.

[0009] According to another aspect of the present disclosure, there is provided a computer program product including a computer program, where the computer program implements the method according to the present disclosure when executed by a processor.

[0010] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understandable through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0012] Figure 1 is a schematic diagram of an exemplary system architecture to which an image rendering method and apparatus according to an embodiment of the present disclosure can be applied;

[0013] Figure 2 is a flowchart of an image rendering method according to an embodiment of the present disclosure;

[0014] Figure 3 schematically shows the flow of a target mesh model generation method according to an embodiment of the present disclosure;

[0015] Figure 4 schematically shows the flow of an initial normal direction determination method according to an embodiment of the present disclosure;

[0016] Figure 5 schematically shows the effect diagram after dithering processing of transparency according to an embodiment of the present disclosure;

[0017] Figure 6 is a block diagram of an image rendering apparatus according to an embodiment of the present disclosure;

[0018] Figure 7 is a block diagram of an electronic device according to an embodiment of the present disclosure; and

[0019] Figure 8 shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement embodiments of the present disclosure. Detailed implementation manners

[0020] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to assist understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] In actual application scenarios, the 3DGS shadow rendering method may include a translucent shadow rendering method based on ray casting, a shadow rendering method based on a 3DGS scene depth map, and a shadow rendering method based on ambient occlusion.

[0022] The translucent shadow rendering method based on ray casting emits rays from each Gaussian point to detect whether the point is occluded, and calculates the shadow intensity by accumulating transparency. This method requires ray tracing for each pixel in the scene and calculating the intersection points of the rays and the objects in the scene. This process involves a large amount of geometric calculations and floating-point operations, resulting in a large amount of computation. Especially for complex scenes and high-quality rendering requirements, the computational amount of the ray casting method will further increase. Due to the large amount of computation, the ray casting scheme usually requires a long rendering time and is difficult to meet the requirements of real-time rendering. In interactive applications or real-time rendering scenarios, the ray casting scheme may cause obvious delays or stuttering phenomena.

[0023] The shadow rendering method based on the 3DGS scene depth map renders the depth map of the 3DGS scene as a shadow map from the perspective of the light source, and detects whether each Gaussian point is in the shadow during the rendering of the main window. When dealing with translucent objects, this method needs to perform transparency aliasing (or transparency blending), that is, synthesize the colors of different objects according to the transparency values of the objects. Transparency aliasing will increase the complexity of rendering and the rendering efficiency is low.

[0024] The shadow rendering method based on ambient occlusion calculates the ambient occlusion values around each Gaussian point in the three-dimensional Gaussian sputtering model (3DGS model), and calculates the shadow effect in combination with the pre-computed ambient occlusion map. This method relies on pre-computation, and in order to obtain a high-quality occlusion effect, it is necessary to densely sample the object surface. The higher the sampling density, the greater the amount of computation, and the higher the computational complexity, resulting in low rendering efficiency.

[0025] In view of this, embodiments of the present disclosure provide an image rendering method, which includes: generating a target mesh model based on a three-dimensional Gaussian sputtering model, where the three-dimensional Gaussian sputtering model includes a plurality of Gaussian points for describing an object to be rendered, the target mesh model includes target patches corresponding to each Gaussian point, and the transparency of the middle region of the target patches in the target mesh model is lower than that of the surrounding regions of the target patches; projecting a shadow using the target mesh model as a mask to obtain a shadow map of the object to be rendered at a predetermined resolution; rendering the object to be rendered based on the three-dimensional Gaussian sputtering model and the shadow map to obtain a rendered target image; and displaying the rendered target image.

[0026] Figure 1 FIG. is a schematic diagram of an exemplary system architecture to which an image rendering method and apparatus according to an embodiment of the present disclosure can be applied. It should be noted that Figure 1 the figure shown is only an example of the system architecture to which embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0027] As Figure 1 shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting 2D display and 3D display, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0029] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for rendering requests sent by users using the terminal devices 101, 102, 103. The background management server may, in response to a received rendering request, render the object to be rendered and feedback the rendering result to the terminal devices 101, 102, 103 for display on the display screens of the terminal devices 101, 102, 103.

[0030] Note that the image rendering method provided by the embodiments of the present disclosure can generally be executed by the server 105. Correspondingly, the image rendering apparatus provided by the embodiments of the present disclosure can generally be disposed in the server 105. The image rendering method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103, and / or the server 105. Correspondingly, the image rendering apparatus provided by the embodiments of the present disclosure can also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103, and / or the server 105.

[0031] It should be understood that Figure 1 the number and types of the terminal devices, networks, and servers in

[0032] Figure 2 is a flowchart of an image rendering method according to an embodiment of the present disclosure.

[0033] As Figure 2 shown, the image rendering method 200 may include operations S210 to S240.

[0034] In operation S210, a target mesh model is generated based on a three-dimensional Gaussian sputtering model.

[0035] According to an embodiment of the present disclosure, the 3DGS model may include a plurality of Gaussian points for describing the object to be rendered, and the object to be rendered may be a three-dimensional object. The 3DGS model can use a Gaussian function to describe the point cloud data of the object to be rendered, and the object to be rendered may include various information such as objects, lights, and materials. These scenes need to be converted into two-dimensional images through the rendering process so as to be presented in the corresponding display area on the screen or monitor. The 3DGS model fits the observed data (such as multi-view images) by optimizing Gaussian parameters, thereby realizing the representation of three-dimensional scenes. This representation method can not only capture the geometric information of the scene but also handle complex factors such as lights and materials. During the rendering process, the 3DGS model converts the objects and volumes in three-dimensional space into a series of Gaussian points (each Gaussian point is represented by a Gaussian function), and each Gaussian point can contain multiple attributes, such as parameters describing position (XYZ), covariance (3x3 matrix, indicating how to stretch or scale), color (RGB), and transparency (Alpha), etc. During rendering, these Gaussian points are projected and overlapped on the image plane to form the final rendered image.

[0036] According to an embodiment of the present disclosure, the target mesh model may include target patches for respective Gaussian points, that is, in the target mesh model, target patches are used to replace Gaussian points to describe the object to be rendered. The patch can be any polygon, such as a triangle or a quadrilateral. Triangular meshes have the advantages of simplicity, flexibility, and ease of processing, while quadrilateral meshes may be easier to subdivide and smooth in some cases. Each patch consists of a certain number of vertices (e.g., 3 for a triangle and 4 for a quadrilateral) and edges. These vertices and edges define the shape and position of the patch. Patches can be used to represent details on the surface of the 3DGS model, such as textures, colors, lighting effects, etc. Each patch has a normal vector, which represents the orientation of the patch. The normal vector is crucial for lighting and rendering effects because it determines how light interacts with the patch. Patches can be used to construct and represent the basic geometric units of the three-dimensional model surface. By combining and varying multiple target patches, a complex target mesh model (three-dimensional model) can be constructed. During the rendering process, the computer graphics system can calculate the color and brightness of each pixel based on the geometric shape, material, and lighting conditions of the patch. The number and density of patches directly affect the quality and performance of the rendering.

[0037] The transparency of the middle region of the target patch in the target mesh model can be lower than that of the surrounding region of the target patch, or in other words, the opacity of the middle region of the target patch in the target mesh model can be higher than that of the surrounding region of the target patch. This can form a target patch with a "solid center and blurred edges", which can then be directly used as a translucent mask material to project shadows. The translucent mask material sets a mask threshold, and the part of the object with an opacity greater than the threshold is displayed, while the part less than the threshold is hidden. The translucent mask material has better rendering performance compared to the translucent material.

[0038] In operation S220, the target mesh model is used as a mask to project shadows, obtaining a shadow map of the object to be rendered at a predetermined resolution.

[0039] According to an embodiment of the present disclosure, since the target mesh model is generated based on the 3DGS model composed of Gaussian points used to describe respective objects to be rendered, the target mesh model can contain attributes such as base color, opacity, glossiness, etc., and can be used as a translucent mask material for shadow projection.

[0040] Before shadow projection, a light source needs to be constructed. Based on the constructed light source, the target mesh model is used as a mask to project shadows. Since the opacity of the middle region of the target patch is higher than that of the surrounding region, a shadow is formed in the middle region, and the surrounding region is culled to generate a shadow map.

[0041] Users can set the size of the predetermined resolution according to the actual usage scenario. The higher the resolution of the shadow map, the richer the details of the shadow and the higher the accuracy, which means that the edges of the shadow will be smoother and more geometric details can be captured. High-resolution shadow maps can provide clearer and more realistic shadow effects, thereby improving the rendering quality of the overall scene. However, high resolution may increase the computational burden of rendering. This may result in longer rendering time or require more powerful hardware support. Low-resolution shadow maps have lower hardware requirements and faster rendering speed. Therefore, users can set the resolution of the shadow map according to the rendering effect and rendering efficiency. For example, if the user pursues a higher rendering effect, a shadow map with a high resolution can be obtained. For another example, to pursue a balance between rendering effect and rendering efficiency, a shadow map with a moderate resolution can be obtained.

[0042] In operation S230 , the object to be rendered is rendered based on the three-dimensional Gaussian splash model and the shadow map to obtain a rendered target image.

[0043] According to the embodiments of the present disclosure, by rendering a three-dimensional Gaussian sputtering model, a rough image of the object to be rendered can be obtained, and then shadow rendering is performed based on the shadow map to obtain a target image of the object to be rendered.

[0044] In operation S240, the rendered target image is displayed.

[0045] According to the implementation of the present disclosure, the rendered target image can be output to a display device, and the target image can be displayed in a display area specified by the display device, thereby realizing visualization or three-dimensional reconstruction of the object to be rendered.

[0046] For example, in an e-commerce digital human scenario, the object to be rendered is a digital human. Based on the above method, a digital human can be rendered and reconstructed, and corresponding e-commerce activities can be carried out based on the digital human instead of humans.

[0047] Through the image rendering method of the embodiment of the present invention, a target network model with multiple intermediate real and virtual edge patches is generated in advance based on the three-dimensional Gaussian sputtering model, and the target network model is used as a mask to directly project shadows, and a shadow map is obtained for subsequent shadow rendering. Since the target network model is used as a mask to directly project shadows, only the transmittance and mask effects need to be considered, which greatly reduces the complexity of shadow calculation. While ensuring the rendering effect, the complexity of rendering is greatly reduced, the rendering efficiency is improved, and hardware resources can be saved.

[0048] The following will be combined Figure 3 A method for generating a target grid model according to an embodiment of the present disclosure is schematically described. Figure 3 The flow of the target grid model generation method according to an embodiment of the present disclosure is schematically shown.

[0049] As Figure 3 shown, the target mesh model generation method 310 may include operation S311 to operation S313.

[0050] In operation S311, an initial mesh model is generated based on a three-dimensional Gaussian sputtering model.

[0051] According to an embodiment of the present disclosure, the initial mesh model may include initial patches for each Gaussian point. That is, in the initial mesh model, the initial patches are used to replace the Gaussian points to describe the object to be rendered. The transparency of the initial patches in the initial mesh model may be linearly distributed, and the transparency of the initial patches does not satisfy that the transparency in the middle region is lower than the transparency in the surrounding regions of the initial patches, or in other words, does not satisfy that the opacity in the middle region may be higher than the opacity in the surrounding regions of the initial patches. At this time, it cannot be well used as a mask for shadow projection.

[0052] In operation S312, Gaussian distribution simulation is performed on the transparency represented by the vertices of the initial patches to make the transparency in the middle region of the initial patches lower than the transparency in the surrounding regions of the initial patches.

[0053] According to an embodiment of the present disclosure, Gaussian distribution simulation can be performed on the transparency of the entire region of the initial patches according to the transparency represented by the vertices of the initial patches, so that the transparency of the entire region of the initial patches presents a Gaussian distribution form, that is, the transparency in the middle region is lower than the transparency in the surrounding regions of the initial patches.

[0054] In operation S313, jitter processing is respectively performed on the transparency after Gaussian distribution simulation of each initial patch to obtain the target mesh model.

[0055] According to an embodiment of the present disclosure, in the actual rendering process, the objects to be rendered may be diverse. For example, there are natural phenomena such as flames, smoke, and water flows in the objects to be rendered, and these phenomena usually have irregular movements and changes. By performing jitter processing on the transparency of the patches, the semi-transparency and transparency changes in these natural phenomena can be simulated; for another example, in some cases, such as transparency fade or overlapping rendering, rendering problems may occur, such as color band separation or abrupt transitions. At this time, jitter processing can be performed on the transparency of the patches to simulate a more regular and uniform distribution of the transparency values of the patches, ensuring that the masked-out areas are as close to the actual situation as possible.

[0056] Through the image rendering method of the embodiments of the present disclosure, by simulating a Gaussian distribution for the initial patch, it is possible to better generate a target patch with a low transparency in the middle region and a high transparency in the surrounding regions, improving the effect of the subsequent mask projection shadow. On this basis, further performing dithering processing on the transparency after Gaussian distribution simulation to obtain a more regular and uniform distribution of the transparency values of the patches, so that the changes in translucency and transparency can be better simulated, making the rendered image more realistic and vivid, and the transparency transition can be smoother and more natural, making the rendering effect softer. In addition, performing dithering processing on the patch transparency can reduce the computational amount and improve the rendering efficiency.

[0057] Based on the above embodiments, in some embodiments, generating an initial mesh model based on a three-dimensional Gaussian sputtering model may include:

[0058] Obtain the respective scaling information and rotation information of each Gaussian point in the three-dimensional Gaussian sputtering model.

[0059] Respectively determine the respective initial normal directions of each Gaussian point according to the respective scaling information of each Gaussian point, and rotate the respective initial normal directions according to the respective rotation information of each Gaussian point to obtain the respective target normal directions.

[0060] Respectively construct initial patches perpendicular to the respective target normal directions according to the respective scaling information of each Gaussian point, and add the color and transparency of the object to be rendered to the vertices of the respective initial patches to obtain the initial mesh model

[0061] According to the embodiments of the present disclosure, the scaling information can be used to characterize the size and shape changes of the object to be rendered in space. For the 3DGS model, the scaling information is a parameter for size adjustment, which determines the size of the model in three-dimensional space, that is, it determines the size of the constructed initial patch.

[0062] The volume, position, shape, etc. of an object in the object to be rendered can be described by one Gaussian point. There may be multiple identical objects or different objects in the object to be rendered. That is, the parameters such as the Gaussian point positions, covariances, colors, and transparencies corresponding to different objects may be the same or different, and the scaling information corresponding to different Gaussian points may be the same or different. Therefore, the sizes of the initial patches corresponding to different Gaussian points may be the same or different.

[0063] The rotation information is used to characterize the orientation of the object to be rendered in space. Similarly, there may be multiple identical objects or different objects in the object to be rendered. That is, the orientations of different objects may be the same or different, the initial normal directions corresponding to different Gaussian points may be the same or different, and the rotation information corresponding to different Gaussian points may be the same or different. Therefore, the target normal directions corresponding to different Gaussian points may be the same or different.

[0064] By generating patches corresponding to each Gaussian point in the 3DGS model, an initial mesh model can be obtained by combination.

[0065] It should be noted that in some scenarios, the 3DGS model obtained through training itself has multiple attributes. If the normal information of each Gaussian point is already included in these attributes, there is no need to use the above method to obtain the initial normal direction. The initial normal direction can be directly obtained by acquiring the normal information.

[0066] Through the image rendering method of the embodiments of the present disclosure, the 3DGS model is transformed into a mesh model based on the scaling information and rotation information of the Gaussian points, and a mesh model that can accurately describe the object to be rendered can be obtained. In this way, based on the mask projection of the mesh model subsequently, a shadow map that fits the shadow rendering of the object to be rendered can be obtained, and the authenticity of the shadow rendering effect.

[0067] The following will be combined with Figure 4 The method for determining the initial normal direction of the embodiments of the present disclosure will be described schematically. Figure 4 Schematically shows the flow of the method for determining the initial normal direction according to the embodiments of the present disclosure.

[0068] In some embodiments, respectively determining the initial normal direction of each Gaussian point according to the respective scaling information of each Gaussian point may include:

[0069] Project the scaling information onto the first direction, the second direction, and the third direction respectively to obtain the first scaling scale, the second scaling scale, and the third scaling scale, where the first direction, the second direction, and the third direction are orthogonal to each other.

[0070] Determine the direction corresponding to the minimum scaling scale among the first scaling scale, the second scaling scale, and the third scaling scale as the initial normal direction.

[0071] As Figure 4 shown, for example, each Gaussian point can be regarded as an ellipsoid. The first direction can be the X-axis direction, the second direction can be the Y-axis direction, and the third direction can be the Z-axis direction. Place the ellipsoid representing the Gaussian point in the coordinate system, and the center of the ellipsoid is located at the origin of the coordinate system. The first scaling scale, the second scaling scale, and the third scaling scale obtained by projecting the scaling information onto the first direction, the second direction, and the third direction respectively can be understood as the protruding distances of the ellipsoid in the X-axis direction, the Y-axis direction, and the Z-axis direction respectively. For example, if the protruding distance of a certain ellipsoid in the X-axis direction is the smallest, it can be considered that the initial normal direction of the Gaussian point represented by the ellipsoid is along the X-axis direction.

[0072] Through the image rendering method of the embodiments of the present disclosure, selecting the direction with the smallest scaling scale as the normal direction can avoid the situation of too large or too small values during the calculation process, thereby improving the numerical stability of the entire rendering process. Moreover, selecting the direction with the smallest scaling scale as the normal direction means that the possibility of change in this direction is relatively small, which can simplify the normal calculation process, reduce the consumption of computing resources, and improve the rendering efficiency. In addition, by selecting the dimension with the smallest scaling information as the initial direction, the direction of the normal can be more effectively controlled, thereby more accurately representing the surface shape and characteristics of the object.

[0073] Based on the above embodiments, in some embodiments, simulating the transparency of the initial patch vertex representation with a Gaussian distribution may include:

[0074] Fitting the transparency of the initial patch vertex representation into a Gaussian distribution form of transparency.

[0075] Obtaining a preset transparency control threshold and the texture coordinates of the object to be rendered.

[0076] Performing a Gaussian distribution simulation on the transparency of the initial patch according to the Gaussian distribution form of transparency, the preset transparency control threshold, and the texture coordinates.

[0077] Taking a rectangular patch as an example, the transparency of the initial patch can be linearly distributed, for example. It can be interpolated based on the transparency values of the four vertices, and the linearly distributed transparency values are fitted and transformed into a Gaussian distribution form of transparency.

[0078] For example, the texture coordinates of the object to be rendered can be processed into values between -1 and 1:

[0079]

[0080] where RemapVal is the processed texture coordinate and TexCoord is the texture coordinate before processing.

[0081] For example, performing a Gaussian distribution simulation on the transparency of the initial patch according to the Gaussian distribution form of transparency, the preset transparency control threshold, and the texture coordinates can be:

[0082]

[0083] where FinalAlpha is the transparency after Gaussian distribution simulation, CustomScale is the preset transparency control threshold, Alpha is the transparency represented by the vertex, and RemapVal is the processed texture coordinate.

[0084] It should be understood that the above calculation method of texture coordinates and the calculation method of transparency after Gaussian distribution simulation are for more clearly illustrating the process of Gaussian distribution simulation of transparency, and are not used to limit the present disclosure.

[0085] Through the image rendering method of the embodiments of the present disclosure, the transparency of the straight-line distribution of patches is fitted to a Gaussian distribution, avoiding the sudden changes that may be brought about by the straight-line distribution of transparency, thereby improving the accuracy of shadow calculation. And it can make the shadow edge more natural, reducing the problems of jagged edges or abrupt transitions, and can calculate the masked shadow more precisely. In addition, fitting the transparency with a Gaussian distribution can simplify the process of shadow calculation, reducing the number of transparency levels to be processed, thereby reducing the computational burden in the rendering process and improving the rendering efficiency.

[0086] The following will be combined with Figure 5 A schematic description of dithering the transparency in the embodiments of the present disclosure. Figure 5 A schematic diagram shows the effect diagram after dithering the transparency according to the embodiments of the present disclosure.

[0087] Based on the above embodiments, in some embodiments, dithering the transparency of each initial patch after Gaussian distribution simulation may include:

[0088] Obtain the pixel position of the object to be rendered in the display area.

[0089] Determine the random perturbation value of transparency according to the pixel position and the random noise extracted from the noise map.

[0090] According to the embodiments of the present disclosure, the pixel position of the object to be rendered in the display area can be understood as the projection point obtained by projecting the points of the mesh model in three-dimensional space onto a two-dimensional plane (display area).

[0091] A noise map can be obtained in advance. The noise map can be a two-dimensional texture containing pseudo-random noise, and the size and resolution of the noise map can match the object to be rendered or be appropriately scaled.

[0092] For each selected pixel position, a noise value can be sampled from the noise map. This noise value will be used to generate the random perturbation of transparency.

[0093] When performing dithering processing, the anti-aliasing sampling index value (TAASampleIndex) can be used to read the corresponding transparency value from the buffer of the previous frame, and combine the read transparency value with the random noise value obtained from the noise map to calculate the dithered transparency value.

[0094] For example, based on the pixel position (ScreenPosition), the sampling index value (TAASampleIndex) of TAA anti-aliasing, and a noise texture, the process of adding dithering to the transparency (FinalAlpha) value after Gaussian distribution simulation can be as follows:

[0095] First, calculate the texture coordinates UV of the noise texture. For example:

[0096]

[0097] Among them, ScreenPosition is the pixel coordinate of the display area on the screen (if the screen resolution is 1080 * 720, the pixel position of the center point pixel can be (540, 360)). TAASampleIndex is related to the TAA anti-aliasing algorithm and will return a value between 0 and the maximum number of samples. The result is finally divided by (64.0, 64.0), which can immediately make the resolution of the noise texture (which can be replaced with a perturbation texture of different resolutions) 64 * 64. The fract() function can be used to generate texture coordinates to achieve the effect of texture repetition, tiling, or offset.

[0098] Then, sample the noise texture using the UV coordinates calculated in the previous step to obtain the perturbation value (AlphaNoise) of the transparency.

[0099] As Figure 5 shown, the left figure schematically shows the effect diagram without dithering the transparency, and the right figure shows the effect diagram without dithering the transparency. From Figure 5 it can be seen that by adding random value perturbations to the transparency, the dithering effect at the patch edge is achieved.

[0100] It should be understood that the above method for calculating the texture perturbation value is for the purpose of more clearly explaining the process of simulating the transparency by Gaussian distribution, and is not used to limit the present disclosure.

[0101] Through the image rendering method of the embodiments of the present disclosure, adding a perturbation value to the transparency based on the random noise extracted from the noise texture can introduce tiny changes between consecutive frames, which helps to break the temporal correlation. And the noise texture can add randomness to the transparency, and this randomness can well simulate natural phenomena, making the rendered scene more vivid and realistic. In addition, compared with complex physical simulations, using the noise texture to add a perturbation value to the transparency is a relatively simple and efficient method. It can add rich details and changes to the scene without adding too much computational burden.

[0102] Based on the above embodiments, in some embodiments, rendering the object to be rendered based on a three-dimensional Gaussian sputtering model and a shadow map, and outputting a rendered image may include:

[0103] Render the three-dimensional Gaussian sputtering model to generate a first rendered image.

[0104] Perform shadow rendering on the first rendered image based on the shadow map to obtain a target image.

[0105] According to an embodiment of the present disclosure, a shadow map with a predetermined resolution of the entire scene can be rendered at the light source.

[0106] Through the image rendering method of the embodiments of the present disclosure, the original three-dimensional Gaussian sputtering model is used for scene rendering, and shadow rendering is performed based on the shadow map, which not only ensures the overall rendering effect but also ensures the shadow rendering effect.

[0107] Based on the above embodiments, in some embodiments, the three-dimensional Gaussian sputtering model can be rendered to generate a first rendered image, which may include:

[0108] Perform scene rendering based on the three-dimensional Gaussian sputtering model to obtain a scene rendering result.

[0109] Sample the scene rendering result based on the pixel positions of the three-dimensional Gaussian sputtering model projected on the display area to obtain a sampling result.

[0110] Perform a multiplication operation on the sampling result and the illumination color used for rendering as the base color, and simulate the transparency of the first rendered image using a Gaussian distribution to generate the first rendered image.

[0111] According to an embodiment of the present disclosure, the 3DGS model can be imported into the particle system node in the engine for rendering processing. Each Gaussian point can create a semi-transparent GPU particle, and fixed parameters such as color value, size, and initial position are assigned to it. At the same time, the particle dynamic parameters related to the main window rendering camera information, such as transparency and covariance value, are dynamically updated every frame. Render the 3DGS particle system, and output the scene rendering result to a rendering target (RenderTarget) with a predetermined resolution for use in illumination calculation.

[0112] A RenderTarget can be a continuous memory area where the graphics API can "draw" things. During the 3D graphics rendering process, the render target acts as an intermediate storage role for recording the rendered output results. These output results can be information such as color, depth, and stencil, which are stored in the render target texture (Render TargetTexture, RTT) for subsequent processing or display.

[0113] According to an embodiment of the present disclosure, when simulating the transparency of the first rendered image using a Gaussian distribution and no longer adding dithering processing to the transparency, it is to ensure that the rendering can cover the full screen without voids.

[0114] Through the image rendering method of the embodiments of the present disclosure, by sampling the rendering result of the 3DGS scene, existing rendering data can be efficiently utilized, avoiding unnecessary repeated calculations. The sampling method based on the pixel positions in the display area avoids complex geometric calculations, thereby reducing the computational burden during the rendering process. By multiplying the light color by the sampled color value, the interaction between light and the object surface can be simulated, making the rendering result more conform to the lighting effect in the real world.

[0115] Based on the above embodiments, in some embodiments, shadow rendering of the first rendered image based on a shadow map to obtain a target image may include:

[0116] Comparing the first depth value at each pixel position in the first rendered image with the second depth value at the corresponding position in the shadow map.

[0117] In response to the first depth value at the pixel position being greater than the second depth value, determining that the pixel position is in the shadow.

[0118] According to an embodiment of the present disclosure, a soft shadow method can be used for shadow rendering. This method automatically adjusts the filter kernel size according to the distance from the light source and the shading point to the occluder, thereby obtaining a more realistic soft shadow effect. The larger the light source, the softer the shadow edge transition; the farther the shading point is from the occluder, the softer the shadow edge transition. The realization of this principle depends on the simulation of the blurring effect of the shadow edge and the adaptive adjustment of the filter kernel size according to the light source size, the distance between the light source and the occluder, and the distance between the shading plane and the occluder.

[0119] During the rendering process, physically based rendering and deferred rendering can also be performed to further enhance the rendering effect and obtain the final 3DGS shadow and relighting results.

[0120] Figure 6 It is a block diagram of an image rendering device according to an embodiment of the present disclosure.

[0121] As Figure 6 shown, the image rendering device 600 may include a generation module 610, a projection module 620, a rendering module 630, and a display module 640.

[0122] A generation module 610 is configured to generate a target mesh model based on a three-dimensional Gaussian sputtering model. The three-dimensional Gaussian sputtering model includes a plurality of Gaussian points for describing an object to be rendered. The target mesh model includes target patches corresponding to respective Gaussian points, and the transparency of the middle region of the target patches in the target mesh model is lower than that of the surrounding regions of the target patches.

[0123] A projection module 620 is configured to project a shadow using the target mesh model as a mask, so as to obtain a shadow map of the object to be rendered at a predetermined resolution.

[0124] A rendering module 630 is configured to render the object to be rendered based on the three-dimensional Gaussian sputtering model and the shadow map, so as to obtain a rendered target image.

[0125] A display module 640 is configured to display the rendered target image.

[0126] According to an embodiment of the present disclosure, the generation module 610 generating the target mesh model based on the three-dimensional Gaussian sputtering model may include:

[0127] Generating an initial mesh model based on the three-dimensional Gaussian sputtering model. The initial mesh model includes initial patches corresponding to respective Gaussian points, and the vertices of the initial patches are used to represent the color and transparency of the corresponding Gaussian points.

[0128] Performing a Gaussian distribution simulation on the transparency represented by the vertices of the initial patches, so that the transparency of the middle region of the initial patches is lower than that of the surrounding regions of the initial patches.

[0129] Performing a dithering process on the transparency of each of the initial patches after the Gaussian distribution simulation respectively, so as to obtain the target mesh model.

[0130] According to an embodiment of the present disclosure, the generation module 610 generating the initial mesh model based on the three-dimensional Gaussian sputtering model may include:

[0131] Obtaining the scaling information and rotation information of each Gaussian point in the three-dimensional Gaussian sputtering model. The scaling information is used to represent the size and shape changes of the object to be rendered in space, and the rotation information is used to represent the orientation of the object to be rendered in space.

[0132] Respectively determining the initial normal direction of each Gaussian point according to the scaling information of each Gaussian point, and rotating the initial normal direction of each Gaussian point according to the rotation information of each Gaussian point, so as to obtain the target normal direction of each Gaussian point.

[0133] Respectively constructing initial patches perpendicular to the target normal direction of each Gaussian point according to the scaling information of each Gaussian point, and adding the color and transparency of the object to be rendered to the vertices of the respective initial patches, so as to obtain the initial mesh model.

[0134] According to an embodiment of the present disclosure, the generation module 610 determines the initial normal direction of each Gaussian point according to the respective scaling information of each Gaussian point, including:

[0135] Project the scaling information onto the first direction, the second direction, and the third direction respectively to obtain the first scaling scale, the second scaling scale, and the third scaling scale, where the first direction, the second direction, and the third direction are orthogonal to each other.

[0136] Determine the direction corresponding to the minimum scaling scale among the first scaling scale, the second scaling scale, and the third scaling scale as the initial normal direction.

[0137] According to an embodiment of the present disclosure, the generation module 610 performs Gaussian distribution simulation on the transparency of the initial patch vertex representation, including: fitting the transparency of the initial patch vertex representation into a Gaussian distribution form of transparency. Obtain a preset transparency control threshold and the texture coordinates of the object to be rendered. Perform Gaussian distribution simulation on the transparency of the initial patch according to the Gaussian distribution form of transparency, the preset transparency control threshold, and the texture coordinates.

[0138] According to an embodiment of the present disclosure, the generation module 610 performs dithering processing on the transparency after Gaussian distribution simulation of each initial patch respectively, including: obtaining the pixel position of the object to be rendered in the display area. Determining a random perturbation value of the transparency according to the pixel position and the random noise extracted from the noise map. Adding the random perturbation value to the transparency after Gaussian distribution simulation.

[0139] According to an embodiment of the present disclosure, the generation module 610 renders the object to be rendered based on the three-dimensional Gaussian sputtering model and the shadow map, and outputs the rendered image, including:

[0140] Fit the transparency of the initial patch vertex representation into a Gaussian distribution form of transparency.

[0141] Obtain a preset transparency control threshold and the texture coordinates of the object to be rendered.

[0142] Perform Gaussian distribution simulation on the transparency of the initial patch according to the Gaussian distribution form of transparency, the preset transparency control threshold, and the texture coordinates.

[0143] According to an embodiment of the present disclosure, the generation module 610 performs dithering processing on the transparency after Gaussian distribution simulation of each initial patch respectively, including:

[0144] Obtain the pixel position of the object to be rendered in the display area.

[0145] Determine a random perturbation value of the transparency according to the pixel position and the random noise extracted from the noise map.

[0146] Add the random perturbation value to the transparency after Gaussian distribution simulation.

[0147] According to an embodiment of the present disclosure, the rendering module 630 renders the object to be rendered based on a three-dimensional Gaussian sputtering model and a shadow map, and outputs a rendered image, including:

[0148] Render the three-dimensional Gaussian sputtering model to generate a first rendered image.

[0149] Perform shadow rendering on the first rendered image based on the shadow map to obtain a target image.

[0150] According to an embodiment of the present disclosure, the rendering module 630 renders the three-dimensional Gaussian sputtering model to generate a first rendered image, including:

[0151] Perform scene rendering based on the three-dimensional Gaussian sputtering model to obtain a scene rendering result.

[0152] Sample the scene rendering result based on the pixel positions projected by the three-dimensional Gaussian sputtering model on the display area to obtain a sampling result.

[0153] Perform a multiplication operation on the sampling result and the lighting color used for rendering as the base color, and simulate the transparency of the first rendered image using a Gaussian distribution to generate the first rendered image.

[0154] According to an embodiment of the present disclosure, the rendering module 630 performs shadow rendering on the first rendered image based on the shadow map to obtain a target image, including:

[0155] Compare the first depth value at each pixel position in the first rendered image with the second depth value at the corresponding position in the shadow map.

[0156] In response to the first depth value at the pixel position being greater than the second depth value, determine that the pixel position is in the shadow.

[0157] It should be noted that for the details not described in the image rendering device according to the embodiments of the present disclosure, please refer to the corresponding parts of the foregoing image rendering method embodiments, which will not be elaborated herein.

[0158] Figure 7 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0159] As Figure 7 shown, the electronic device 700 may include

[0160] at least one display device 710 and a processor 720 communicatively connected to the at least one display device 710.

[0161] The processor 720 is configured to execute the following method:

[0162] Generate a target mesh model based on a three-dimensional Gaussian sputtering model, where the three-dimensional Gaussian sputtering model includes a plurality of Gaussian points for describing the object to be rendered, the target mesh model includes target patches corresponding to each Gaussian point, and the transparency of the middle region of the target patches in the target mesh model is lower than that of the surrounding regions of the target patches; project a shadow using the target mesh model as a mask to obtain a shadow map of the object to be rendered at a predetermined resolution; render the object to be rendered based on the three-dimensional Gaussian sputtering model and the shadow map to obtain a rendered target image; output the rendered target image to the at least one display device for display.

[0163] According to an embodiment of the present disclosure, the processor 720 generating a target mesh model based on a three-dimensional Gaussian sputtering model may include:

[0164] Generate an initial mesh model based on the three-dimensional Gaussian sputtering model, where the initial mesh model includes initial patches corresponding to each Gaussian point, and the vertices of the initial patches are used to represent the color and transparency of the corresponding Gaussian points.

[0165] Perform Gaussian distribution simulation on the transparency represented by the vertices of the initial patches, so that the transparency of the middle region of the initial patches is lower than that of the surrounding regions of the initial patches.

[0166] Perform dithering processing on the transparency after Gaussian distribution simulation of each initial patch respectively to obtain the target mesh model.

[0167] According to an embodiment of the present disclosure, the processor 720 generating an initial mesh model based on the three-dimensional Gaussian sputtering model may include:

[0168] Obtain the scaling information and rotation information of each Gaussian point in the three-dimensional Gaussian sputtering model, where the scaling information is used to represent the size and shape changes of the object to be rendered in space, and the rotation information is used to represent the orientation of the object to be rendered in space.

[0169] Determine the initial normal direction of each Gaussian point respectively according to the scaling information of each Gaussian point, and rotate the initial normal direction of each Gaussian point according to the rotation information of each Gaussian point to obtain the target normal direction of each Gaussian point.

[0170] Construct initial patches perpendicular to the respective target normal directions according to the scaling information of each Gaussian point respectively, and add the color and transparency of the object to be rendered to the vertices of the respective initial patches to obtain the initial mesh model.

[0171] According to an embodiment of the present disclosure, the processor 720 determining the initial normal direction of each Gaussian point respectively according to the scaling information of each Gaussian point includes:

[0172] Project the scaling information onto the first direction, the second direction, and the third direction respectively to obtain a first scaling factor, a second scaling factor, and a third scaling factor, where the first direction, the second direction, and the third direction are orthogonal to each other.

[0173] Determine the direction corresponding to the minimum scaling factor among the first scaling factor, the second scaling factor, and the third scaling factor as the initial normal direction.

[0174] According to an embodiment of the present disclosure, the processor 720 performs Gaussian distribution simulation on the transparency of the initial patch vertices, including: fitting the transparency of the initial patch vertices into a Gaussian distribution form of transparency. Obtain a preset transparency control threshold and the texture coordinates of the object to be rendered. Perform Gaussian distribution simulation on the transparency of the initial patch according to the Gaussian distribution form of transparency, the preset transparency control threshold, and the texture coordinates.

[0175] According to an embodiment of the present disclosure, the processor 720 performs dithering processing on the transparency after Gaussian distribution simulation of each initial patch respectively, including: obtaining the pixel position of the object to be rendered in the display area. Determining a random perturbation value of the transparency according to the pixel position and the random noise extracted from the noise map. Adding the random perturbation value to the transparency after Gaussian distribution simulation.

[0176] According to an embodiment of the present disclosure, the processor 720 renders the object to be rendered based on a three-dimensional Gaussian sputtering model and a shadow map, and outputs a rendered image, including:

[0177] Fit the transparency of the initial patch vertices into a Gaussian distribution form of transparency.

[0178] Obtain a preset transparency control threshold and the texture coordinates of the object to be rendered.

[0179] Perform Gaussian distribution simulation on the transparency of the initial patch according to the Gaussian distribution form of transparency, the preset transparency control threshold, and the texture coordinates.

[0180] According to an embodiment of the present disclosure, the processor 720 performs dithering processing on the transparency after Gaussian distribution simulation of each initial patch respectively, including:

[0181] Obtain the pixel position of the object to be rendered in the display area.

[0182] Determine a random perturbation value of the transparency according to the pixel position and the random noise extracted from the noise map.

[0183] Add the random perturbation value to the transparency after Gaussian distribution simulation.

[0184] According to an embodiment of the present disclosure, the processor 720 renders the object to be rendered based on a three-dimensional Gaussian sputtering model and a shadow map, and outputs a rendered image, including:

[0185] Render the three-dimensional Gaussian sputtering model to generate a first rendered image.

[0186] Perform shadow rendering on the first rendered image based on the shadow map to obtain the target image.

[0187] According to an embodiment of the present disclosure, the processor 720 renders the three-dimensional Gaussian sputtering model to generate a first rendered image, including:

[0188] Perform scene rendering based on the three-dimensional Gaussian sputtering model to obtain a scene rendering result.

[0189] Sample the scene rendering result based on the pixel positions of the three-dimensional Gaussian sputtering model projected on the display area to obtain a sampling result.

[0190] Perform a multiplication operation on the sampling result and the lighting color used for rendering as the base color, and simulate the transparency of the first rendered image using a Gaussian distribution to generate the first rendered image.

[0191] According to an embodiment of the present disclosure, the processor 720 performs shadow rendering on the first rendered image based on the shadow map to obtain the target image, including:

[0192] Compare the first depth value of each pixel position in the first rendered image with the second depth value at the corresponding position in the shadow map.

[0193] In response to the first depth value of the pixel position being greater than the second depth value, determine that the pixel position is in the shadow.

[0194] It should be noted that for the details not described in the electronic device of the embodiments of the present disclosure, please refer to the embodiments of the foregoing image rendering method, which will not be elaborated here.

[0195] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0196] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0197] Figure 8FIG. 0 shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementations of the present disclosure described and / or claimed herein.

[0198] As Figure 8 shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0199] A plurality of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0200] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the text processing method and / or the deployment method of a deep learning framework. For example, in some embodiments, the text processing method and / or the deployment method of a deep learning framework can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the text processing method and / or the deployment method of a deep learning framework described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the text processing method and / or the deployment method of a deep learning framework by any other suitable means (e.g., by means of firmware).

[0201] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip (SOC) systems, complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0202] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on the remote machine or server.

[0203] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0204] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) monitor or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0205] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of a communication network include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0206] A computer system may include a client and a server. The client and the server are generally far from each other and typically interact via a communication network. The relationship between the client and the server is generated by computer programs running on respective computers and having a client-server relationship with each other.

[0207] It should be understood that various forms of the processes shown above may be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitation is imposed herein.

[0208] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. An image rendering method, comprising: Generate a target mesh model based on a three-dimensional Gaussian sputtering model, wherein the three-dimensional Gaussian sputtering model includes a plurality of Gaussian points for describing an object to be rendered, the target mesh model includes a target facet for each Gaussian point, and the transparency of a middle region of the target facet in the target mesh model is lower than the transparency of a surrounding region of the target facet; Using the target mesh model as a mask to cast a shadow, obtaining a shadow map of the object to be rendered at a predetermined resolution; Rendering the object to be rendered based on the three-dimensional Gaussian sputtering model and the shadow map to obtain a rendered target image; and Displays the rendered target image.

2. The method according to claim 1, wherein: The method of generating a target grid model based on a three-dimensional Gaussian sputtering model comprises: Generate an initial mesh model based on the three-dimensional Gaussian sputtering model, wherein the initial mesh model includes initial facets corresponding to respective Gaussian points, and the vertices of the initial facets are used to characterize the color and transparency of the corresponding Gaussian points; Performing Gaussian distribution simulation on the transparency represented by the vertices of the initial face patch, so that the transparency of the middle area of ​​the initial face patch is lower than the transparency of the surrounding areas of the initial face patch; and The transparency of each initial face after Gaussian distribution simulation is jittered to obtain the target mesh model.

3. The method according to claim 2, wherein: The generating an initial mesh model based on the three-dimensional Gaussian sputtering model comprises: Obtaining scaling information and rotation information of each Gaussian point in the three-dimensional Gaussian sputtering model, wherein the scaling information is used to characterize the size and shape changes of the object to be rendered in space, and the rotation information is used to characterize the orientation of the object to be rendered in space; Determine the initial normal direction of each Gauss point according to the scaling information of each Gauss point, and rotate the initial normal direction according to the rotation information of each Gauss point to obtain the target normal direction; and Initial patches perpendicular to the respective target normal directions are constructed according to the respective scaling information of the respective Gaussian points, and the colors and transparency of the objects to be rendered are added to the vertices of the respective initial patches to obtain the initial mesh model.

4. The method according to claim 3, wherein: The determining the initial normal direction of each Gauss point according to the scaling information of each Gauss point respectively includes: Projecting the scaling information to a first direction, a second direction, and a third direction respectively to obtain a first scaling scale, a second scaling scale, and a third scaling scale, wherein the first direction, the second direction, and the third direction are orthogonal to each other; and A direction corresponding to a minimum scaling scale among the first scaling scale, the second scaling scale, and the third scaling scale is determined as the initial normal direction.

5. The method according to claim 2, wherein: The Gaussian distribution simulation of the transparency represented by the initial face vertex comprises: Fitting the transparency represented by the initial facet vertices to the transparency in the form of Gaussian distribution; Obtaining a preset transparency control threshold and texture coordinates of the object to be rendered; and The transparency of the initial patch is simulated by Gaussian distribution according to the transparency in the form of Gaussian distribution, the preset transparency control threshold and the texture coordinates.

6. The method according to claim 2, wherein: The dithering process is performed on the transparency of each initial surface piece after Gaussian distribution simulation, including: Obtaining the pixel position of the object to be rendered in the display area; Determining a random perturbation value of transparency based on the pixel position and random noise extracted from the noise map; and Adds the random perturbation value to the transparency after Gaussian distribution simulation.

7. The method according to claim 1, wherein: The step of rendering the object to be rendered based on the three-dimensional Gaussian sputtering model and the shadow map and outputting a rendered image comprises: Rendering the three-dimensional Gaussian sputtering model to generate a first rendered image; Shadow rendering is performed on the first rendered image based on the shadow map to obtain the target image.

8. The method according to claim 7, wherein: The step of rendering the three-dimensional Gaussian sputtering model to generate a first rendered image includes: Perform scene rendering based on a three-dimensional Gaussian sputtering model to obtain a scene rendering result; Based on the pixel position of the three-dimensional Gaussian sputtering model projected on the display area, sampling the scene rendering result to obtain a sampling result; The sampling result is multiplied by the illumination color used for rendering to obtain a basic color, and Gaussian distribution is used to simulate the transparency of the first rendered image to generate the first rendered image.

9. The method according to claim 7, wherein: The shadow rendering of the first rendered image based on the shadow map to obtain the target image includes: Comparing a first depth value of each pixel position in the first rendered image with a second depth value of a corresponding position in the shadow map; In response to the first depth value of a pixel location being greater than the second depth value, it is determined that the pixel location is in shadow.

10. An image rendering device, comprising: A generating module, configured to generate a target mesh model based on a three-dimensional Gaussian sputtering model, wherein the three-dimensional Gaussian sputtering model includes a plurality of Gaussian points for describing an object to be rendered, the target mesh model includes a target facet for each Gaussian point, and the transparency of a middle region of the target facet in the target mesh model is lower than the transparency of a region surrounding the target facet; A projection module, used to use the target mesh model as a mask to project a shadow, so as to obtain a shadow map of the object to be rendered at a predetermined resolution; A rendering module, used for rendering the object to be rendered based on the three-dimensional Gaussian sputtering model and the shadow map to obtain a rendered target image; The display module is used to display the rendered target image.

11. An electronic device comprising at least one display device; and a processor in communication with the at least one display device; wherein, The processor executes the method according to any one of claims 1 to 9 to obtain a rendered target image, and outputs the rendered target image to the at least one display device for display.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

14. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images