Rendering methods, devices, equipment, and storage media for 3D models

Abstract

A rendering method, apparatus, device, and storage medium for a 3D model are disclosed, relating to the field of computer graphics technology. The method includes: acquiring vertex position data of a 3D model, wherein the vertex position data indicates a first position of at least one vertex of the 3D model in 3D space; projecting at least one vertex from 3D space to screen space according to camera parameters of a virtual camera in 3D space, obtaining projection point position data, wherein the projection point position data indicates a second position of the corresponding projection point of at least one vertex in screen space; predicting projection point color data corresponding to a preset lighting effect of the 3D model in 3D space using a light gradient model based on the projection point position data and camera parameters; and rendering the 3D model using the projection point position data and projection point color data to obtain a rendered image of the model. This application enables automatic rendering of lighting effects of 3D models, improving the rendering efficiency of lighting effects.

Description

Technical Field

[0001] This application relates to the field of computer graphics technology, and in particular to a method, apparatus, device, and storage medium for rendering three-dimensional models. Background Technology

[0002] With the development of computer technology, users have increasingly higher demands for special effects in videos or specific scenes. High-quality 3D model rendering can provide users with a better viewing experience.

[0003] Currently, to ensure the rendering quality of 3D models, professionals typically perform lighting rendering on them, giving the rendered 3D models lighting effects and thus achieving a more realistic visual effect. However, the rendering process of 3D models is mostly handled manually; for example, the rendering of lighting effects largely relies on manual artists. Manual rendering of 3D models is inefficient and not suitable for many scenarios. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for rendering three-dimensional models. The technical solution provided by this application can be as follows:

[0005] According to one aspect of the embodiments of this application, a method for rendering a three-dimensional model is provided, the method comprising:

[0006] Obtain vertex position data of the three-dimensional model, wherein the vertex position data is used to indicate the first position of at least one vertex of the three-dimensional model in three-dimensional space;

[0007] Based on the camera parameters of the virtual camera in the three-dimensional space, the at least one vertex is projected from the three-dimensional space to the screen space to obtain projection point position data. The projection point position data is used to indicate the second position of the projection point corresponding to the at least one vertex in the screen space.

[0008] Based on the projection point location data and the camera parameters, the projection point color data corresponding to the preset lighting effect of the three-dimensional model in the three-dimensional space is predicted by the light gradient model;

[0009] The three-dimensional model is rendered using the projection point position data and the projection point color data to obtain a rendered image of the model in the screen space.

[0010] According to one aspect of the embodiments of this application, a rendering apparatus for a three-dimensional model is provided, the apparatus comprising:

[0011] A vertex data acquisition module is used to acquire vertex position data of the three-dimensional model, wherein the vertex position data is used to indicate the first position of at least one vertex of the three-dimensional model in three-dimensional space.

[0012] The projection point data acquisition module is used to project at least one vertex from the three-dimensional space to the screen space according to the camera parameters of the virtual camera in the three-dimensional space, and obtain projection point position data. The projection point position data is used to indicate the second position of the projection point corresponding to the at least one vertex in the screen space.

[0013] The color data prediction module is used to predict the color data of the projection point corresponding to the preset lighting effect of the three-dimensional model in the three-dimensional space based on the projection point position data and the camera parameters and through the light gradient model.

[0014] The 3D model rendering module is used to render the 3D model using the projection point position data and the projection point color data to obtain the model rendering image in the screen space.

[0015] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described rendering method for a three-dimensional model.

[0016] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the above-described rendering method for a three-dimensional model.

[0017] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program stored in a computer-readable storage medium, and a processor reading from the computer-readable storage medium and executing the computer program to implement the above-described three-dimensional model rendering method.

[0018] The technical solutions provided in this application have at least the following beneficial effects:

[0019] On the one hand, after projecting each vertex of the 3D model from 3D space to screen space according to camera parameters to obtain each projection point, it supports predicting the projection point color data corresponding to the preset lighting effect through a light gradient model based on the projection points and camera parameters, thereby realizing the automatic prediction of the color value to be assigned at the projection point. On the other hand, it supports rendering the model rendering image of the 3D model using the second position of the projection point and the projection point color data corresponding to the preset lighting effect. Since the projection point color data corresponding to the preset lighting effect can make the 3D model in the model rendering image have the preset lighting effect during the rendering process, the lighting effect of the 3D model can be automatically rendered without the need for manual rendering of the lighting effect by artists, which is conducive to improving the rendering efficiency of the lighting effect of the 3D model.

[0020] In addition, since this application can avoid the problem of inaccurate lighting effect rendering caused by errors easily introduced by manual art, it is beneficial to improve the lighting effect rendering quality of 3D models. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of a rendering method for a three-dimensional model provided in one embodiment of this application;

[0022] Figure 2 This is a schematic diagram of a computer system provided in one embodiment of this application;

[0023] Figure 3 This is a flowchart of a three-dimensional model rendering method provided in one embodiment of this application;

[0024] Figure 4 This is a schematic diagram of a three-dimensional model located in three-dimensional space provided in one embodiment of this application;

[0025] Figure 5 This is a flowchart of a method for obtaining projection point position data according to an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of a model projection provided in one embodiment of this application;

[0027] Figure 7 This is a flowchart of a method for obtaining projection point color data according to an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of a sampling point acquisition method provided in one embodiment of this application;

[0029] Figure 9 This is a schematic diagram of a model rendering image provided in one embodiment of this application;

[0030] Figure 10This is a schematic diagram of a simplified three-dimensional model provided in one embodiment of this application;

[0031] Figure 11 This is a flowchart of a three-dimensional model rendering method provided in another embodiment of this application;

[0032] Figure 12 This is a block diagram of a rendering apparatus for a three-dimensional model provided in one embodiment of this application;

[0033] Figure 13 This is a block diagram of a rendering apparatus for a three-dimensional model provided in another embodiment of this application;

[0034] Figure 14 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0036] Before introducing the technical solutions of this application, some terms involved in this application will be explained. The following related explanations are optional solutions and can be arbitrarily combined with the technical solutions of the embodiments of this application, all of which fall within the protection scope of the embodiments of this application. The embodiments of this application include at least some of the following contents.

[0037] A 3D model is a polygonal representation of an object. 3D models can be used to represent real-world entities or virtual objects. A 3D model can be a model with 3D data constructed in a virtual 3D space.

[0038] A mesh, also known as a mesh surface, is the basic unit that makes up a 3D model. The surface of a 3D model can be composed of multiple meshes, which together construct the shape of the 3D model. Mesh types can include triangular patches, quadrilateral patches, and any other polygonal patches. A mesh can consist of vertices, edges, and faces.

[0039] Vertex: The smallest unit of geometry, a single point in three-dimensional space, used to define the corners and intersections of a mesh. Vertices connect to form edges and faces, collectively constituting the surface and shape of a 3D model. The vertices of a 3D model are the vertices of its mesh.

[0040] Rendering a 3D model refers to the process of converting a 3D model into a 2D image. The rendering process simulates how light interacts with the 3D model in a virtual 3D space, thus giving the rendered 2D image (hereinafter referred to as the rendered image) visual representations such as depth, lighting, and shadow effects. Rendering of a 3D model can be performed on a vertex or mesh basis.

[0041] Lighting and shadow effects play crucial roles in the rendering process, jointly determining the quality and realism of the rendered image. Lighting rendering refers to simulating the effect of light on the surfaces of objects (such as 3D models) in a scene (such as a virtual 3D space), which is then implemented as lighting effects. Shadow effects refer to the shadows formed by the occlusion relationships between objects during the lighting rendering process.

[0042] To ensure the rendering quality of 3D models, professionals typically perform lighting rendering on them, enabling the rendered image to present a 3D model with lighting effects, thus achieving a more realistic visual effect. For example, shadow effects are an important component of lighting effects, and shadow effects include soft shadow effects. Professionals usually perform lighting rendering on 3D models to enable the rendered image to present a 3D model with soft shadow effects. Among them, soft shadow effects are used in computer graphics to simulate the shadow gradient effects in real life. They have soft boundaries and a certain transition between light and dark (forming the "soft" characteristic), which makes the 3D model appear more natural and realistic.

[0043] In related technologies, the rendering process of 3D models is mostly handled manually. For example, the rendering of lighting effects (such as soft shadow effects) largely relies on manual artists. For instance, after obtaining sufficient geometric samples for the 3D model, the relevant personnel mainly analyze the geometric samples to model the lighting effects in a manual art style, such as continuously adjusting the lighting effects by hand to make the lighting effects match the 3D model.

[0044] However, manual adjustments are complex, difficult, and inefficient, resulting in low efficiency in modeling lighting effects. Furthermore, manual adjustments are prone to introducing errors, leading to inaccurate rendering of lighting effects and consequently, poor rendering quality.

[0045] The technical solution provided in this application embodiment achieves automatic rendering of the lighting effects of the 3D model by supporting automatic vertex projection, automatic color prediction, and automatic rendering of the 3D model.

[0046] Please refer to Figure 1 The diagram illustrates a rendering method for a three-dimensional model according to an embodiment of this application. The method includes at least one of the following steps:

[0047] 1. Obtain the three-dimensional model: Obtain the first position of at least one vertex of the three-dimensional model 101 in the three-dimensional space 100. The first position is used to refer to the position of the vertex in the three-dimensional space 100.

[0048] 2. Model Simplification: Merge at least one vertex of 3D model 101 to obtain simplified 3D model 102. The number of vertices in simplified 3D model 102 is less than the number of vertices in 3D model 101.

[0049] 3. Projection Modeling: At least one vertex (e.g., vertex P) of the simplified 3D model 102 is projected from 3D space 100 to screen space 300 (i.e., the 2D space corresponding to the screen), obtaining the second position of the projection point (e.g., projection point P″) of each vertex in the simplified 3D model 102 in screen space 300. The second position refers to the position of the projection point in screen space 300. For example, the vertex is first projected from 3D space 100 to the standardized device coordinate space 200 corresponding to the virtual camera 106 (e.g., intermediate projection point P′), and then transformed to the screen space 300 of the screen to obtain the projection point corresponding to the vertex. The projection point refers to the point corresponding to the vertex in screen space 200.

[0050] 4. Color Prediction: The ray gradation model predicts the color value (i.e., projection point color data) of at least one projection point corresponding to the preset lighting effect of the 3D model 101 in the 3D space 100. For example, if the ray gradation model predicts that projection point P″ corresponds to the preset lighting effect, then the predicted color value of projection point P″ corresponding to the preset lighting effect is the same as the predicted color value of the intermediate projection point P′ corresponding to the preset lighting effect. The ray gradation model is used to simulate the influence of volumetric light 103 on the 3D space 100. Volumetric light 103 is the simulated light when rendering the preset lighting effect. Volumetric light 103 can be implemented as a series of lighting parameters, and the preset lighting effect can be implemented as the color assigned to each point in the 3D space 100.

[0051] 5. Model Rendering: Using the second position corresponding to at least one projection point and the predicted color value corresponding to at least one projection point, the 3D model 101 is rendered to obtain the model rendering image 104 in the screen space 300. The model rendering image 104 is used to present the 3D model 101 with preset lighting effects (such as soft shadow effect 105) in the screen space 300. The soft shadow effect 105 is rendered by simulating the effect of volumetric light 103 in the 3D space 100 on the 3D model 101. For example, for any projection point, the predicted color value corresponding to the projection point is assigned to the projection point, so that the 3D model 101 in the model rendering image 104 has preset lighting effects. For the shadow of the 3D model 101, the predicted color values ​​assigned to each projection point corresponding to the shadow have a gradient effect due to the effect of simulated volumetric light 103, thereby rendering the soft shadow effect 105.

[0052] Because the technical solution provided in this application can automatically render the lighting effects of 3D models, it avoids the problem of low lighting effect modeling efficiency caused by the high complexity, difficulty, and low efficiency of manual adjustments, thus improving the lighting effect rendering efficiency of 3D models. Furthermore, because this application can avoid the problem of inaccurate lighting effect rendering caused by errors easily introduced by manual artists, it helps to improve the lighting effect rendering quality of 3D models. Especially when simulating volumetric light for lighting rendering of 3D models, it can improve the rendering efficiency and quality of soft shadow effects.

[0053] The following will use method embodiments to specifically describe the technical solutions provided in the embodiments of this application.

[0054] Please refer to Figure 2 This diagram illustrates a computer system provided in one embodiment of this application. The computer system can implement the technical solutions provided in the embodiments of this application within a specific environment. The computer system may include: a terminal device 10 and a server 20.

[0055] Terminal device 10 can be an electronic device such as a mobile phone, tablet computer, multimedia playback device, PC (Personal Computer), wearable device, in-vehicle terminal device, VR (Virtual Reality) device, AR (Augmented Reality) device, MR (Mixed Reality) device, etc. Terminal device 10 can run a client of the target application. The target application can be an application that requires the use of a 3D model, or it can be an application used to construct a 3D model; this embodiment does not limit the scope. For example, the target application can include at least one of the following: image application (such as an image generated for rendering a 3D model), video application, engine application, or modeling application. In some embodiments, terminal device 10 also has a rendering application installed for automatically rendering 3D models, and the 3D model in the target application is exported to the rendering application for rendering. In some embodiments, the target application may have the ability to render 3D models itself.

[0056] The embodiments of this application do not limit the implementation form of the target application described above. For example, it can be an application that needs to be downloaded and installed, a mini-program that does not need to be installed, a web application, etc.

[0057] The target application may also include game applications, which can be any of the following: Massive Multiplayer Online Role-Playing Game (MMORPG), casual games, party games, sandbox games, tower defense games, action-adventure games, multiplayer online battle arena (MOBA) games, first-person shooter (FPS) games, multiplayer shooting survival games, third-person shooter (TPS) games, simulation games (SLG), simulation management games, survival building games, real-time strategy games, etc. This application embodiment does not limit this.

[0058] Server 20 can be used to provide background services for the clients of the aforementioned target applications (such as image-based applications). For example, server 20 can be a background server for the aforementioned target applications. Server 20 can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center.

[0059] Terminal device 10 can communicate with server 20 via a network, such as a wireless or wired network.

[0060] The technical solutions provided in this application can be applied to any scenario that requires rendering three-dimensional models, such as image rendering, three-dimensional model construction and other scenarios that require real-time rendering of three-dimensional models, as well as scenarios that can render three-dimensional models offline, such as images, animations, videos, movies, etc., and other scenarios such as rendering repair (such as three-dimensional model rendering failure requiring re-rendering), rendering update, application development (such as involving three-dimensional model construction and rendering).

[0061] For example, taking real-time rendering of a 3D model as an example, during the operation of the target application, the client of the target application can upload the 3D model to be rendered (such as an unrendered 3D model, a 3D model that failed to render, etc.) to the server 20. The server 20 performs real-time rendering of the 3D model to obtain a rendered image of the 3D model (i.e., the model rendering image mentioned above). The server 20 returns the rendered image to the client of the target application, and the client realizes real-time display of the 3D model through the rendered image.

[0062] Optionally, the rendering process of the 3D model can also be executed by the client of the target application, such as the client of the target application rendering the 3D model in real time to obtain the rendered image of the 3D model. This application embodiment does not limit this.

[0063] Please refer to Figure 3 The diagram illustrates a flowchart of a three-dimensional model rendering method according to an embodiment of this application. The execution entity for each step of this method is a computer device; for example, the execution entity for each step of this method could be... Figure 2 The terminal device 10 in the computer system shown can be a client, such as a server 20. Taking the server as the execution subject of each step of the method as an example, the method may include at least one of the following steps 310 to 340.

[0064] Step 310: Obtain vertex position data of the 3D model. The vertex position data is used to indicate the first position of at least one vertex of the 3D model in the 3D space.

[0065] The aforementioned 3D model can refer to a 3D model to be rendered, which may consist of multiple meshes, each mesh being enclosed by at least three vertices. The vertices of the meshes in the 3D model are the vertices of the 3D model. A vertex in the 3D model can refer to a 3D point in space, used to indicate the position of the 3D model in space and the area it occupies. This application does not limit the data for at least one vertex; at least one vertex can refer to all vertices of the 3D model or only some vertices, and can be set and adjusted according to actual usage requirements.

[0066] This application does not limit the source of the 3D model. For example, the 3D model mentioned above can refer to a 3D model provided by a specific application, such as a 3D model of a virtual character, virtual object (such as a virtual prop), virtual environment (such as a virtual mountain, virtual tree, virtual island, virtual river, etc.); the 3D model mentioned above can also refer to a 3D model provided by a modeling application, such as a 3D model constructed based on a real object.

[0067] The aforementioned three-dimensional space can refer to the virtual three-dimensional space in which the three-dimensional model resides. For example, the aforementioned three-dimensional model can refer to the world space provided by a specific application, in which the three-dimensional models of virtual characters, virtual objects, virtual environments, etc., are located. This world space has a corresponding world coordinate system to define the position of objects within it. The aforementioned three-dimensional model can also refer to the world space provided by a modeling application, used to place three-dimensional models constructed from real objects, and for constructing three-dimensional models of real objects.

[0068] In one example, the vertex position data mentioned above may include the position information of each vertex. The vertex position information is used to indicate the first position of the vertex in three-dimensional space, and the first position refers to the position of the vertex in three-dimensional space. For example, the vertex position information may include the coordinates of the vertex in the three-dimensional coordinate system corresponding to the three-dimensional space. For example, the vertex position information may include at least one of the following: the three-dimensional coordinates of the vertex in the three-dimensional coordinate system, and the homogeneous coordinates of the vertex in the three-dimensional coordinate system.

[0069] For example, refer to Figure 4 Within the three-dimensional space 400, there exists a three-dimensional model 401. The three-dimensional model 401 is composed of multiple triangular facets, each enclosed by three vertices 4011. The three-dimensional space 400 corresponds to a three-dimensional coordinate system O. w -X w Y w Z wThe three-dimensional spatial coordinate system can be set and adjusted according to actual usage requirements, and this application embodiment does not limit this. For example, the z-axis of the three-dimensional spatial coordinate system is vertically upward, the y-axis is horizontally to the right, and the x-axis is perpendicular to both the y-axis and z-axis. The origin of the three-dimensional spatial coordinate system can be set and adjusted according to requirements.

[0070] Optionally, for each vertex 4011, the homogeneous coordinates of vertex 4011 in the three-dimensional spatial coordinate system can be determined as the position information of vertex 4011. By packaging the position information of each vertex 4011, the vertex position data of the three-dimensional model 401 can be obtained.

[0071] Step 320: Based on the camera parameters of the virtual camera in the three-dimensional space, at least one vertex is projected from the three-dimensional space to the screen space to obtain projection point position data. The projection point position data is used to indicate the second position of the projection point corresponding to the at least one vertex in the screen space.

[0072] A virtual camera can refer to a fictional camera that simulates the shooting principle of a real camera. It can be used to observe three-dimensional space to capture two-dimensional images of that space. For example, a virtual camera can be implemented as a component that simulates the behavior and function of a real camera, and this component can generate two-dimensional images based on the shooting principle of a real camera. In the embodiments of this application, the virtual camera can be used for projection modeling of a three-dimensional model to obtain the projection points of at least one vertex of the three-dimensional model in screen space.

[0073] Optionally, the virtual camera can be fixed at a certain position in three-dimensional space, and its intrinsic parameters (such as focal length, principal point, distortion coefficient, etc.) and rotation attitude are also fixed. The virtual camera can also move along with the movement of the three-dimensional model to observe the model in real time. The virtual camera's intrinsic parameters (such as focal length, principal point, distortion coefficient, etc.) and rotation attitude can also be dynamically adjusted as needed; this embodiment does not limit this. For example, refer to... Figure 4 A virtual camera 402 is set at a certain point in the three-dimensional space 400. The virtual camera 402 is used to observe the three-dimensional model 401.

[0074] The extrinsic parameters of a virtual camera may include: the virtual camera's position information, the virtual camera's rotation parameters, and the virtual camera's view orientation. The virtual camera's position information indicates its position in the three-dimensional coordinate system, the virtual camera's rotation parameters indicate its rotation angle in the three-dimensional coordinate system, and the virtual camera's view orientation indicates its orientation in the three-dimensional coordinate system.

[0075] Screen space refers to the space associated with the pixels on a screen. For example, it can be the two-dimensional space of a computer device's screen, the same size as the screen, and measured in pixels. The screen coordinate system of screen space can be a three-dimensional coordinate system. For instance, the screen coordinate system of screen space might have its origin at the top-left corner of the screen, with the y-axis pointing vertically downwards and the x-axis pointing horizontally to the right. The coordinate range of the y-axis and x-axis represents the screen's resolution, and the z-axis value represents depth, such as from 0 to 1, corresponding to the near and far clipping planes of a virtual camera, respectively. For example, see reference... Figure 1 Screen space 300 can be used to represent the two-dimensional space of the screen. The screen coordinate system of screen space 300 can take the upper left corner of the screen as the origin, the y-axis is vertically downward, the x-axis is horizontally to the right, the coordinate range of the y-axis and x-axis is the screen resolution, and the z-axis is perpendicular to the x-axis and y-axis respectively, with a range of 0 to 1.

[0076] In this embodiment, the screen space can be set on the near clipping plane (i.e., the film plane) of the virtual camera. The virtual camera can project objects in the camera space onto the near clipping plane, thereby converting the objects into the screen space.

[0077] A projection point can refer to the pixel corresponding to a vertex in screen space. For example, a vertex can be projected by a virtual camera into the following standardized device coordinate space to obtain an intermediate projection point. This intermediate projection point can then be transformed into screen space to obtain the projection point corresponding to the vertex in screen space. Optionally, a pixel in screen space can correspond to at least one vertex.

[0078] In one example, the aforementioned projection point position data may include position information of each projection point. This position information indicates a second position of the projection point in screen space, whereby the second position refers to the location of the projection point in screen space. For example, the projection point position information may include the coordinates of the projection point in the screen coordinate system corresponding to the screen space. The projection point position information may include at least one of the following: three-dimensional coordinates of the projection point in the screen coordinate system, or homogeneous coordinates of the projection point in the screen coordinate system.

[0079] The above-mentioned projection of at least one vertex from three-dimensional space to screen space can refer to the process of camera projection modeling. Camera projection modeling is one of the important steps in the rendering pipeline. Its purpose is to project objects in three-dimensional space onto a two-dimensional screen, that is, to project three-dimensional models in three-dimensional space onto screen space to generate the rendered image seen by the user.

[0080] In one example, the camera parameters of the virtual camera described above may include: the virtual camera's position information and the virtual camera's view orientation. (Reference) Figure 5Step 320 may also include the following sub-steps:

[0081] Step 321: Construct the projection matrix of the virtual camera based on the virtual camera's position information and view orientation.

[0082] The position and orientation of the virtual camera can be determined by parameters of the Unreal Engine, a real-time 3D creation tool used in application development, film and television production, architectural design, virtual reality, and other fields. In this embodiment, the Unreal Engine can be used for rendering 3D models. For example, taking an engine-type application as an example, the engine parameters may include the virtual camera's position information and view orientation.

[0083] The view orientation of a virtual camera can be described by three vectors: the forward vector f, the rightward vector r, and the upward vector u.

[0084] Forward vector f: Represents the virtual camera's line of sight. It can be calculated using the target point's position and the virtual camera's position information. For example, the ratio of the difference between the target point's position and the virtual camera's position information to the absolute value of that difference is determined as the forward vector f.

[0085] Where T is the position information of the target point (such as a position vector formed by coordinates), the target point can refer to the reference observation point in three-dimensional space (such as the origin of the three-dimensional space coordinate system), and C is the position vector of the virtual camera (i.e. the position information mentioned above).

[0086] The rightward vector r represents the direction to the right of the virtual camera. It can be calculated using the cross product of the forward vector f and the upper-right vector in the 3D coordinate system. For example, the ratio of the absolute value of the cross product between the forward vector f and the upper-right vector in the 3D coordinate system is used to determine the rightward vector r.

[0087] Among them, w u It is the upper right vector in the three-dimensional coordinate system.

[0088] The upward vector u represents the direction of the top of the virtual camera. It can be calculated based on the cross product of the rightward vector r and the forward vector f. For example, the cross product between the rightward vector r and the forward vector f can be determined as the upward vector u: u = r × f.

[0089] In one example, the projection matrix of the virtual camera includes a view transformation matrix and a projection transformation matrix. The view transformation matrix is ​​used to transform points in the 3D coordinate system corresponding to the 3D space to the camera coordinate system corresponding to the virtual camera. The projection transformation matrix is ​​used to project points in the camera coordinate system to the normalized device coordinate system corresponding to the screen space.

[0090] The camera coordinate system is a three-dimensional coordinate system constructed with the virtual camera as the origin. The space indicated by the camera coordinate system is called camera space. Camera space interacts with the aforementioned three-dimensional space and can also be called view space. Specifically, the z-axis of the camera coordinate system is mapped to the viewing direction (coinciding with the optical axis), the y-axis is mapped to the upward direction, and the x-axis is perpendicular to both the z-axis and y-axis.

[0091] For example, refer to Figure 6 The virtual camera 402 has a corresponding camera coordinate system O. c -X c Y c Z c Camera coordinate system O c -X c Y c Z c The origin of the camera coordinate system O is the point in three-dimensional space where the virtual camera is located. c -X c Y c Z c The indicated space is the camera space 500.

[0092] A virtual camera has a certain field of view, which can be defined by a view frustum (such as a cube). All objects within the view frustum are visible to the virtual camera (i.e., ultimately displayed on the screen), while objects outside the view frustum are invisible to the virtual camera.

[0093] A projection transformation matrix can be used to transform the coordinates within a created viewing box to a normalized device coordinate system; this process is called projection. Coordinates in the normalized device coordinate system are easily mapped to the screen coordinate system.

[0094] The space indicated by the standardized device coordinate system is called the Normalized Device Coordinates (NDC) space. The NDC space can be in the form of a cube (such as a cubic prism), a truncated pyramid, etc. For example, refer to... Figure 6 The virtual camera 402 corresponds to a standardized device coordinate space 600 (shaped like a truncated square pyramid) with a corresponding standardized device coordinate system O. n -X n Y n Z nThe origin of the standardized equipment coordinate system is the upper left corner of the near-cut plane. The x-axis and y-axis of the standardized equipment coordinate system range from (0, 0) to (1, 1), and the z-axis value (i.e., the depth value) is from 0 to 1, corresponding to points on the near-cut plane and the far-cut plane, respectively.

[0095] Optionally, the screen coordinate system O corresponding to screen space 700 s -X s Y s Z s It can be set in the near clipping plane, and the z-axis value (i.e., the depth value) is also 0 to 1, corresponding to points on the near clipping plane and the far clipping plane, respectively.

[0096] Optionally, the camera coordinate system and the standardized device coordinate system share the z-axis, and the screen coordinate system can also share the z-axis with the camera coordinate system and the standardized device coordinate system. This application does not limit this aspect.

[0097] In one example, the view transformation matrix can be constructed based on the extrinsic parameters of the virtual camera. For instance, it can be constructed based on the virtual camera's position information and view orientation. For example, the view transformation matrix can be constructed using the virtual camera's position vector, as well as its rightward, upward, and forward vectors, as elements. The view transformation matrix can then be represented as follows:

[0098] Where, r x r y r z These are the three elements of the right vector r, u x u y u z These are the three elements of the upward vector u, f x f y f z These are the three elements in the forward vector f.

[0099] In one example, the projection transformation matrix can be implemented as one of the following: orthographic projection matrix, perspective projection matrix.

[0100] An orthogonal projection matrix is ​​used to linearly transform points within a cube (i.e., the observation box mentioned above) to a normalized device coordinate system. The cube can be obtained by clipping from the view frustum of a virtual camera, in which case the normalized device coordinate space is also cube-shaped. If the cube's range is [-l, l], [-r, r], and [-n, f], then the elements of the orthogonal projection matrix can be constructed based on the parameters l, r, n, and f, and the orthogonal projection matrix can be constructed using these elements. The orthogonal projection matrix can then be represented as follows:

[0101]

[0102] The perspective projection matrix can be used to linearly transform points within a truncated pyramid (i.e., the observation box mentioned above) to a normalized device coordinate system. The cube can be obtained by clipping from the view frustum of the virtual camera, in which case the normalized device coordinate space also takes the form of a truncated pyramid. If the near-plane distance of the truncated pyramid is n, the far-plane distance is f, the viewing angle is θ, and the aspect ratio is α, then the elements of the perspective projection matrix can be constructed based on these parameters n, f, θ, and α, and the perspective projection matrix can be constructed using these elements. The perspective projection matrix can then be represented as follows:

[0103]

[0104] Step 322: Use the projection matrix to project at least one vertex from the three-dimensional space to the screen space to obtain the projection point position data.

[0105] Optionally, based on the view transformation matrix and the projection transformation matrix, vertices in the three-dimensional space are projected into the normalized device coordinate space, and then the vertices are transformed from the normalized device coordinate space into the screen space.

[0106] In one example, the process of obtaining the projection point location data may include the following:

[0107] 1. Use the view transformation matrix to transform at least one vertex in the three-dimensional spatial coordinate system to the camera coordinate system to obtain camera projection position data. The camera projection position data is used to indicate the fourth position of at least one vertex in the camera coordinate system.

[0108] Optionally, the camera projection data includes the position information of each vertex of the 3D model in the camera coordinate system. The position information of the vertices in the camera coordinate system may include the coordinates of the vertices in the camera coordinate system, such as 3D coordinates and homogeneous coordinates. A fourth position can be used to refer to the position of the vertex in camera space.

[0109] For each vertex of the 3D model, the product of the view transformation matrix and the vertex's position information in the 3D spatial coordinate system can be directly used to determine the vertex's position information in the camera coordinate system.

[0110] For example, refer to Figure 6 Taking any vertex 4011 as an example, vertex 4011 in the three-dimensional coordinate system O w -X w Y w Z w The homogeneous coordinates in P are w =[x w y w , z w[1]T, vertex 4011 can be transformed in the three-dimensional coordinate system O through the view transformation matrix V of the virtual camera 402. w -X w Y w Z w Homogeneous coordinates P in w Convert vertex 4011 to the camera coordinate system O of virtual camera 402. c -X c Y c Z c Homogeneous coordinates P in c =[x c y c , z c ,1] T =VP w .

[0111] 2. Use the projection transformation matrix to project at least one vertex in the camera coordinate system onto the standardized device coordinate system to obtain standardized projection point position data. The standardized projection point position data is used to indicate the fifth position of at least one projection point in the standardized device coordinate system.

[0112] The projection transformation matrix can project the vertices in the observation box onto the standardized device coordinate space, obtaining the intermediate projection point of the vertex in the standardized device coordinate space. The position of this intermediate projection point in the standardized device coordinate system is the position of the projection point of the vertex in the screen space in the standardized device coordinate system.

[0113] Optionally, the standardized projection point location data includes the location information of at least one projection point in the standardized device coordinate system. The location information of the projection point in the standardized device coordinate system may include the coordinates of the projection point in the standardized device coordinate system, such as three-dimensional coordinates and homogeneous coordinates. The fifth location can be used to refer to the position of the projection point in the standardized device coordinate space.

[0114] For each projection point, the product of the projection transformation matrix and the position information of the vertex in the camera coordinate system can be directly used to determine the position information of the projection point in the standardized device coordinate system, so as to convert the camera projection data into standardized projection point position data.

[0115] For example, refer to Figure 6 For any vertex 4011, the projection transformation matrix P of the virtual camera 402 can be used to project vertex 4011 onto the camera coordinate system O. c -X c Y c Z c Homogeneous coordinates P in c Converted to projection point 4012 in the standardized equipment coordinate system O n-X n Y n Z n Homogeneous coordinates P in n =[x n y n , z n w n ] T =PP c , where w n This indicates the depth of 600 in the standardized equipment coordinate space.

[0116] 3. Using the parameters corresponding to the screen space, transform at least one projection point in the standardized device coordinate system to the screen coordinate system corresponding to the screen space to obtain the projection point position data.

[0117] The parameters corresponding to the screen space can refer to the parameters of the screen on which the image to be displayed and rendered is located. Optionally, the parameters corresponding to the screen space can include the screen resolution. The resolution is measured in pixels.

[0118] Optionally, based on the screen resolution, the position information of the projection point in the standardized device coordinate system can be converted into the position information of the projection point in the screen coordinate system. If the position information of the projection point in the standardized device coordinate system is homogeneous coordinates, the standardized projection point position data can first be normalized to obtain normalized standardized projection point position data. Then, based on the screen resolution, the normalized standardized projection point position data can be converted back into projection point position data. Optionally, the normalized standardized projection point position data can be obtained by dividing each element of the homogeneous coordinates of the projection point in the standardized device coordinate system by the depth of 600 in the standardized device coordinate space.

[0119] For example, refer to Figure 6 For projection point 4012 in the standardized equipment coordinate system O n -X n Y n Z n Homogeneous coordinates P in n Normalization is performed to obtain normalized coordinates.

[0120] Optionally, the second position of the projection point in screen space can be obtained by linearly scaling the normalized coordinates to the screen width. For example, by linearly scaling the sum of the normalized coordinates and 1 to half the screen width, the second position of the projection point in screen space can be obtained. For instance, with a screen resolution of width W and height H, the projection point 4012 in the screen coordinate system O of screen space 700... s -X s Y sZ s The homogeneous coordinates in the middle are It is used to indicate the position of the projection point 4012 corresponding to vertex 4011 in screen space 700.

[0121] This application embodiment constructs an accurate projection matrix (such as a view transformation matrix or perspective projection matrix) by combining the perspective transformation of a virtual camera. This accurate projection matrix ensures the accuracy and realism of vertex projection, thereby improving the accuracy of light capture and, consequently, the accuracy of soft shadow generation.

[0122] Step 330: Based on the projection point location data and camera parameters, the projection point color data corresponding to the preset lighting effect of the 3D model in 3D space is predicted by the light gradient model.

[0123] Lighting effects are used to simulate various visual effects produced when light illuminates an object; that is, they are the effects obtained by rendering lighting on a 3D model. For example, based on the type of light source simulated in the lighting rendering, lighting effects include at least one of the following: global illumination and volumetric lighting. Global illumination simulates the complex reflection and scattering of light in a scene, achieving realistic lighting. Volumetric lighting simulates the interaction and movement of light with particles in the air, achieving volumetric lighting effects. Optionally, shadow effects are simulated by lighting and can be understood as a type of lighting effect or included within lighting effects.

[0124] The aforementioned preset lighting effect can refer to the lighting effect to be rendered onto the 3D model. This application embodiment does not limit the type of the preset lighting effect, which can be set and adjusted according to actual usage requirements. Optionally, the preset lighting effect can be implemented as the color assigned to each point in 3D space; that is, assigning a color corresponding to the preset lighting effect to each point in 3D space can achieve the rendering of the preset lighting effect.

[0125] Projection point color data can be used to indicate the predicted color value of at least one projection point corresponding to a preset lighting effect. The predicted color value refers to the predicted color value used to indicate the color of a point in three-dimensional space, such as being represented by RGB (Red-Green-Blue) values. In the embodiments of this application, the color value can refer to the color value that the projection point needs to be assigned to when the three-dimensional model has a preset lighting effect.

[0126] The preset lighting effect corresponding to a 3D model in 3D space can refer to the visual effect produced when a preset light source illuminates a 3D model in 3D space, that is, the effect obtained by rendering a 3D model with preset lighting. The predicted color value corresponding to the preset lighting effect can refer to the predicted color value that the projection point needs to be assigned when the 3D model has the preset lighting effect. For example, the projection point color data can include the color information of at least one projection point. The color information of the projection point is used to indicate the predicted color value of the projection point corresponding to the preset lighting effect. The color information of the projection point can be represented by RGB values.

[0127] In this embodiment, the predicted color value of the projection point corresponding to the preset lighting effect can be predicted based on the propagation mode of light (such as volumetric light) simulated in three-dimensional space when rendering a preset lighting effect. That is, by simulating the relationship between the observation point and each point in three-dimensional space under the influence of the simulated light when rendering the preset lighting effect. The observation point can be any point in three-dimensional space, such as the point where the virtual camera is located. Volumetric light refers to the simulation in computer graphics of light encountering impurities and being reflected in other directions when propagating in a medium, thus forming a bright beam of light. For example, in the field of image rendering, volumetric light is a commonly used lighting effect that can be used to represent the beam of light leaking from the translucent part of an occluded object when light shines on it, realistically reproducing the changes in brightness caused by the obstruction of light.

[0128] In simulating the effect of light on a 3D model when rendering preset lighting effects, each point in 3D space can be defined with volume density and color values ​​in various directions. Volume density is the differential probability that a ray terminates at a certain location in an infinitesimal particle; it can be used to represent the density of each point in 3D space, similar to a differential opacity.

[0129] The color assigned to the projection point observed by the virtual camera (i.e., the viewpoint) is related to the volume density and color value of each point on the ray emanating from the virtual camera. Therefore, based on the position information of the virtual camera and the projection point, the predicted color value of the projection point corresponding to the preset lighting effect can be predicted. By simulating the propagation of light in three-dimensional space during the rendering of the preset lighting effect, the predicted color value of the projection point corresponding to that preset lighting effect can be predicted, which helps improve the accuracy and realism of the predicted color of the projection point.

[0130] Optionally, the above-mentioned light gradient model can be used to simulate the effect of simulated light on the three-dimensional model when rendering preset lighting effects. For example, the light gradient model can be used to simulate the propagation of fictitious volumetric light in three-dimensional space (that is, the interaction between fictitious volumetric light and three-dimensional model) to simulate the effect of fictitious volumetric light on the three-dimensional model. The preset lighting effect can be the effect rendered by simulating the effect of fictitious volumetric light on the three-dimensional model.

[0131] In one example, the ray gradation model could be a machine learning model that learns the relationship between the color and volume density of sampled points and the position information of camera rays and sampled points when simulating the effect of light on a 3D model during the rendering of preset lighting effects. For example, it could take the direction information of camera rays and the position information of sampled points as inputs and the color and volume density of sampled points as outputs.

[0132] Optionally, the light gradient model can be constructed based on at least one of the following: MLP (Multilayer Perceptron), CNN (Convolutional Neural Networks), DNN (Deep Neural Networks), Transformer (a self-attention neural network), and ResNet (Residual Networks).

[0133] For example, consider constructing a ray gradation model based on an MLP. An MLP consists of several fully connected layers, which can be divided into an input layer, hidden layers, and an output layer for the ray gradation model. The input layer receives the input data, the hidden layers extract latent features from the input data and perform some nonlinear transformations, and the output layer generates the prediction result. Optionally, the ray gradation model may include multiple hidden layers. For example, after obtaining the input data from camera rays and sampling point data through the input layer, the ray gradation model performs feature extraction and some nonlinear transformations on the input data through multiple hidden layers to obtain a feature map. Finally, the output layer performs logistic regression on the feature map to output the prediction result.

[0134] For example, a light gradient model can be represented as follows:

[0135] (c,σ)=MLP(γ(x),γ(d));

[0136] Where c = (r, g, b) is the color of the sampling point, σ is the volume density of the sampling point, and γ(x) is the location information of the sampling point. w y w , z wThe encoded data of ) , where γ(d) is the orientation information of the virtual camera d=(d x d y d z ) encoded data.

[0137] In one example, reference Figure 7 Step 330 may also include the following sub-steps:

[0138] Step 331: For each projection point, based on the second position of the projection point and the camera parameters, obtain the camera ray corresponding to the projection point. The camera ray is used to simulate the ray path when the virtual camera captures the vertex corresponding to the projection point.

[0139] Standardized projection point position data can be obtained by transforming the projection point position data. This standardized projection point position data can be used to indicate the position of the projection point in the standardized device coordinate system. In other words, the fifth position of the projection point in the standardized device coordinate system can be obtained by transforming the second position of the projection point. The camera parameters of the virtual camera include the position information of the virtual camera, which is used to indicate the position of the virtual camera in the three-dimensional spatial coordinate system.

[0140] Optionally, when both the standardized device coordinate system and the camera coordinate system are constructed based on the virtual camera in the three-dimensional spatial coordinate system, the ray pointing from the position of the virtual camera in the three-dimensional spatial coordinate system to the fifth position of the projection point in the standardized device coordinate system can be determined as the camera ray corresponding to the projection point.

[0141] For example, refer to Figure 8 After constructing a standardized device coordinate system based on the position of the virtual camera 402 in the three-dimensional space 400, the ray that originates from the virtual camera 402 (such as the camera optical center) and passes through the projection point 4012 in the standardized device coordinate system can be used as the camera ray 4013. The camera ray 4013 propagates in a straight line in the three-dimensional space 400 and passes through the vertex 4011 on the three-dimensional model 401.

[0142] Step 332: Sample along the camera light in three-dimensional space to obtain sampling point position data. The sampling point position data is used to indicate the third position of at least two sampling points in three-dimensional space.

[0143] A sampling point can refer to a point in three-dimensional space. Optionally, at least two sampling points are obtained by randomly sampling along the camera ray in three-dimensional space. The coordinates of these sampling points in three-dimensional space are then used to determine their position information. Combining the position information of the at least two sampling points yields the sampling point position data. A third position can be used to indicate the location of the sampling point in three-dimensional space. Alternatively, at least two sampling points can be obtained by sampling along the camera ray in three-dimensional space according to preset rules. These preset rules include at least one of the following: equal spacing, increasing spacing, and decreasing spacing. Spacing refers to the distance between sampling points. If sampling points can be initiated from a virtual camera, the spacing of the first sampling point indicates the distance between the first sampling point and the virtual camera.

[0144] For example, refer to Figure 8 Multiple sampling points 4014 can be randomly sampled in the virtual space 400 along the camera ray 4013, and the coordinates of the sampling points 4014 in the three-dimensional coordinate system are used as the position information of the sampling points 4014. The position information of the multiple sampling points 4014 is combined to obtain the sampling point position data corresponding to the projection point 4012.

[0145] Step 333: Based on camera light and sampling point position data, predict the color information of the projection point through a light gradient model. The color information of the projection point is used to indicate the predicted color value of the projection point corresponding to the preset lighting effect.

[0146] Optionally, the color and volume density of the sampling points are predicted using a light gradient model, and then the color information of the projection points is calculated based on the color and volume density of the sampling points.

[0147] For example, the process of obtaining the color information of the projection point may include the following:

[0148] 1. Based on camera light and sampling point location data, the gradation model predicts at least two sampling points that correspond to the color and volume density of the preset lighting effect.

[0149] Optionally, camera rays can be characterized by their direction information. This direction information can be calculated based on the position information of the virtual camera in the three-dimensional coordinate system and the position information of the projection point in the standardized device coordinate system. For example, the difference between the three-dimensional coordinates of the virtual camera in the three-dimensional coordinate system and the three-dimensional coordinates of the projection point in the standardized device coordinate system can be determined as the direction information of the camera rays, d = (d...). x d y d z ), d x d y d zThese represent the differences along the x-axis, y-axis, and z-axis, respectively. The sampling point can be characterized using its positional information, such as its three-dimensional coordinates in a three-dimensional coordinate system: x = (x... w y w , z w This is used to characterize the sampling point.

[0150] Therefore, for any sampling point, the color and volume density corresponding to the preset lighting effect can be predicted using a light gradient model based on the direction information of the camera rays and the position information of the sampling point. The color and volume density corresponding to the preset lighting effect of the sampling point can refer to the color and volume density that need to be assigned to the sampling point when rendering the preset lighting effect.

[0151] In one example, the color and volume density of multiple sampling points can be predicted at once using a light gradient model to improve the prediction efficiency of the color and volume density of the sampling points, thereby improving the color prediction efficiency of the projection points. Therefore, embodiments of this application may further include the following:

[0152] (1) Encode the direction information of the camera light and the sampling point position data to obtain the input data.

[0153] The sampling point location data includes the location information of at least two sampling points, which indicates the third position of the sampling point in three-dimensional space. Since at least two sampling points are distributed on the same camera ray, the ray gradient model can use the direction information of the camera ray when predicting each sampling point.

[0154] The input data is the input to the light gradient model, which can be obtained by combining coded data of direction information and coded data of position information. In the embodiments of this application, the input data includes coded data of direction information and coded data of position information of at least two sampling points.

[0155] Optionally, to enhance the expressive power of the neural network, high-frequency encoding can be used to encode both the direction information and the position information. For example, sine and cosine encoding can be used to encode both the direction information and the position information.

[0156] For example, by sequentially using sine and cosine functions as input to the position information, and encoding each element in the coded data, the position information x = (x w y w , z w The encoded data of ) can be represented as follows:

[0157]

[0158] Where L is the number of frequency layers in the encoding.

[0159] And by sequentially using sine and cosine functions as input to encode the direction information, each element in the encoded data is obtained. Then the direction information d = (d... x d y d z The encoded data of ) can be represented as follows:

[0160] γ(d)=(sin(2 0 πd), cos(2 0 πd), ..., sin(2 L-1 πd)).

[0161] (2) Input the input data into the light gradient model. Based on the input data, the light gradient model predicts at least two sampling points that correspond to the color and volume density of the preset lighting effect.

[0162] The light gradient model first extracts a series of features from the input data, and then performs a series of linear transformations to obtain the prediction result. This prediction result can be used to indicate the color and volume density of at least two sampling points corresponding to the preset lighting effect.

[0163] In one example, the color and volume density of a sampling point can be predicted one at a time using a light gradient model to avoid the influence of other sampling points, thereby improving the accuracy of color prediction for the projection point. Therefore, embodiments of this application may further include the following:

[0164] (1) For any one of the at least two sampling points, encode the direction information of the camera light and the position information of the sampling point to obtain the input data.

[0165] Optionally, sine coding is used to encode the direction information of the camera light to obtain coded data of the direction information, and cosine coding is used to encode the position information of the sampling point to obtain coded data of the position information. Then, the coded data of the direction information and the coded data of the position information are concatenated together to obtain the input data.

[0166] (2) Input the input data into the light gradient model, and the light gradient model predicts the color and volume density of the sampling point corresponding to the preset lighting effect based on the input data.

[0167] The light gradient model first extracts a series of features from the input data, and then performs a series of linear transformations to obtain the prediction result. This prediction result can be used to indicate the color and volume density of a sampling point corresponding to a preset lighting effect.

[0168] 2. Based on the color and volume density of at least two sampling points corresponding to the preset lighting effect, the color information of the projection point is calculated.

[0169] Optionally, the color information of the projection point is calculated using a volumetric rendering formula based on the color and volume density of the two sampling points corresponding to a preset lighting effect, respectively. The volumetric rendering formula can be used to characterize a physical model of light absorption; that is, the RGB value of each pixel (i.e., the projection point) in the rendered image is determined by the colors of all sampling points on its corresponding camera ray.

[0170] For example, for each sampling point, a weight parameter is calculated based on the volume density of the sampling point corresponding to the preset lighting effect, the transmittance of the virtual camera, and the distance between the sampling point and the previous sampling point. Then, based on the weight parameter, the colors corresponding to the preset lighting effect for each sampling point are weighted and summed to obtain the predicted color value of the projection point corresponding to the preset lighting effect. For example, if there are K sampling points on the camera light, and the color of each sampling point corresponding to the preset lighting effect is c... i and the bulk density is σ i Then the predicted color value of the projection point corresponding to the preset lighting effect is:

[0171]

[0172] Where, δ i It is the distance between adjacent sampling points (such as the distance between the i-th sampling point and the previous sampling point). It is the transmittance from the virtual camera to the i-th sampling point. The transmittance reflects the proportion of light remaining after passing through the first i-1 sampling points.

[0173] This application embodiment can automatically predict the color and volume density of sampling points corresponding to preset lighting effects through a machine learning model, thereby achieving automatic prediction of the predicted color value of the projection point corresponding to the preset lighting effect, without the need for manual rendering of lighting effects by artists, thus improving the rendering efficiency of lighting effects in 3D models. Simultaneously, the machine learning model can learn realistic light gradient effects, predicting accurate and realistic projection point colors. Rendering preset lighting effects based on accurate and realistic projection point colors further improves the rendering quality of lighting effects in 3D models. Furthermore, the parameters of the light gradient model can be dynamically adjusted for different scenes (i.e., 3D space), allowing the light gradient model to adapt to different scene requirements, thereby improving the generalizability of the technical solution provided in this application embodiment.

[0174] Step 334: Combine the color information of each projection point to obtain the projection point color data.

[0175] The projection point color data includes the color information of each projection point. Optionally, the projection point color data can be obtained by arranging the color information of each projection point in relation to the position information of each projection point.

[0176] Step 340: Render the 3D model using the projection point position data and projection point color data to obtain the model rendering image in screen space.

[0177] The model rendering image is used to present a 3D model with preset lighting effects in screen space. The preset lighting effects can refer to the effects presented by the color data of the projection point under the constraints of the projection point position data. That is, under the constraints of the second position of the projection point, the surface of the 3D model is presented by the predicted color value of the projection point corresponding to the preset lighting effect.

[0178] For example, for each projection point, the color information of the projection point in the projection point color data can be assigned to the projection point (i.e., pixel) based on the position information of the projection point included in the projection point position data, so as to obtain the model rendering image, that is, to render a three-dimensional model with preset lighting effects.

[0179] A model rendering image can be a rendering image obtained only from a 3D model, excluding the projection points of 3D spatial points outside the 3D model. The model rendering image is the image that is ultimately displayed on the screen.

[0180] Optionally, when simulating volumetric light during the rendering of preset lighting effects, hard shadows of the 3D model can be rendered using only the projection point position data. Since the projection point color data is obtained under the constraint of the volumetric light's gradient, combined with the projection point color data, the edges of the hard shadow effect can be given a color transition effect, resulting in a soft transition between light and dark and a smooth boundary. This also gives the hard shadow effect a penumbra, thus forming a soft shadow effect. The hard shadow effect refers to a uniformly dark, clearly defined shadow produced by ideal light (i.e., a single light source with good focusing performance), which only includes the object's umbra.

[0181] For example, refer to Figure 4 and Figure 9 Volumetric light 403 is used to simulate the light rays used in rendering preset lighting effects. When volumetric light 403 is used to render the 3D model 401, the 3D model 401 forms a soft shadow effect under the lighting rendering of volumetric light 403. For the projection points of each vertex of the 3D model 401 in the model rendering image 404, the predicted color values ​​obtained by the light gradient model are assigned to each projection point, thus obtaining the model rendering image 404. The model rendering image 404 can be used to present the 3D model 401 with soft shadow effect 405.

[0182] In summary, the technical solution provided by the embodiments of this application, on the one hand, after projecting each vertex of the 3D model from 3D space to screen space according to camera parameters to obtain each projection point, supports predicting the projection point color data corresponding to the preset lighting effect through a light gradient model based on the projection points and camera parameters, thereby realizing the automatic prediction of the color value to be assigned at the projection point; on the other hand, it supports rendering a model rendering image of the 3D model using the second position of the projection point and the projection point color data corresponding to the preset lighting effect. Since the projection point color data corresponding to the preset lighting effect can make the 3D model in the model rendering image have the preset lighting effect during the rendering process, the lighting effect of the 3D model can be automatically rendered without the need for manual rendering of the lighting effect by artists, thereby improving the rendering efficiency of the lighting effect of the 3D model.

[0183] In addition, since this application can avoid the problem of inaccurate lighting effect rendering caused by errors easily introduced by manual art, it is beneficial to improve the lighting effect rendering quality of 3D models.

[0184] In some embodiments, since the above-mentioned light gradient model is used to simulate the propagation of light (such as volumetric light) in three-dimensional space when rendering preset lighting effects, training data can be collected corresponding to the preset lighting effects of the three-dimensional model in three-dimensional space, and then the light gradient model can be trained based on the training data. This allows the light gradient model to learn the propagation of light simulated when rendering preset lighting effects in three-dimensional space, that is, to learn the interaction between the light simulated when rendering preset lighting effects and the three-dimensional model in three-dimensional space.

[0185] The execution entity for each step in this application embodiment can be a computer device. Content not described in this application embodiment can be referred to the above embodiments, and will not be repeated here. For example, the training process of the above-described light gradient model may include the following:

[0186] 1. Obtain training data for the light gradient model.

[0187] Here, training data refers to the data used to train the light gradient model. The training data is related to the aforementioned three-dimensional space and preset lighting effects, or it can be related to any three-dimensional model within that space and the preset lighting effects. For example, the training data could be based on multiple rendered images acquired from different viewpoints in the three-dimensional space, along with camera parameters of a virtual camera related to the viewpoints, where simulated light rays are present in the three-dimensional space during the rendering of the preset lighting effects. Optionally, the rendered images can refer to images projected into the three-dimensional space using a virtual camera, and these rendered images can be located in the standardized device coordinate system of the virtual camera.

[0188] For example, the training data described above may include at least two sample projection point data, as well as camera ray and sample sampling point position data for each of the at least two sample projection point data. Multiple rendered images acquired can be used as sample images to determine at least two sample projection point data, such as each sample projection point data being used to indicate the position and true color information of a sample projection point obtained from the sample image.

[0189] A sample projection point can refer to any pixel in the sample image, that is, any point in the standardized device coordinate system. For example, the sample projection point can be any three-dimensional point projected by the virtual camera onto the standardized device coordinate system. The position of the sample projection point can refer to its location in the three-dimensional coordinate system, such as its three-dimensional coordinates. The true color information of the sample projection point can be used to indicate the color assigned to the sample projection point when rendering a preset lighting effect, that is, the true color value corresponding to the preset lighting effect. Optionally, the above sample projection point data includes the three-dimensional coordinates of the sample projection point in the three-dimensional coordinate system and the true color value of the sample projection point corresponding to the preset lighting effect.

[0190] The camera rays for the sample projection point data can be determined based on camera parameters, such as the position of the virtual camera in three-dimensional space. For example, the camera rays for the sample projection point data can be rays pointing from the position of the virtual camera in three-dimensional space to the position of the sample projection point in three-dimensional space. The camera rays can be represented by direction information, which can be expressed as the difference between the position of the sample projection point in three-dimensional space and the position of the virtual camera in three-dimensional space.

[0191] The aforementioned sample sampling point location data is used to indicate the positions of at least two sample sampling points obtained by sampling the same sample projection point in three-dimensional space. For example, for any sample projection point, sampling along the camera ray from the sample projection point in three-dimensional space can yield at least two sample sampling points. A sample sampling point is a point in three-dimensional space, and its position can refer to its location in three-dimensional space, such as being represented by its three-dimensional coordinates in a three-dimensional coordinate system. Optionally, the aforementioned sample sampling point data includes the three-dimensional coordinates of at least two sample sampling points in a three-dimensional coordinate system.

[0192] 2. For any sample projection point data in at least two sample projection point data, based on the direction information of the camera light corresponding to the sample projection point data and the position data of the sample sampling point corresponding to the sample projection point data, the color information of the sample projection point corresponding to the sample projection point data is predicted by the light gradient model. The color information of the sample projection point is used to indicate the predicted color value of the sample projection point corresponding to the preset lighting effect.

[0193] The predicted color value of the sample projection point corresponding to the preset lighting effect is the predicted color value, and the real color value of the sample projection point corresponding to the preset lighting effect is the real color value.

[0194] Optionally, for each sample projection point data, the direction information of the camera light corresponding to the sample projection point data is encoded to obtain encoded data of the direction information, and the position data of the sample sampling point corresponding to the sample projection point data is encoded to obtain encoded data of the sample sampling point position data; the encoded data of the direction information and the encoded data of the sample sampling point position data are spliced ​​together to obtain the input data of the light gradient model; the input data is input into the light gradient model, and the light gradient model predicts the color and volume density of at least two sample sampling points corresponding to the preset lighting effect based on the input data; based on the color and volume density of at least two sample sampling points corresponding to the preset lighting effect, the color information of the sample projection point corresponding to the sample projection point data is calculated.

[0195] In a feasible example, for each sample projection point data, the direction information of the camera light corresponding to the sample projection point data is encoded to obtain encoded direction information data, and the position information of any sample sampling point in the sample sampling point position data corresponding to the sample projection point data is encoded to obtain encoded position information data; the encoded direction information data and the encoded position information data are spliced ​​together to obtain the input data of the light gradient model; the input data is input into the light gradient model, and the light gradient model predicts the color and volume density of the sample sampling point corresponding to the preset lighting effect based on the input data; based on the color and volume density of at least two sample sampling points corresponding to the preset lighting effect respectively, the color information of the sample projection point corresponding to the sample projection point data is calculated.

[0196] 3. Adjust the parameters of the light gradient model based on the color information of at least two sample projection point data and the true color information of at least two sample projection point data.

[0197] Training a light gradient model is an iterative process. For example, each iteration iterates the parameters of the light gradient model using a batch of training data, where each batch of training data includes at least one training data point. This application embodiment uses a specific iterative process of the light gradient model as an example for illustration. Optionally, the termination condition for iteration includes at least one of the following: minimizing the loss function value, the number of iterations being greater than or equal to a threshold, or the loss function value being less than or equal to a threshold, etc. This application embodiment does not limit this to any particular condition.

[0198] Optionally, a loss function for the ray gradation model can be constructed based on the prediction results of the ray gradation model (i.e., the color information of the predicted sample projection point data) and the label data (i.e., the true color information of the sample projection point data), and then the parameters of the ray gradation model can be adjusted based on the loss function.

[0199] In one example, the parameter adjustment process for a light gradient model may include the following:

[0200] (1) For any sample projection point data in at least two sample projection point data, the color reconstruction loss of the sample projection point data is obtained based on the color information of the sample projection point data and the true color information of the sample projection point data.

[0201] Among them, the color reconstruction loss of the sample projection point data is used to indicate the difference between the color information of the sample projection point data and the true color information of the sample projection point data, that is, it can be used to indicate the accuracy of the prediction results of the light gradient model.

[0202] The color reconstruction loss of the sample projection point data can be constructed based on the difference between the color information of the sample projection point data and the true color information of the sample projection point data. For example, the loss function used to construct the color reconstruction loss includes at least one of the following: Cross Entropy (CE) loss function, Binary Cross Entropy (BCE) loss function, Mean Absolute Loss (MAL) loss function, and Mean Square Error (MSE) loss function.

[0203] For example, taking the mean squared error loss function as an example, the square of the difference between the color information of the sample projection point data and the true color information of the sample projection point data can be determined as the color reconstruction loss of the sample projection point data. Then the color reconstruction loss of the sample projection point data can be expressed as follows:

[0204]

[0205] Among them, C i It is the color information predicted for the i-th sample projection point. It is the true color information of the i-th sample projection point.

[0206] (2) Based on the color reconstruction loss of at least two sample projection point data, the color reconstruction loss of the light gradient model is obtained.

[0207] Optionally, the color reconstruction loss of the light gradient model can be obtained by summing the color reconstruction losses of at least two sample projection point data; alternatively, the color reconstruction loss of the light gradient model can be obtained by summing the color reconstruction losses of at least two sample projection point data and then averaging them. This application embodiment does not limit this.

[0208] For example, taking the mean squared error loss function as an example, the sum of the color reconstruction losses of at least two sample projection point data can be averaged to obtain the color reconstruction loss of the ray gradation model. The color reconstruction loss of the ray gradation model can then be expressed as follows:

[0209]

[0210] Where N is the number of sample projection points (i.e., sample projection point data).

[0211] (3) Based on the color reconstruction loss of the light gradient model, the parameters of the light gradient model are iteratively adjusted.

[0212] Training the ray gradation model involves forward propagation, loss calculation, and backpropagation. Forward propagation is the process of predicting the color information of the sample projection point data, loss calculation is the process of calculating the color reconstruction loss of the ray gradation model, and backpropagation is the process of adjusting the parameters of the ray gradation model.

[0213] In backpropagation, the gradient of each parameter of the ray gradation model can be calculated using the backpropagation algorithm based on the color reconstruction loss of the ray gradation model, and each parameter can be adjusted using gradient descent. The key to backpropagation is calculating the derivative of each parameter with respect to the loss function (such as the color reconstruction loss mentioned above), and the parameters can be updated based on the derivative of each parameter with respect to the loss function (such as the color reconstruction loss mentioned above).

[0214] For example, if the parameters of the light gradient model are represented by θ and the loss function is L, then we can first obtain the product between the bias of the loss function with respect to the parameter features and the learning rate, and then determine the updated parameters by the difference between the feature parameters and this product. The parameter update method can be expressed as follows:

[0215] Where η is the learning rate.

[0216] Optionally, by minimizing the color reconstruction loss of the ray gradation model, the parameters of the ray gradation model can be iteratively adjusted to obtain the trained ray gradation model, which can then be used to predict the color information of the projection point.

[0217] This application embodiment adjusts the parameters of the ray gradient model by adjusting the color reconstruction loss based on the ray gradient model, so that the ray gradient model learns the complex propagation of light (such as volumetric light) in three-dimensional space when rendering preset lighting effects. This enables the ray gradient model to accurately predict the predicted color value of the projection point corresponding to the preset lighting effect, thereby improving the rendering quality of the lighting effect of the three-dimensional model.

[0218] In one example, the visual consistency (i.e. feature consistency) between the predicted color values ​​of the sample projection point data and the true color values ​​of the sample projection point data can also be enhanced by constructing a perceptual loss. This is beneficial to enhancing the visual consistency of the rendered image obtained based on the light gradient model, which is conducive to better reconstructing the color and texture of the 3D model, thereby improving the rendering quality of the lighting effect of the 3D model.

[0219] For example, the parameter adjustment process for the light gradient model may also include the following:

[0220] (1) For any sample projection point data in at least two sample projection point data, extract the color information features of the sample projection point data to obtain the first feature data, and extract the real color information features of the sample projection point data to obtain the second feature data.

[0221] Optionally, feature extraction is performed using a feature extraction network, which may be a pre-trained network for direct use in feature extraction. For example, the feature extraction network may include at least one of the following: CNN, DNN, Transformer, ResNet.

[0222] The first feature data is used to indicate the potential features of the color information of the sample projection point data, and the second feature data is used to indicate the potential features of the true color information of the sample projection point data. These potential features can be features at any scale.

[0223] (2) Based on the first feature data and the second feature data, the perceptual loss of the sample projection point data is obtained. The perceptual loss of the sample projection point data is used to indicate the difference between the first feature data and the second feature data.

[0224] The aforementioned perceptual loss can be used to measure the difference between the color information of the sample projection point data and the true color information of the sample projection point data at the feature level. This helps to maintain the visual consistency between the color information of the sample projection point data and the true color information of the sample projection point data, thereby improving the fidelity of the predicted color and ensuring the naturalness and realism of the visual effect.

[0225] Optionally, the perceptual loss of the sample projection point data can be constructed based on the difference between the first feature data and the second feature data. For example, the loss function used to construct the perceptual loss includes at least one of the following: cross-entropy loss function, binary cross-entropy loss function, mean absolute error loss function, mean squared error loss function, and variance loss function.

[0226] For example, taking the variance loss function as an example, the square of the difference between the first feature data and the second feature data can be determined as the perceptual loss of the sample projection point data. Then, the perceptual loss of the sample projection point data can be expressed as follows:

[0227]

[0228] Wherein, Φ(C i ) is the first feature data of the i-th sample projection point. It is the second feature data of the i-th sample projection point.

[0229] (3) Based on the perceptual loss of at least two sample projection point data, the perceptual loss of the light gradient model is obtained.

[0230] Optionally, the perceptual loss of the light gradient model can be obtained by summing the perceptual losses of at least two sample projection point data; alternatively, the perceptual loss of the light gradient model can be obtained by summing the perceptual losses of at least two sample projection point data and then averaging them. This application does not limit the specific method used.

[0231] For example, taking the variance loss function as an example, the perceptual loss of the light gradient model can be obtained by summing the perceptual losses of at least two sample projection point data. The perceptual loss of the light gradient model can then be expressed as follows:

[0232]

[0233] Where N is the number of sample projection points (i.e., sample projection point data), and Φ is the pre-trained feature extraction network (also known as a perceptual network).

[0234] (4) Based on the color reconstruction loss of the light gradient model and the perceptual loss of the light gradient model, the parameters of the light gradient model are iteratively adjusted.

[0235] Optionally, based on the color reconstruction loss and the perceptual loss of the ray gradation model, the total loss of the ray gradation model is constructed. With the goal of minimizing the total loss, the parameters of the ray gradation model are iteratively adjusted to obtain the trained ray gradation model.

[0236] For example, the total loss of the light gradient model can be obtained by summing the color reconstruction loss and the perceptual loss of the light gradient model, or by weighted summing the color reconstruction loss and the perceptual loss of the light gradient model. This application does not limit the specific method used.

[0237] For example, the total loss of the light gradient model can be expressed as follows:

[0238]

[0239] Where λ is the weight of the perceptual loss.

[0240] In summary, the technical solution provided in this application adjusts the parameters of the ray gradation model by adjusting the color reconstruction loss or perceptual loss based on the ray gradation model. This enables the ray gradation model to learn the complex propagation of light in three-dimensional space (including the complex relationship between the light and shadow) simulated when rendering preset lighting effects. Based on the ray gradation model, it can predict the accurate and realistic color of the projection point corresponding to the preset lighting effect. This is beneficial for generating realistic ray gradation effects, thereby improving the rendering quality of lighting effects in three-dimensional models and giving users a realistic visual experience.

[0241] In some embodiments, processing and rendering highly complex 3D models is extremely challenging. To achieve smooth 3D model rendering with limited hardware resources and provide users with a high-quality visual experience, the 3D model can be simplified and reconstructed before rendering. Simplifying the 3D model not only significantly reduces the computational load during 3D model rendering but also significantly improves the rendering efficiency and reduces the storage requirements for data related to the 3D model.

[0242] The embodiments of this application do not limit the simplification method of the 3D model, which can be set and adjusted according to actual usage requirements. For example, the simplification method of the 3D model includes at least one of the following: vertex merging (such as vertex downsampling), mesh merging (such as mesh simplification), and error optimization (optimizing vertices based on error).

[0243] In one example, taking error optimization as an example to illustrate the simplification of the 3D model, step 310 above may also include the following:

[0244] 1. Obtain the initial vertex position data of the 3D model. The initial vertex position data is used to indicate the sixth position of each initial vertex of the 3D model in 3D space. The number of initial vertices is greater than the number of vertices.

[0245] The initial vertices corresponding to the initial vertex data can refer to the vertices of the 3D model before simplification, and the vertices corresponding to the vertex data can refer to the vertices of the simplified 3D model. Optionally, the 3D model in the above method embodiments can refer to the simplified 3D model.

[0246] The initial vertex position data can include the position information of each initial vertex. This initial vertex position information indicates the sixth position of the initial vertex in three-dimensional space, and the sixth position can be used to refer to the position of the initial vertex in three-dimensional space. For example, the initial vertex position information includes the coordinates of the initial vertex in three-dimensional space, such as 3D coordinates, homogeneous coordinates, etc.

[0247] In this embodiment, the number of vertices in the original 3D model is greater than the number of vertices in the simplified 3D model. Reducing the number of vertices helps to reduce the workload required to render the 3D model, thereby improving the rendering efficiency of the 3D model.

[0248] 2. Merge the initial vertices of the 3D model to obtain the vertex position data of the 3D model.

[0249] Merging initial vertices can refer to the process of replacing at least two initial vertices with new nodes. In the embodiments of this application, merging initial vertices can refer to the process of replacing two initial vertices with new nodes.

[0250] Optionally, the initial vertex position data is merged through multiple iterative merging processes to obtain vertex position data. Each iterative merging process corresponds to one merging of the initial vertices, meaning that the number of vertices decreases by one with each iterative merging process. The methods used in each iterative merging process are the same. This embodiment of the application uses a certain iterative merging process as an example for illustration, and the process of obtaining vertex position data may include the following:

[0251] (1) During the nth iteration merging process, obtain the error matrix of each transition vertex indicated by the nth-order vertex position data.

[0252] The nth iteration merging process can refer to any one of the multiple iteration merging processes, where n is a positive integer. The nth-order vertex position data can refer to the vertex position data that needs to be processed in the nth iteration merging process, that is, the nth-order vertex position data is used to indicate the multiple transition vertices participating in the merging process in the nth iteration merging process.

[0253] Optionally, when n is 1, the nth-order vertex position data is the initial vertex position data. When n is greater than 1, the nth-order vertex position data is the data obtained after the (n-1)th-order vertex position data through the nth iteration merging process. During multiple iterations of merging, the vertices of the 3D model can be recorded as transition vertices.

[0254] The aforementioned error matrix can be used to indicate the distance between a transition vertex and at least two adjacent meshes of the transition vertex. Adjacent meshes of a transition vertex are those that include the transition vertex, and these meshes are used to construct the 3D model. For example, refer to... Figure 10 For the transition vertex 1011 of the 3D model 1010, the transition vertex 1011 includes 7 meshes (such as triangular patches) with the transition vertex 1011 as the vertex. For the transition vertex 1012 of the 3D model 1010, the transition vertex 1012 includes 5 meshes with the transition vertex 1012 as the vertex.

[0255] For example, the process of obtaining the error matrix of the transition vertex may include the following: for any transition vertex indicated by the nth-order vertex position data, obtain the error matrix of each of the at least two adjacent grids of the transition vertex, the error matrix of the adjacent grids is used to indicate the distance between the adjacent grids and the transition vertex; and determine the error matrix of the transition vertex based on the error matrices of the at least two adjacent grids.

[0256] Optionally, the error matrix of a transition vertex describes the sum of squared distances from the transition vertex to all its adjacent grids. Then, the error matrix of each transition vertex can be the sum of the error matrices of all its adjacent grids. For example, the error matrix of the transition vertex can be obtained by summing the error matrices of at least two adjacent grids.

[0257] For example, refer to Figure 10 Taking a triangular mesh as an example, for each adjacent mesh of transition vertex 1011, the plane equation of that adjacent mesh can be defined as ax + by + cz + d = 0 using the normal vector n = (a, b, c) of the adjacent mesh and a point (x, y, z) passing through that adjacent mesh. Then, the elements of the error matrix can be constructed using the parameters a, b, c, and d. The error matrix of the adjacent mesh can then be constructed using the constructed elements. Thus, the error matrix of the i-th adjacent mesh of transition vertex 1011 can be expressed as follows:

[0258]

[0259] The error matrix Q of transition vertex 1011 is the sum of the error matrices of all its adjacent grids. Therefore, the error matrix of transition vertex 1011 can be represented as follows:

[0260] Where Q is the error matrix of transition vertex 1011, K i This is the error matrix of the i-th neighboring grid of transition vertex 1011, where n is the number of neighboring grids of transition vertex 1011 (e.g., 7).

[0261] (2) Determine at least two compliant point pairs from each of the transition vertices indicated by the nth-order vertex position data.

[0262] Here, a compliant point pair refers to a pair of transition vertices that meet certain set conditions. Optionally, the distance between the two transition vertices in a compliant point pair is less than or equal to a first threshold. The first threshold can be set and adjusted according to actual usage requirements. Any two transition nodes with a distance less than or equal to the first threshold can be combined into a compliant point pair to avoid introducing large distance errors. This ensures that merging compliant point pairs will not significantly affect the topology of the 3D model, thereby ensuring the topology and visual effect of the 3D model.

[0263] Alternatively, the two transition vertices in a compliant point pair belong to the same edge, which means that two transition vertices on any edge can be combined to form a compliant point pair, thus avoiding the introduction of large distance errors.

[0264] For example, referring to 10, if transition vertex 1011 and transition vertex 1012 belong to the same edge, then transition vertex 1011 and transition vertex 1012 can be combined into a compliant vertex pair.

[0265] (3) For any of the at least two compliance point pairs, determine the shrinkage error of the compliance point pair based on the error matrix of each of the two transition vertices in the compliance point pair. The shrinkage error is used to indicate the priority of merging compliance point pairs.

[0266] Shrinkage error can also indicate the impact of merging compliant point pairs on the network topology of the 3D model. Optionally, the smaller the shrinkage error of a compliant point pair, the higher the priority of merging that compliant point pair; the larger the shrinkage error of a compliant point pair, the lower the priority of merging that compliant point pair.

[0267] For example, for any pair of compliant points, the shrinkage error of the compliant point pair can be obtained by summing the error matrices of the two transition vertices in the pair.

[0268] For example, the shrinkage error of a compliant point pair can be expressed as follows:

[0269] Q′=Q i +Q j ; where Q i and Q j These are the error matrices of the two transition vertices in the compliant point pair.

[0270] (4) The compliance point pair with the smallest shrinkage error among at least two compliance point pairs is determined as the target compliance point pair.

[0271] Optionally, at least two compliant point pairs are sorted in ascending order of shrinkage error to obtain a compliant point pair sequence; the first compliant point pair in the compliant point pair sequence is determined as the target compliant point pair. Alternatively, at least two compliant point pairs can be sorted in descending order of shrinkage error to obtain a compliant point pair sequence; the last compliant point pair in the compliant point pair sequence is determined as the target compliant point pair.

[0272] Since the shrinkage error of the target compliance point pair is minimized, merging the two transition vertices in the target compliance point pair has the least impact on the network topology of the 3D model, thus ensuring the topological structure and visual effect of the 3D model.

[0273] (5) Based on the shrinkage error of the target compliance point pair, determine the new vertex of the target compliance point pair. The new vertex of the target compliance point pair is used to replace the two transition vertices in the target compliance point pair.

[0274] The new vertex can be represented by its position information in three-dimensional space, which can be determined by minimizing the shrinkage error of the target compliant point pair.

[0275] For example, the position information of the new vertex can be represented as follows:

[0276]

[0277] Where v′ is the v when Q′ is minimized.

[0278] (6) Replace the data in the nth-order vertex position data of the target compliant point pair with the position information of the new vertex of the target compliant point pair to obtain the (n+1)th-order vertex position data.

[0279] In other words, by replacing the position information of the two transition vertices in the nth-order vertex position data of the target compliant point pair with the position information of the new vertex of the target compliant point pair, the (n+1)th-order vertex position data can be obtained.

[0280] By iteratively selecting and merging target compliance point pairs with the minimum shrinkage error, the number of vertices in the 3D model can be gradually reduced, thereby simplifying the 3D model.

[0281] (7) If the minimum shrinkage error corresponding to the (n+1)th order vertex position data is greater than or equal to the error threshold, the (n+1)th order vertex position data shall be determined as vertex position data.

[0282] The error threshold can be set and adjusted according to actual usage requirements, and this application embodiment does not limit this. By setting the error threshold, only small geometric errors can be introduced during the simplification of the 3D model, thereby ensuring the topological structure and visual effect of the 3D model.

[0283] Optionally, if the minimum shrinkage error corresponding to the vertex position data of the (n+1)th order is greater than or equal to the error threshold, the simplification speed of the 3D model can be slowed down (e.g., ensuring that the number of vertices is greater than the number threshold). A more conservative simplification method can also be adopted (e.g., further limiting the above-mentioned first threshold, or constructing compliant point pairs only with the first threshold, or filtering by the first threshold based on constructing compliant point pairs based on edges). This application embodiment does not limit this.

[0284] For example, refer to Figure 10 By merging transition vertex 1011 and transition vertex 1012, a simplified 3D model 1020 can be obtained, with transition vertex 1011 and transition vertex 1012 replaced by vertex 1021.

[0285] Alternatively, unnecessary details corresponding to the 3D model can be removed to reduce the complexity of the 3D model. For example, vertices that cannot be observed by the virtual camera can be deleted from the 3D model to reduce the complexity of the 3D model.

[0286] In one example, during the simplification of a 3D model, the normal direction of the 3D model may change due to the removal of vertices. To ensure that the simplified 3D model still correctly reflects the preset lighting effect, this embodiment also supports recalculating the normal of each vertex. For example, for each vertex, the new normal of the vertex can be determined based on the normals of each of its adjacent meshes and the area of ​​each of its adjacent meshes. For instance, for each adjacent mesh, the product of the normal and area of ​​the adjacent meshes is obtained, and then the products corresponding to each adjacent mesh are summed to obtain a sum value. Finally, the ratio of this sum value to the absolute value of the sum value is determined as the new normal of the vertex.

[0287]

[0288] Where, n i A is the normal to the i-th adjacent grid of vertex v. i It is the area of ​​the i-th adjacent grid.

[0289] Optionally, during the simplification of the 3D model, due to the removal of vertices, the UV coordinates of the vertices may need to be adjusted to ensure the correctness of texture mapping. For example, the UV coordinates of the new vertex can be calculated by interpolation. A first sum is obtained by weighted summation of the UV coordinates of the two transition vertices in the target vertex pair, and a second sum is obtained by summing the UV coordinates of the two transition vertices according to their respective weights. The ratio of the first sum to the second sum is then used to determine the UV coordinates of the new vertex.

[0290]

[0291] Where (u1, v1) and (u2, v2) are the UV coordinates of the two transition vertices in the target compliance point pair, respectively, and w1 and w2 are weights, which can usually be determined based on the position of the vertex on the folded edge (such as the edge formed by the two transition vertices in the target compliance point pair).

[0292] In summary, the technical solution provided in this application, by iteratively merging the vertices of a 3D model based on a vertex error matrix, can reduce the number of vertices in a 3D model while maintaining its topological structure and visual effects. This helps to reduce the rendering workload of the 3D model and thus improve its rendering efficiency.

[0293] In addition, for scenarios that require real-time rendering of 3D models, simplifying the 3D model before rendering helps ensure the smoothness and real-time performance of the application.

[0294] In some embodiments, the technical solutions provided in this application are applicable to the scenario of "detection and reconstruction of shadow effects missing in 3D models under lighting rendering".

[0295] Many 3D models undergo lighting rendering, but due to issues such as computer device type or the rendering order of 3D models, shadow effects may not be rendered. Related techniques rely on extensive contour sampling for soft shadow rendering, but due to the limitations of the virtual camera's field of view, these techniques may lose light characteristics in the target scene. Therefore, in the re-rendering process of soft shadow effects, these techniques also need to rely on extensive contour sampling to re-render the soft shadow effects, resulting in low efficiency.

[0296] The technical solution provided in this application can improve the re-rendering efficiency of soft shadow effects. For example, refer to... Figure 11 The embodiments of this application may also include the following:

[0297] Optionally, in this embodiment of the application, the debug toolbox of UE4 (Unreal Engine 4), which can be used to help designers find and fix problems in the code, can be used to extract and export the rendered image of the 3D model in real time, as well as mark the 3D model that failed to render. Then, the technical solution provided in this embodiment of the application can be used to repair the 3D model, that is, to re-render the 3D model, and update the re-rendering result into the code for real-time viewing. The re-rendering process of the 3D model may include the following:

[0298] 1. Obtain the coordinates of each vertex 1111 of the 3D model 1110 (corresponding to the initial vertices mentioned above) in the 3D coordinate system O corresponding to the 3D space 1100. w -X w Y w Z w Homogeneous coordinates P in w =[x w y w , z w ,1] T .

[0299] The 3D model 1110 is composed of multiple triangular facets, each of which is enclosed by three vertices 1111. The position of vertex 1111 in the 3D space 100 can be represented by the homogeneous coordinates of vertex 1111 in the 3D space coordinate system.

[0300] 2. Iteratively merge the vertices of the 3D model 1110 to obtain the simplified 3D model 1140. The simplified 3D model 1140 has multiple vertices 1141, and the number of multiple vertices 1141 is less than the number of multiple vertices 1111.

[0301] For example, for each iteration of the merging process, first obtain the error matrix of each triangular facet; for each vertex 1111, accumulate the error matrices of the adjacent triangular faces of vertex 1111 to obtain the error matrix of vertex 1111; for each pair of vertices 1111 on each edge, construct a compliant point pair respectively; for any compliant point pair, sum the error matrices of the two vertices 1111 in the compliant point pair to obtain the shrinkage error of the compliant point pair; merge the two vertices 1111 in the target compliant point pair with the smallest shrinkage error to obtain a new vertex; replace the two vertices 1111 in the target compliant point pair with the new vertex to proceed to the next iteration of the merging process; if the shrinkage error of the target compliant point is greater than or equal to the error threshold, stop iteratively merging the vertices of the 3D model 1110 to obtain the simplified 3D model 1140.

[0302] By minimizing the shrinkage error of the target vertex pair, the coordinates P′ of the new vertex in the three-dimensional coordinate system are calculated. w =[xw y w , z w ,1] T .

[0303] 3. Project each vertex 1141 of the simplified 3D model 1140 from the 3D space 1100 to the screen space 1400 to obtain multiple projection points 1142.

[0304] Virtual camera 1120 is placed at a specific point in three-dimensional space 1100. The camera coordinate system O of virtual camera 1120 is... c -X c Y c Z c With the position of the virtual camera 1120 as the origin, the z-axis is mapped to the viewing direction (coinciding with the optical axis), the y-axis is mapped to the upward direction, and the x-axis is perpendicular to the z-axis and y-axis.

[0305] Taking any vertex 1141 as an example, the projection process of vertex 1141 can include the following:

[0306] (1) Transform vertex 1141 from 3D space 1100 to camera space 1200 of virtual camera 1120.

[0307] Using the view transformation matrix V of the virtual camera 1120, vertex 1141 is projected into the three-dimensional coordinate system O. w -X w Y w Z w The homogeneous coordinates of vertex 1141 are converted to camera coordinates O in camera space 1200. c -X c Y c Z c Homogeneous coordinates P in c =[x c y c , z c ,1] T =VP′ w .

[0308] (2) Project vertex 1141 from camera space 1200 to normalized device coordinate space 1300.

[0309] Using the projection transformation matrix P of the virtual camera 1120, vertex 1141 is projected onto the camera coordinate system O. c -X c Y c Z c The homogeneous coordinates of vertex 1141 are converted to the normalized device coordinate system O in the normalized device coordinate space 1300. n -Xn Y n Z n Homogeneous coordinates P in n =[x n y n , z n w n ] T =PP c .

[0310] Standardized equipment coordinate system O n -X n Y n Z n The x-axis and y-axis range from (0, 0) to (1, 1), with the origin at the top left corner of the near-cut plane. Depth values ​​(i.e., z-values) range from 0 to 1, corresponding to points on the near and far cut planes, respectively. Standardized equipment coordinate system O n -X n Y n Z n z-axis and camera coordinate system O c -X c Y c Z c It is coaxial with the z-axis.

[0311] (3) Project vertex 1141 from the normalized device coordinate space 1300 to the screen space 1400.

[0312] For vertex 1141 in the normalized device coordinate system O n -X n Y n Z n Normalize the homogeneous coordinates in the matrix to obtain normalized coordinates.

[0313] Given a screen resolution of width W and height H, vertex 1141 corresponds to a screen coordinate system O with a screen space of 1400. s -X s Y s Z s The homogeneous coordinates in the middle are That is, the homogeneous coordinates of the projection point 1142 of vertex 1141 in screen space 1400 (i.e., the projection point data mentioned above).

[0314] Screen coordinate system O s -X s Y s Z sIt can be defined on the film plane (i.e., the near-cut plane), and its depth value (i.e., the z-value) is the same as the depth value in the standardized device coordinate system. The x-axis is parallel to the x-axis of the standardized device coordinate system, and the y-axis is parallel to the y-axis of the standardized device coordinate system. The origin is at the upper left corner of the screen. Screen coordinate system O s -X s Y s Z s z-axis and camera coordinate system O c -X c Y c Z c It is coaxial with the z-axis.

[0315] 4. Predict the predicted color values ​​(i.e., the color data of the projection points) of each projection point 1142 corresponding to the volumetric lighting effect (i.e., the preset lighting effect mentioned above) using the light gradient model.

[0316] This volumetric lighting effect can be rendered by simulating the effect of volumetric light 1130 on the 3D model 1110.

[0317] Taking any projection point 1142 as an example, the color prediction process for projection point 1142 may include the following:

[0318] (1) Obtain the K sampling points 1144 corresponding to the projection point 1142 in the three-dimensional space 100.

[0319] For example, taking the ray from the optical center of the virtual camera 1120 to the projection point 1142 as camera ray 1143, K sampling points 1144 are randomly sampled on camera ray 1143, and the three-dimensional coordinates x = (x_k + x_k + x_k) of the K sampling points 1144 in the three-dimensional space 1100 are obtained respectively. w y w , z w K is an integer greater than 1.

[0320] Obtain the viewing direction of camera ray 1143 (i.e., the direction information mentioned above): azimuth and polar angle. The viewing direction of camera ray 1143 can also be expressed as a unit vector d = (d... x d y d z The value represents the difference between the position of projection point 1142 in three-dimensional space and the position of virtual camera 1120 in three-dimensional space.

[0321] (2) For any sampling point 1144, sinusoidal coding is used to obtain the encoded data of sampling point 1144:

[0322]

[0323] And using cosine coding, the encoded data of the direction information of camera ray 1143 is obtained:

[0324] γ(d)=(sin(2 0 πd), cos(2 0 πd), ..., sin(2 L-1 πd));

[0325] (3) Based on γ(x) and γ(d) using the light gradient model, the sampling point 1144 is predicted to correspond to the color and volume density (c, σ) of the volume lighting effect, respectively.

[0326] The ray gradient model can be constructed using machine learning, and can be represented as:

[0327] (c,σ)=MLP(γ(x),γ(d)).

[0328] (4) The colors corresponding to the volumetric lighting effects at the K sampling points 1144 are weighted and summed to obtain the predicted color value of the volumetric lighting effect corresponding to the projection point 1142:

[0329]

[0330] Among them, c i The i-th sampling point 1144 corresponds to the color of the volumetric lighting effect, σ i δ is the volume density of the volumetric lighting effect corresponding to the i-th sampling point. i T is the distance between adjacent sampling points. i It is the transmittance from virtual camera 1120 to the i-th sampling point.

[0331] 5. For any projection point 1142, assign the predicted color value C corresponding to the volumetric lighting effect of projection point 1142 to the pixel point indicated by projection point 1142 to obtain the model rendering image 1500. The model rendering image 1500 presents a 3D model 1110 with a soft shadow effect 1600 (belonging to the volumetric lighting effect).

[0332] Since the predicted color value C of the volumetric lighting effect corresponding to the projection point 1142 is predicted under the light gradient constraint of the volumetric light 1130, the pixel point referred to by the projection point 1142 is assigned the predicted color value C of the volumetric lighting effect corresponding to the projection point 1142. This can give the edge of the shadow of the three-mode model 1130 a color transition effect, so that the edge of the shadow of the three-mode model 1130 presents a light-dark transition and a soft boundary effect, thus forming a soft shadow effect 1600.

[0333] In summary, the technical solution provided by the embodiments of this application, on the one hand, after projecting each vertex of the 3D model from 3D space to screen space based on camera parameters to obtain each projection point, supports predicting the predicted color value corresponding to the volumetric lighting effect of the projection point based on the projection point and camera parameters, thereby realizing the automatic prediction of the color value to be assigned at the projection point; on the other hand, supports rendering the model rendering image of the 3D model based on the projection point and the predicted color value corresponding to the volumetric lighting effect at the projection point. Since the predicted color value corresponding to the volumetric lighting effect of the projection point can make the 3D model in the model rendering image have a soft shadow effect during the rendering process, the automatic rendering of the soft shadow effect of the 3D model is realized without the need for manual rendering of soft shadows by artists, which is beneficial to improving the rendering efficiency of the soft shadow effect of the 3D model.

[0334] In addition, since this application can avoid the problem of inaccurate soft shadow effect rendering caused by errors easily introduced by manual art, it is beneficial to improve the soft shadow effect rendering quality of 3D models.

[0335] In addition, the technical solution provided in this application embodiment enables rapid diagnosis of the soft shadow effect of a 3D model and rapid reconstruction of the soft shadow effect of a 3D model. This is beneficial for relevant technicians to make reasonable quality judgments on the 3D model and obtain effect comparisons. This is beneficial to improve the efficiency and accuracy of the detection and preview of the soft shadow effect of the 3D model.

[0336] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0337] Please refer to Figure 12 This diagram illustrates a block diagram of a rendering apparatus for a three-dimensional model according to an embodiment of this application. The apparatus has the function of implementing the aforementioned rendering method for the three-dimensional model. This function can be implemented in hardware or by hardware executing corresponding software. The apparatus 1200 can be a computer device or can be installed within a computer device. The apparatus 1200 may include: a vertex data acquisition module 1201, a projection point data acquisition module 1202, a color data prediction module 1203, and a three-dimensional model rendering module 1204.

[0338] Vertex data acquisition module 1201 is used to acquire vertex position data of the three-dimensional model, wherein the vertex position data is used to indicate the first position of at least one vertex of the three-dimensional model in three-dimensional space.

[0339] The projection point data acquisition module 1202 is used to project at least one vertex from the three-dimensional space to the screen space according to the camera parameters of the virtual camera in the three-dimensional space, and obtain projection point position data. The projection point position data is used to indicate the second position of the projection point corresponding to the at least one vertex in the screen space.

[0340] The color data prediction module 1203 is used to predict the color data of the projection point corresponding to the preset lighting effect of the three-dimensional model in the three-dimensional space based on the projection point position data and the camera parameters through a light gradient model.

[0341] The 3D model rendering module 1204 is used to render the 3D model using the projection point position data and the projection point color data to obtain the model rendering image in the screen space.

[0342] In some embodiments, such as Figure 13 As shown, the color data prediction module 1203 includes: a camera light determination submodule 1203a, a sampling point acquisition submodule 1203b, a color information prediction submodule 1203c, and a color data prediction submodule 1203d.

[0343] The camera ray determination submodule 1203a is used to obtain the camera ray corresponding to each projection point based on the second position of the projection point and the camera parameters. The camera ray is used to simulate the ray path when the virtual camera captures the vertex corresponding to the projection point.

[0344] The sampling point acquisition submodule 1203b is used to sample along the camera ray in the three-dimensional space to obtain sampling point position data, which is used to indicate the third position of at least two sampling points in the three-dimensional space.

[0345] The color information prediction submodule 1203c is used to predict the color information of the projection point based on the camera light and the sampling point position data through the light gradient model. The color information of the projection point is used to indicate the predicted color value of the projection point corresponding to the preset lighting effect.

[0346] The color data prediction submodule 1203d is used to combine the color information of each projection point to obtain the color data of the projection point.

[0347] In some embodiments, the color information prediction submodule 1203c is used for:

[0348] Based on the camera light and the sampling point location data, the light gradient model predicts the color and volume density of the at least two sampling points corresponding to the preset lighting effect, respectively.

[0349] The color information of the projection point is calculated based on the color and volume density of the at least two sampling points corresponding to the preset lighting effect.

[0350] In some embodiments, the color information prediction submodule 1203c is further configured to:

[0351] The direction information of the camera rays and the location data of the sampling points are encoded to obtain the input data;

[0352] The input data is input into the light gradient model, and the light gradient model predicts the color and volume density of the at least two sampling points corresponding to the preset lighting effect based on the input data.

[0353] In some embodiments, the sampling point location data includes location information of at least two sampling points, the location information being used to indicate a third location of the sampling point in the three-dimensional space; the color information prediction submodule 1203c is further used for:

[0354] The step of predicting the color and volume density of at least two sampling points corresponding to the preset lighting effect based on the camera light and the sampling point position data using the light gradient model includes:

[0355] For any one of the at least two sampling points, the direction information of the camera light and the position information of the sampling point are encoded to obtain input data;

[0356] The input data is input into the light gradient model, and the light gradient model predicts the color and volume density of the sampling point corresponding to the preset lighting effect based on the input data.

[0357] In some embodiments, the training process of the light gradient model is as follows:

[0358] The training data of the light gradient model is obtained. The training data includes at least two sample projection point data, and camera light and sample sampling point position data of each of the at least two sample projection point data. Each sample projection point data is used to indicate the position and true color information of a sample projection point obtained from the sample image. The sample sampling point position data is used to indicate the position of at least two sample sampling points obtained by sampling the same sample projection point in the three-dimensional space.

[0359] For any of the at least two sample projection point data, based on the direction information of the camera light corresponding to the sample projection point data and the position data of the sample sampling point corresponding to the sample projection point data, the color information of the sample projection point corresponding to the sample projection point data is predicted by the light gradient model. The color information of the sample projection point is used to indicate the predicted color value of the sample projection point corresponding to the preset lighting effect.

[0360] The parameters of the light gradient model are adjusted based on the color information of each of the at least two sample projection point data and the true color information of each of the at least two sample projection point data.

[0361] Optionally, for any one of the at least two sample projection point data, a color reconstruction loss of the sample projection point data is obtained based on the color information of the sample projection point data and the true color information of the sample projection point data. The color reconstruction loss of the sample projection point data is used to indicate the difference between the color information of the sample projection point data and the true color information of the sample projection point data.

[0362] Based on the color reconstruction loss of each of the at least two sample projection point data, the color reconstruction loss of the light gradient model is obtained;

[0363] Based on the color reconstruction loss of the light gradient model, the parameters of the light gradient model are iteratively adjusted.

[0364] Optionally, for any one of the at least two sample projection point data, color information features of the sample projection point data are extracted to obtain first feature data, and real color information features of the sample projection point data are extracted to obtain second feature data.

[0365] Based on the first feature data and the second feature data, the perceptual loss of the sample projection point data is obtained, and the perceptual loss of the sample projection point data is used to indicate the difference between the first feature data and the second feature data.

[0366] Based on the perceptual loss of each of the at least two sample projection point data, the perceptual loss of the light gradient model is obtained;

[0367] Based on the color reconstruction loss and the perceptual loss of the light gradient model, the parameters of the light gradient model are iteratively adjusted.

[0368] In some embodiments, the camera parameters include: the position information and view orientation of the virtual camera; such as Figure 13As shown, the projection point data acquisition module 1202 includes: a projection matrix construction submodule 1202a and a projection point data acquisition submodule 1202b.

[0369] The projection matrix construction submodule 1202a is used to construct the projection matrix of the virtual camera based on the position information and the view direction.

[0370] The projection point data acquisition submodule 1202b is used to project the at least one vertex from the three-dimensional space to the screen space using the projection matrix to obtain the projection point position data.

[0371] In some embodiments, the projection matrix includes a view transformation matrix and a projection transformation matrix. The view transformation matrix is ​​used to transform points in the three-dimensional spatial coordinate system corresponding to the three-dimensional space to the camera coordinate system corresponding to the virtual camera. The projection transformation matrix is ​​used to project points in the camera coordinate system to the normalized device coordinate system corresponding to the screen space. The projection point data acquisition submodule 1202b is further used for:

[0372] The view transformation matrix is ​​used to transform at least one vertex in the three-dimensional spatial coordinate system to the camera coordinate system to obtain camera projection position data. The camera projection position data is used to indicate the fourth position of the at least one vertex in the camera coordinate system.

[0373] The projection transformation matrix is ​​used to project at least one vertex in the camera coordinate system onto the standardized device coordinate system to obtain standardized projection point position data. The standardized projection point position data is used to indicate the fifth position of at least one of the projection points in the standardized device coordinate system.

[0374] Using the parameters corresponding to the screen space, at least one of the projection points in the standardized device coordinate system is transformed to the screen coordinate system corresponding to the screen space to obtain the projection point position data.

[0375] In some embodiments, such as Figure 13 As shown, the vertex data acquisition module 1201 includes: an initial data acquisition submodule 1201a and a vertex data acquisition submodule 1201b.

[0376] The initial data acquisition submodule 1201a is used to acquire the initial vertex position data of the three-dimensional model. The initial vertex position data is used to indicate the sixth position of each initial vertex of the three-dimensional model in the three-dimensional space. The number of initial vertices is greater than the number of vertices.

[0377] The vertex data acquisition submodule 1201b is used to merge the initial vertices of the three-dimensional model to obtain the vertex position data of the three-dimensional model.

[0378] In some embodiments, the initial vertex position data is obtained after multiple iterative merging processes; the vertex data acquisition submodule 1201b is further configured to:

[0379] During the nth iteration merging process, the error matrix of each transition vertex indicated by the nth-order vertex position data is obtained. The error matrix is ​​used to indicate the distance between the transition vertex and at least two adjacent grids of the transition vertex. The adjacent grids of the transition vertex refer to the grids that include the transition vertex. The grids are used to construct the three-dimensional model. When n is 1, the nth-order vertex position data is the initial vertex data, and n is a positive integer.

[0380] From the transition vertices indicated by the nth-order vertex position data, at least two compliant point pairs are determined, wherein the distance between the two transition vertices in the compliant point pair is less than or equal to a first threshold or the two transition vertices belong to the same edge;

[0381] For any of the at least two compliance point pairs, a shrinkage error of the compliance point pair is determined based on the error matrices of the two transition vertices in the compliance point pair. The shrinkage error is used to indicate the priority of merging the compliance point pairs.

[0382] The compliance point pair with the smallest shrinkage error among the at least two compliance point pairs is determined as the target compliance point pair;

[0383] Based on the shrinkage error of the target compliance point pair, a new vertex of the target compliance point pair is determined, and the new vertex of the target compliance point pair is used to replace the two transition vertices in the target compliance point pair;

[0384] Replace the data in the nth-order vertex position data of the target compliance point pair with the position information of the new vertex of the target compliance point pair to obtain the (n+1)th-order vertex position data;

[0385] If the minimum shrinkage error corresponding to the (n+1)th vertex position data is greater than or equal to the error threshold, the (n+1)th vertex position data is determined as the vertex position data.

[0386] In some embodiments, the vertex data acquisition submodule 1201b is further configured to:

[0387] For any transition vertex indicated by the nth-order vertex position data, obtain the error matrix of each of the at least two adjacent grids of the transition vertex, and the error matrix of the adjacent grids is used to indicate the distance between the adjacent grids and the transition vertex;

[0388] The error matrix of the transition vertex is determined based on the error matrices of each of the at least two adjacent grids.

[0389] In summary, the technical solution provided by the embodiments of this application, on the one hand, after projecting each vertex of the 3D model from 3D space to screen space according to camera parameters to obtain each projection point, supports predicting the projection point color data corresponding to the preset lighting effect through a light gradient model based on the projection points and camera parameters, thereby realizing the automatic prediction of the color value to be assigned at the projection point; on the other hand, it supports rendering a model rendering image of the 3D model using the second position of the projection point and the projection point color data corresponding to the preset lighting effect. Since the projection point color data corresponding to the preset lighting effect can make the 3D model in the model rendering image have the preset lighting effect during the rendering process, the lighting effect of the 3D model can be automatically rendered without the need for manual rendering of the lighting effect by artists, thereby improving the rendering efficiency of the lighting effect of the 3D model.

[0390] In addition, since this application can avoid the problem of inaccurate lighting effect rendering caused by errors easily introduced by manual art, it is beneficial to improve the lighting effect rendering quality of 3D models.

[0391] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0392] Please refer to Figure 14 It shows a structural block diagram of a computer device provided in one embodiment of this application.

[0393] Typically, computer device 1400 includes a processor 1401 and a memory 1402.

[0394] Processor 1401 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1401 may integrate a GPU, which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0395] The memory 1402 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1402 stores a computer program that is loaded and executed by the processor 1401 to implement the rendering method of the three-dimensional model described above.

[0396] Those skilled in the art will understand that Figure 14 The structure shown does not constitute a limitation on the computer device 1400, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0397] In some embodiments, a computer-readable storage medium is also provided, wherein a computer program is stored therein, the computer program being loaded and executed by a processor to implement the above-described method for rendering the three-dimensional model.

[0398] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0399] In some embodiments, a computer program product is also provided, the computer program product including a computer program stored in a computer-readable storage medium, and a processor reading from the computer-readable storage medium and executing the computer program to implement the above-described method for rendering the three-dimensional model.

[0400] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0401] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. A method for rendering a three-dimensional model, characterized in that, The method includes: Obtain vertex position data of the three-dimensional model, wherein the vertex position data is used to indicate the first position of at least one vertex of the three-dimensional model in three-dimensional space; Based on the camera parameters of the virtual camera in the three-dimensional space, the at least one vertex is projected from the three-dimensional space to the screen space to obtain projection point position data. The projection point position data is used to indicate the second position of the projection point corresponding to the at least one vertex in the screen space. Based on the projection point location data and the camera parameters, the projection point color data corresponding to the preset lighting effect of the three-dimensional model in the three-dimensional space is predicted by the light gradient model; The three-dimensional model is rendered using the projection point position data and the projection point color data to obtain a rendered image of the model in the screen space.

2. The method according to claim 1, characterized in that, The step of predicting the projection point color data corresponding to the preset lighting effect of the 3D model in the 3D space based on the projection point position data and the camera parameters using a light gradient model includes: For each projection point, based on the second position of the projection point and the camera parameters, the camera ray corresponding to the projection point is obtained. The camera ray is used to simulate the ray path when the virtual camera captures the vertex corresponding to the projection point. Sampling is performed along the camera ray in the three-dimensional space to obtain sampling point position data, which is used to indicate the third position of at least two sampling points in the three-dimensional space; Based on the camera light and the sampling point position data, the color information of the projection point is predicted by the light gradient model. The color information of the projection point is used to indicate the predicted color value of the projection point corresponding to the preset lighting effect. The color information of each projection point is combined to obtain the color data of the projection point.

3. The method according to claim 2, characterized in that, The step of predicting the color information of the projection point based on the camera light and the sampling point position data using the light gradient model includes: Based on the camera light and the sampling point location data, the light gradient model predicts the color and volume density of the at least two sampling points corresponding to the preset lighting effect, respectively. The color information of the projection point is calculated based on the color and volume density of the at least two sampling points corresponding to the preset lighting effect.

4. The method according to claim 3, characterized in that, The step of predicting the color and volume density of at least two sampling points corresponding to the preset lighting effect based on the camera light and the sampling point position data using the light gradient model includes: The direction information of the camera rays and the location data of the sampling points are encoded to obtain the input data; The input data is input into the light gradient model, and the light gradient model predicts the color and volume density of the at least two sampling points corresponding to the preset lighting effect based on the input data.

5. The method according to claim 3, characterized in that, The sampling point location data includes the location information of at least two sampling points, and the location information is used to indicate the third location of the sampling point in the three-dimensional space; The step of predicting the color and volume density of at least two sampling points corresponding to the preset lighting effect based on the camera light and the sampling point position data using the light gradient model includes: For any one of the at least two sampling points, the direction information of the camera light and the position information of the sampling point are encoded to obtain input data; The input data is input into the light gradient model, and the light gradient model predicts the color and volume density of the sampling point corresponding to the preset lighting effect based on the input data.

6. The method according to any one of claims 3 to 5, characterized in that, The training process for the light gradient model is as follows: The training data of the light gradient model is obtained. The training data includes at least two sample projection point data, and camera light and sample sampling point position data of each of the at least two sample projection point data. Each sample projection point data is used to indicate the position and true color information of a sample projection point obtained from the sample image. The sample sampling point position data is used to indicate the position of at least two sample sampling points obtained by sampling the same sample projection point in the three-dimensional space. For any of the at least two sample projection point data, based on the direction information of the camera light corresponding to the sample projection point data and the position data of the sample sampling point corresponding to the sample projection point data, the color information of the sample projection point corresponding to the sample projection point data is predicted by the light gradient model. The color information of the sample projection point is used to indicate the predicted color value of the sample projection point corresponding to the preset lighting effect. The parameters of the light gradient model are adjusted based on the color information of each of the at least two sample projection point data and the true color information of each of the at least two sample projection point data.

7. The method according to claim 6, characterized in that, The step of adjusting the parameters of the light gradient model based on the color information of each of the at least two sample projection points and the true color information of each of the at least two sample projection points includes: For any of the at least two sample projection point data, the color reconstruction loss of the sample projection point data is obtained based on the color information of the sample projection point data and the true color information of the sample projection point data. The color reconstruction loss of the sample projection point data is used to indicate the difference between the color information of the sample projection point data and the true color information of the sample projection point data. Based on the color reconstruction loss of each of the at least two sample projection point data, the color reconstruction loss of the light gradient model is obtained; Based on the color reconstruction loss of the light gradient model, the parameters of the light gradient model are iteratively adjusted.

8. The method according to claim 7, characterized in that, The method further includes: For any one of the at least two sample projection point data, color information features of the sample projection point data are extracted to obtain first feature data, and real color information features of the sample projection point data are extracted to obtain second feature data. Based on the first feature data and the second feature data, the perceptual loss of the sample projection point data is obtained, and the perceptual loss of the sample projection point data is used to indicate the difference between the first feature data and the second feature data. Based on the perceptual loss of each of the at least two sample projection point data, the perceptual loss of the light gradient model is obtained; The color reconstruction loss based on the light gradient model, iteratively adjusting the parameters of the light gradient model, includes: Based on the color reconstruction loss and the perceptual loss of the light gradient model, the parameters of the light gradient model are iteratively adjusted.

9. The method according to any one of claims 1 to 8, characterized in that, The camera parameters include: the position information and view orientation of the virtual camera; The step of projecting at least one vertex from the three-dimensional space to the screen space based on the camera parameters of the virtual camera in the three-dimensional space to obtain the projection point position data includes: Based on the location information and the view direction, construct the projection matrix of the virtual camera; The projection matrix is ​​used to project at least one vertex from the three-dimensional space to the screen space to obtain the projection point position data.

10. The method according to claim 9, characterized in that, The projection matrix includes a view transformation matrix and a projection transformation matrix. The view transformation matrix is ​​used to transform points in the three-dimensional spatial coordinate system corresponding to the three-dimensional space to the camera coordinate system corresponding to the virtual camera. The projection transformation matrix is ​​used to project points in the camera coordinate system to the normalized device coordinate system corresponding to the screen space. The step of projecting at least one vertex from the three-dimensional space to the screen space using the projection matrix to obtain the projection point position data includes: The view transformation matrix is ​​used to transform at least one vertex in the three-dimensional spatial coordinate system to the camera coordinate system to obtain camera projection position data. The camera projection position data is used to indicate the fourth position of the at least one vertex in the camera coordinate system. The projection transformation matrix is ​​used to project at least one vertex in the camera coordinate system onto the standardized device coordinate system to obtain standardized projection point position data. The standardized projection point position data is used to indicate the fifth position of at least one of the projection points in the standardized device coordinate system. Using the parameters corresponding to the screen space, at least one of the projection points in the standardized device coordinate system is transformed to the screen coordinate system corresponding to the screen space to obtain the projection point position data.

11. The method according to any one of claims 1 to 10, characterized in that, The step of obtaining the vertex position data of the 3D model includes: Obtain the initial vertex position data of the three-dimensional model. The initial vertex position data is used to indicate the sixth position of each initial vertex of the three-dimensional model in the three-dimensional space. The number of initial vertices is greater than the number of vertices. The initial vertices of the 3D model are merged to obtain the vertex position data of the 3D model.

12. The method according to claim 11, characterized in that, The initial vertex position data is obtained after multiple iterative merging processes; The process of merging the initial vertices of the 3D model to obtain the vertex position data of the 3D model includes: During the nth iteration merging process, the error matrix of each transition vertex indicated by the nth-order vertex position data is obtained. The error matrix is ​​used to indicate the distance between the transition vertex and at least two adjacent grids of the transition vertex. The adjacent grids of the transition vertex refer to the grids that include the transition vertex. The grids are used to construct the three-dimensional model. When n is 1, the nth-order vertex position data is the initial vertex data, and n is a positive integer. From the transition vertices indicated by the nth-order vertex position data, at least two compliant point pairs are determined, wherein the distance between the two transition vertices in the compliant point pair is less than or equal to a first threshold or the two transition vertices belong to the same edge; For any of the at least two compliance point pairs, a shrinkage error of the compliance point pair is determined based on the error matrices of the two transition vertices in the compliance point pair. The shrinkage error is used to indicate the priority of merging the compliance point pairs. The compliance point pair with the smallest shrinkage error among the at least two compliance point pairs is determined as the target compliance point pair; Based on the shrinkage error of the target compliance point pair, a new vertex of the target compliance point pair is determined, and the new vertex of the target compliance point pair is used to replace the two transition vertices in the target compliance point pair; Replace the data in the nth-order vertex position data of the target compliance point pair with the position information of the new vertex of the target compliance point pair to obtain the (n+1)th-order vertex position data; If the minimum shrinkage error corresponding to the (n+1)th vertex position data is greater than or equal to the error threshold, the (n+1)th vertex position data is determined as the vertex position data.

13. The method according to claim 12, characterized in that, The error matrix of each transition vertex indicated by the acquisition of the nth-order vertex position data includes: For any transition vertex indicated by the nth-order vertex position data, obtain the error matrix of each of the at least two adjacent grids of the transition vertex, and the error matrix of the adjacent grids is used to indicate the distance between the adjacent grids and the transition vertex; The error matrix of the transition vertex is determined based on the error matrices of each of the at least two adjacent grids.

14. A rendering device for a three-dimensional model, characterized in that, The device includes: A vertex data acquisition module is used to acquire vertex position data of the three-dimensional model, wherein the vertex position data is used to indicate the first position of at least one vertex of the three-dimensional model in three-dimensional space. The projection point data acquisition module is used to project at least one vertex from the three-dimensional space to the screen space according to the camera parameters of the virtual camera in the three-dimensional space, and obtain projection point position data. The projection point position data is used to indicate the second position of the projection point corresponding to the at least one vertex in the screen space. The color data prediction module is used to predict the color data of the projection point corresponding to the preset lighting effect of the three-dimensional model in the three-dimensional space based on the projection point position data and the camera parameters and through the light gradient model. The 3D model rendering module is used to render the 3D model using the projection point position data and the projection point color data to obtain the model rendering image in the screen space.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the method as claimed in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the method as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, which a processor reads from and executes to implement the method as described in any one of claims 1 to 13.

Images