Shadow generation method and apparatus for three-dimensional model, and device, medium and program product

Abstract

A shadow generation method for a three-dimensional model, the method comprising: acquiring light source information of a virtual light source and an observation parameter of a virtual camera, wherein the light source information is used for forming an illumination effect of the virtual light source in a virtual scene, and the observation parameter is used for forming a field-of-view range of the virtual camera in the virtual scene for a three-dimensional model; on the basis of the light source information and the observation parameter, generating a multi-angle contour map corresponding to the three-dimensional model, wherein the multi-angle contour map is used for representing an initial shadow; on the basis of the light source information and the observation parameter, projecting the multi-angle contour map onto a reference plane, so as to obtain a hard shadow image corresponding to the three-dimensional model; on the basis of the light source information and material attributes of the three-dimensional model, obtaining a shadow feature of the three-dimensional model; and inputting the hard shadow image and the shadow feature into a trained shadow prediction model based on a convolutional neural network, and on the basis of the hard shadow image and the shadow feature and by means of the shadow prediction model, outputting a soft shadow image corresponding to the three-dimensional model.

Description

Shadow generation method, device, equipment, medium and program product of three-dimensional model

[0001] The present application claims priority to the Chinese patent application No. 202410444986.5, filed on April 12, 2024, and entitled "Shadow generation method, device, equipment, medium and program product of three-dimensional model", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of computer, in particular to a shadow generation method, device, computer equipment, storage medium and computer program product of three-dimensional model. BACKGROUND

[0003] Adjusting shadow effect is an important step to add reality and depth to three-dimensional model. Proper shadow can make three-dimensional model more stereoscopic and improve visualization effect. Generally, corresponding shadow effect can be added according to light source in virtual scene. In rendering of virtual scene, many three-dimensional models are rendered with shadow of light, but due to the influence of model or rendering sequence of three-dimensional model, there is a problem of failure of shadow effect rendering.

[0004] In some technologies, for shadow reconstruction of three-dimensional model in virtual scene, a geometric shadow map algorithm based on triangle reconstruction of computer graphics is often used, however, this way needs a lot of calculation and the shadow generation efficiency is low, which is difficult to realize real-time shadow reconstruction in virtual scene. SUMMARY

[0005] According to various embodiments of the present application, a shadow generation method of three-dimensional model is provided. The method comprises:

[0006] Obtaining light source information of virtual light source and observation parameter of virtual camera, the light source information is used to form light effect of the virtual light source in virtual scene, and the observation parameter is used to form field of view range of the virtual camera to three-dimensional model in virtual scene;

[0007] Generating multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameter, the multi-view contour map is used to represent initial shadow;

[0008] Projecting the multi-view contour map to reference plane according to the light source information and the observation parameter to obtain hard shadow image corresponding to the three-dimensional model;

[0009] Obtaining shadow feature of the three-dimensional model according to the light source information and material property of the three-dimensional model;

[0010] input the hard shadow image and the shadow feature into a trained shadow prediction model based on a convolutional neural network, and output a soft shadow image corresponding to the three-dimensional model through the shadow prediction model according to the hard shadow image and the shadow feature.

[0011] According to various embodiments of the present application, the present application further provides a shadow generation device of a three-dimensional model. The device comprises:

[0012] The acquisition module is configured to acquire light source information of a virtual light source and observation parameters of a virtual camera, the light source information being used to form a lighting effect of the virtual light source in a virtual scene, and the observation parameters being used to form a field of view range of the virtual camera on the three-dimensional model in the virtual scene.

[0013] The contour extraction module is configured to generate a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters, the multi-view contour map being used to represent an initial shadow.

[0014] The hard shadow generation module is configured to project the multi-view contour map to a reference plane according to the light source information and the observation parameters, to obtain a hard shadow image corresponding to the three-dimensional model.

[0015] The shadow feature extraction module is configured to obtain a shadow feature of the three-dimensional model according to the light source information and a material attribute of the three-dimensional model.

[0016] The prediction module is configured to input the hard shadow image and the shadow feature into a trained shadow prediction model based on a convolutional neural network, and output a soft shadow image corresponding to the three-dimensional model through the shadow prediction model according to the hard shadow image and the shadow feature.

[0017] According to various embodiments of the present application, the present application further provides a computer device. The computer device comprises a memory and a processor, the memory stores computer readable instructions, and the processor implements the steps of the shadow generation method of the three-dimensional model when executing the computer readable instructions.

[0018] According to various embodiments of the present application, the present application further provides a computer readable storage medium. The computer readable storage medium stores computer readable instructions, and the computer readable instructions implement the steps of the shadow generation method of the three-dimensional model when executed by a processor.

[0019] According to various embodiments of the present application, the present application further provides a computer program product. The computer program product comprises computer readable instructions, and the computer readable instructions implement the steps of the shadow generation method of the three-dimensional model when executed by a processor.

[0020] The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the application will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0022] Fig. 1 is an application environment diagram of the shadow generation method of the three-dimensional model in some embodiments;

[0023] Fig. 2 is a flowchart of the shadow generation method of the three-dimensional model in some embodiments;

[0024] Fig. 3 is a schematic diagram of hard shadow and soft shadow in some embodiments;

[0025] Fig. 4 is a flowchart of generating the multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters in some embodiments;

[0026] Fig. 5 is a schematic diagram of the binary contour images corresponding to different views of the three-dimensional model in some embodiments;

[0027] Fig. 6 is a schematic diagram of the multi-view contour map corresponding to the three-dimensional model in some embodiments;

[0028] Fig. 7 is a flowchart of extracting the binary contour images from multiple views in some embodiments;

[0029] Fig. 8 is a flowchart of projecting the multi-view contour map to the reference plane to obtain the hard shadow image in some embodiments;

[0030] Fig. 9 is a flowchart of obtaining the shadow feature map of the three-dimensional model according to the light source information and the material properties of the three-dimensional model in some embodiments;

[0031] Fig. 10 is a schematic diagram of the framework of the shadow generation method of the three-dimensional model in some embodiments;

[0032] Fig. 11 is a structural block diagram of the shadow generation device of the three-dimensional model in some embodiments;

[0033] Fig. 12 is an internal structure diagram of a computer device in some embodiments;

[0034] Fig. 13 is an internal structure diagram of a computer device in some other embodiments. DETAILED DESCRIPTION

[0035] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0036] The scheme provided by the embodiments of the present application relates to the convolutional neural network and other technologies of artificial intelligence, which is specifically explained by the following embodiments.

[0037] The shadow generation method of the three-dimensional model provided by the embodiments of the present application can be applied to the application environment as shown in FIG. 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can obtain the light source information of the virtual light source and the observation parameter of the virtual camera. The light source information is used to represent the lighting effect of the virtual light source in the virtual scene, and the observation parameter is used to represent the field of view range of the virtual camera to the three-dimensional model in the virtual scene. According to the light source information and the observation parameter, a multi-view contour map corresponding to the three-dimensional model is generated, and the multi-view contour map is used to represent the initial shadow. According to the light source information and the observation parameter, the multi-view contour map is projected to the reference plane to obtain a hard shadow image corresponding to the three-dimensional model. According to the light source information and the material properties of the three-dimensional model, the shadow feature of the three-dimensional model is obtained. The hard shadow image and the shadow feature are input into the trained shadow prediction model based on the convolutional neural network. According to the hard shadow image and the shadow feature, the shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model. That is, only one low-consumption multi-view contour map generation process is needed, and then the hard shadow image of the three-dimensional model is generated according to the multi-view contour map. According to the hard shadow image and the shadow feature, the soft shadow image of the three-dimensional model, that is, the shadow effect of the three-dimensional model, can be efficiently generated, which reduces the data processing amount when generating the shadow and improves the shadow generation efficiency.

[0038] Of course, the shadow generation method of the three-dimensional model described above can also be executed by the server 104. The terminal 102 can obtain the generated soft shadow image from the server 104. Optionally, the terminal 102 can also render the shadow effect according to the soft shadow image, so that the display of the three-dimensional model in the virtual scene is more realistic and natural.

[0039] The terminal 102 can be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle-mounted device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be a standalone physical server, a server cluster composed of multiple physical servers, or a distributed system. The server 104 can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc.

[0040] In an embodiment, the terminal 102 is installed and runs an application (or a client) that supports a virtual scene. The application can be a competition program or a game program. When the terminal 102 runs the application, an interface of the application is displayed on a screen of the terminal 102, and the virtual scene is displayed in the interface. Optionally, the terminal 102 can be a terminal used by a current player. The current player can use the terminal 102 to control a virtual object in the virtual scene. The virtual object can be a virtual person, a virtual animal, or a virtual animated character, etc. The virtual object is implemented by a three-dimensional model.

[0041] In an embodiment, as shown in FIG. 2, a shadow generation method of a three-dimensional model is provided. The method is executed by a computer device (for example, the terminal 102 or the server 104) in FIG. 1. The method includes the following steps:

[0042] In step 202, light source information of a virtual light source and observation parameters of a virtual camera are obtained. The light source information is used to form a lighting effect of the virtual light source in the virtual scene. The observation parameters are used to form a field of view of the virtual camera in the virtual scene.

[0043] In three-dimensional graphics, a virtual light source is an important basis for generating shadows. The virtual light source determines the intensity and color of light received by an object by simulating the interaction of light and the object. Based on the virtual light source, a high-quality shadow can be presented, thereby significantly improving the texture and realism of the virtual scene. The types of virtual light sources usually include ambient light, parallel light, point light source, spotlight, and the like. Light source information is used to form the lighting effect of the virtual light source in the virtual scene. The light source information of the virtual light source includes at least one of the following: the position of the light source; the direction of the light source; the view angle of the light source; the near plane distance of the light source; the far plane distance of the light source; and the aspect ratio of the light source. These information determines the irradiation mode, illumination angle and illumination intensity distribution of the light irradiation, thereby affecting the generation of shadows. By adjusting the position and angle of the virtual light source, the shape and length of the shadow can be changed, thereby enhancing the three-dimensional sense and realism of the three-dimensional model. The intensity of the virtual light source also affects the light and dark degree of the shadow.

[0044] In three-dimensional graphics modeling, a virtual camera can simulate the camera mechanism in the real world to obtain better effects in the modeling, rendering and animation production process. Observation parameters are used to form the field of view of the virtual camera in the virtual scene. The observation parameters of the virtual camera include at least one of the following: camera position, i.e., the position of the virtual camera; camera view point; camera view direction vector, i.e., a ray pointing to the observation target point from the camera view point; camera upward direction vector, which specifies the upward direction of the virtual camera; camera view angle; near plane distance of the camera; far plane distance of the camera; and aspect ratio of the camera.

[0045] A virtual scene is a virtual picture displayed by an application running on a terminal. The virtual scene can be a simulated scene of a real scene, a fictitious scene, a virtual scene obtained by fusing a simulated scene of a real scene, or a purely fictitious scene.

[0046] In the three-dimensional graphics modeling process, a user can edit and set the light source information of the virtual light source and the observation parameters of the virtual camera in the view. Therefore, the computer device can obtain the light source information of the virtual light source and the observation parameters of the virtual camera from the setting parameters of the view.

[0047] In step 204, a multi-view contour map corresponding to the three-dimensional model is generated according to the light source information and the observation parameters. The multi-view contour map is used to represent the initial shadow.

[0048] The multi-view profile image is an image formed by profile lines of the three-dimensional model under multiple different views, and reflects the shape and structure of the three-dimensional model, and can be used to represent the initial shadow of the three-dimensional model. The computer device can sum the profile lines of the three-dimensional model under different views generated by the GPU-based renderer according to the light source information of the virtual light source and the observation parameters of the virtual camera, to obtain the multi-view profile image of multiple angles. Since rendering a profile shadow on the GPU only needs to occupy a very small consumption, the rendering can be completed within a few milliseconds, and the generation efficiency is high. With the multi-view profile image, the multi-view profile image is used as the input data for soft shadow generation, that is, the initial shape of the shadow, which can ensure the effect of the subsequently generated soft shadow image, and the entire process of generating the soft shadow image is also very efficient.

[0049] In step 206, the multi-view profile image is projected to the reference plane according to the light source information and the observation parameters, to obtain a hard shadow image corresponding to the three-dimensional model.

[0050] In order to obtain the spatial relationship between the three-dimensional model and the reference plane, after obtaining the multi-view profile image, the computer device projects the multi-view profile image to the reference plane according to the light source information and the observation parameters, to obtain a hard shadow image corresponding to the three-dimensional model. The reference plane can be a simple horizontal plane or a virtual scene with texture and material, and the three-dimensional model will be placed on the reference plane.

[0051] The softness and hardness of the shadow are important factors affecting the shadow effect, and the softness and hardness of the shadow are related to the size and shape of the virtual light source in the virtual scene. By adjusting the softness and hardness of the shadow, the shadow effect can be more realistic. The edge of the hard shadow represented by the hard shadow image is very clear, without gradual change or blur effect, and the gray scale of the shadow part is very uniform. The hard shadow image can reflect the shape and contour of the three-dimensional model. The edge of the soft shadow represented by the soft shadow image has a gradual change or blur effect, and the gray scale of the shadow part is not uniform. It can increase the depth and realism of the shadow. As shown in FIG. 3, which is a schematic diagram of hard shadow and soft shadow in an embodiment, it can be seen that the boundary of the hard shadow is relatively clear, while the gray scale of the soft shadow is not uniform, and there is a shadow transition.

[0052] Optionally, after generating the hard shadow image, the computer device can also correct and optimize the hard shadow image according to the shape and material properties of the reference plane, such as eliminating jaggedness, increasing shadow intensity, etc.

[0053] In step 208, the shadow characteristics of the three-dimensional model are obtained according to the light source information and the material properties of the three-dimensional model.

[0054] The material of the three-dimensional model has an impact on the shadow effect. Different materials have different abilities of reflection and absorption of light, thereby affecting the brightness and texture of the shadow. The material properties at least include at least one of the following: diffuse reflectance, specular reflectance, roughness and metallicity of the surface material of the three-dimensional model. In the embodiments of the present application, considering the light source information of the virtual light source in the virtual scene and the material properties of the three-dimensional model, which will affect the shape and intensity of the shadow, some features, i.e. shadow features, are extracted therefrom. The hard shadow image of the three-dimensional model and the extracted shadow features are combined as inputs required for soft shadow prediction. That is, the soft shadow image is predicted based on the hard shadow image and the shadow features by the convolutional neural network model, which can ensure the quality of the reconstruction of the shadow of the three-dimensional model and obtain a more natural and realistic shadow effect.

[0055] In step 210, the hard shadow image and the shadow features are input into the trained shadow prediction model based on the convolutional neural network. The shadow prediction model outputs the corresponding soft shadow image of the three-dimensional model according to the hard shadow image and the shadow features.

[0056] The shadow prediction model based on the convolutional neural network has the ability to predict the soft shadow image based on the hard shadow image and the shadow features. The shadow prediction model is obtained based on the initial convolutional neural network and continuous learning and training based on the training samples. The computer device inputs the hard shadow image and the shadow features into the trained shadow prediction model based on the convolutional neural network. The shadow prediction model outputs the corresponding soft shadow image of the three-dimensional model according to the hard shadow image and the shadow features.

[0057] In the related art, based on the three-dimensional model and the virtual scene, the rendering and processing of the shadow effect can be realized by using complex computer graphics. There is a problem of large amount of calculation and low efficiency of shadow generation, which is difficult to apply in real-time shadow reconstruction of the virtual scene. In the embodiments of the present application, only the hard shadow image and the shadow features are used, and the shadow prediction model is used to quickly generate the corresponding soft shadow image of the three-dimensional model, which has high efficiency of shadow generation and can be applied in real-time shadow reconstruction of the virtual scene.

[0058] In one embodiment, further, the computer device runs the rendering program to render according to the soft shadow image, which can achieve a high-quality, natural and realistic shadow effect. Optionally, the rendering is combined with the base map or the texture map of the virtual scene, which can efficiently display a natural shadow effect.

[0059] For example, the computer device can input the obtained soft shadow image as a shadow map into a resource set in a debugging module in the game engine, so that the game engine can generate a shadow effect corresponding to the three-dimensional model according to the soft shadow image. Illustratively, the game engine superimposes and renders the soft shadow image and a scene texture corresponding to the virtual scene, and displays the shadow effect in the virtual scene.

[0060] The above application scenarios are only illustrative, and the shadow generation method of the model can also be applied to the shadow generation process of a model in other virtual scenes, which is not limited here.

[0061] In the above shadow generation method of the three-dimensional model, the light source information of the virtual light source and the observation parameter of the virtual camera are obtained, the light source information is used to represent the lighting effect of the virtual light source in the virtual scene, and the observation parameter is used to represent the field of view range of the virtual camera to the three-dimensional model in the virtual scene. According to the light source information and the observation parameter, a multi-view contour map corresponding to the three-dimensional model is generated, and the multi-view contour map is used to represent an initial shadow. According to the light source information and the observation parameter, the multi-view contour map is projected to a reference plane to obtain a hard shadow image corresponding to the three-dimensional model. According to the light source information and the material property of the three-dimensional model, a shadow feature of the three-dimensional model is obtained. The hard shadow image and the shadow feature are input into a trained shadow prediction model based on a convolutional neural network. According to the hard shadow image and the shadow feature, the shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model. The soft shadow image can be used to generate a more realistic and natural soft shadow effect. That is, only one low-consumption multi-view contour generation process is needed, and then a hard shadow image of the three-dimensional model is generated according to the multi-view contour. According to the hard shadow image and the shadow feature, the shadow effect of the three-dimensional model can be efficiently generated, reducing the data processing amount during shadow generation and improving the shadow generation efficiency.

[0062] In one embodiment, as shown in FIG. 4, according to the light source information and the observation parameter, a multi-view contour map corresponding to the three-dimensional model is generated, including:

[0063] In step 402, based on the light source information and the observation parameter, a binary contour image of the three-dimensional model is extracted from multiple views. The binary contour image is an image obtained by mapping the three-dimensional model to a two-dimensional plane and performing binary distinction on the image foreground and image background mapped to the two-dimensional plane.

[0064] The binarized contour image is obtained by rendering the three-dimensional model to obtain a two-dimensional image, and then extracting the edge contour line of the two-dimensional image. The computer device can map the three-dimensional model to a two-dimensional plane from multiple different perspectives of the three-dimensional model to obtain a two-dimensional image, and then extract the edge contour line of each two-dimensional image, that is, perform binarization division of the image foreground and the image background, so as to obtain the binarized contour image of each three-dimensional model under multiple different perspectives. As shown in FIG. 5, it is a schematic diagram of the binarized contour image of the three-dimensional model under multiple different perspectives in an embodiment.

[0065] In step 404, the binarized contour images corresponding to the perspectives are merged to obtain a multi-perspective contour image corresponding to the three-dimensional model.

[0066] In an embodiment, the binarized contour images corresponding to the perspectives are merged to obtain a multi-perspective contour image corresponding to the three-dimensional model, including: performing logical OR operation on the binarized contour images corresponding to the perspectives to obtain the multi-perspective contour image corresponding to the three-dimensional model.

[0067] As shown in FIG. 6, it is a schematic diagram of a multi-perspective contour image corresponding to a three-dimensional model in an embodiment. Referring to FIG. 6, the diagram includes contour images of the three-dimensional model under different perspectives, in which the three-dimensional model part is a blackened area, and the background part is a white area.

[0068] In this embodiment, the edge contour lines of the three-dimensional model are extracted from different perspectives, and these edge contour lines are merged into a binary image, which can accurately reflect the shape of the three-dimensional model and can be used as input data for shadow generation, which is helpful for subsequent generation of high-quality soft shadow images.

[0069] In an embodiment, as shown in FIG. 7, based on the light source information and the observation parameters, the binarized contour images of the three-dimensional model are extracted from multiple perspectives, including:

[0070] In step 702, the three-dimensional model is converted into a triangular mesh composed of triangular patches.

[0071] A triangular patch is a two-dimensional geometric figure composed of three vertices and three edges, each edge connecting two vertices to form a closed triangle. Triangular patches are usually used to describe the surface of a three-dimensional object, and a large number of adjacent triangular patches are combined to construct complex shapes. The triangular mesh composed of triangular patches can be used to simulate the surface of a complex object. In order to facilitate subsequent projection transformation and image processing, the computer device can first convert the three-dimensional model into a triangular mesh, that is, a mesh structure composed of a series of triangular patches, which can well maintain the shape and details of the three-dimensional model, and is also convenient for geometric operations and rendering.

[0072] At step 704, a projection matrix of the three-dimensional model at different viewing angles is calculated according to the light source information and the observation parameter, the different viewing angles being different rotation angles when the three-dimensional model is rotated with the specified coordinate axis as the center.

[0073] The specified coordinate axis can be a longitudinal axis of the three-dimensional model, the three-dimensional model can be rotated with the longitudinal axis of the three-dimensional model as the center, and the projection matrix corresponding to different rotation angles in the rotation process is calculated according to the light source information and the observation parameter, the projection matrix reflecting the spatial relationship between the three-dimensional model and the two-dimensional plane at different viewing angles.

[0074] At step 706, for each viewing angle, the corresponding projection matrix is used to project the triangular mesh onto the two-dimensional plane to obtain a two-dimensional polygon at the corresponding viewing angle.

[0075] At step 708, for each viewing angle, an edge contour line of the two-dimensional polygon at the corresponding viewing angle is extracted to obtain a binary contour image of the three-dimensional model at the corresponding viewing angle.

[0076] The computer device can use an image processing algorithm to extract the edge of the two-dimensional polygon to obtain a binary image, which can reflect the edge contour of the three-dimensional model, that is, a binary contour image.

[0077] In this embodiment, by extracting the edge contour line of the three-dimensional model from different viewing angles, the shape of the three-dimensional model can be accurately reflected.

[0078] In one embodiment, at step 702, the three-dimensional model is converted into a triangular mesh composed of triangular facets, including: obtaining a vertex set and a triangular facet set of the three-dimensional model, each triangular facet in the triangular facet set being represented by three vertex indexes, and the three vertices identified by the three vertex indexes all belonging to the vertex set; and generating the triangular mesh according to the vertex set and the triangular facet set.

[0079] wherein, for a three-dimensional model, its corresponding vertices and triangular facets are known, the vertex set and the triangular facet set are denoted by V and F respectively, wherein V is an n x 3 matrix, each row represents the coordinates (x, y, z) of a vertex, F is an m x 3 matrix, each row represents the indices of the three vertices of a triangular facet, i.e. vertex identification or vertex serial number. The triangular mesh M generated according to V and F needs to satisfy the following conditions: the vertex set of the triangular mesh M is the same as V, i.e. M has n vertices, and the coordinates of each vertex are the same as the corresponding row in V; the triangular facet set of M is the same as F, i.e. M has m triangular facets, and the three vertices of each triangular facet are the same as the corresponding row in F; each triangular facet of M is a triangle, i.e. each row in F is three different integers, and the three integers are all between 1 and n; each triangular facet of M is counterclockwise, i.e. each row (i, j, k) in F satisfies Vi x Vj · Vk > 0, wherein i, j, k represent three different vertices, x represents vector cross product, and · represents vector dot product.

[0080] In one embodiment, generating a triangular mesh according to a vertex set and a triangular facet set comprises: for each triangular facet in the triangular facet set, generating a corresponding normal vector, which is a unit vector perpendicular to the triangular facet; judging whether the facet direction of the triangular facet is counterclockwise according to the direction vector of the light source and the normal vector, the facet direction being determined by the three vertices constituting the triangular facet; when the facet direction of the triangular facet is not counterclockwise, then exchanging the order of the last two vertices in the triangular facet to obtain a triangular facet with counterclockwise facet direction; storing the vertex set and the triangular facet with counterclockwise facet direction into a triangular mesh object to obtain the triangular mesh corresponding to the three-dimensional model.

[0081] Specifically, first, the computer device calculates the normal vector corresponding to each triangular facet, i.e. a unit vector perpendicular to the triangular facet, which can be calculated using the coordinates of the three vertices, and the formula is as follows:

[0082] wherein Vi, Vj and Vk represent the three-dimensional coordinates of the three vertices i, j and k, and ||·|| represents the length of the vector, i.e. the Euclidean distance.

[0083] Next, the computer device judges the direction of each triangular facet, i.e. whether it is counterclockwise, which can be judged using the position and direction of the virtual light source, and the formula is as follows: D_ijk=N_ijk·L

[0084] wherein L represents the direction vector of the virtual light source, if D_ijk>0, then it indicates that the triangular facet (i, j, k) is counterclockwise, otherwise if D_ijk≤0, then exchange the order of the vertex j and the vertex k to make it counterclockwise.

[0085] Then, the computer device stores the information of the vertices and the triangular facets into a triangular mesh object, which can be implemented by some data structures, such as an adjacency list, a half-edge structure, etc., and thus a triangular mesh object M is generated, which can store and access the information of the vertices and the triangular facets.

[0086] In this embodiment, by converting the three-dimensional model into a triangular mesh composed of triangular facets, the surface morphology of the three-dimensional model can be well simulated, and subsequent geometric operations and rendering are facilitated.

[0087] In one embodiment, as shown in FIG. 8, in step 206, the multi-view contour map is projected onto the reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model, including:

[0088] In step 802, a projection matrix corresponding to the multi-view contour map on the reference plane is calculated according to the light source information and the observation parameters.

[0089] The reference plane is a preset plane, which can be, for example, the ground in the virtual scene. The computer device can calculate the projection matrix corresponding to the multi-view contour map on the reference plane according to the light source information and the observation parameters, and the projection matrix can be used to reflect the spatial relationship between the three-dimensional model and the reference plane.

[0090] In step 804, the multi-view contour map is projected onto the reference plane using the projection matrix to obtain a hard shadow image corresponding to the three-dimensional model.

[0091] According to the projection matrix corresponding to the multi-view contour map on the reference plane, the computer device can project the multi-view contour map onto the reference plane. Through this method based on projection transformation and image processing, the hard shadow image of the three-dimensional model can be quickly generated from the multi-view contour map, without complex rendering and processing of the three-dimensional model and the virtual scene, and only a binary image is needed as input, which is efficient and convenient.

[0092] In one embodiment, in step 802, the projection matrix corresponding to the multi-view contour map on the reference plane is calculated according to the light source information and the observation parameters, including: calculating a first transformation matrix for converting the two-dimensional coordinates of the pixel points in the multi-view contour map into three-dimensional coordinates in a world coordinate system according to the observation parameters; calculating a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane according to the light source information; and calculating the projection matrix according to the first transformation matrix and the second transformation matrix.

[0093] In order to project the multi-view profile map onto the reference plane, a projection matrix needs to be calculated, which can map each pixel point in the multi-view profile map to a position on the reference plane. It is assumed that the reference plane is a plane, and its equation is ax+by+cz+d=0, where a, b, c, and d are known constants. It is assumed that the light source information of the virtual light source and the observation parameters of the virtual camera are known, and according to these parameters, a 4×4 projection matrix P is calculated, which needs to satisfy the following conditions at the same time:

[0094] For any pixel point (u, v) in the multi-view profile map, its corresponding three-dimensional coordinates are (x, y, z, 1), where x, y, and z are calculated according to the observation parameters of the virtual camera;

[0095] For the three-dimensional coordinates (x, y, z, 1), its projection coordinates on the reference plane are (x', y', z', 1), where x', y', and z' are calculated according to the light source information of the virtual light source.

[0096] That is, the projection matrix P can project any pixel point in the multi-view profile map to the reference plane.

[0097] Wherein, the first transformation matrix is used to convert the multi-view profile map located in the camera coordinate system to the world coordinate system, and the second transformation matrix is used to convert the world coordinate system to the projection coordinate system corresponding to the reference plane.

[0098] In one embodiment, the projection matrix is calculated according to the first transformation matrix and the second transformation matrix, comprising: multiplying the second transformation matrix and the first transformation matrix to obtain the projection matrix.

[0099] In one embodiment, the first transformation matrix for converting the two-dimensional coordinates of the pixel points in the multi-view profile map to the three-dimensional coordinates in the world coordinate system is calculated according to the observation parameters, comprising: calculating the front orthogonal basis vector of the virtual camera according to the line-of-sight direction vector of the virtual camera; calculating the right orthogonal basis vector of the virtual camera according to the front orthogonal basis vector and the upward direction vector of the virtual camera; calculating the upward orthogonal basis vector of the virtual camera according to the right orthogonal basis vector and the front orthogonal basis vector, and the front orthogonal basis vector, the right orthogonal basis vector, and the upward orthogonal basis vector of the virtual camera constitute the camera coordinate system of the virtual camera; calculating the first transformation matrix according to the position of the virtual camera and the front orthogonal basis vector, the right orthogonal basis vector, and the upward orthogonal basis vector of the virtual camera, and the first transformation matrix is used to convert the camera coordinate system to the world coordinate system.

[0100] To calculate the projection matrix, first calculate the three-dimensional coordinates (x, y, z, 1) corresponding to each pixel point in the multi-view contour map, which can be calculated by the observation parameters of the virtual camera. Assuming that the position of the virtual camera is (Cx, Cy, Cz), the direction of the virtual camera is (Dx, Dy, Dz), the upward direction of the virtual camera is (Ux, Uy, Uz), the view angle of the virtual camera is θ, the near plane distance of the virtual camera is n, the far plane distance of the virtual camera is f, and the width-height ratio of the virtual camera is r, a 4x4 first transformation matrix V is calculated according to the above parameters, which satisfies the following conditions:

[0101] For any pixel point (u, v) in the binary image, the three-dimensional coordinates corresponding to it are (x, y, z, 1), where x, y, z satisfy the following formula:

[0102] V satisfies (x, y, z, 1) = V (Cx, Cy, Cz, 1), that is, the first transformation matrix V can convert the position coordinates of the camera into three-dimensional coordinates. Where w and h are the width and height of the multi-view contour map, u and v are the horizontal and vertical coordinates of the pixel point, and tan(θ / 2) is the tangent value of the half vertex angle of the virtual camera's view cone. To calculate the matrix V, first calculate the three orthogonal basis vectors of the virtual camera, that is, the right orthogonal basis vector of the virtual camera, the upper right orthogonal basis vector and the front right orthogonal basis vector, which are R=(R x ,R y ,R z ), U=(U x ,U y ,U z ), F=(F x ,F y ,F z ), which satisfy the following formula: U=RxF

[0103] Where × represents vector cross product, ||·|| represents the length of the vector, and D=(D x ,D y ,D z ) is the view direction vector of the virtual camera. According to the position of the virtual camera and the three orthogonal basis vectors, the first transformation matrix V can be calculated, and the formula is as follows:

[0104] In one embodiment, the light source information includes a position of the virtual light source, a light source direction vector, and based on the light source information, a second transformation matrix for converting a three-dimensional coordinate into a projection coordinate on the reference plane is calculated, including: calculating a front orthogonal basis vector of the virtual light source based on the light source direction vector of the virtual light source; calculating a right orthogonal basis vector of the virtual light source based on the right orthogonal basis vector of the virtual camera; calculating an upper orthogonal basis vector of the virtual light source based on the front orthogonal basis vector of the virtual light source and the front orthogonal basis vector of the virtual light source, the front orthogonal basis vector, the right orthogonal basis vector and the upper orthogonal basis vector of the virtual light source constitute a light source coordinate system of the virtual light source; and calculating the second transformation matrix based on the position of the virtual light source and the front orthogonal basis vector, the right orthogonal basis vector and the upper orthogonal basis vector of the virtual light source, the second transformation matrix being used for converting a world coordinate system into a projection coordinate system corresponding to the reference plane.

[0105] In order to calculate the projection coordinate (x', y', z', 1) of the three-dimensional coordinate on the reference plane, it can be calculated based on the light source information of the virtual light source. Assuming that the position of the virtual light source is (Lx, Ly, Lz), the light source direction vector of the virtual light source is (Sx, Sy, Sz), the viewing angle of the virtual light source is φ, the near plane distance of the virtual light source is n', the far plane distance of the virtual light source is f', and the aspect ratio of the virtual light source is r', based on these parameters, a 4x4 matrix second transformation matrix P' is calculated, and the second transformation matrix P' satisfies the following conditions at the same time:

[0106] For a three-dimensional coordinate (x, y, z, 1), its projection coordinate on the reference plane is (x', y', z', 1), wherein x', y', z' satisfy the following formula:

[0107] Wherein, tan(φ / 2) is the tangent value of the half vertex angle of the virtual light source. The second transformation matrix P' satisfies (x', y', z', 1) = P'(x, y, z, 1), that is, P' can convert the three-dimensional coordinate into the projection coordinate.

[0108] In order to calculate P', three orthogonal basis vectors of the virtual light source need to be calculated first, that is, the right orthogonal basis vector of the virtual light source, the upper right orthogonal basis vector and the front right orthogonal basis vector, which are R' = (Rx', Ry', Rz'), U' = (Ux', Uy', Uz'), F' = (Fx', Fy', Fz') respectively, and they satisfy the following formula: U' = R' x F'

[0109] Wherein, S = (Sx, Sy, Sz) is the light source direction vector of the virtual light source.

[0110] Then, based on the position of the virtual light source and the three orthogonal basis vectors, the second transformation matrix P' can be calculated, and the formula is as follows:

[0111] The first transformation matrix V and the second transformation matrix P' are multiplied to obtain a final projection matrix P, as follows:

[0112] P = P'V

[0113] According to the projection matrix P, each pixel point in the multi-view contour map can be mapped to a position on the reference plane, thereby generating a hard shadow image.

[0114] In one embodiment, as shown in FIG. 9, step 208, according to the light source information and the material properties of the three-dimensional model, the shadow features of the three-dimensional model are obtained, including:

[0115] Step 902, the coordinates of each vertex in the three-dimensional model are obtained, and the normal vector corresponding to each vertex is calculated according to the coordinates of the adjacent vertices.

[0116] Suppose the vertex set of the three-dimensional model is represented by C, C is an n x 3 matrix, each row represents the x, y, z coordinates of a vertex. According to the coordinates of each vertex in the vertex set, the normal vector of each vertex can be calculated. The normal vectors of all vertices can reflect the shape and details of the three-dimensional model, as well as the incidence and reflection angles of the light. Therefore, using the normal vector as the shadow feature of the three-dimensional model can help improve the quality of the generated soft shadow image.

[0117] According to the vertex set corresponding to the three-dimensional model, adjacent vertices can be found, and the normal vector of each vertex can be calculated according to the coordinates of the adjacent vertices, as follows:

[0118] where × represents the cross product of vectors, ∥·∥ represents the length of the vector, N(i) represents the neighbor set of vertex i, and k is any vertex not equal to i and j. An n x 3 matrix N can be used to store the normal vectors of all vertices, with each row representing the normal vector of a vertex.

[0119] Step 904, the light source radius of the virtual light source and the light source distance from the virtual light source to the three-dimensional model are obtained, and the light source size of the virtual light source is calculated according to the light source radius and the light source distance.

[0120] The light source size of the virtual light source can affect the blur degree of the shadow. The larger the light source, the more blurred the shadow. Therefore, the light source size of the virtual light source is also a factor that can affect shadow generation. The computer device can calculate the light source size of the virtual light source according to the light source radius of the virtual light source and the light source distance from the virtual light source to the three-dimensional model, as follows: S = 2r / d;

[0121] where r is the radius or width of the light source, d is the light source distance from the light source to the three-dimensional model, and S is the light source size of the virtual light source, that is, the scalar S can store the light source size of the virtual light source.

[0122] In step 906, the material attribute of the three-dimensional model is obtained, the material attribute representing the diffuse reflectance, the specular reflectance, the roughness, and the metallicity corresponding to each vertex in the three-dimensional model, and the comprehensive reflectance corresponding to each vertex is calculated according to the diffuse reflectance, the specular reflectance, the roughness, and the metallicity corresponding to each vertex.

[0123] The reflectance refers to the reflection ability of the material to light, which can affect the light and shade of the shadow, and the higher the reflectance, the darker the shadow. The computer device can calculate the reflectance of each vertex according to the diffuse reflectance, the specular reflectance, the roughness, and the metallicity of the material on the surface of the three-dimensional model, and the formula is as follows:

[0124] M is an n x 4 matrix, each row representing the diffuse reflectance, the specular reflectance, the roughness, and the metallicity of a vertex, is the diffuse reflectance of vertex i, is the specular reflectance of vertex i, θ is the angle between the light ray of the virtual light source and the normal vector corresponding to vertex i, and Ri is the reflectance of vertex i. The computer device can store the reflectance information of all vertices using an n x 1 vector R, each row representing the reflectance of a vertex.

[0125] In step 909, the shadow feature corresponding to the three-dimensional model is obtained according to the normal vector corresponding to each vertex, the light source size of the virtual light source, and the corresponding comprehensive reflectance.

[0126] In an embodiment, in order to use these information as input of the shadow prediction model together with the hard shadow image, the computer device can also convert these information to have the same resolution and format as the hard shadow image, that is, a w x h x c tensor, where w and h are the width and height of the hard shadow image, and c is the number of channels. Alternatively, the computer device can use some interpolation or resampling methods to map the features from the vertex space to the pixel space, which is not limited in the embodiments of the present application. For example, it is assumed that there is a function F that can convert these information to have the same resolution and format as the hard shadow image, and the formula is as follows: Fi = F(Ni, S, Ri)

[0127] where Ni is the normal vector of vertex i, S is the light source size of the virtual light source, Ri is the reflectance information of vertex i, and Fi is the shadow feature map of pixel i. The computer device can store the shadow features of all pixels using a w x h x c tensor F, and thus the extraction of the shadow features is completed.

[0128] In this embodiment, in order to generate a soft shadow image, some factors affecting the shape and intensity of the shadow are considered, such as the normal vector of the three-dimensional model, the light source size of the virtual light source, the material attribute of the three-dimensional model, etc. By extracting some features based on these factors and providing them as input together with the hard shadow image to the shadow prediction model based on the convolutional neural network, the quality of the generated soft shadow image can be improved.

[0129] In one embodiment, step 210, the hard shadow image and the shadow feature are input into the trained shadow prediction model based on the convolutional neural network. The shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model according to the hard shadow image and the shadow feature, including the following steps:

[0130] After the shadow feature is mapped from the vertex space to the pixel space to obtain a shadow feature map with the same data size as the hard shadow image, the hard shadow image and the shadow feature map are spliced by the input layer of the shadow prediction model to obtain an input tensor, and the input tensor is input into the trained shadow prediction model based on the convolutional neural network;

[0131] The input tensor is sequentially processed by the encoder of the shadow prediction model to obtain high-resolution encoded features, and the encoder includes a plurality of alternatingly connected dilated convolution layers and pooling layers.

[0132] The high-resolution encoded features are sequentially processed by the decoder of the shadow prediction model to obtain low-resolution decoded features, and the decoder includes a plurality of alternatingly connected dilated convolution layers and up-sampling layers.

[0133] The low-resolution decoded features are converted into an output tensor with the same data size as the hard shadow image by the output layer of the shadow prediction model, and the output tensor is a soft shadow image corresponding to the three-dimensional model.

[0134] The input layer of the shadow prediction model receives the hard shadow image and the shadow feature map as input, and splices them into a w x h x (c + 1) tensor, where w and h are the width and height of the hard shadow image, c is the number of channels of the feature, and +1 is the number of channels of the hard shadow image.

[0135] The encoder of the shadow prediction model is composed of a plurality of dilated convolution layers and pooling layers, which gradually down-samples the tensor of the input layer to extract high-level features and increase the size of the receptive field. By using different dilation rates, each dilated convolution layer can cover different spatial ranges, thereby increasing the diversity and flexibility of convolution. By using batch normalization and activation functions, the non-linear expression capability can be enhanced.

[0136] The decoder of the shadow prediction model is composed of several deconvolutional layers and up-sampling layers, which gradually up-sample the tensor of the encoder to restore the original resolution while preserving high-level features. By using the opposite deconvolutional rate as the encoder, each deconvolutional layer can be matched with the corresponding layer of the encoder to achieve a skip connection, which fuses low-level and high-level features. By using batch normalization and activation functions, the non-linear expression capability is enhanced.

[0137] The output layer of the shadow prediction model is a deconvolutional layer, which converts the tensor of the decoder into a w x h x 1 tensor, i.e., a soft shadow image. By using a convolutional kernel with a deconvolutional rate of 1, the output layer can maintain the same resolution as the input layer, and by using the hyperbolic tangent function as the activation function, the output value is limited between -1 and 1, representing the intensity of the shadow.

[0138] By concatenating the hard shadow image and the shadow feature map by channel, the trained reference prediction model can directly output the soft shadow image corresponding to the three-dimensional model.

[0139] In one embodiment, the training steps of the shadow prediction model based on the convolutional neural network include the following steps:

[0140] Obtain a plurality of training samples, each training sample including a hard shadow image corresponding to a three-dimensional model, a shadow feature, and a sample soft shadow image corresponding to the hard shadow image;

[0141] For each training sample, input the corresponding shadow feature map into the initial shadow prediction model, and output the corresponding predicted soft shadow image through the shadow prediction model;

[0142] For each training sample, calculate the difference between the corresponding predicted soft shadow image and the sample soft shadow image, and construct a loss function according to the differences corresponding to the plurality of samples;

[0143] Update the shadow prediction model according to the loss function.

[0144] In this embodiment, in order to train the above-mentioned shadow prediction model, training sample data is needed, i.e., a hard shadow image corresponding to a three-dimensional model, a shadow feature, and a sample soft shadow image corresponding to the hard shadow image. The hard shadow image corresponding to the three-dimensional model and the shadow feature are used as the input of the shadow prediction model, and the sample soft shadow image is used to compare with the predicted soft shadow image output by the model to update the shadow prediction model. Among them, the sample soft shadow image corresponding to the hard shadow image of the three-dimensional model can be generated using some soft shadow generation algorithm. During the training process, some data enhancement methods such as rotation, scaling, cropping, noise, etc. can be used to increase the diversity and robustness of the training data.

[0145] The input data and the sample soft shadow image in the plurality of training samples are denoted by X and Y respectively, where X is an n x w x h x (c+1) tensor, each row represents the input data of a training sample, i.e. the combination of the hard shadow image and the shadow feature, and Y is an n x w x h x 1 tensor, each row represents the soft shadow image in a training sample. According to X and Y, the model parameters of the shadow prediction model based on the dilated convolutional neural network can be optimized, so that it can map X to Y as much as possible, i.e. minimize the following loss function:

[0146] Where F represents the shadow prediction model based on the dilated convolutional neural network, and L represents the loss function. Some optimization algorithms such as stochastic gradient descent, Adam, etc. can be used to update the model parameters of the shadow prediction model, so as to reduce the value of the loss function. Suppose there is a function O that can update the model parameters of the shadow prediction model according to X and Y, the formula is as follows: O(X,Y)=arg min FL;

[0147] Where O represents the optimization function, and argminF represents finding the F that minimizes L among all possible values of F.

[0148] The loss function can be a mean square error, which can measure the difference between the output predicted soft shadow image and the sample soft shadow image, and the formula is as follows:

[0149] Where Oij is the pixel value of the i-th row and j-th column of the output predicted soft shadow image, Tij is the pixel value of the i-th row and j-th column of the sample soft shadow image, and L is the value of the loss function.

[0150] In a specific embodiment, the shadow generation method of the three-dimensional model can be executed by a computer device, comprising the following steps:

[0151] 1. Obtain light source information of a virtual light source and observation parameters of a virtual camera, the light source information is used to form the lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form the field of view range of the virtual camera in the virtual scene for the three-dimensional model;

[0152] 2. Convert the three-dimensional model into a triangular mesh composed of triangular patches, calculate the projection matrix of the three-dimensional model under different viewing angles according to the light source information and the observation parameters, and the different viewing angles are different rotation angles of the three-dimensional model when rotating around the specified coordinate axis;

[0153] 3. For each viewing angle, use the corresponding projection matrix to project the triangular mesh onto a two-dimensional plane to obtain a two-dimensional polygon under the corresponding viewing angle;

[0154] 4. For each view angle, an edge contour line of the two-dimensional polygon under the corresponding view angle is extracted to obtain a binary contour image of the three-dimensional model under the corresponding view angle, the binary contour image being an image obtained by mapping the three-dimensional model to a two-dimensional plane and performing binary distinction on the image foreground and image background mapped to the two-dimensional plane;

[0155] 5. The binary contour images corresponding to the view angles are subjected to logical OR operation to obtain a multi-view contour graph corresponding to the three-dimensional model, the multi-view contour graph being used to represent an initial shadow;

[0156] 6. A first transformation matrix for converting two-dimensional coordinates of a pixel point in the multi-view contour graph into three-dimensional coordinates in a world coordinate system is calculated according to observation parameters; a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on a reference plane is calculated according to light source information; and the second transformation matrix is multiplied by the first transformation matrix to obtain a projection matrix;

[0157] 7. The multi-view contour graph is projected onto the reference plane using the projection matrix to obtain a hard shadow image corresponding to the three-dimensional model;

[0158] 8. Coordinates of each vertex in the three-dimensional model are obtained, a normal vector corresponding to each vertex is calculated according to coordinates of adjacent vertices, a light source radius of a virtual light source and a light source distance of the virtual light source to the three-dimensional model are obtained, a light source size of the virtual light source is calculated according to the light source radius and the light source distance, material properties of the three-dimensional model are obtained, the material properties representing diffuse reflectivity, specular reflectivity, roughness and metallicity corresponding to each vertex in the three-dimensional model, a comprehensive reflectivity corresponding to each vertex is calculated according to the diffuse reflectivity, the specular reflectivity, the roughness and the metallicity corresponding to each vertex, and a shadow feature corresponding to the three-dimensional model is obtained according to the normal vector corresponding to each vertex, the light source size of the virtual light source and the comprehensive reflectivity corresponding to each vertex;

[0159] 9. The hard shadow image and the shadow feature are input into a trained shadow prediction model based on a convolutional neural network, and a soft shadow image corresponding to the three-dimensional model is output by the shadow prediction model according to the hard shadow image and the shadow feature.

[0160] As shown in FIG. 10, it is a schematic diagram of the framework of the shadow generation method of the three-dimensional model in an embodiment. Referring to FIG. 10, the schematic diagram includes a plurality of modules. First, multi-view silhouette shadow extraction: for a three-dimensional model, render its silhouettes from different viewpoints, and then combine these silhouettes into a binary image, i.e. multi-view silhouette shadow, in which the model part is black and the background part is white, which can be used as the initial shape of the shadow. Second, hard shadow image generation: according to the light source information and camera observation parameters, project the multi-view silhouette shadow onto the reference plane to obtain a hard shadow image, which can be used as the initial intensity of the shadow. Then, shadow prediction model: design a model based on a dilated convolutional neural network, the input is the hard shadow image and the shadow feature corresponding to the three-dimensional model, and the output is a soft shadow image. The goal of this network is to learn how to generate more realistic and natural soft shadow effects according to the shape of the model, the light source information of the virtual light source, the observation parameters of the virtual camera, and the material properties of the three-dimensional model. Finally, soft shadow image generation: use the improved dilated convolutional neural network to convert the hard shadow image of each three-dimensional model to obtain a soft shadow image, which can be used as the final result of the shadow and fused with the virtual scene.

[0161] The shadow generation method of the three-dimensional model can realize fast shadow map generation only by relying on the black and white map or texture of the three-dimensional model, and the corresponding shadow map is non-colored. Only by combining rendering with the shadow map, the rendering effect can be perfectly achieved. By combining the rendering program and the bottom map or texture color superposition, the shadow effect can be efficiently displayed and rendered.

[0162] It should be understood that although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0163] Based on the same inventive concept, the embodiments of the present application also provide a device for generating shadows of a three-dimensional model, which is used to implement the method for generating shadows of a three-dimensional model described above. The device provides a solution to the problem similar to the solution described in the method, and therefore the specific limitations in one or more device embodiments for generating shadows of a three-dimensional model provided below can refer to the limitations of the method for generating shadows of a three-dimensional model described above, which will not be described here again.

[0164] In one embodiment, as shown in FIG. 11, a device 1100 for generating shadows of a three-dimensional model is provided, which includes an acquisition module 1102, a contour extraction module 1104, a hard shadow generation module 1106, a shadow feature extraction module 1108, and a prediction module 1110, wherein:

[0165] The acquisition module 1102 is configured to acquire light source information of a virtual light source and observation parameters of a virtual camera, the light source information being used to form an illumination effect of the virtual light source in a virtual scene, and the observation parameters being used to form a field of view range of the virtual camera on the three-dimensional model in the virtual scene.

[0166] The contour extraction module 1104 is configured to generate a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters, the multi-view contour map being used to represent an initial shadow.

[0167] The hard shadow generation module 1106 is configured to project the multi-view contour map to a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model.

[0168] The shadow feature extraction module 1108 is configured to obtain a shadow feature of the three-dimensional model according to the light source information and a material attribute of the three-dimensional model.

[0169] The prediction module 1110 is configured to input the hard shadow image and the shadow feature into a trained shadow prediction model based on a convolutional neural network, and output a soft shadow image corresponding to the three-dimensional model by the shadow prediction model according to the hard shadow image and the shadow feature.

[0170] In one embodiment, the contour extraction module 1104 is further configured to extract a binary contour image of the three-dimensional model from multiple views based on the light source information and the observation parameters, the binary contour image being an image obtained by mapping the three-dimensional model to a two-dimensional plane and performing binary distinction on an image foreground and an image background mapped to the two-dimensional plane; and merge the binary contour images corresponding to the multiple views to obtain the multi-view contour map corresponding to the three-dimensional model.

[0171] In an embodiment, the contour extraction module 1104 is further configured to convert the three-dimensional model into a triangular mesh composed of triangular facets; calculate projection matrices of the three-dimensional model under different viewing angles according to the light source information and the observation parameters, the different viewing angles being different rotation angles of the three-dimensional model centered on a specified coordinate axis; for each viewing angle, project the triangular mesh onto a two-dimensional plane using a corresponding projection matrix to obtain a two-dimensional polygon under the corresponding viewing angle; and for each viewing angle, extract an edge contour line of the two-dimensional polygon under the corresponding viewing angle to obtain a binary contour image of the three-dimensional model under the corresponding viewing angle.

[0172] In an embodiment, the contour extraction module 1104 is further configured to perform a logical OR operation on the binary contour images corresponding to the different viewing angles to obtain a multi-view contour map corresponding to the three-dimensional model.

[0173] In an embodiment, the contour extraction module 1104 is further configured to obtain a vertex set and a triangular facet set of the three-dimensional model, each triangular facet in the triangular facet set being represented by three vertex indices, and the three vertices identified by the three vertex indices all belonging to the vertex set; and generate a triangular mesh according to the vertex set and the triangular facet set.

[0174] In an embodiment, the contour extraction module 1104 is further configured to, for each triangular facet in the triangular facet set, generate a corresponding normal vector, the normal vector being a unit vector perpendicular to the triangular facet; determine whether a facet direction of the triangular facet is counterclockwise according to a direction vector of the light source and the normal vector, the facet direction being determined by the three vertices constituting the triangular facet; when the facet direction of the triangular facet is not counterclockwise, exchange the order of the last two vertices in the triangular facet to obtain a triangular facet with a counterclockwise facet direction; and store the vertex set and the triangular facet with the counterclockwise facet direction into a triangular mesh object to obtain a triangular mesh corresponding to the three-dimensional model.

[0175] In an embodiment, the hard shadow generation module 1106 is further configured to calculate a projection matrix of the multi-view contour map corresponding to the reference plane according to the light source information and the observation parameters; and project the multi-view contour map onto the reference plane using the projection matrix to obtain a hard shadow image corresponding to the three-dimensional model.

[0176] In an embodiment, the hard shadow generation module 1106 is further configured to calculate a first transformation matrix for converting two-dimensional coordinates of a pixel point in the multi-view contour map into three-dimensional coordinates in a world coordinate system according to the observation parameters; calculate a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane according to the light source information; and calculate the projection matrix according to the first transformation matrix and the second transformation matrix.

[0177] In an embodiment, the hard shadow generation module 1106 is further configured to calculate a front orthogonal basis vector of the virtual camera according to the view line direction vector of the virtual camera; calculate a right orthogonal basis vector of the virtual camera according to the front orthogonal basis vector and an upward direction vector of the virtual camera; calculate an upward orthogonal basis vector of the virtual camera according to the right orthogonal basis vector and the front orthogonal basis vector, the front orthogonal basis vector, the right orthogonal basis vector and the upward orthogonal basis vector of the virtual camera forming a camera coordinate system of the virtual camera; and calculate a first transformation matrix according to the position of the virtual camera and the front orthogonal basis vector, the right orthogonal basis vector and the upward orthogonal basis vector of the virtual camera, the first transformation matrix being used to convert the camera coordinate system into the world coordinate system.

[0178] In an embodiment, the light source information includes a position of the virtual light source and a light source direction vector of the virtual light source, and the hard shadow generation module 1106 is further configured to calculate a front orthogonal basis vector of the virtual light source according to the light source direction vector of the virtual light source; calculate a right orthogonal basis vector of the virtual light source according to the right orthogonal basis vector of the virtual camera; calculate an upward orthogonal basis vector of the virtual light source according to the front orthogonal basis vector of the virtual light source and the front orthogonal basis vector of the virtual light source, the front orthogonal basis vector, the right orthogonal basis vector and the upward orthogonal basis vector of the virtual light source forming a light source coordinate system of the virtual light source; and calculate a second transformation matrix according to the position of the virtual light source and the front orthogonal basis vector, the right orthogonal basis vector and the upward orthogonal basis vector of the virtual light source, the second transformation matrix being used to convert the world coordinate system into a projection coordinate system corresponding to the reference plane.

[0179] In an embodiment, the hard shadow generation module 1106 is further configured to multiply the second transformation matrix and the first transformation matrix to obtain a projection matrix.

[0180] In an embodiment, the shadow feature extraction module 1108 is further configured to obtain coordinates of each vertex in the three-dimensional model, calculate a normal vector corresponding to each vertex according to coordinates of adjacent vertices; obtain a light source radius of the virtual light source and a light source distance from the virtual light source to the three-dimensional model, calculate a light source size of the virtual light source according to the light source radius and the light source distance; obtain material properties of the three-dimensional model, the material properties representing diffuse reflectance, specular reflectance, roughness and metallicity corresponding to each vertex in the three-dimensional model, calculate a comprehensive reflectance corresponding to each vertex according to the diffuse reflectance, the specular reflectance, the roughness and the metallicity corresponding to each vertex; and obtain a shadow feature corresponding to the three-dimensional model according to the normal vector corresponding to each vertex, the light source size of the virtual light source and the corresponding comprehensive reflectance.

[0181] In an embodiment, the prediction module 1110 is further configured to map the shadow feature from the vertex space to the pixel space to obtain a shadow feature map with the same size as the hard shadow image data, concatenate the hard shadow image and the shadow feature map by channel through an input layer of the shadow prediction model to obtain an input tensor, input the input tensor into the trained shadow prediction model based on the convolutional neural network, perform feature encoding and feature down-sampling on the input tensor successively through an encoder of the shadow prediction model to obtain high-resolution encoded features, the encoder comprising a plurality of alternatively connected hollow convolutional layers and pooling layers, perform feature decoding and feature up-sampling on the high-resolution encoded features successively through a decoder of the shadow prediction model to obtain low-resolution decoded features, the decoder comprising a plurality of alternatively connected hollow convolutional layers and up-sampling layers, and convert the low-resolution decoded features into an output tensor with the same size as the hard shadow image data through an output layer of the shadow prediction model, the output tensor being a soft shadow image corresponding to the three-dimensional model.

[0182] In an embodiment, the shadow generation apparatus 1100 further comprises:

[0183] a training module configured to obtain a plurality of training samples, each training sample comprising a hard shadow image corresponding to a three-dimensional model, a shadow feature, and a sample soft shadow image corresponding to the hard shadow image, input the corresponding hard shadow image and shadow feature map into an initial shadow prediction model for each training sample, output a corresponding predicted soft shadow image through the shadow prediction model, calculate the difference between the corresponding predicted soft shadow image and the sample soft shadow image for each training sample, construct a loss function according to the differences corresponding to the plurality of samples, and update the shadow prediction model according to the loss function.

[0184] The shadow generation apparatus 1100 of the three-dimensional model obtains light source information of a virtual light source and observation parameters of a virtual camera, the light source information is used to represent a lighting effect of the virtual light source in a virtual scene, and the observation parameters are used to represent a field of view range of the virtual camera to the three-dimensional model in the virtual scene. According to the light source information and the observation parameters, a multi-view contour map corresponding to the three-dimensional model is generated, and the multi-view contour map is used to represent an initial shadow. According to the light source information and the observation parameters, the multi-view contour map is projected to a reference plane to obtain a hard shadow image corresponding to the three-dimensional model. According to the light source information and material properties of the three-dimensional model, a shadow feature of the three-dimensional model is obtained. The hard shadow image and the shadow feature are input into a trained shadow prediction model based on a convolutional neural network. According to the hard shadow image and the shadow feature, the shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model. The soft shadow image can be used to generate a more realistic and natural soft shadow effect. That is, only one low-consumption multi-view contour map generation process is needed, and then a hard shadow image of the three-dimensional model is generated according to the multi-view contour map. According to the hard shadow image and the shadow feature, the shadow effect of the three-dimensional model can be efficiently generated, the data processing amount during shadow generation is reduced, and the shadow generation efficiency is improved.

[0185] The modules in the shadow generation apparatus of the three-dimensional model can be realized by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the modules.

[0186] In an embodiment, a computer device is provided, which can be a server. An internal structure diagram of the computer device can be as shown in FIG. 12. The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for running the operating system and the computer readable instructions in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. The computer readable instructions are executed by the processor to implement a shadow generation method of a three-dimensional model.

[0187] In one embodiment, a computer device is provided, which can be a terminal, and its internal structure diagram can be as shown in FIG. 13. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer readable instructions. The internal memory provides an environment for the operating system and computer readable instructions in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer readable instructions are executed by the processor to implement a shadow generation method of a three-dimensional model. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0188] Those skilled in the art can understand that the structures shown in FIG. 12 and FIG. 13 are only block diagrams of part of the structures related to the scheme of the present application, and do not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0189] In one embodiment, a computer device is provided, which includes a memory and a processor, the memory stores computer readable instructions, and the processor executes the computer readable instructions to implement the steps of the shadow generation method of the three-dimensional model provided by the embodiments of the present application.

[0190] In one embodiment, a computer readable storage medium is provided, which stores computer readable instructions, and the computer readable instructions are executed by the processor to implement the steps of the shadow generation method of the three-dimensional model provided by the embodiments of the present application.

[0191] In one embodiment, a computer program product is provided, which includes computer readable instructions, and the computer readable instructions are executed by the processor to implement the steps of the shadow generation method of the three-dimensional model provided by the embodiments of the present application.

[0192] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions.

[0193] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through computer readable instructions, and the computer readable instructions can be stored in a non-volatile computer readable storage medium. When the computer readable instructions are executed, the processes of the above-mentioned embodiments of the methods can be included. In the embodiments provided in the present application, any reference to memory, database or other medium can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0194] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0195] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for generating a shadow of a three-dimensional model, executed by a computer device, the method comprising: Acquiring light source information of a virtual light source and observation parameters of a virtual camera, wherein the light source information is used to form a lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form a field of view of the virtual camera on the three-dimensional model in the virtual scene; generating a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters, wherein the multi-view contour map is used to represent an initial shadow; Projecting the multi-view contour image onto a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model; Obtaining shadow features of the three-dimensional model according to the light source information and material properties of the three-dimensional model; The hard shadow image and the shadow features are input into a trained shadow prediction model based on a convolutional neural network, and the shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model according to the hard shadow image and the shadow features.

2. The method according to claim 1, characterized in that Generating a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters includes: Extracting a binary contour image of the three-dimensional model from multiple viewing angles based on the light source information and the observation parameters, wherein the binary contour image is an image in which the three-dimensional model is mapped onto a two-dimensional plane and a foreground image and a background image mapped onto the two-dimensional plane are binary-differentiated; The binary contour images corresponding to each viewing angle are merged to obtain a multi-view contour image corresponding to the three-dimensional model.

3. The method according to claim 2, characterized in that The step of extracting a binary contour image of the three-dimensional model from multiple perspectives based on the light source information and the observation parameters includes: Converting the three-dimensional model into a triangular mesh composed of triangular facets; Calculating, based on the light source information and the observation parameters, a projection matrix of the three-dimensional model at different viewing angles, wherein the different viewing angles are different rotation angles when the three-dimensional model is rotated about a specified coordinate axis; For each viewing angle, project the triangular mesh onto the two-dimensional plane using a corresponding projection matrix to obtain a two-dimensional polygon at the corresponding viewing angle; For each viewing angle, edge contour lines of the two-dimensional polygon at the corresponding viewing angle are extracted to obtain a binary contour image of the three-dimensional model at the corresponding viewing angle.

4. The method according to claim 3, characterized in that The step of merging the binary contour images corresponding to the respective viewing angles to obtain a multi-view contour image corresponding to the three-dimensional model includes: A logical OR operation is performed on the binary contour images corresponding to each viewing angle to obtain a multi-view contour image corresponding to the three-dimensional model.

5. The method according to claim 3, characterized in that The converting of the three-dimensional model into a triangular mesh composed of triangular facets comprises: Obtaining a vertex set and a triangle face set of the three-dimensional model, where each triangle face in the triangle face set is represented by three vertex indices, and the three vertices identified by the three vertex indices all belong to the vertex set; The triangular mesh is generated according to the vertex set and the triangular facet set.

6. The method according to claim 5, characterized in that Generating the triangular mesh according to the vertex set and the triangular facet set includes: For each triangle in the set of triangles, generate a corresponding normal vector, where the normal vector is a unit vector perpendicular to the triangle; Determining whether a facet direction of the triangular facet is counterclockwise according to a direction vector of the light source and the normal vector, wherein the facet direction is determined by three vertices constituting the triangular facet; When the face direction of the triangular face is not counterclockwise, the order of the last two vertices in the triangular face is swapped to obtain a triangular face with a counterclockwise face direction; The vertex set and the counterclockwise triangular facets are stored in a triangular mesh object to obtain a triangular mesh corresponding to the three-dimensional model.

7. The method according to claim 1, characterized in that The step of projecting the multi-view contour image onto a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model includes: Calculating a projection matrix of the multi-view profile image corresponding to the reference plane according to the light source information and the observation parameters; The multi-view contour image is projected onto the reference plane using the projection matrix to obtain a hard shadow image corresponding to the three-dimensional model.

8. The method according to claim 7, characterized in that Calculating the projection matrix corresponding to the multi-view profile image on the reference plane according to the light source information and the observation parameters includes: Calculating a first transformation matrix for converting the two-dimensional coordinates of the pixel points in the multi-view contour image into three-dimensional coordinates in a world coordinate system according to the observation parameters; Calculating, according to the light source information, a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane; A projection matrix is ​​calculated according to the first transformation matrix and the second transformation matrix.

9. The method according to claim 8, characterized in that The step of calculating, based on the observation parameters, a first transformation matrix for converting the two-dimensional coordinates of the pixel points in the multi-view contour image into three-dimensional coordinates in a world coordinate system comprises: Calculate the front orthogonal basis vector of the virtual camera according to the sight direction vector of the virtual camera; Calculating a right orthogonal basis vector of the virtual camera according to the front orthogonal basis vector and the upper vector of the virtual camera; Calculating an upper orthogonal basis vector of the virtual camera according to the right orthogonal basis vector and the front orthogonal basis vector, wherein the front orthogonal basis vector, the right orthogonal basis vector and the upper orthogonal basis vector constitute a camera coordinate system of the virtual camera; A first transformation matrix is ​​calculated based on the position of the virtual camera and the front orthogonal basis vector, the right orthogonal basis vector and the top orthogonal basis vector of the virtual camera. The first transformation matrix is ​​used to convert the camera coordinate system into a world coordinate system.

10. The method according to claim 8, characterized in that The light source information includes the position of the virtual light source and the light source direction vector. The calculating, based on the light source information, a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane includes: Calculating the forward orthogonal basis vectors of the virtual light source according to the light source direction vector of the virtual light source; Calculating the right orthogonal basis vectors of the virtual light source according to the right orthogonal basis vectors of the virtual camera; Calculating an upper orthogonal basis vector of the virtual light source according to the front orthogonal basis vector of the virtual light source and the front orthogonal basis vector of the virtual light source, wherein the front orthogonal basis vector of the virtual light source, the right orthogonal basis vector and the upper orthogonal basis vector constitute a light source coordinate system of the virtual light source; A second transformation matrix is ​​calculated based on the position of the virtual light source and the front orthogonal basis vector of the virtual light source, the right orthogonal basis vector and the upper orthogonal basis vector. The second transformation matrix is ​​used to convert the world coordinate system into a projection coordinate system corresponding to the reference plane.

11. The method according to claim 8, characterized in that The calculating the projection matrix according to the first transformation matrix and the second transformation matrix includes: The second transformation matrix is ​​multiplied by the first transformation matrix to obtain the projection matrix.

12. The method according to any one of claims 1 to 11, characterized in that Obtaining the shadow features of the three-dimensional model according to the light source information and the material properties of the three-dimensional model includes: Obtaining the coordinates of each vertex in the three-dimensional model, and calculating the normal vector corresponding to each vertex based on the coordinates of adjacent vertices; Obtaining a light source radius of the virtual light source and a light source distance from the virtual light source to the three-dimensional model, and calculating a light source size of the virtual light source according to the light source radius and the light source distance; Obtaining material properties of the three-dimensional model, where the material properties represent diffuse reflectivity, specular reflectivity, roughness, and metallicity corresponding to each vertex in the three-dimensional model, and calculating a comprehensive reflectivity corresponding to each vertex based on the diffuse reflectivity, specular reflectivity, roughness, and metallicity corresponding to each vertex; The shadow feature corresponding to the three-dimensional model is obtained according to the normal vector corresponding to each vertex, the light source size of the virtual light source and the corresponding comprehensive reflectivity.

13. The method according to any one of claims 1 to 11, characterized in that Inputting the hard shadow image and the shadow features into a trained shadow prediction model based on a convolutional neural network, and outputting a soft shadow image corresponding to the three-dimensional model according to the hard shadow image and the shadow features by the shadow prediction model, comprises: After mapping the shadow features from vertex space to pixel space to obtain a shadow feature map having the same size as the hard shadow image data, the hard shadow image and the shadow feature map are concatenated channel by channel through the input layer of the shadow prediction model to obtain an input tensor, and the input tensor is input into the trained convolutional neural network-based shadow prediction model; The encoder of the shadow prediction model successively performs feature encoding and feature downsampling processing on the input tensor to obtain high-resolution encoded features, wherein the encoder includes a plurality of alternately connected dilated convolutional layers and pooling layers; The high-resolution encoded features are sequentially decoded and upsampled by a decoder of the shadow prediction model to obtain low-resolution decoded features, wherein the decoder includes a plurality of alternately connected dilated convolutional layers and upsampling layers; The low-resolution decoded features are converted into an output tensor having the same size as the hard shadow image data through the output layer of the shadow prediction model, and the output tensor is a soft shadow image corresponding to the three-dimensional model.

14. The method according to any one of claims 1 to 11, characterized in that The training steps of the shadow prediction model based on the convolutional neural network include: Acquire multiple training samples, each training sample including a hard shadow image corresponding to a three-dimensional model, a shadow feature, and a sample soft shadow image corresponding to the hard shadow image; For each training sample, the corresponding hard shadow image and shadow feature map are input into the initial shadow prediction model, and the shadow prediction model outputs the corresponding predicted soft shadow image; For each training sample, calculating the difference between the corresponding predicted soft shadow image and the sample soft shadow image, and constructing a loss function based on the difference corresponding to the multiple samples; The shadow prediction model is updated according to the loss function.

15. A shadow generating device for a three-dimensional model, the device comprising: an acquisition module, configured to acquire light source information of a virtual light source and observation parameters of a virtual camera, wherein the light source information is used to form a lighting effect of the virtual light source in a virtual scene, and the observation parameters are used to form a field of view of the virtual camera on a three-dimensional model in the virtual scene; a contour extraction module, configured to generate a multi-view contour map corresponding to the three-dimensional model based on the light source information and the observation parameters, wherein the multi-view contour map is used to represent an initial shadow; a hard shadow generation module, configured to project the multi-view contour image onto a reference plane according to the light source information and the observation parameters, to obtain a hard shadow image corresponding to the three-dimensional model; a shadow feature extraction module, configured to obtain shadow features of the three-dimensional model based on the light source information and the material properties of the three-dimensional model; A prediction module is used to input the hard shadow image and the shadow features into a trained shadow prediction model based on a convolutional neural network, and output a soft shadow image corresponding to the three-dimensional model based on the hard shadow image and the shadow features through the shadow prediction model.

16. The device according to claim 15, characterized in that The contour extraction module is also used to extract the binary contour image of the three-dimensional model from multiple perspectives based on the light source information and the observation parameters. The binary contour image is an image in which the three-dimensional model is mapped onto a two-dimensional plane and the image foreground and image background mapped onto the two-dimensional plane are binary-distinguished. The binary contour images corresponding to each perspective are merged to obtain a multi-perspective contour map corresponding to the three-dimensional model.

17. The device according to claim 16, characterized in that The contour extraction module is further configured to convert the three-dimensional model into a triangular mesh composed of triangular facets; and calculate, based on the light source information and the observation parameters, a projection matrix of the three-dimensional model at different viewing angles, wherein the different viewing angles are different rotation angles when the three-dimensional model is rotated about a specified coordinate axis. For each viewing angle, the triangular mesh is projected onto the two-dimensional plane using the corresponding projection matrix to obtain a two-dimensional polygon at the corresponding viewing angle; for each viewing angle, the edge contour line of the two-dimensional polygon at the corresponding viewing angle is extracted to obtain a binary contour image of the three-dimensional model at the corresponding viewing angle.

18. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the method according to any one of claims 1 to 14 when executing the computer-readable instructions.

19. A computer-readable storage medium having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.

20. A computer program product comprising computer readable instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 14.

Images