Shadow generation method and device for three-dimensional model, equipment, medium and program product

Abstract

The invention relates to a shadow generation method and device for a three-dimensional model, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring light source information of a virtual light source and observation parameters of a virtual camera; according to the light source information and the observation parameters, a multi-view profile diagram corresponding to the three-dimensional model is generated, and the multi-view profile diagram is used for representing an initial shadow; projecting the multi-view profile diagram to a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model; obtaining shadow features of the three-dimensional model according to the light source information and material attributes of the three-dimensional model; and inputting the hard shadow image and the shadow features into a trained shadow prediction model based on a convolutional neural network, and outputting a soft shadow image corresponding to the three-dimensional model through the shadow prediction model according to the hard shadow image and the shadow features. By adopting the method, the shadow of the three-dimensional model can be quickly generated.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for generating shadows of a three-dimensional model. Background Art

[0002] Adjusting shadow effects is an important step in adding realism and depth to 3D models. Appropriate shadows can make 3D models more three-dimensional and enhance visualization. Typically, shadow effects are applied based on the light sources in the virtual scene. When rendering virtual scenes in games, many 3D models use lighting shadow rendering. However, due to the influence of the machine model or the order in which the 3D models are rendered, shadow rendering may fail.

[0003] In related technologies, a geometric shadow map algorithm based on triangle reconstruction is often used to reconstruct shadows of three-dimensional models in virtual scenes. However, this method requires a lot of calculations, has low shadow generation efficiency, and is difficult to achieve real-time shadow reconstruction in virtual scenes. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for generating shadows of three-dimensional models in order to address the above technical problems, so as to quickly generate shadows of three-dimensional models.

[0005] In a first aspect, the present application provides a method for generating shadows of a three-dimensional model. The method comprises:

[0006] Acquiring light source information of a virtual light source and observation parameters of a virtual camera, wherein the light source information is used to form a lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form a field of view of the virtual camera on the three-dimensional model in the virtual scene;

[0007] generating a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters, wherein the multi-view contour map is used to represent an initial shadow;

[0008] Projecting the multi-view contour image onto a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model;

[0009] Obtaining shadow features of the three-dimensional model according to the light source information and material properties of the three-dimensional model;

[0010] The hard shadow image and the shadow features are input into a trained shadow prediction model based on a convolutional neural network, and the shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model according to the hard shadow image and the shadow features.

[0011] In a second aspect, the present application further provides a device for generating shadows of a three-dimensional model. The device comprises:

[0012] an acquisition module, configured to acquire light source information of a virtual light source and observation parameters of a virtual camera, wherein the light source information is used to form a lighting effect of the virtual light source in a virtual scene, and the observation parameters are used to form a field of view of the virtual camera on a three-dimensional model in the virtual scene;

[0013] a contour extraction module, configured to generate a multi-view contour map corresponding to the three-dimensional model based on the light source information and the observation parameters, wherein the multi-view contour map is used to represent an initial shadow;

[0014] a hard shadow generation module, configured to project the multi-view contour image onto a reference plane according to the light source information and the observation parameters, to obtain a hard shadow image corresponding to the three-dimensional model;

[0015] a shadow feature extraction module, configured to obtain shadow features of the three-dimensional model based on the light source information and the material properties of the three-dimensional model;

[0016] A prediction module is used to input the hard shadow image and the shadow features into a trained shadow prediction model based on a convolutional neural network, and output a soft shadow image corresponding to the three-dimensional model based on the hard shadow image and the shadow features through the shadow prediction model.

[0017] In one embodiment, the contour extraction module is also used to extract the binary contour image of the three-dimensional model from multiple perspectives based on the light source information and the observation parameters. The binary contour image is an image in which the three-dimensional model is mapped onto a two-dimensional plane and the image foreground and image background mapped onto the two-dimensional plane are binary-distinguished. The binary contour images corresponding to each perspective are merged to obtain a multi-perspective contour map corresponding to the three-dimensional model.

[0018] In one embodiment, the contour extraction module is also used to convert the three-dimensional model into a triangular mesh composed of triangular facets; based on the light source information and the observation parameters, the projection matrix of the three-dimensional model at different viewing angles is calculated, and the different viewing angles are different rotation angles when the three-dimensional model is rotated around the specified coordinate axis; for each viewing angle, the corresponding projection matrix is ​​used to project the triangular mesh onto the two-dimensional plane to obtain a two-dimensional polygon at the corresponding viewing angle; for each viewing angle, the edge contour line of the two-dimensional polygon at the corresponding viewing angle is extracted to obtain a binary contour image of the three-dimensional model at the corresponding viewing angle.

[0019] In one embodiment, the contour extraction module is further configured to perform a logical OR operation on the binary contour images corresponding to each viewing angle to obtain a multi-view contour image corresponding to the three-dimensional model.

[0020] In one embodiment, the contour extraction module is also used to obtain a vertex set and a triangle face set of the three-dimensional model, each triangle face in the triangle face set is represented by three vertex indices, and the three vertices identified by the three vertex indices all belong to the vertex set; and the triangular mesh is generated based on the vertex set and the triangle face set.

[0021] In one embodiment, the contour extraction module is also used to generate a corresponding normal vector for each triangular facet in the triangular facet set, where the normal vector is a unit vector perpendicular to the triangular facet; based on the direction vector of the light source and the normal vector, determine whether the facet direction of the triangular facet is counterclockwise, and the facet direction is determined by the three vertices that constitute the triangular facet; when the facet direction of the triangular facet is not counterclockwise, exchange the order of the last two vertices in the triangular facet to obtain a triangular facet with a counterclockwise facet direction; store the vertex set and the counterclockwise triangular facet in a triangular mesh object to obtain a triangular mesh corresponding to the three-dimensional model.

[0022] In one embodiment, the hard shadow generation module is also used to calculate the projection matrix corresponding to the multi-perspective contour image on the reference plane based on the light source information and the observation parameters; and use the projection matrix to project the multi-perspective contour image onto the reference plane to obtain a hard shadow image corresponding to the three-dimensional model.

[0023] In one embodiment, the hard shadow generation module is further used to calculate a first transformation matrix for converting the two-dimensional coordinates of the pixel points in the multi-view contour image into three-dimensional coordinates in the world coordinate system based on the observation parameters; calculate a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane based on the light source information; and calculate a projection matrix based on the first transformation matrix and the second transformation matrix.

[0024] In one embodiment, the hard shadow generation module is further used to calculate the front orthogonal basis vectors of the virtual camera based on the line of sight direction vector of the virtual camera; calculate the right orthogonal basis vectors of the virtual camera based on the front orthogonal basis vectors and the top vector of the virtual camera; calculate the top orthogonal basis vectors of the virtual camera based on the right orthogonal basis vectors and the front orthogonal basis vectors, and the front orthogonal basis vectors, the right orthogonal basis vectors and the top orthogonal basis vectors of the virtual camera constitute the camera coordinate system of the virtual camera; calculate the first transformation matrix based on the position of the virtual camera and the front orthogonal basis vectors, the right orthogonal basis vectors and the top orthogonal basis vectors of the virtual camera, and the first transformation matrix is ​​used to convert the camera coordinate system into the world coordinate system.

[0025] In one embodiment, the light source information includes the position and light source direction vector of the virtual light source, and the hard shadow generation module is further used to calculate the front orthogonal basis vectors of the virtual light source based on the light source direction vector of the virtual light source; calculate the right orthogonal basis vectors of the virtual light source based on the right orthogonal basis vectors of the virtual camera; calculate the upper orthogonal basis vectors of the virtual light source based on the front orthogonal basis vectors of the virtual light source and the front orthogonal basis vectors of the virtual light source, and the front orthogonal basis vectors of the virtual light source, the right orthogonal basis vectors and the upper orthogonal basis vectors constitute the light source coordinate system of the virtual light source; calculate the second transformation matrix based on the position of the virtual light source and the front orthogonal basis vectors of the virtual light source, the right orthogonal basis vectors and the upper orthogonal basis vectors, and the second transformation matrix is ​​used to convert the world coordinate system into the projection coordinate system corresponding to the standard plane.

[0026] In one embodiment, the hard shadow generation module is further configured to multiply the second transformation matrix by the first transformation matrix to obtain the projection matrix.

[0027] In one embodiment, the shadow feature extraction module is also used to obtain the coordinates of each vertex in the three-dimensional model, and calculate the normal vector corresponding to each vertex based on the coordinates of adjacent vertices; obtain the light source radius of the virtual light source and the light source distance from the virtual light source to the three-dimensional model, and calculate the light source size of the virtual light source based on the light source radius and the light source distance; obtain the material properties of the three-dimensional model, and the material properties represent the diffuse reflectivity, specular reflectivity, roughness and metallicity corresponding to each vertex in the three-dimensional model, and calculate the comprehensive reflectivity corresponding to each vertex based on the diffuse reflectivity, specular reflectivity, roughness and metallicity corresponding to each vertex; obtain the shadow features corresponding to the three-dimensional model based on the normal vector corresponding to each vertex, the light source size of the virtual light source and the corresponding comprehensive reflectivity.

[0028] In one embodiment, the prediction module is also used to map the shadow features from vertex space to pixel space, and after obtaining a shadow feature map with the same size as the hard shadow image data, the hard shadow image and the shadow feature map are spliced ​​by channel through the input layer of the shadow prediction model to obtain an input tensor, and the input tensor is input into a trained shadow prediction model based on a convolutional neural network; the input tensor is successively feature encoded and down-sampled through the encoder of the shadow prediction model to obtain high-resolution encoded features, and the encoder includes a plurality of hole convolution layers and pooling layers that are alternately connected in sequence; the high-resolution encoded features are successively feature decoded and up-sampled through the decoder of the shadow prediction model to obtain low-resolution decoded features, and the decoder includes a plurality of hole convolution layers and upsampling layers that are alternately connected in sequence; the low-resolution decoded features are converted into an output tensor with the same size as the hard shadow image data through the output layer of the shadow prediction model, and the output tensor is the soft shadow image corresponding to the three-dimensional model.

[0029] In one embodiment, the apparatus further comprises:

[0030] A training module is used to obtain multiple training samples, each training sample includes a hard shadow image, shadow features and a sample soft shadow image corresponding to the three-dimensional model; for each training sample, the corresponding hard shadow image and shadow feature map are input into an initial shadow prediction model, and the corresponding predicted soft shadow image is output through the shadow prediction model; for each training sample, the difference between the corresponding predicted soft shadow image and the sample soft shadow image is calculated, and a loss function is constructed according to the difference corresponding to the multiple samples; and the shadow prediction model is updated according to the loss function.

[0031] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0032] Acquiring light source information of a virtual light source and observation parameters of a virtual camera, wherein the light source information is used to form a lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form a field of view of the virtual camera on the three-dimensional model in the virtual scene;

[0033] generating a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters, wherein the multi-view contour map is used to represent an initial shadow;

[0034] Projecting the multi-view contour image onto a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model;

[0035] Obtaining shadow features of the three-dimensional model according to the light source information and material properties of the three-dimensional model;

[0036] The hard shadow image and the shadow features are input into a trained shadow prediction model based on a convolutional neural network, and the shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model according to the hard shadow image and the shadow features.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0038] Acquiring light source information of a virtual light source and observation parameters of a virtual camera, wherein the light source information is used to form a lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form a field of view of the virtual camera on the three-dimensional model in the virtual scene;

[0039] generating a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters, wherein the multi-view contour map is used to represent an initial shadow;

[0040] Projecting the multi-view contour image onto a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model;

[0041] Obtaining shadow features of the three-dimensional model according to the light source information and material properties of the three-dimensional model;

[0042] The hard shadow image and the shadow features are input into a trained shadow prediction model based on a convolutional neural network, and the shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model according to the hard shadow image and the shadow features.

[0043] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0044] Acquiring light source information of a virtual light source and observation parameters of a virtual camera, wherein the light source information is used to form a lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form a field of view of the virtual camera on the three-dimensional model in the virtual scene;

[0045] generating a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters, wherein the multi-view contour map is used to represent an initial shadow;

[0046] Projecting the multi-view contour image onto a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model;

[0047] Obtaining shadow features of the three-dimensional model according to the light source information and material properties of the three-dimensional model;

[0048] The hard shadow image and the shadow features are input into a trained shadow prediction model based on a convolutional neural network, and the shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model according to the hard shadow image and the shadow features.

[0049] The shadow generation method, device, computer equipment, storage medium and computer program product of the above-mentioned three-dimensional model obtain light source information of the virtual light source and observation parameters of the virtual camera, the light source information is used to represent the lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to represent the field of view of the virtual camera on the three-dimensional model in the virtual scene. According to the light source information and observation parameters, a multi-perspective contour map corresponding to the three-dimensional model is generated, and the multi-perspective contour map is used to represent the initial shadow; according to the light source information and observation parameters, the multi-perspective contour map is projected onto the reference plane to obtain a hard shadow image corresponding to the three-dimensional model, and according to the light source information and the material properties of the three-dimensional model, the shadow features of the three-dimensional model are obtained, and the hard shadow image and shadow features are input into a trained shadow prediction model based on a convolutional neural network. The shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model based on the hard shadow image and shadow features. The soft shadow image can be used to generate a more realistic and natural soft shadow effect. In other words, only one low-consumption multi-view contour map generation process is required, and then a hard shadow image of the three-dimensional model is generated based on the multi-view contour map. The shadow effect of the three-dimensional model can be efficiently generated based on the hard shadow image and shadow features, which reduces the amount of data processing during shadow generation and improves the efficiency of shadow generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A diagram illustrating an application environment of a method for generating a shadow of a three-dimensional model in one embodiment;

[0051] Figure 2 1 is a flow chart of a method for generating a shadow of a three-dimensional model in one embodiment;

[0052] Figure 3 is a schematic diagram of hard shadows and soft shadows in one embodiment;

[0053] Figure 4 A schematic diagram of a process for generating a multi-view contour map corresponding to a three-dimensional model based on light source information and observation parameters in one embodiment;

[0054] Figure 5is a schematic diagram of binary contour images corresponding to different viewing angles of a three-dimensional model in one embodiment;

[0055] Figure 6 is a schematic diagram of a multi-view contour map corresponding to a three-dimensional model in one embodiment;

[0056] Figure 7 A schematic diagram of a process for extracting a binary contour image from multiple perspectives in one embodiment;

[0057] Figure 8 1 is a schematic diagram of a process for projecting a multi-view contour image onto a reference plane to obtain a hard shadow image in one embodiment;

[0058] Figure 9 1 is a flow chart of obtaining a shadow feature map of a three-dimensional model based on light source information and material properties of the three-dimensional model in one embodiment;

[0059] Figure 10 Schematic diagram of a framework of a method for generating shadows of a three-dimensional model in one embodiment;

[0060] Figure 11 is a structural block diagram of a shadow generating device for a three-dimensional model in one embodiment;

[0061] Figure 12 is a diagram of the internal structure of a computer device in one embodiment;

[0062] Figure 13 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0065] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and smart transportation.

[0066] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0067] The solutions provided in the embodiments of this application involve technologies such as convolutional neural networks of artificial intelligence, which are specifically illustrated by the following embodiments.

[0068] The shadow generation method of the three-dimensional model provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store data that server 104 needs to process. The data storage system can be integrated with server 104, or placed on a cloud or other server. Terminal 102 can obtain light source information of a virtual light source and observation parameters of a virtual camera. The light source information is used to represent the lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to represent the field of view of the virtual camera on the three-dimensional model in the virtual scene. Based on the light source information and observation parameters, a multi-view contour map corresponding to the three-dimensional model is generated. The multi-view contour map is used to represent the initial shadow. Based on the light source information and observation parameters, the multi-view contour map is projected onto a reference plane to obtain a hard shadow image corresponding to the three-dimensional model. Based on the light source information and material properties of the three-dimensional model, shadow features of the three-dimensional model are obtained. The hard shadow image and shadow features are input into a trained shadow prediction model based on a convolutional neural network. The shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model based on the hard shadow image and shadow features. Of course, the above-mentioned shadow generation method for the three-dimensional model can also be performed by a server. The terminal 102 may obtain the generated soft shadow image from the server 104. Optionally, the terminal 102 may also render a shadow effect based on the soft shadow image to make the display of the three-dimensional model in the virtual scene more realistic and natural.

[0069] Terminal 102 may include, but is not limited to, various desktop computers, laptops, smartphones, tablet computers, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers.

[0070] In one embodiment, terminal 102 has an application (or client) installed and running that supports virtual scenes. This application can be a competition or athletic program. When terminal 110 runs this application, the application interface is displayed on the screen of terminal 102, which displays the virtual scene. Optionally, terminal 102 can be used by the current player. The current player can use terminal 110 to control the activities of virtual objects in the virtual scene. These virtual objects can be virtual people, virtual animals, virtual animated characters, etc., and are implemented as three-dimensional models.

[0071] In one embodiment, Figure 2 As shown, a shadow generation method for a three-dimensional model is provided, and the method is applied to Figure 1 Taking the computer device (such as terminal 102 or server 104) in the example as an example, the following steps are included:

[0072] Step 202 : Obtain light source information of a virtual light source and observation parameters of a virtual camera. The light source information is used to form the lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form the field of view of the virtual camera on the three-dimensional model in the virtual scene.

[0073] In 3D graphics, virtual light sources are crucial for generating shadows. By simulating the interaction of light with objects, virtual light sources determine the intensity and color of the illumination they receive. Using virtual light sources, high-quality shadows can be achieved, significantly enhancing the quality and realism of virtual scenes. Typical virtual light source types include ambient light, directional light, point light, and spotlight. Light source information is used to create the lighting effects of virtual light sources within a virtual scene. This information includes at least one of the following: the position of the light source; its direction; its viewing angle; its near-plane distance; its far-plane distance; and its aspect ratio. This information determines the illumination pattern, angle, and intensity distribution of the light, thus influencing shadow generation. By adjusting the position and angle of the virtual light source, the shape and length of the shadow can be altered, enhancing the three-dimensionality and realism of the 3D model. The intensity of the virtual light source also affects the brightness of the shadow.

[0074] In 3D graphics modeling, a virtual camera can simulate real-world camera mechanisms, achieving better results during modeling, rendering, and animation. Observation parameters are used to define the virtual camera's field of view of the 3D model in the virtual scene. These parameters include at least one of the following: camera position, i.e., the location of the virtual camera; camera viewpoint; camera sight vector, i.e., the ray from the camera viewpoint to the observation target point; camera up vector, which specifies the virtual camera's up direction; camera angle of view; camera near plane distance; camera far plane distance; and camera aspect ratio.

[0075] A virtual scene is a virtual screen displayed when an application is running on a terminal. The virtual scene can be an imitation of a real scene, a virtual scene obtained by fusion of a fictional scene and an imitation of a real scene, or a purely fictional scene.

[0076] During the 3D graphics modeling process, users can edit and set the light source information of the virtual light source and the observation parameters of the virtual camera in the view. Therefore, the computer device can obtain the light source information of the virtual light source and the observation parameters of the virtual camera from the setting parameters of the view.

[0077] Step 204 : generating a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters. The multi-view contour map is used to represent the initial shadow.

[0078] The multi-perspective contour map is an image formed by the contour lines of a three-dimensional model from multiple different perspectives. The multi-perspective contour map reflects the shape and structure of the three-dimensional model and can be used to represent the initial shadow of the three-dimensional model. The computer device can sum the contour lines of the three-dimensional model from different perspectives generated by the GPU-based renderer based on the light source information of the virtual light source and the observation parameters of the virtual camera to obtain a multi-perspective contour map from various angles. Since rendering a contour shadow once on the GPU only requires a small amount of power and can be completed in a few milliseconds, the generation efficiency is high. With the multi-perspective contour map, the multi-perspective contour map is used as input data for soft shadow generation, that is, the initial shape of the shadow, which can ensure the effect of the subsequently generated soft shadow image. Moreover, the entire process of generating the soft shadow image will also be very efficient.

[0079] Step 206 : Project the multi-view contour image onto a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model.

[0080] In order to obtain the spatial relationship between the three-dimensional model and the reference plane, after obtaining the multi-view contour map, the computer equipment projects the multi-view contour map onto the reference plane according to the light source information and observation parameters to obtain the hard shadow image corresponding to the three-dimensional model.

[0081] The hardness of the shadow is an important factor affecting the shadow effect. The hardness of the shadow is related to the size and shape of the virtual light source in the virtual scene. By adjusting the hardness of the shadow, the shadow effect can be made more realistic. The hard shadow represented by the hard shadow image has very clear edges, no gradient or blur effect, and the grayscale of the shadow part is very uniform. The hard shadow image can reflect the shape and outline of the three-dimensional model. The soft shadow represented by the soft shadow image has a gradient or blur effect on the edges, and the grayscale of the shadow part is uneven, which can increase the depth and realism of the shadow. Figure 3 , which is a schematic diagram of hard shadows and soft shadows in one embodiment. It can be seen that the boundaries of hard shadows are relatively clear, while the grayscale of soft shadows is uneven and there is a shadow transition.

[0082] Optionally, after generating the hard shadow image, the computer device may further correct and optimize the hard shadow image according to the shape and material properties of the standard plane, such as eliminating aliasing, increasing shadow intensity, etc.

[0083] Step 208 : Obtain shadow features of the three-dimensional model based on the light source information and the material properties of the three-dimensional model.

[0084] The material of the three-dimensional model will have a certain impact on the shadow effect. Different materials have different reflection and absorption capabilities for light, which affects the brightness and texture of the shadow. The material properties include at least one of the following: diffuse reflectivity, specular reflectivity, roughness and metallicity of the surface material of the three-dimensional model. In the embodiment of the present application, considering the light source information of the virtual light source in the virtual scene and the material properties of the three-dimensional model, which will affect the shape and intensity of the shadow, some features are extracted therefrom, namely shadow features, and the hard shadow image of the three-dimensional model and the extracted shadow features are combined as the input for soft shadow prediction, that is, the soft shadow image is predicted based on the hard shadow image and the shadow features through the convolutional neural network model, which can ensure the quality of the reconstruction of the shadow of the three-dimensional model and obtain a more natural and realistic shadow effect.

[0085] In step 210 , the hard shadow image and shadow features are input into a trained shadow prediction model based on a convolutional neural network. The shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model based on the hard shadow image and shadow features.

[0086] The convolutional neural network-based shadow prediction model is capable of predicting soft shadow images based on hard shadow images and shadow features. This shadow prediction model is derived from the initial convolutional neural network through continuous learning and training based on training samples. The computer inputs the hard shadow images and shadow features into the trained convolutional neural network-based shadow prediction model, which then outputs a soft shadow image corresponding to the 3D model based on the hard shadow images and shadow features.

[0087] In related technologies, complex computer graphics can be used to render and process shadow effects based on 3D models and virtual scenes. In the embodiments of the present application, only hard shadow images and shadow features are used, and a shadow prediction model is used to quickly generate soft shadow images corresponding to the 3D model.

[0088] In one embodiment, further, the computer device runs a rendering program to render according to the soft shadow image, which can achieve high-quality, natural and realistic shadow effects. Optionally, the rendering is combined with the base map or texture map of the virtual scene to efficiently display natural shadow effects.

[0089] For example, a computer device can input the acquired soft shadow image as a shadow map into a resource collection in a debugging module within a game engine, enabling the game engine to generate a corresponding shadow effect for the three-dimensional model based on the soft shadow image. In an exemplary embodiment, the game engine overlays and renders the soft shadow image with a scene texture corresponding to the virtual scene, displaying the shadow effect within the virtual scene.

[0090] The above application scenarios are merely illustrative. The shadow generation method of the model can also be applied to the shadow generation process of models in other virtual scenes, and is not specifically limited here.

[0091] In the above-mentioned shadow generation method for a 3D model, light source information of a virtual light source and observation parameters of a virtual camera are obtained. The light source information is used to represent the lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to represent the field of view of the virtual camera on the 3D model in the virtual scene. Based on the light source information and observation parameters, a multi-view contour map corresponding to the 3D model is generated. The multi-view contour map represents the initial shadow. Based on the light source information and observation parameters, the multi-view contour map is projected onto a reference plane to obtain a hard shadow image corresponding to the 3D model. Shadow features of the 3D model are obtained based on the light source information and material properties of the 3D model. The hard shadow image and shadow features are input into a trained shadow prediction model based on a convolutional neural network. The shadow prediction model outputs a soft shadow image corresponding to the 3D model based on the hard shadow image and shadow features. The soft shadow image can be used to generate a more realistic and natural soft shadow effect. In other words, only a low-cost multi-view contour map generation process is required. Then, a hard shadow image of the 3D model is generated based on the multi-view contour map. Based on this hard shadow image and shadow features, the shadow effect of the 3D model can be efficiently generated, reducing the amount of data processing during shadow generation and improving shadow generation efficiency.

[0092] In one embodiment, Figure 4 As shown in the figure, according to the light source information and observation parameters, a multi-view contour map corresponding to the 3D model is generated, including:

[0093] Step 402: extracting a binary outline image of the 3D model from multiple viewing angles based on the light source information and the observation parameters. The binary outline image is an image in which the 3D model is mapped onto a 2D plane and the foreground and background of the image mapped onto the 2D plane are binary-separated.

[0094] The binary contour image is obtained by rendering the 3D model to obtain a 2D image, and then extracting the edge contour lines of the 2D image. The computer device can map the 3D model onto a 2D plane from multiple different perspectives of the 3D model to obtain a 2D image, and then extract the edge contour lines of each 2D image, that is, perform a binary distinction between the image foreground and the image background, thereby obtaining the binary contour images of each 3D model from multiple different perspectives. Figure 5 , which is a schematic diagram of binary contour images corresponding to a three-dimensional model at multiple different viewing angles in one embodiment.

[0095] Step 404 : merge the binary contour images corresponding to the respective viewing angles to obtain a multi-view contour image corresponding to the three-dimensional model.

[0096] In one embodiment, the binary contour images corresponding to each perspective are merged to obtain a multi-perspective contour map corresponding to the three-dimensional model, including: performing a logical OR operation on the binary contour images corresponding to each perspective to obtain a multi-perspective contour map corresponding to the three-dimensional model.

[0097] like Figure 6 FIG. 1 is a schematic diagram of a multi-view contour map corresponding to a three-dimensional model in an embodiment. Figure 6 ,The figure includes the outline of the three-dimensional model under different ,viewing angles, where the three-dimensional model part is painted black and the background part is painted white.

[0098] In this embodiment, by extracting the edge contour lines of the three-dimensional model from different perspectives and merging these edge contour lines into a binary image, the shape of the three-dimensional model can be accurately reflected, which can be used as input data for shadow generation, and helps to subsequently generate high-quality soft shadow images.

[0099] In one embodiment, Figure 7 As shown, based on the light source information and observation parameters, the binary contour images of the 3D model are extracted from multiple perspectives, including:

[0100] Step 702: Convert the three-dimensional model into a triangular mesh composed of triangular facets.

[0101] A triangular facet is a two-dimensional geometric shape consisting of three vertices and three edges, with each edge connecting two vertices to form a closed triangle. Triangular facets are commonly used to describe the surface of three-dimensional objects. Complex shapes can be constructed by combining a large number of adjacent triangular facets. A triangular mesh composed of triangular facets can be used to simulate the surface of complex objects. To facilitate subsequent projection transformations and image processing, computers can first convert a three-dimensional model into a triangular mesh, a mesh structure composed of a series of triangular facets. This can effectively preserve the shape and details of the three-dimensional model while also facilitating geometric operations and rendering.

[0102] Step 704 , calculating the projection matrix of the three-dimensional model at different viewing angles based on the light source information and the observation parameters. Different viewing angles are different rotation angles when the three-dimensional model is rotated around the specified coordinate axis.

[0103] Among them, the specified coordinate axis can be the longitudinal axis of the three-dimensional model. The three-dimensional model can be rotated with the longitudinal axis of the three-dimensional model as the rotation center, and the projection matrix corresponding to different rotation angles during the rotation process is calculated according to the light source information and observation parameters. The projection matrix can reflect the spatial relationship between the three-dimensional model and the two-dimensional plane at different viewing angles.

[0104] Step 706: For each viewing angle, use the corresponding projection matrix to project the triangular mesh onto a two-dimensional plane to obtain a two-dimensional polygon at the corresponding viewing angle.

[0105] Step 708 : For each viewing angle, extract the edge contour lines of the two-dimensional polygon at the corresponding viewing angle to obtain a binary contour image of the three-dimensional model at the corresponding viewing angle.

[0106] Computer equipment can use image processing algorithms to extract the edges of two-dimensional polygons to obtain a binary image. This binary image can reflect the edge contour of the three-dimensional model, which is called a binary contour image.

[0107] In this embodiment, by extracting edge contour lines of the three-dimensional model from different viewing angles, the shape of the three-dimensional model can be accurately reflected.

[0108] In one embodiment, step 702 converts the three-dimensional model into a triangular mesh composed of triangular facets, including: obtaining a vertex set and a triangular facet set of the three-dimensional model, each triangular face in the triangular facet set is represented by three vertex indices, and the three vertices identified by the three vertex indices all belong to the vertex set; generating a triangular mesh based on the vertex set and the triangular facet set.

[0109] For a 3D model, its corresponding vertices and triangles are known. The vertex set and triangle set are denoted by V and F, respectively. V is an n×3 matrix, with each row representing the coordinates (x, y, z) of a vertex. F is an m×3 matrix, with each row representing the indices of the three vertices of a triangle, i.e., vertex identifiers or vertex indices. The triangle mesh M generated from V and F must meet the following conditions: the vertex set of triangle mesh M is the same as that of V, i.e., M has n vertices, and the coordinates of each vertex are the same as the corresponding row in V; the triangle set of M is the same as that of F, i.e., M has m triangles, and the three vertices of each triangle are the same as the corresponding row in F; each triangle of M is a triangle, i.e., each row in F contains three different integers between 1 and n; and each triangle of M is oriented counterclockwise, i.e., each row (i, j, k) in F satisfies Vi×Vj•Vk>0, where i, j, k represent three different vertices, × denotes the vector cross product, and • denotes the vector dot product.

[0110] In one embodiment, a triangular mesh is generated based on a vertex set and a triangle face set, including: for each triangle face in the triangle face set, generating a corresponding normal vector, where the normal vector is a unit vector perpendicular to the triangle face; judging whether the face direction of the triangle face is counterclockwise based on the direction vector of the light source and the normal vector, where the face direction is determined by the three vertices that make up the triangle face; when the face direction of the triangle face is not counterclockwise, exchanging the order of the last two vertices in the triangle face to obtain a triangle face with a counterclockwise face direction; storing the vertex set and the counterclockwise triangle face in a triangular mesh object to obtain a triangular mesh corresponding to the three-dimensional model.

[0111] Specifically, first, the computer device calculates the normal vector corresponding to each triangle, that is, the unit vector perpendicular to the triangle. It can be calculated using the coordinates of the three vertices, and the formula is as follows:

[0112] ;

[0113] Among them, Vi, Vj and Vk represent the three-dimensional coordinates of the three vertices i, j, and k, and ||•|| represents the modulus of the vector, that is, the Euclidean distance.

[0114] Next, the computer device determines the direction of each triangle, that is, whether it is counterclockwise. This can be determined using the position and direction of the virtual light source. The formula is as follows:

[0115]

[0116] Where L represents the direction vector of the virtual light source. If D_ijk>0, it means that the triangle (i, j, k) is counterclockwise. Otherwise, if D_ijk≤0, the order of vertex j and vertex k is swapped to make it counterclockwise.

[0117] Then, the computer device stores the information of vertices and triangles in a triangle mesh object, which can be implemented using some data structures, such as adjacency lists, half-edge structures, etc. At this point, a triangle mesh object M is generated, which can store and access the information of vertices and triangles.

[0118] In this embodiment, by converting the three-dimensional model into a triangular mesh composed of triangular facets, the surface morphology of the three-dimensional model can be well simulated, and subsequent geometric operations and rendering are also facilitated.

[0119] In one embodiment, Figure 8 As shown, step 206, based on the light source information and observation parameters, projects the multi-view contour image onto the reference plane to obtain a hard shadow image corresponding to the three-dimensional model, including:

[0120] Step 802: Calculate the projection matrix corresponding to the multi-view profile image on the reference plane according to the light source information and the observation parameters.

[0121] The reference plane is a pre-defined plane, such as the ground in a virtual scene. Based on light source information and observation parameters, the computer can calculate the projection matrix of the multi-view profile image on the reference plane. This projection matrix can be used to reflect the spatial relationship between the 3D model and the reference plane.

[0122] Step 804 : Project the multi-view contour image onto a reference plane using a projection matrix to obtain a hard shadow image corresponding to the three-dimensional model.

[0123] Based on the projection matrix corresponding to the multi-view contour image on the reference plane, the computer device can project the multi-view contour image onto the reference plane. This method, based on projection transformation and image processing, can quickly generate a hard-shadow image of the 3D model from the multi-view contour image. This method does not require complex rendering and processing of the 3D model and virtual scene, and only requires a binary image as input, making it both efficient and convenient.

[0124] In one embodiment, step 802 calculates the projection matrix corresponding to the multi-view contour image on the reference plane based on the light source information and the observation parameters, including: calculating a first transformation matrix for converting the two-dimensional coordinates of the pixel points in the multi-view contour image into three-dimensional coordinates in the world coordinate system based on the observation parameters; calculating a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane based on the light source information; and calculating the projection matrix based on the first transformation matrix and the second transformation matrix.

[0125] To project the multi-view profile onto a standard plane, a projection matrix needs to be calculated that maps each pixel in the multi-view profile to a position on the standard plane. Assuming the standard plane is a plane, its equation is ax+by+cz+d=0, where a, b, c, and d are known constants. Assuming the light source information of the virtual light source and the observation parameters of the virtual camera are known, a 4×4 projection matrix P is calculated based on these parameters. The projection matrix P needs to satisfy the following conditions:

[0126] For any pixel point (u, v) in the multi-view contour map, its corresponding three-dimensional coordinates are (x, y, z, 1), where x, y, z are calculated based on the observation parameters of the virtual camera;

[0127] For the three-dimensional coordinate (x, y, z, 1), its projection coordinate on the standard plane is (x′, y′, z′, 1), where x′, y′, z′ are all calculated based on the light source information of the virtual light source.

[0128] That is, the projection matrix P can project any pixel point in the multi-view contour image to the standard plane.

[0129] The first transformation matrix is ​​used to transform the multi-view contour image in the camera coordinate system into the world coordinate system, and the second transformation matrix is ​​used to transform the world coordinate system into the projection coordinate system corresponding to the standard plane.

[0130] In one embodiment, calculating the projection matrix according to the first transformation matrix and the second transformation matrix includes: multiplying the second transformation matrix by the first transformation matrix to obtain the projection matrix.

[0131] In one embodiment, based on observation parameters, a first transformation matrix for converting the two-dimensional coordinates of pixel points in a multi-view contour image into three-dimensional coordinates in a world coordinate system is calculated, including: calculating the front orthogonal basis vectors of the virtual camera based on the line of sight direction vector of the virtual camera; calculating the right orthogonal basis vectors of the virtual camera based on the front orthogonal basis vectors and the top vector of the virtual camera; calculating the top orthogonal basis vectors of the virtual camera based on the right orthogonal basis vectors and the front orthogonal basis vectors, the front orthogonal basis vectors, the right orthogonal basis vectors and the top orthogonal basis vectors of the virtual camera constitute the camera coordinate system of the virtual camera; calculating the first transformation matrix based on the position of the virtual camera and the front orthogonal basis vectors, the right orthogonal basis vectors and the top orthogonal basis vectors of the virtual camera, the first transformation matrix is ​​used to convert the camera coordinate system into the world coordinate system.

[0132] In order to calculate the projection matrix, first calculate the three-dimensional coordinates (x, y, z, 1) corresponding to each pixel in the multi-view contour map, which can be calculated using the observation parameters of the virtual camera. Assume that the position of the virtual camera is (Cx, Cy, Cz), the direction of the virtual camera is (Dx, Dy, Dz), the up vector of the virtual camera is (Ux, Uy, Uz), the viewing angle of the virtual camera is θ, the near plane distance of the virtual camera is n, the far plane distance of the virtual camera is f, and the aspect ratio of the virtual camera is r. According to the above parameters, a 4×4 first transformation matrix V is calculated. The first transformation matrix V satisfies the following conditions:

[0133] For any pixel point (u, v) in a binary image, its corresponding three-dimensional coordinates are (x, y, z, 1), where x, y, and z satisfy the following formula:

[0134] ;

[0135] V satisfies (x, y, z, 1) = V (Cx, Cy, Cz, 1), that is, the first transformation matrix V can convert the camera's position coordinates into three-dimensional coordinates. Among them, w and h are the width and height of the multi-view contour map, u and v are the horizontal and vertical coordinates of the pixel point, and tan (θ / 2) is the tangent value of the half-vertex angle of the virtual camera's viewing cone. In order to calculate the matrix V, first calculate the three orthogonal basis vectors of the virtual camera, namely the right orthogonal basis vector of the virtual camera, the upper right orthogonal basis vector and the front right orthogonal basis vector, which are R = (R x ,R y ,R z ), U=(U x ,U y ,U z ), F=(F x ,F y ,F z ), which satisfy the following formula:

[0136]

[0137] Among them, × represents vector cross product, ||⋅|| represents the modulus of the vector, D=(D x ,D y ,D z ) is the sight direction vector of the virtual camera. According to the position of the virtual camera and the three orthogonal basis vectors, the first transformation matrix V can be calculated as follows:

[0138] .

[0139] In one embodiment, the light source information includes the position and direction vector of the virtual light source. According to the light source information, a second transformation matrix for converting three-dimensional coordinates into projection coordinates on a reference plane is calculated, including: calculating the front orthogonal basis vectors of the virtual light source according to the light source direction vector of the virtual light source; calculating the right orthogonal basis vectors of the virtual light source according to the right orthogonal basis vectors of the virtual camera; calculating the top orthogonal basis vectors of the virtual light source according to the front orthogonal basis vectors of the virtual light source and the front orthogonal basis vectors of the virtual light source, the front orthogonal basis vectors, the right orthogonal basis vectors and the top orthogonal basis vectors of the virtual light source constitute the light source coordinate system of the virtual light source; calculating the second transformation matrix according to the position of the virtual light source and the front orthogonal basis vectors, the right orthogonal basis vectors and the top orthogonal basis vectors of the virtual light source, the second transformation matrix is ​​used to convert the world coordinate system into the projection coordinate system corresponding to the standard plane.

[0140] To calculate the projection coordinates (x′, y′, z′, 1) of the three-dimensional coordinates on the reference plane, we can calculate them based on the light source information of the virtual light source. Assuming the position of the virtual light source is (Lx, Ly, Lz), the light source direction vector of the virtual light source is (Sx, Sy, Sz), the viewing angle of the virtual light source is φ, the near plane distance of the virtual light source is n′, the far plane distance of the virtual light source is f′, and the aspect ratio of the virtual light source is r′, based on these parameters, a 4×4 matrix second transformation matrix P′ is calculated. The second transformation matrix P′ satisfies the following conditions:

[0141] For a three-dimensional coordinate (x, y, z, 1), its projection coordinate on the reference plane is (x′, y′, z′, 1), where x′, y′, z′ satisfies the following formula:

[0142] ;

[0143] Where tan(ϕ / 2) is the tangent of the half-vertex angle of the virtual light source's viewing cone. The second transformation matrix P′ satisfies (x′, y′, z′, 1) = P′(x, y, z, 1), meaning that P′ can convert three-dimensional coordinates into projected coordinates.

[0144] In order to calculate P', it is necessary to first calculate the three orthogonal basis vectors of the virtual light source, namely the right orthogonal basis vector of the virtual light source, the upper right orthogonal basis vector and the front right orthogonal basis vector, which are R'=(Rx', Ry', Rz'), U'=(Ux', Uy', Uz'), and F'=(Fx', Fy', Fz'), which satisfy the following formula:

[0145]

[0146] Among them, S=(Sx,Sy,Sz) is the light source direction vector of the virtual light source.

[0147] Next, based on the position of the virtual light source and the three orthogonal basis vectors, the second transformation matrix P′ can be calculated as follows:

[0148] ;

[0149] Multiply the first transformation matrix V and the second transformation matrix P' to obtain the final projection matrix P, as shown in the following formula:

[0150]

[0151] According to the projection matrix P, each pixel in the multi-view contour image can be mapped to a position on the standard plane to generate a hard shadow image.

[0152] In one embodiment, Figure 9 As shown, step 208, obtaining the shadow features of the 3D model based on the light source information and the material properties of the 3D model, includes:

[0153] Step 902: Obtain the coordinates of each vertex in the three-dimensional model, and calculate the normal vector corresponding to each vertex based on the coordinates of adjacent vertices.

[0154] Assume that the vertex set of the 3D model is represented by C, which is an n×3 matrix. Each row represents the x, y, and z coordinates of a vertex. Based on the coordinates of each vertex in the vertex set, the normal vector of each vertex can be calculated. The normal vectors of all vertices can reflect the shape and details of the 3D model, as well as the incident and reflection angles of light. Therefore, using the normal vector as the shadow feature of the 3D model can help improve the quality of the generated soft shadow image.

[0155] According to the vertex set corresponding to the three-dimensional model, the adjacent vertices can be found. According to the coordinates of the adjacent vertices, the normal vector of each vertex can be calculated. The formula is as follows:

[0156] ;

[0157] Where × represents the vector cross product, ∥·∥ represents the vector modulus, N(i) represents the set of neighbors of vertex i, and k is any vertex not equal to i and j. We can use an n×3 matrix N to store the normal vectors of all vertices, with each row representing the normal vector of a vertex.

[0158] Step 904 : Obtain the light source radius of the virtual light source and the light source distance from the virtual light source to the three-dimensional model, and calculate the light source size of the virtual light source based on the light source radius and the light source distance.

[0159] The size of a virtual light source can affect the blurriness of shadows. The larger the light source, the blurrier the shadow. Therefore, the size of a virtual light source can also affect shadow generation. Computers can calculate the size of a virtual light source based on its radius and the distance from the virtual light source to the 3D model. The formula is as follows:

[0160] S=2r / d;

[0161] Among them, r is the radius or width of the light source, d is the light source distance from the light source to the three-dimensional model, and S is the light source size of the virtual light source, that is, the scalar S can store the light source size of the virtual light source.

[0162] Step 906: Obtain the material properties of the three-dimensional model. The material properties represent the diffuse reflectivity, specular reflectivity, roughness, and metallicity corresponding to each vertex in the three-dimensional model. Calculate the comprehensive reflectivity corresponding to each vertex based on the diffuse reflectivity, specular reflectivity, roughness, and metallicity corresponding to each vertex.

[0163] Reflectivity refers to a material's ability to reflect light, which affects the brightness of shadows. The higher the reflectivity, the darker the shadow. Computers can calculate the reflectivity of each vertex based on the diffuse reflectivity, specular reflectivity, roughness, and metallic properties of the material on the surface of a 3D model. The formula is as follows:

[0164] ;

[0165] M is an n×4 matrix, each row represents the diffuse reflectivity, specular reflectivity, roughness and metalness of a vertex, is the diffuse reflectance of vertex i, is the specular reflectivity of vertex i, θ is the angle between the light of the virtual light source and the normal vector corresponding to vertex i, and Ri is the reflectivity of vertex i. Computer devices can use an n×1 vector R to store the reflectivity information of all vertices, with each row representing the reflectivity of a vertex.

[0166] Step 909 : Obtain shadow features corresponding to the three-dimensional model according to the normal vector corresponding to each vertex, the light source size of the virtual light source, and the corresponding comprehensive reflectivity.

[0167] In one embodiment, in order to use this information together with the hard shadow image as input to the shadow prediction model, the computer device can also convert this information into a tensor with the same resolution and format as the hard shadow image, that is, a w×h×c tensor, where w and h are the width and height of the hard shadow image, and c is the number of channels. Optionally, the computer device can use some interpolation or resampling methods to map features from vertex space to pixel space, which is not limited in this embodiment of the present application. For example, suppose there is a function F that can convert this information into the same resolution and format as the hard shadow image, and the formula is as follows:

[0168] Fi=F(Ni,S,Ri)

[0169] Where Ni is the normal vector of vertex i, S is the size of the virtual light source, Ri is the reflectivity information of vertex i, and Fi is the shadow feature map of pixel i. The computer device can use a w×h×c tensor F to store the shadow features of all pixels, thereby completing the shadow feature extraction.

[0170] In this embodiment, in order to generate a soft shadow image, some factors affecting the shape and intensity of the shadow are considered, such as the normal vector of the three-dimensional model, the light source size of the virtual light source, the material properties of the three-dimensional model, etc. By extracting some features based on these factors and providing them together with the hard shadow image as input to the shadow prediction model based on the convolutional neural network, the quality of the generated soft shadow image can be improved.

[0171] In one embodiment, step 210, inputting the hard shadow image and shadow features into a trained shadow prediction model based on a convolutional neural network, and outputting a soft shadow image corresponding to the three-dimensional model based on the hard shadow image and shadow features by the shadow prediction model, includes the following steps:

[0172] After mapping the shadow features from vertex space to pixel space to obtain a shadow feature map with the same size as the hard shadow image data, the hard shadow image and the shadow feature map are concatenated channel by channel through the input layer of the shadow prediction model to obtain the input tensor. This input tensor is then input into the trained convolutional neural network-based shadow prediction model.

[0173] The encoder of the shadow prediction model performs feature encoding and feature downsampling on the input tensor one by one to obtain high-resolution encoded features. The encoder consists of multiple alternately connected dilated convolutional layers and pooling layers.

[0174] The high-resolution encoded features are decoded and upsampled one by one through the decoder of the shadow prediction model to obtain low-resolution decoded features. The decoder includes multiple alternately connected dilated convolutional layers and upsampling layers.

[0175] Through the output layer of the shadow prediction model, the low-resolution decoded features are converted into an output tensor with the same size as the hard shadow image data. The output tensor is the soft shadow image corresponding to the 3D model.

[0176] Among them, the input layer of the shadow prediction model receives the hard shadow image and the shadow feature map as input, and splices them into a w×h×(c+1) tensor, where w and h are the width and height of the hard shadow image, c is the number of channels of the feature, and +1 is the number of channels of the hard shadow image.

[0177] The shadow prediction model's encoder consists of several atrous convolutional layers and pooling layers. Its function is to gradually downsample the input tensor to extract high-level features while increasing the receptive field. By using different atrous rates, each atrous convolutional layer can cover different spatial ranges, thereby increasing the diversity and flexibility of convolution. The use of batch normalization and activation functions can enhance nonlinear expression capabilities.

[0178] The decoder of the shadow prediction model consists of several dilated convolutional layers and upsampling layers. Its function is to gradually upsample the encoder tensor to restore the original resolution while preserving high-level features. By using a dilation ratio opposite to that of the encoder, each dilated convolutional layer is matched to the corresponding layer of the encoder, thus achieving skip connections and fusing low-level and high-level features. Batch normalization and activation functions are used to enhance nonlinear expression capabilities.

[0179] The output layer of the shadow prediction model is a dilated convolutional layer, which converts the decoder's tensor into a w×h×1 tensor, representing the soft shadow image. Using a convolution kernel with a dilation ratio of 1 allows the output layer to maintain the same resolution as the input layer. The hyperbolic tangent function is used as the activation function, limiting the output value to between -1 and 1, representing the shadow intensity.

[0180] By splicing the hard shadow image and shadow feature map by channel and inputting them into the trained reference prediction model, the soft shadow image corresponding to the three-dimensional model can be directly output.

[0181] In one embodiment, the training step of the shadow prediction model based on the convolutional neural network includes the following steps:

[0182] Acquire multiple training samples, each training sample including a hard shadow image corresponding to the three-dimensional model, a shadow feature, and a sample soft shadow image corresponding to the hard shadow image;

[0183] For each training sample, the corresponding shadow feature map is input into the initial shadow prediction model, and the shadow prediction model outputs the corresponding predicted soft shadow image;

[0184] For each training sample, the difference between the corresponding predicted soft shadow image and the sample soft shadow image is calculated, and a loss function is constructed based on the differences corresponding to multiple samples;

[0185] Update the shadow prediction model according to the loss function.

[0186] In this embodiment, in order to train the above-mentioned shadow prediction model, it is necessary to prepare training sample data, namely, hard shadow images and shadow features corresponding to the three-dimensional model and sample soft shadow images corresponding to the hard shadow images. The hard shadow images and shadow features corresponding to the three-dimensional model serve as the input of the shadow prediction model, and the sample soft shadow images are used to compare with the predicted soft shadow images output by the model to update the shadow prediction model. Among them, the sample soft shadow images corresponding to the hard shadow images corresponding to the three-dimensional model can be generated using some soft shadow generation algorithms. During the training process, some data enhancement methods, such as rotation, scaling, cropping, noise, etc., can also be used to increase the diversity and robustness of the training data.

[0187] The input data and sample soft shadow images in multiple training samples are represented by X and Y, respectively, where X is a tensor of n×w×h×(c+1), each row of which represents the input data of a training sample, that is, the combination of hard shadow images and shadow features, and Y is a tensor of n×w×h×1, each row of which represents the soft shadow image in a training sample. Based on X and Y, the model parameters of the shadow prediction model based on the void convolutional neural network can be optimized so that it can map X to Y as much as possible, that is, minimize the following loss function:

[0188] ;

[0189] Where F represents the function of our shadow prediction model based on the dilated convolutional neural network, and L represents the loss function. We can use optimization algorithms, such as stochastic gradient descent and Adam, to update the model parameters of the shadow prediction model, thereby reducing the value of the loss function. Assume that there exists a function O that can update the model parameters of the shadow prediction model based on X and Y. The formula is as follows:

[0190] ;

[0191] Where O represents the optimization function, and argminF means finding the F that minimizes L among all possible values ​​of F.

[0192] The loss function can be a mean squared error, which measures the difference between the output predicted soft shadow image and the sample soft shadow image, and the formula is as follows:

[0193]

[0194] Among them, Oij is the pixel value of the i-th row and j-th column of the output predicted soft shadow image, Tij is the pixel value of the i-th row and j-th column of the sample soft shadow image, and L is the value of the loss function.

[0195] In a specific embodiment, the shadow generation method of the three-dimensional model includes the following steps:

[0196] 1. Obtain the light source information of the virtual light source and the observation parameters of the virtual camera. The light source information is used to form the lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form the field of view of the virtual camera on the 3D model in the virtual scene.

[0197] 2. Convert the 3D model into a triangular mesh composed of triangular facets. Based on the light source information and observation parameters, calculate the projection matrix of the 3D model at different viewing angles. Different viewing angles are different rotation angles when the 3D model is rotated around the specified coordinate axis.

[0198] 3. For each viewpoint, use the corresponding projection matrix to project the triangular mesh onto a two-dimensional plane to obtain a two-dimensional polygon at the corresponding viewpoint;

[0199] 4. For each viewing angle, extract the edge contour lines of the two-dimensional polygon at the corresponding viewing angle to obtain a binary contour image of the three-dimensional model at the corresponding viewing angle. The binary contour image is an image in which the three-dimensional model is mapped onto a two-dimensional plane and the foreground and background of the image mapped onto the two-dimensional plane are binary-separated.

[0200] 5. Perform a logical OR operation on the binary contour images corresponding to each perspective to obtain a multi-perspective contour map corresponding to the 3D model. The multi-perspective contour map is used to represent the initial shadow.

[0201] 6. Calculate, based on the observation parameters, a first transformation matrix for converting the two-dimensional coordinates of the pixel points in the multi-view profile image into three-dimensional coordinates in the world coordinate system; calculate, based on the light source information, a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane; multiply the second transformation matrix by the first transformation matrix to obtain a projection matrix;

[0202] 7. Use the projection matrix to project the multi-view contour image onto the reference plane to obtain the hard shadow image corresponding to the 3D model;

[0203] 8. Obtain the coordinates of each vertex in the three-dimensional model, and calculate the normal vector corresponding to each vertex based on the coordinates of adjacent vertices; obtain the light source radius of the virtual light source and the light source distance from the virtual light source to the three-dimensional model, and calculate the light source size of the virtual light source based on the light source radius and the light source distance; obtain the material properties of the three-dimensional model, which represent the diffuse reflectivity, specular reflectivity, roughness, and metallicity corresponding to each vertex in the three-dimensional model, and calculate the comprehensive reflectivity corresponding to each vertex based on the diffuse reflectivity, specular reflectivity, roughness, and metallicity corresponding to each vertex; obtain the shadow characteristics corresponding to the three-dimensional model based on the normal vector corresponding to each vertex, the light source size of the virtual light source, and the corresponding comprehensive reflectivity;

[0204] 9. Input the hard shadow image and shadow features into the trained shadow prediction model based on the convolutional neural network. The shadow prediction model outputs the soft shadow image corresponding to the three-dimensional model based on the hard shadow image and shadow features.

[0205] like Figure 10 FIG. 1 is a schematic diagram of a shadow generation method for a three-dimensional model in one embodiment. Figure 10 The framework diagram includes multiple modules. First, multi-view silhouette shadow extraction: For a 3D model, its silhouette is rendered from different viewpoints. These silhouettes are then merged into a binary image, the multi-view silhouette shadow, where the model is black and the background is white. This binary image serves as the initial shadow shape. Second, hard shadow image generation: Based on the light source information and camera observation parameters, the multi-view silhouette shadow is projected onto a reference plane to obtain a hard shadow image, which serves as the initial shadow intensity. Next, the shadow prediction model is designed: a model based on a dilated convolutional neural network is designed. It takes as input the hard shadow image and the shadow features corresponding to the 3D model, and outputs a soft shadow image. The goal of this network is to learn how to generate more realistic and natural soft shadow effects based on the model's shape, the virtual light source information, the virtual camera observation parameters, and the 3D model's material properties. Finally, soft shadow image generation: Using an improved dilated convolutional neural network, the hard shadow image of each 3D model is transformed to obtain a soft shadow image. This image serves as the final shadow result and can be integrated into the virtual scene.

[0206] This shadow generation method for three-dimensional models can quickly generate shadow maps by relying solely on the black and white maps or textures of the three-dimensional model. The corresponding shadow maps are also non-color. Relying solely on this shadow map, combined with rendering, the rendering effect can be perfectly achieved. Combined with the rendering program and the base map or texture color overlay, the shadow effect can be efficiently displayed and rendered.

[0207] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0208] Based on the same inventive concept, embodiments of the present application also provide a device for generating shadows for a three-dimensional model, for implementing the aforementioned method for generating shadows for a three-dimensional model. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the device for generating shadows for a three-dimensional model provided below can be found in the aforementioned limitations of the method for generating shadows for a three-dimensional model, and will not be further elaborated here.

[0209] In one embodiment, Figure 11 As shown, a shadow generation device 1100 for a three-dimensional model is provided, comprising: an acquisition module 1102, a contour extraction module 1104, a hard shadow generation module 1106, a shadow feature extraction module 1108, and a prediction module 1110, wherein:

[0210] An acquisition module 1102 is configured to acquire light source information of a virtual light source and observation parameters of a virtual camera. The light source information is used to form a lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form a field of view of the virtual camera on the three-dimensional model in the virtual scene.

[0211] The contour extraction module 1104 is used to generate a multi-view contour map corresponding to the three-dimensional model based on the light source information and the observation parameters, and the multi-view contour map is used to represent the initial shadow;

[0212] A hard shadow generation module 1106 is used to project the multi-view profile image onto a reference plane based on light source information and observation parameters to obtain a hard shadow image corresponding to the three-dimensional model;

[0213] A shadow feature extraction module 1108 is used to obtain shadow features of the 3D model based on light source information and material properties of the 3D model;

[0214] The prediction module 1110 is used to input the hard shadow image and shadow features into a trained shadow prediction model based on a convolutional neural network, and output a soft shadow image corresponding to the three-dimensional model based on the hard shadow image and shadow features through the shadow prediction model.

[0215] In one embodiment, the contour extraction module 1104 is also used to extract a binary contour image of the three-dimensional model from multiple perspectives based on light source information and observation parameters. The binary contour image is an image in which the three-dimensional model is mapped onto a two-dimensional plane and the image foreground and image background mapped onto the two-dimensional plane are binary-distinguished. The binary contour images corresponding to each perspective are merged to obtain a multi-perspective contour map corresponding to the three-dimensional model.

[0216] In one embodiment, the contour extraction module 1104 is also used to convert the three-dimensional model into a triangular mesh composed of triangular facets; based on the light source information and observation parameters, the projection matrix of the three-dimensional model at different viewing angles is calculated, and different viewing angles are different rotation angles when the three-dimensional model is rotated around the specified coordinate axis; for each viewing angle, the corresponding projection matrix is ​​used to project the triangular mesh onto a two-dimensional plane to obtain a two-dimensional polygon at the corresponding viewing angle; for each viewing angle, the edge contour line of the two-dimensional polygon at the corresponding viewing angle is extracted to obtain a binary contour image of the three-dimensional model at the corresponding viewing angle.

[0217] In one embodiment, the contour extraction module 1104 is further configured to perform a logical OR operation on the binary contour images corresponding to each viewing angle to obtain a multi-view contour image corresponding to the three-dimensional model.

[0218] In one embodiment, the contour extraction module 1104 is also used to obtain a vertex set and a triangle face set of a three-dimensional model, where each triangle face in the triangle face set is represented by three vertex indices, and the three vertices identified by the three vertex indices all belong to the vertex set; and a triangular mesh is generated based on the vertex set and the triangle face set.

[0219] In one embodiment, the contour extraction module 1104 is also used to generate a corresponding normal vector for each triangle in the triangle facet set, where the normal vector is a unit vector perpendicular to the triangle facet; based on the direction vector of the light source and the normal vector, determine whether the facet direction of the triangle facet is counterclockwise, and the facet direction is determined by the three vertices that make up the triangle facet; when the facet direction of the triangle facet is not counterclockwise, the order of the last two vertices in the triangle facet is exchanged to obtain a triangle facet with a counterclockwise facet direction; store the vertex set and the counterclockwise triangle facet in a triangle mesh object to obtain a triangle mesh corresponding to the three-dimensional model.

[0220] In one embodiment, the hard shadow generation module 1106 is further used to calculate the projection matrix corresponding to the multi-view contour image on the reference plane based on the light source information and observation parameters; and use the projection matrix to project the multi-view contour image onto the reference plane to obtain a hard shadow image corresponding to the three-dimensional model.

[0221] In one embodiment, the hard shadow generation module 1106 is also used to calculate a first transformation matrix for converting the two-dimensional coordinates of pixel points in the multi-view contour image into three-dimensional coordinates in the world coordinate system based on the observation parameters; calculate a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane based on the light source information; and calculate the projection matrix based on the first transformation matrix and the second transformation matrix.

[0222] In one embodiment, the hard shadow generation module 1106 is further used to calculate the front orthogonal basis vectors of the virtual camera based on the line of sight direction vector of the virtual camera; calculate the right orthogonal basis vectors of the virtual camera based on the front orthogonal basis vectors and the top vector of the virtual camera; calculate the top orthogonal basis vectors of the virtual camera based on the right orthogonal basis vectors and the front orthogonal basis vectors, and the front orthogonal basis vectors, the right orthogonal basis vectors and the top orthogonal basis vectors of the virtual camera constitute the camera coordinate system of the virtual camera; calculate the first transformation matrix based on the position of the virtual camera and the front orthogonal basis vectors, the right orthogonal basis vectors and the top orthogonal basis vectors of the virtual camera, and the first transformation matrix is ​​used to convert the camera coordinate system into the world coordinate system.

[0223] In one embodiment, the light source information includes the position and light source direction vector of the virtual light source. The hard shadow generation module 1106 is also used to calculate the front orthogonal basis vectors of the virtual light source based on the light source direction vector of the virtual light source; calculate the right orthogonal basis vectors of the virtual light source based on the right orthogonal basis vectors of the virtual camera; calculate the top orthogonal basis vectors of the virtual light source based on the front orthogonal basis vectors of the virtual light source and the front orthogonal basis vectors of the virtual light source, and the front orthogonal basis vectors, right orthogonal basis vectors and top orthogonal basis vectors of the virtual light source constitute the light source coordinate system of the virtual light source; calculate the second transformation matrix based on the position of the virtual light source and the front orthogonal basis vectors, right orthogonal basis vectors and top orthogonal basis vectors of the virtual light source, and the second transformation matrix is ​​used to convert the world coordinate system into the projection coordinate system corresponding to the standard plane.

[0224] In one embodiment, the hard shadow generation module 1106 is further configured to multiply the second transformation matrix by the first transformation matrix to obtain a projection matrix.

[0225] In one embodiment, the shadow feature extraction module 1108 is also used to obtain the coordinates of each vertex in the three-dimensional model, and calculate the normal vector corresponding to each vertex based on the coordinates of adjacent vertices; obtain the light source radius of the virtual light source and the light source distance from the virtual light source to the three-dimensional model, and calculate the light source size of the virtual light source based on the light source radius and the light source distance; obtain the material properties of the three-dimensional model, the material properties represent the diffuse reflectivity, specular reflectivity, roughness and metallicity corresponding to each vertex in the three-dimensional model, and calculate the comprehensive reflectivity corresponding to each vertex based on the diffuse reflectivity, specular reflectivity, roughness and metallicity corresponding to each vertex; obtain the shadow features corresponding to the three-dimensional model based on the normal vector corresponding to each vertex, the light source size of the virtual light source and the corresponding comprehensive reflectivity.

[0226] In one embodiment, the prediction module 1110 is also used to map shadow features from vertex space to pixel space, and after obtaining a shadow feature map with the same size as the hard shadow image data, the hard shadow image and the shadow feature map are spliced ​​by channel through the input layer of the shadow prediction model to obtain an input tensor, and the input tensor is input into a trained shadow prediction model based on a convolutional neural network; the input tensor is successively feature encoded and down-sampled through the encoder of the shadow prediction model to obtain high-resolution encoded features, and the encoder includes a plurality of hole convolution layers and pooling layers that are alternately connected in sequence; the high-resolution encoded features are successively feature decoded and up-sampled through the decoder of the shadow prediction model to obtain low-resolution decoded features, and the decoder includes a plurality of hole convolution layers and upsampling layers that are alternately connected in sequence; the low-resolution decoded features are converted into an output tensor with the same size as the hard shadow image data through the output layer of the shadow prediction model, and the output tensor is a soft shadow image corresponding to the three-dimensional model.

[0227] In one embodiment, the apparatus 1100 further includes:

[0228] A training module is used to obtain multiple training samples, each training sample includes a hard shadow image, shadow features and a sample soft shadow image corresponding to the three-dimensional model; for each training sample, the corresponding hard shadow image and shadow feature map are input into the initial shadow prediction model, and the corresponding predicted soft shadow image is output through the shadow prediction model; for each training sample, the difference between the corresponding predicted soft shadow image and the sample soft shadow image is calculated, and a loss function is constructed according to the differences corresponding to multiple samples; and the shadow prediction model is updated according to the loss function.

[0229] The shadow generation device 1100 for a 3D model obtains light source information of a virtual light source and observation parameters of a virtual camera. The light source information represents the lighting effect of the virtual light source in the virtual scene, and the observation parameters represent the field of view of the virtual camera on the 3D model in the virtual scene. Based on the light source information and observation parameters, a multi-view contour map corresponding to the 3D model is generated. The multi-view contour map represents the initial shadow. Based on the light source information and observation parameters, the multi-view contour map is projected onto a reference plane to obtain a hard shadow image corresponding to the 3D model. Based on the light source information and material properties of the 3D model, shadow features of the 3D model are obtained. The hard shadow image and shadow features are input into a trained shadow prediction model based on a convolutional neural network. The shadow prediction model outputs a soft shadow image corresponding to the 3D model based on the hard shadow image and shadow features. The soft shadow image can be used to generate a more realistic and natural soft shadow effect. In other words, only a low-cost multi-view contour map generation process is required. A hard shadow image of the 3D model is then generated based on the multi-view contour map. Based on the hard shadow image and shadow features, the shadow effect of the 3D model can be efficiently generated, reducing the amount of data processing required for shadow generation and improving shadow generation efficiency.

[0230] Each module in the above-mentioned shadow generation device for a three-dimensional model can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0231] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 12 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a shadow generation method of a three-dimensional model is implemented.

[0232] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 13 As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication, and the wireless communication can be achieved via Wi-Fi, a mobile cellular network, NFC (near field communication), or other technologies. When executed by the processor, the computer program implements a method for generating shadows of a three-dimensional model. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0233] Those skilled in the art will understand that Figure 12 、 Figure 13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0234] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the shadow generation method of the three-dimensional model provided in the embodiment of the present application are implemented.

[0235] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the shadow generation method of the three-dimensional model provided in the embodiment of the present application are implemented.

[0236] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the shadow generation method for a three-dimensional model provided in an embodiment of the present application.

[0237] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0238] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processors (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0239] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0240] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for generating shadows of a three-dimensional model, characterized in that: The method comprises: Acquiring light source information of a virtual light source and observation parameters of a virtual camera, wherein the light source information is used to form a lighting effect of the virtual light source in the virtual scene, and the observation parameters are used to form a field of view of the virtual camera on the three-dimensional model in the virtual scene; generating a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters, wherein the multi-view contour map is used to represent an initial shadow; Projecting the multi-view contour image onto a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model; Obtaining shadow features of the three-dimensional model according to the light source information and material properties of the three-dimensional model; The hard shadow image and the shadow features are input into a trained shadow prediction model based on a convolutional neural network, and the shadow prediction model outputs a soft shadow image corresponding to the three-dimensional model according to the hard shadow image and the shadow features.

2. The method according to claim 1, characterized in that Generating a multi-view contour map corresponding to the three-dimensional model according to the light source information and the observation parameters includes: Extracting a binary contour image of the three-dimensional model from multiple viewing angles based on the light source information and the observation parameters, wherein the binary contour image is an image in which the three-dimensional model is mapped onto a two-dimensional plane and a foreground image and a background image mapped onto the two-dimensional plane are binary-differentiated; The binary contour images corresponding to each viewing angle are merged to obtain a multi-view contour image corresponding to the three-dimensional model.

3. The method according to claim 2, characterized in that The step of extracting a binary contour image of the three-dimensional model from multiple perspectives based on the light source information and the observation parameters includes: Converting the three-dimensional model into a triangular mesh composed of triangular facets; Calculating, based on the light source information and the observation parameters, a projection matrix of the three-dimensional model at different viewing angles, wherein the different viewing angles are different rotation angles when the three-dimensional model is rotated about a specified coordinate axis; For each viewing angle, project the triangular mesh onto the two-dimensional plane using a corresponding projection matrix to obtain a two-dimensional polygon at the corresponding viewing angle; For each viewing angle, edge contour lines of the two-dimensional polygon at the corresponding viewing angle are extracted to obtain a binary contour image of the three-dimensional model at the corresponding viewing angle.

4. The method according to claim 3, characterized in that The step of merging the binary contour images corresponding to the respective viewing angles to obtain a multi-view contour image corresponding to the three-dimensional model includes: A logical OR operation is performed on the binary contour images corresponding to each viewing angle to obtain a multi-view contour image corresponding to the three-dimensional model.

5. The method according to claim 3, characterized in that The converting of the three-dimensional model into a triangular mesh composed of triangular facets comprises: Obtaining a vertex set and a triangle face set of the three-dimensional model, where each triangle face in the triangle face set is represented by three vertex indices, and the three vertices identified by the three vertex indices all belong to the vertex set; The triangular mesh is generated according to the vertex set and the triangular facet set.

6. The method according to claim 5, characterized in that Generating the triangular mesh according to the vertex set and the triangular facet set includes: For each triangle in the set of triangles, generate a corresponding normal vector, where the normal vector is a unit vector perpendicular to the triangle; Determining whether a facet direction of the triangular facet is counterclockwise according to a direction vector of the light source and the normal vector, wherein the facet direction is determined by three vertices constituting the triangular facet; When the face direction of the triangular face is not counterclockwise, the order of the last two vertices in the triangular face is swapped to obtain a triangular face with a counterclockwise face direction; The vertex set and the counterclockwise triangular facets are stored in a triangular mesh object to obtain a triangular mesh corresponding to the three-dimensional model.

7. The method according to claim 1, characterized in that The step of projecting the multi-view contour image onto a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model includes: Calculating a projection matrix of the multi-view profile image corresponding to the reference plane according to the light source information and the observation parameters; The multi-view contour image is projected onto the reference plane using the projection matrix to obtain a hard shadow image corresponding to the three-dimensional model.

8. The method according to claim 7, characterized in that Calculating the projection matrix corresponding to the multi-view profile image on the reference plane according to the light source information and the observation parameters includes: Calculating a first transformation matrix for converting the two-dimensional coordinates of the pixel points in the multi-view contour image into three-dimensional coordinates in a world coordinate system according to the observation parameters; Calculating, according to the light source information, a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane; A projection matrix is ​​calculated according to the first transformation matrix and the second transformation matrix.

9. The method according to claim 8, characterized in that The step of calculating, based on the observation parameters, a first transformation matrix for converting the two-dimensional coordinates of the pixel points in the multi-view contour image into three-dimensional coordinates in a world coordinate system comprises: Calculate the front orthogonal basis vector of the virtual camera according to the sight direction vector of the virtual camera; Calculating a right orthogonal basis vector of the virtual camera according to the front orthogonal basis vector and the upper vector of the virtual camera; Calculating an upper orthogonal basis vector of the virtual camera according to the right orthogonal basis vector and the front orthogonal basis vector, wherein the front orthogonal basis vector, the right orthogonal basis vector and the upper orthogonal basis vector constitute a camera coordinate system of the virtual camera; A first transformation matrix is ​​calculated based on the position of the virtual camera and the front orthogonal basis vector, the right orthogonal basis vector and the top orthogonal basis vector of the virtual camera. The first transformation matrix is ​​used to convert the camera coordinate system into a world coordinate system.

10. The method according to claim 8, characterized in that The light source information includes the position of the virtual light source and the light source direction vector. The calculating, based on the light source information, a second transformation matrix for converting the three-dimensional coordinates into projection coordinates on the reference plane includes: Calculating the forward orthogonal basis vectors of the virtual light source according to the light source direction vector of the virtual light source; Calculating the right orthogonal basis vectors of the virtual light source according to the right orthogonal basis vectors of the virtual camera; Calculating an upper orthogonal basis vector of the virtual light source according to the front orthogonal basis vector of the virtual light source and the front orthogonal basis vector of the virtual light source, wherein the front orthogonal basis vector of the virtual light source, the right orthogonal basis vector and the upper orthogonal basis vector constitute a light source coordinate system of the virtual light source; A second transformation matrix is ​​calculated based on the position of the virtual light source and the front orthogonal basis vector of the virtual light source, the right orthogonal basis vector and the upper orthogonal basis vector. The second transformation matrix is ​​used to convert the world coordinate system into the projection coordinate system corresponding to the standard plane.

11. The method according to claim 8, characterized in that The calculating the projection matrix according to the first transformation matrix and the second transformation matrix includes: The second transformation matrix is ​​multiplied by the first transformation matrix to obtain the projection matrix.

12. The method according to claim 1, characterized in that Obtaining the shadow features of the three-dimensional model according to the light source information and the material properties of the three-dimensional model includes: Obtaining the coordinates of each vertex in the three-dimensional model, and calculating the normal vector corresponding to each vertex based on the coordinates of adjacent vertices; Obtaining a light source radius of the virtual light source and a light source distance from the virtual light source to the three-dimensional model, and calculating a light source size of the virtual light source according to the light source radius and the light source distance; Obtaining material properties of the three-dimensional model, where the material properties represent diffuse reflectivity, specular reflectivity, roughness, and metallicity corresponding to each vertex in the three-dimensional model, and calculating a comprehensive reflectivity corresponding to each vertex based on the diffuse reflectivity, specular reflectivity, roughness, and metallicity corresponding to each vertex; The shadow feature corresponding to the three-dimensional model is obtained according to the normal vector corresponding to each vertex, the light source size of the virtual light source and the corresponding comprehensive reflectivity.

13. The method according to claim 1, wherein Inputting the hard shadow image and the shadow features into a trained shadow prediction model based on a convolutional neural network, and outputting a soft shadow image corresponding to the three-dimensional model according to the hard shadow image and the shadow features by the shadow prediction model, comprises: After mapping the shadow features from vertex space to pixel space to obtain a shadow feature map having the same size as the hard shadow image data, the hard shadow image and the shadow feature map are concatenated channel by channel through the input layer of the shadow prediction model to obtain an input tensor, and the input tensor is input into the trained convolutional neural network-based shadow prediction model; The encoder of the shadow prediction model successively performs feature encoding and feature downsampling processing on the input tensor to obtain high-resolution encoded features, wherein the encoder includes a plurality of alternately connected dilated convolutional layers and pooling layers; The high-resolution encoded features are sequentially decoded and upsampled by a decoder of the shadow prediction model to obtain low-resolution decoded features, wherein the decoder includes a plurality of alternately connected dilated convolutional layers and upsampling layers; The low-resolution decoded features are converted into an output tensor having the same size as the hard shadow image data through the output layer of the shadow prediction model, and the output tensor is a soft shadow image corresponding to the three-dimensional model.

14. The method according to any one of claims 1 to 13, characterized in that The training steps of the shadow prediction model based on the convolutional neural network include: Acquire multiple training samples, each training sample including a hard shadow image corresponding to a three-dimensional model, a shadow feature, and a sample soft shadow image corresponding to the hard shadow image; For each training sample, the corresponding hard shadow image and shadow feature map are input into the initial shadow prediction model, and the shadow prediction model outputs the corresponding predicted soft shadow image; For each training sample, calculating the difference between the corresponding predicted soft shadow image and the sample soft shadow image, and constructing a loss function based on the difference corresponding to the multiple samples; The shadow prediction model is updated according to the loss function.

15. A shadow generating device for a three-dimensional model, characterized in that: The device comprises: an acquisition module, configured to acquire light source information of a virtual light source and observation parameters of a virtual camera, wherein the light source information is used to form a lighting effect of the virtual light source in a virtual scene, and the observation parameters are used to form a field of view of the virtual camera on a three-dimensional model in the virtual scene; a contour extraction module, configured to generate a multi-view contour map corresponding to the three-dimensional model based on the light source information and the observation parameters, wherein the multi-view contour map is used to represent an initial shadow; a hard shadow generation module, configured to project the multi-view contour image onto a reference plane according to the light source information and the observation parameters, to obtain a hard shadow image corresponding to the three-dimensional model; a shadow feature extraction module, configured to obtain shadow features of the three-dimensional model based on the light source information and the material properties of the three-dimensional model; A prediction module is used to input the hard shadow image and the shadow features into a trained shadow prediction model based on a convolutional neural network, and output a soft shadow image corresponding to the three-dimensional model based on the hard shadow image and the shadow features through the shadow prediction model.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 14 are implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.

18. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.

Images