Three-dimensional scene rendering method, device, equipment and storage medium

Abstract

The present invention is applicable to the field of image processing and provides a three-dimensional scene rendering method, device, equipment, and storage medium. The method includes: obtaining a shading point on the surface of a three-dimensional object corresponding to a screen pixel, obtaining feature information of the shading point from a geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set, calculating a feature code of the shading point according to the two-dimensional coordinates of the shading point, inputting the feature information, the feature code, and the light angle from the shading point to the camera into a trained illumination prediction network to obtain the illumination effect of the shading point. The scheme reduces the three-dimensional coordinates of the shading point to two-dimensional coordinates through geometric images, thereby reducing the complexity of information processing in the rendering process and making the feature representation more efficient.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a three-dimensional scene rendering method, device, equipment and storage medium. Background Art

[0002] Traditional light field rendering methods require the use of ray tracing technology to accurately calculate and simulate the light in the scene, so as to calculate the graphics rendering effect. However, the cost of calculating and simulating light is very high, which causes the light field rendering to run very slowly. In order to reduce this calculation cost in real-time rendering, the lighting cache technology can be used to cache the calculated ray tracing information in advance, so that the existing lighting information can be quickly called when rendering. However, this method of directly caching lighting also has a disadvantage, which is that it requires a huge amount of memory space to cache all the accurate lighting information.

[0003] When using light fields for graphics rendering, in order to reduce the huge memory consumption caused by directly caching lighting, some scholars have proposed using neural networks to store and predict lighting information. The input position, direction, material and other information are transmitted to the neural network, and then the neural network predicts the lighting information. However, although the change of lighting with surface material and normal is low frequency, the change of lighting with position may be high frequency. Therefore, the performance of directly using simple neural networks to predict lighting information is very low. The lighting information cannot be stored and predicted accurately at all, and therefore has no practical value.

[0004] In order to solve this difficulty, some scholars have proposed the NRC algorithm in recent years. It uses the sin sine function to perform high-dimensional encoding expansion on the three-dimensional coordinates of the position information to improve the prediction performance of the neural network. However, the illumination information may change with the position, which may cause high frequency and the prediction performance of the neural network is insufficient. The complexity of the three-dimensional coordinate position information is a bit high, and the representation performance of the sin sine function encoding is insufficient, resulting in low performance. Summary of the invention

[0005] The purpose of the present invention is to provide a three-dimensional scene rendering method, device, equipment and storage medium, aiming to solve the problem of high information processing complexity in existing three-dimensional scene rendering technology.

[0006] In one aspect, the present invention provides a three-dimensional scene rendering method, the method comprising the following steps:

[0007] Get the shading point on the surface of the three-dimensional object corresponding to the screen pixel;

[0008] Acquire feature information of the shading point from the geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set;

[0009] Calculate the feature code of the shading point according to the two-dimensional coordinates;

[0010] The feature information, the feature code and the light angle from the shading point to the camera are input into a trained lighting prediction network to obtain the lighting effect of the shading point.

[0011] Optionally, before the step of obtaining the shading point on the surface of the three-dimensional object corresponding to the screen pixel, the step includes:

[0012] The geometric image set is generated according to the feature information of the three-dimensional object, wherein a geometric image in the geometric image set stores a feature information of the three-dimensional object, wherein the feature information of the three-dimensional object includes surface shape, normal direction, color texture and roughness.

[0013] Optionally, the step of generating the geometric image set according to the feature information of the three-dimensional object includes:

[0014] Acquire a triangulated network model of the three-dimensional object;

[0015] The triangulated mesh model is converted into a geometric image for storing the surface shape.

[0016] Optionally, the feature code of the shading point is calculated using a bilinear interpolation algorithm, and the step of calculating the feature code of the shading point according to the two-dimensional coordinates includes:

[0017] Acquire feature codes of four adjacent coded grid points of the shading point according to the two-dimensional coordinates, wherein the feature code of each coded grid point includes feature information of the coded grid point;

[0018] The feature codes of the four coded grid points are subjected to bilinear interpolation calculation to obtain the feature code of the shading point.

[0019] Optionally, the calculation formula of the feature code of the colored point is:

[0020] ,in, represents the two-dimensional coordinates of the shading point, Represents the feature code of the coloring point, u and v are both in the range of 0~1, and the coding grid size is , , Express Round down, Express Round up; Express Round down, Express Round up, the coding grid points a, b, c, and d are , , , , Represent the feature codes of the coded grid points a, b, c, and d respectively.

[0021] Optionally, the method further comprises: training the illumination prediction network, and the step of training the illumination prediction network comprises:

[0022] Obtain a training data set, where each training data set sample includes input data and output data, wherein the input data includes the two-dimensional coordinates of a shading point, feature information, feature coding, and the light angle from the shading point to the camera, and the output data is the actual lighting effect, which is calculated using a preset rendering equation;

[0023] The training data set is used as the input of the illumination prediction network, and the illumination prediction network is trained according to the label of each training data set sample to obtain a trained illumination prediction network.

[0024] Optionally, the illumination prediction network is a multi-layer perceptron neural network.

[0025] Optionally, the hidden layer of the illumination prediction network is 2 or 3 layers.

[0026] Optionally, the activation function between each layer of the illumination prediction network adopts a ReLU function.

[0027] Optionally, the loss function used in training the illumination prediction network is a MES loss function.

[0028] In another aspect, the present invention provides a three-dimensional scene rendering device, the device comprising:

[0029] A shading point acquisition unit, used to acquire shading points on the surface of a three-dimensional object corresponding to a screen pixel;

[0030] A feature information acquisition unit, configured to acquire feature information of the shading point from the geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set;

[0031] a feature code calculation unit, configured to calculate a feature code of the shading point according to the two-dimensional coordinates; and

[0032] The lighting calculation unit is used to input the feature information, the feature code and the light angle from the shading point to the camera into a trained lighting prediction network to obtain the lighting effect of the shading point.

[0033] On the other hand, the present invention also provides a three-dimensional scene rendering device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0034] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0035] The present invention obtains the shading point on the surface of the three-dimensional object corresponding to the screen pixel, obtains the feature information of the shading point from the geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set, calculates the feature code of the shading point according to the two-dimensional coordinates, and inputs the feature information, feature code and the light angle from the shading point to the camera into the trained illumination prediction network to obtain the illumination effect of the shading point. The present scheme reduces the three-dimensional coordinates of the shading point to two-dimensional coordinates through the geometric image, thereby reducing the complexity of information processing in the rendering process and making the feature representation more efficient. In addition, since the present scheme adopts hierarchical encoding, the prediction performance of the illumination prediction network is greatly improved, thereby improving the calculation speed of the rendering process. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flowchart of the implementation of the three-dimensional scene rendering method provided in the first embodiment of the present invention;

[0037] Figure 2 is a diagram showing an example of a structure of a geometric image set provided in the first embodiment of the present invention;

[0038] Figure 3 is an example diagram of a position representation method of a shading point provided in the first embodiment of the present invention;

[0039] Figure 4 This is an example diagram of a characteristic coding representation method of a colored point provided in the first embodiment of the present invention;

[0040] Figure 5 This is an example diagram of the angle representation of light from a shading point to a camera provided in the first embodiment of the present invention;

[0041] Figure 6 is a flowchart of the implementation of the illumination prediction network training method provided in the second embodiment of the present invention;

[0042] Figure 7 is a flowchart of the implementation of obtaining a training data set provided in the third embodiment of the present invention;

[0043] Figure 8 is a schematic diagram of the structure of a three-dimensional scene rendering device provided in Embodiment 4 of the present invention; and

[0044] Fig. 9 It is a structural schematic diagram of a three-dimensional scene rendering device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0046] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments: Embodiment 1

[0047] Figure 1 The implementation process of the three-dimensional scene rendering method provided in the first embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows.

[0048] In step S101, a shading point on the surface of a three-dimensional object corresponding to a screen pixel is obtained.

[0049] This embodiment is applicable to lighting rendering of a three-dimensional scene, which may include a camera and at least one three-dimensional object. In the embodiment of the present invention, the shading point of the object surface corresponding to the screen pixel can be obtained based on the traditional rendering pipeline, such as rasterization and ray tracing.

[0050] Optionally, before obtaining the shading point on the surface of the three-dimensional object corresponding to the screen pixel, a geometric image set is generated according to the feature information of the three-dimensional object, and a geometric image in the geometric image set stores a kind of feature information of the three-dimensional object, so as to realize the pre-storage of the feature information through the geometric image. Among them, the feature information of the three-dimensional object may include the surface shape, normal direction, color texture and roughness. Accordingly, the geometric image set may include a first geometric image for storing the surface shape, a second geometric image for storing the normal direction, a third geometric image for storing the color texture and a fourth geometric image for storing the roughness. It should be noted here that the feature information of the three-dimensional object may also include features other than those mentioned above, which can be set according to actual needs.

[0051] As an example, Figure 2As shown, the R, G, and B channels in the geometric image used to store the surface shape can store the three-dimensional coordinates of each point of the three-dimensional object, that is, the X coordinate, the Y coordinate, and the Z coordinate; the R, G, and B channels in the geometric image used to store the normal direction can store the normal direction of each three-dimensional coordinate point of the three-dimensional object, that is, the X normal component, the Y normal component, and the Z normal component; the R, G, and B channels in the geometric image used to store the color texture can store the color texture of each three-dimensional coordinate point of the three-dimensional object, that is, the R channel value, the G channel value, and the B channel value; the R, G, and B channels in the geometric image used to store the roughness can store the scattering components of each three-dimensional coordinate point of the three-dimensional object, that is, the R channel scattering component, the G channel scattering component, and the B channel scattering component.

[0052] Optionally, when generating a geometric image set according to feature information of a three-dimensional object, a triangulated mesh model of the three-dimensional object is obtained, and the triangulated mesh model is converted into a geometric image for storing the surface shape, thereby realizing the generation of a first geometric image. In a specific implementation, the conformal geometry method proposed by Gu Xianfeng can be used to convert a traditional triangulated mesh model into a representation of a geometric image.

[0053] In step S102, feature information of the shading point is obtained from the geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set.

[0054] In the embodiment of the present invention, the three-dimensional space coordinates are represented by the two-dimensional coordinates on the geometric image, that is, the three-dimensional coordinates of the shading point can be equivalently represented by the two-dimensional coordinates on the geometric image set. Figure 3 As shown, the three-dimensional object is a rabbit. The three-dimensional coordinates of point P on the surface of the rabbit's three-dimensional model are (x, y, z) = (20, 100, 50), and the coordinates of point P' corresponding to the first geometric image are (u, v) = (0.7, 0.75). The component values ​​of the three channels of the first geometric image at the coordinates (0.7, 0.75) are (20, 100, 50) respectively. Therefore, the three-dimensional coordinates of point P (20, 100, 50) can be equivalently represented by the coordinates (0.7, 0.75) of P' in the first geometric image.

[0055] The characteristic information of the shading point obtained from the geometric image set may include the three-dimensional coordinates, normal direction, color texture and roughness of the shading point. In a specific implementation, the three-dimensional coordinates of the shading point can be obtained from the first geometric image according to the two-dimensional coordinates of the shading point, the normal direction of the shading point can be obtained from the second geometric image, the color texture of the shading point can be obtained from the third geometric image, and the roughness of the shading point can be obtained from the fourth geometric image.

[0056] In step S103, a feature code of the shading point is calculated according to the two-dimensional coordinates of the shading point.

[0057] In an embodiment of the present invention, the feature coding of the shading point can be calculated using a nearest neighbor interpolation algorithm, a linear interpolation algorithm, etc. Optionally, the feature coding of the shading point is calculated using a bilinear interpolation algorithm to improve the prediction performance of the neural network and effectively compress the number of layers of the illumination prediction network. Specifically, the feature coding of the four adjacent coded grid points of the shading point is obtained according to the two-dimensional coordinates of the shading point, and the feature coding of the four coded grid points is bilinearly interpolated to obtain the feature coding of the shading point, thereby realizing hierarchical coding by bilinear interpolation. Among them, the feature coding of each coded grid point contains the feature information of the coded grid point.

[0058] Alternatively, if Figure 4 As shown, the calculation formula for the feature encoding of the colored point is:

[0059] ,in, represents the two-dimensional coordinates of the shading point, Represents the feature code of the coloring point, u and v are both in the range of 0~1, and the coding grid size is , , Express Round down, Express Round up; Express Round down, Express Round up, the coding grid points a, b, c, and d are , , , , Represent the feature codes of the coded grid points a, b, c, and d respectively.

[0060] It should be noted here that the feature encoding of the shading point can also be calculated using linear interpolation methods other than the above, for example, twice linear interpolation or twice bilinear interpolation, which is not limited in this embodiment.

[0061] In step S104, the feature information, feature coding and the light angle from the shading point to the camera are input into the trained illumination prediction network to obtain the illumination effect of the shading point.

[0062] In an embodiment of the present invention, the light angle from the shading point to the camera can be calculated based on the camera position and the three-dimensional coordinates of the shading point in the feature information. Figure 5 As shown, the angle of light from the shading point to the camera , if spherical polar coordinates are used, then Indicates the direction of the light; if three-dimensional coordinates are used, To represent the direction vector of the light.

[0063] The illumination prediction network can be implemented using a neural network such as a DNN, RNN, CNN or GNN. Optionally, the illumination prediction network uses a multi-layer perceptron (MLP) neural network to increase the reasoning speed of the illumination prediction network, thereby achieving maximum acceleration of rendering. Among them, the illumination prediction network is usually a shallow network, and optionally, the hidden layer of the illumination prediction network is usually no more than five layers to ensure the rendering speed.

[0064] Further optionally, the hidden layer of the illumination prediction network is 2 or 3 layers to further improve the reasoning speed of the illumination prediction network. Specifically, the multi-layer perceptron network includes an input layer, a plurality of hidden layers and an output layer connected in sequence. Further optionally, the number of nodes in each hidden layer is 64.

[0065] Further optionally, the activation function between each layer of the illumination prediction network adopts a ReLU function to increase the running speed of the illumination prediction network.

[0066] Further optionally, the loss function used in the illumination prediction network training is the MES loss function. The specific training method of the illumination prediction network can refer to the description of the second embodiment, which will not be repeated here.

[0067] As an example, the illumination prediction network is implemented using an MLP neural network with 2 hidden layers. The ReLU activation function is used between each layer in the illumination prediction network. The illumination effect calculation formula of the shading point is:

[0068] ,

[0069] ,

[0070]

[0071] in, Represents the input of the lighting prediction network, namely the feature information of the shading point, the feature encoding, and the light angle from the shading point to the camera. , denote the weight and bias of the first hidden layer, respectively. , denote the weight and bias of the second hidden layer, respectively. , denote the weight and bias of the output layer respectively, , denote the outputs of the first hidden layer and the second hidden layer respectively, Represents the output of the lighting prediction network, that is, the lighting effect of the shading point.

[0072] In an embodiment of the present invention, a shading point on the surface of a three-dimensional object corresponding to a screen pixel is obtained, feature information of the shading point is obtained from a geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set, feature coding of the shading point is calculated according to the two-dimensional coordinates, and the feature information, feature coding and the light angle from the shading point to the camera are input into a trained illumination prediction network to obtain the illumination effect of the shading point. This scheme reduces the three-dimensional coordinates of the shading point to two-dimensional coordinates through geometric images, thereby reducing the complexity of information processing in the rendering process and making feature representation more efficient. In addition, since this scheme adopts hierarchical coding, the prediction performance of the illumination prediction network is greatly improved, thereby improving the calculation speed of the rendering process. Embodiment 2

[0073] Figure 6 The implementation process of the illumination prediction network training method provided in the second embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows.

[0074] In step S201, a training data set is obtained.

[0075] In the embodiment of the present invention, each training data set sample includes input data and output data, the input data includes the two-dimensional coordinates of a shading point, feature information, feature coding, and the light angle from the shading point to the camera, and the output data is the actual lighting effect corresponding to the training data set sample, and the actual lighting effect can be calculated using a preset rendering equation. The acquisition of the training data set can refer to the description of Example 3, which will not be repeated here.

[0076] In step S202, the input data of the training data set samples are used as the input of the illumination prediction network, and the illumination prediction network is trained according to the output data of the training data set samples to obtain a trained illumination prediction network.

[0077] In an embodiment of the present invention, the illumination prediction network may adopt an MLP neural network. Optionally, the illumination prediction network is trained using an MSE loss:

[0078]

[0079] in, represents the loss value of the illumination prediction network, represents the predicted value of the illumination prediction network, Represents the actual lighting effect.

[0080] When training the illumination prediction network, the back propagation method can be used to update the model weights. At the same time, in order to increase the information encoding capability, the encoding grid is initially , , and The feature codes are all random vectors. Then, in the back propagation, the gradient of the feature code is calculated according to the chain rule, and the learning is updated according to the gradient value.

[0081] In an embodiment of the present invention, a training data set is obtained, and the training data set samples include input data and output data. The input data includes the two-dimensional coordinates of a shading point, feature information, feature coding, and the light angle from the shading point to the camera. The output data is the actual lighting effect corresponding to the training data set sample. The input data of the training data set sample is used as the input of the lighting prediction network. The lighting prediction network is trained according to the output data of each training data set sample to obtain a trained lighting prediction network, thereby realizing the training of the lighting prediction network. Embodiment 3

[0082] Figure 7 The implementation process of obtaining a training data set provided by the third embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows.

[0083] In step S301, a geometric image set of a three-dimensional object is obtained.

[0084] In the embodiment of the present invention, the implementation of this step can refer to the relevant description of step S101 in the first embodiment, and will not be repeated here.

[0085] In step S302 , a large number of random rays are generated from a light source.

[0086] In step S303, the light field of the rendering equation is calculated.

[0087] In the embodiment of the present invention, when a geometric point of a three-dimensional scene is rendered on the screen, the rendering color of the point needs to be accurately calculated. This process is related to the texture color of the object itself, the illumination irradiated to the point, and the diffuse scattering material of the object surface. The calculation process may depend on the following rendering equation:

[0088]

[0089] in, Represents the position (i.e., the coloring point) The angle of the outgoing light The light value of Indicates location From The incident light The energy reflection ratio of the outgoing light, Indicates the position The angle of the incident light is The light intensity, Indicates the proportion of energy that the surface of a three-dimensional object receives when light is irradiated.

[0090] This part of the calculation can use traditional ray path tracing technology to simulate and calculate the light from the light source to the surface of the three-dimensional object, and calculate the reflection effect of the three-dimensional object surface based on the surface normal of the three-dimensional object, the material of the three-dimensional object and other characteristic information. The lighting value of a given position , and from the location The angle of light to the camera After that, the final lighting effect is given when the light reaches the camera.

[0091] In step S304, a training data set is acquired based on the geometric image set and the light field calculation result.

[0092] In the embodiment of the present invention, the input data of each training data set sample, i.e., the two-dimensional coordinates of the shading point, feature information, feature coding, and the light angle from the shading point to the camera, can be obtained based on the geometric image set and the light field calculation result, and the output data of each training data set sample, i.e., the actual lighting effect corresponding to the shading point, i.e., the light field calculation result of the shading point obtained by step S303, can be obtained. The acquisition of feature information and feature coding in this step can refer to the relevant description of steps S102 and S103 in the first embodiment, which will not be repeated here.

[0093] In an embodiment of the present invention, a geometric image set of a three-dimensional object is obtained, a large number of random light rays are generated from a light source, the light field of a rendering equation is calculated, and a training data set is obtained based on the geometric image set and the light field calculation results, thereby achieving the acquisition of a training data set. Embodiment 4

[0094] Figure 8 The structure of the three-dimensional scene rendering device provided by the fourth embodiment of the present invention is shown. For the convenience of description, only the part related to the embodiment of the present invention is shown, including:

[0095] A shading point acquisition unit 41 is used to acquire a shading point on the surface of a three-dimensional object corresponding to a screen pixel;

[0096] A feature information acquisition unit 42, configured to acquire feature information of the shading point from the geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set;

[0097] A feature code calculation unit 43, used to calculate the feature code of the shading point according to the two-dimensional coordinates; and

[0098] The illumination calculation unit 44 is used to input feature information, feature coding and the light angle from the shading point to the camera into the trained illumination prediction network to obtain the illumination effect of the shading point.

[0099] Optionally, the device further comprises:

[0100] The geometric image generation unit is used to generate a geometric image set based on the feature information of the three-dimensional object. A geometric image in the geometric image set stores a kind of feature information of the three-dimensional object, wherein the feature information of the three-dimensional object includes surface shape, normal direction, color texture and roughness.

[0101] Optionally, the geometric image generating unit further includes:

[0102] A model acquisition unit, used to acquire a triangulated network model of a three-dimensional object; and

[0103] The image conversion unit is used for converting the triangulated network model into a geometric image for storing the surface shape.

[0104] Optionally, the feature code of the shading point is calculated using a bilinear interpolation algorithm, and the feature code calculation unit 43 includes:

[0105] A neighborhood code acquisition unit, used for acquiring feature codes of four adjacent coded grid points of the colored point according to the two-dimensional coordinates, wherein the feature code of each coded grid point includes feature information of the coded grid point;

[0106] The calculation subunit is used to perform bilinear interpolation calculation on the feature codes of the four coded grid points to obtain the feature code of the shading point.

[0107] Optionally, the calculation formula of the feature encoding of the colored point is:

[0108]

[0109] in, represents the two-dimensional coordinates of the shading point, Represents the feature code of the coloring point, u and v are both in the range of 0~1, and the coding grid size is , , Express Round down, Express Round up; Express Round down, Express Round up, the coding grid points a, b, c, and d are , , , , Represent the feature codes of the coded grid points a, b, c, and d respectively.

[0110] Optionally, the device further comprises:

[0111] Model training unit, used to train the illumination prediction network.

[0112] Optionally, the model training unit further includes:

[0113] A training data acquisition unit, used to acquire a training data set, each training data set sample includes input data and output data, the input data includes the two-dimensional coordinates of a shading point, feature information, feature coding, and the light angle from the shading point to the camera, and the output data is the actual lighting effect, which is calculated using a preset rendering equation; and

[0114] The model training subunit is used to take the input data as the input of the illumination prediction network, train the illumination prediction network according to the output data, and obtain a trained illumination prediction network.

[0115] Optionally, the illumination prediction network is a multi-layer perceptron neural network.

[0116] Further optionally, the number of hidden layers of the illumination prediction network is 2 or 3.

[0117] Further optionally, the activation function between each layer of the illumination prediction network adopts a ReLU function.

[0118] Further optionally, the loss function used in training the illumination prediction network is a MES loss function.

[0119] In the embodiment of the present invention, each unit of the 3D scene rendering device can be implemented by a corresponding hardware or software unit, and each unit can be an independent software or hardware unit, or can be integrated into a software or hardware unit, which is not intended to limit the present invention. The specific implementation of each unit of the 3D scene rendering device can refer to the description of the aforementioned method embodiment, which will not be repeated here. Embodiment 5

[0120] Fig. 9 The structure of a three-dimensional scene rendering device provided by Embodiment 5 of the present invention is shown. For the sake of ease of explanation, only the parts related to the embodiment of the present invention are shown.

[0121] The three-dimensional scene rendering device 5 of the embodiment of the present invention includes a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps in the above-mentioned method embodiments are implemented, for example Figure 1Alternatively, when the processor 50 executes the computer program 52, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 8 The functions of the units 41 to 44 are shown.

[0122] In an embodiment of the present invention, a shading point on the surface of a three-dimensional object corresponding to a screen pixel is obtained, feature information of the shading point is obtained from a geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set, a feature code of the shading point is calculated according to the two-dimensional coordinates, and the feature information, the feature code and the light angle from the shading point to the camera are input into a trained lighting prediction network to obtain the lighting effect of the shading point. This scheme reduces the three-dimensional coordinates of the shading point to two-dimensional coordinates through geometric images, thereby reducing the complexity of information processing in the rendering process and making feature representation more efficient. In addition, since this scheme adopts hierarchical coding, the prediction performance of the lighting prediction network is greatly improved, thereby improving the calculation speed of the rendering process. Embodiment 6

[0123] In an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above method embodiment are implemented, for example, Figure 1 Alternatively, when the computer program is executed by a processor, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 8 The functions of the units 41 to 43 are shown.

[0124] In an embodiment of the present invention, a shading point on the surface of a three-dimensional object corresponding to a screen pixel is obtained, feature information of the shading point is obtained from a geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set, a feature code of the shading point is calculated according to the two-dimensional coordinates, and the feature information, the feature code and the light angle from the shading point to the camera are input into a trained lighting prediction network to obtain the lighting effect of the shading point. This scheme reduces the three-dimensional coordinates of the shading point to two-dimensional coordinates through geometric images, thereby reducing the complexity of information processing in the rendering process and making feature representation more efficient. In addition, since this scheme adopts hierarchical coding, the prediction performance of the lighting prediction network is greatly improved, thereby improving the calculation speed of the rendering process.

[0125] The computer-readable storage medium of the embodiment of the present invention may include any entity or device or recording medium capable of carrying computer program code, for example, ROM / RAM, magnetic disk, optical disk, flash memory and other memories.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A three-dimensional scene rendering method, characterized in that: The method comprises: Get the shading point on the surface of the three-dimensional object corresponding to the screen pixel; Acquire feature information of the shading point from the geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set; Calculate the feature code of the shading point according to the two-dimensional coordinates; Input the feature information, the feature code and the light angle from the shading point to the camera into a trained illumination prediction network to obtain the illumination effect of the shading point, wherein the illumination prediction network is a multi-layer perceptron neural network, and the input data of each training data set sample of the illumination prediction network includes the two-dimensional coordinates of a shading point, the feature information, the feature code and the light angle from the shading point to the camera; Before the step of obtaining the shading point on the surface of the three-dimensional object corresponding to the screen pixel, it includes: The geometric image set is generated according to the feature information of the three-dimensional object, and a geometric image in the geometric image set stores a kind of feature information of the three-dimensional object.

2. The method according to claim 1, characterized in that The characteristic information of the three-dimensional object includes surface shape, normal direction, color texture and roughness; The step of generating the geometric image set according to the feature information of the three-dimensional object comprises: Acquire a triangulated network model of the three-dimensional object; The triangulated mesh model is converted into a geometric image for storing the surface shape.

3. The method according to claim 1, characterized in that The feature code of the shading point is calculated by using a bilinear interpolation algorithm, and the step of calculating the feature code of the shading point according to the two-dimensional coordinates includes: Acquire feature codes of four adjacent coded grid points of the shading point according to the two-dimensional coordinates, wherein the feature code of each coded grid point includes feature information of the coded grid point; The feature codes of the four coded grid points are subjected to bilinear interpolation calculation to obtain the feature code of the shading point.

4. The method according to claim 3, characterized in that The calculation formula of the feature code of the colored point is: ,in, represents the two-dimensional coordinates of the shading point, Represents the feature code of the coloring point, u and v are both in the range of 0~1, and the coding grid size is , , Express Round down, Express Round up; Express Round down, Express Round up, the coding grid points a, b, c, and d are , , , , Represent the feature codes of the coded grid points a, b, c, and d respectively.

5. The method according to claim 1, characterized in that The training steps of the illumination prediction network include: Acquire a training data set, each training data set sample includes the input data and output data, the output data is the actual lighting effect, and the actual lighting effect is calculated using a preset rendering equation; The input data is used as the input of the illumination prediction network, and the illumination prediction network is trained according to the output data to obtain a trained illumination prediction network.

6. The method according to claim 1, characterized in that The hidden layer of the illumination prediction network is 2 or 3 layers, the activation function between each layer of the illumination prediction network adopts the ReLU function, and the loss function adopted during the training of the illumination prediction network is the MES loss function.

7. A three-dimensional scene rendering device, characterized in that: The device comprises: A shading point acquisition unit, used to acquire shading points on the surface of a three-dimensional object corresponding to a screen pixel; A feature information acquisition unit, configured to acquire feature information of the shading point from the geometric image set according to the two-dimensional coordinates of the shading point on the geometric image set; A feature code calculation unit, used to calculate the feature code of the shading point according to the two-dimensional coordinates; an illumination calculation unit, configured to input the feature information, the feature code, and the light angle from the shading point to the camera into a trained illumination prediction network to obtain the illumination effect of the shading point, wherein the illumination prediction network is a multi-layer perceptron neural network, and the input data of each training data set sample of the illumination prediction network includes the two-dimensional coordinates of a shading point, the feature information, the feature code, and the light angle from the shading point to the camera; and The geometric image generation unit is used to generate the geometric image set according to the feature information of the three-dimensional object, and a geometric image in the geometric image set stores a kind of feature information of the three-dimensional object.

8. A three-dimensional scene rendering device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images