Method and device for adaptive arrangement of light probes in real-time rendering

Abstract

The application discloses a kind of real-time rendering in light probe adaptive arrangement method and device, the present application includes real-time renderer and reinforcement learning network, real-time renderer is according to the position coordinate of light probe, space is divided into several tetrahedron grid, and the space structure information after division is stored, while generating light probe storage structure, store indirect light information and depth information;And the indirect light provided by light probe is combined with direct light to form the final light rendering image;Reinforcement learning network is based on light rendering image according to preset feedback function calculation feedback value;Based on feedback value, the position coordinate of light probe and its strategy are updated;The updated light probe position coordinate is input to real-time renderer;Loop repeatedly executes until the light rendering image generated by real-time renderer reaches preset effect.The application can effectively improve the efficiency and quality of probe arrangement under the condition that the number of light probes is limited.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer graphics, and particularly relates to a light probe adaptive arrangement method and device in real-time rendering. BACKGROUND

[0002] Real-time rendering is an important research field in the graphics and game industry. Light probe is a widely used technology in the field of real-time rendering at present. According to certain rules, a plurality of light probes are generated in space, and the light radiation information near each light probe is sampled and stored. During running, the light information of any point can be calculated by means of nearest interpolation, so that a relatively real global light effect can be approximated with small space and time cost.

[0003] Generally, the placement strategy of light probes in the scene is a uniform grid structure. This structure facilitates the positioning of the shading point, and the light information at the shading point can be directly obtained by tri-linear interpolation. However, the frequency of light field change in the actual scene is often non-uniform, and there are areas where the light change is dense and obvious, and areas where the light change is sparse. Sampling the light distribution of the scene in the same and equal interval form may cause unnecessary space waste. At the same time, neglecting the occlusion in the scene and directly distributing the probes uniformly may easily lead to light leakage. Therefore, although uniform placement saves time cost, it cannot achieve the ideal effect.

[0004] Artists usually choose to manually place light probes non-uniformly in order to achieve the optimal effect of game pictures in real time. However, the distribution of a large number of light probes is often required for constructing large-scale game scenes, and manually placing light probes one by one is time-consuming and laborious. Therefore, the adaptive placement scheme of light probes has been a problem studied by the rendering community.

[0005] Reinforcement learning is a learning mechanism that maps state-action and learns how to obtain the maximum feedback. The learning goal does not know the value of each action in advance, but must try to find out which action will produce the maximum return by executing the action. In the reinforcement learning algorithm, there are three core parameters, namely state (s), action (a) and feedback (r). The state is used to represent the information of the target at a certain time, the action represents the action that the target can take, and the feedback is the immediate income that the object can obtain by taking a certain action in a certain state. Assuming that the function Q(s, a) represents the value obtained by executing the action a in the state s, then the model gradually updates the Q function by continuous learning to adjust its behavior and obtain the maximum value. SUMMARY

[0006] The application aims at the problems in the prior art and provides a light probe adaptive arrangement method and device in real-time rendering.

[0007] The light probe adaptive arrangement method in real-time rendering comprises the following steps:

[0008] (1) constructing and initializing a reinforcement learning network with the light probe position as the state and the moving direction as the action;

[0009] (2) inputting the number of light probes and the initial position coordinates of each light probe to the real-time renderer;

[0010] (3) dividing the space into a plurality of tetrahedral grids according to the position coordinates of the light probes by the real-time renderer, storing the divided space structure information, generating a light probe storage structure, and storing the indirect light information and the depth information;

[0011] (4) combining the indirect light provided by the light probe with the direct light to form a final light rendering image by the real-time renderer;

[0012] (5) calculating a feedback value according to a preset feedback function based on the light rendering image;

[0013] (6) updating the position coordinates of the light probe and the self-strategy of the reinforcement learning network based on the feedback value;

[0014] (7) inputting the updated position coordinates of the light probe to the real-time renderer;

[0015] (8) repeating steps (3) to (7) until the light rendering image generated by the real-time renderer reaches a preset effect.

[0016] Further, step (1) specifically comprises the following steps:

[0017] (1.1) constructing a reinforcement learning network, which is specifically a DQN network, comprising a plurality of sequentially connected fully connected layers, the last fully connected layer being an output layer with a dimension of 6N, N being the number of light probes;

[0018] (1.2) setting the state of the reinforcement learning network as the position coordinates of each light probe and the action as the movement of the light probe in the six directions of up, down, left, right, front and back;

[0019] (1.3) setting the initial exploration rate of the reinforcement learning network and the exploration rate when the exploration is terminated, and randomly obtaining the initial position coordinates of the light probe.

[0020] Further, step (3) specifically comprises the following steps:

[0021] (3.1) The real-time renderer performs Delaunay tetrahedral subdivision with the position of the light probe as the vertex, and obtains several tetrahedral meshes;

[0022] (3.2) Store the spatial structure information of each tetrahedral mesh, including the vertex index and coordinates of the tetrahedron, the index of the light probes it contains, the index of the face it contains, and the index of the four neighboring tetrahedral meshes.

[0023] (3.3) Generate a light probe storage structure to store the indirect light irradiance map, the depth mean map, and the depth square mean map.

[0024] Furthermore, the indirect illumination irradiance map is obtained through the following method:

[0025] A texture is constructed, and the light probe emits light in all directions using the Fibonacci spherical sampling method. The color information at the collision points with the scene is recorded. These collision points are used as secondary light sources, and the irradiance is calculated in each direction. That is, the irradiance provided by the secondary light source is integrated to obtain the indirect lighting irradiance provided by the light probe in each direction. This indirect lighting irradiance is then stored in the constructed texture using an octahedral mapping method, resulting in an indirect lighting irradiance texture.

[0026] Furthermore, the Fibonacci spherical sampling method is specifically as follows:

[0027] Sampling should be performed according to the following formula:

[0028] P i =(cos(φ) i sin(θ) i ),sin(φ i sin(θ) i ),cos(θ i ))

[0029]

[0030]

[0031]

[0032] In the formula, P i Let be the i-th sampling point, and n be the number of sampling points.

[0033] Furthermore, step (4) specifically includes:

[0034] (4.1) For a shading point in the scene, an initial tetrahedral mesh is randomly selected based on the stored spatial structure information, and the barycentric coordinates of the shading point are calculated. If the barycentric coordinates have negative components, the shading point is moved to a neighboring tetrahedral mesh in the direction of the smallest negative component until all components of the barycentric coordinates are positive. At this time, it is indicated that the tetrahedral mesh where the shading point is located is found;

[0035] (4.2) Four light probes on the four vertices of the tetrahedral mesh where the shading point is located are obtained, and the indirect lighting irradiance map of the four light probes is sampled, and the barycentric coordinates are used to interpolate the sampled indirect lighting irradiance;

[0036] (4.3) According to the depth mean map and the square mean map of the depth stored in the light probe storage structure, the mean and variance of the depth of the light probe in the direction of the shading point are obtained, and then the Chebyshev inequality is used to calculate the visibility value of the shading point to the probe in combination with the depth of the shading point relative to the probe;

[0037] (4.4) The interpolated indirect lighting irradiance is weighted and calculated according to the visibility value to obtain the indirect lighting irradiance of the shading point, and then the direct lighting irradiance is added to obtain the final lighting rendering result.

[0038] Optionally, step (5) can include:

[0039] (5.1A) Generate a reference standard Ground-truth;

[0040] (5.2A) Calculate the feedback value based on the lighting rendering image and the reference standard Ground-truth according to the following feedback function:

[0041]

[0042] In the formula, R represents the feedback value, width and height represent the length and width of the lighting rendering image respectively, r i,j , g i,j , b i,j are the color values of the r, g, and b channels of the i-th row and j-th column pixel point of the lighting rendering image, r' i,j , g' i,j , b' i,j are the color values of the r, g, and b channels of the i-th row and j-th column pixel point of the reference standard.

[0043] Optionally, step (5) can also include:

[0044] (5.1B) Render the entire scene onto a map to generate a scene map;

[0045] (5.2B) For each light probe, calculate its visibility value for each pixel on the scene map, merge the pixel set covered by all light probes based on the visibility value, and calculate the number of pixels I contained in the merged set N ;

[0046] (5.3B) For each light probe, calculate the distance from it to all visible pixels on the scene map, and assign a positive or negative value to the distance in the following way to obtain the final distance value d k : If the dot product of the vector from the light probe to the scene surface shading point and the vector of the scene surface normal is positive, that is, the included angle is acute, it indicates that the probe is inside the object, and the distance is recorded as negative; otherwise, the distance is recorded as positive;

[0047] (5.4B) For each light probe, calculate the dot product of the vector from each shading point on the scene map to the light probe and the surface normal, and accumulate the dot product results of all light probes to record F;

[0048] (5.5B) Calculate the feedback value according to the following feedback function:

[0049]

[0050] In the formula, R represents the feedback value, alpha, beta, and gamma represent the corresponding weights, and G represents the set of visible pixels on the scene map.

[0051] The real-time rendering light probe adaptive arrangement device provided by the application comprises:

[0052] A real-time renderer is used to divide the space into a plurality of tetrahedral meshes according to the position coordinates of the light probes, store the space structure information after the division, generate a light probe storage structure, store the indirect light information and depth information, and combine the indirect light provided by the light probe with the direct light to form a final light rendering image;

[0053] A reinforcement learning network is used to calculate a feedback value based on the light rendering image according to a preset feedback function, update the position coordinates of the light probe and its own strategy based on the feedback value, and input the updated position coordinates of the light probe to the real-time renderer; wherein the reinforcement learning network takes the position of the light probe as the state and takes the moving direction as the action.

[0054] Advantages: Compared with the prior art, the application has the following advantages:

[0055] 1. A new idea is provided for the adaptive arrangement of light probes, that is, a reinforcement learning idea is used to train the model in real time, so that the light probe position vector quickly converges to the ideal state.

[0056] 2. This invention can effectively avoid the light leakage problem of existing light probe placement technology through adaptive non-uniform arrangement of reinforcement learning. Moreover, under the premise of limited device memory resources, this invention can achieve better rendering effect with a smaller number of light probes. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the adaptive arrangement method of lighting probes in real-time rendering provided by the present invention.

[0058] Figure 2 This is a module relationship diagram of the present invention;

[0059] Figure 3 This is a schematic diagram illustrating the interpolation of the illumination probe within a tetrahedral mesh according to the present invention;

[0060] Figure 4 This is a schematic diagram illustrating the extrapolation illumination probe of the tetrahedral mesh according to the present invention;

[0061] Figure 5 This is a rendering comparison of the present invention with Ground-truth and uniform probe distribution;

[0062] Figure 6 This is a diagram showing the distribution of the illumination probe positions before and after training according to the present invention;

[0063] Figure 7 This is a comparison chart of the results before and after training according to the present invention. Detailed Implementation

[0064] This embodiment provides a method for adaptive arrangement of lighting probes in real-time rendering, such as... Figure 1 and Figure 2 As shown, it includes the following steps:

[0065] (1) Construct and initialize a reinforcement learning network with the position of the illumination probe as the state and the direction of movement as the action.

[0066] This step specifically includes:

[0067] (1.1) Construct a reinforcement learning network, specifically a DQN network, which includes several fully connected layers connected in sequence, with the last fully connected layer being the output layer. The dimension is 6N, where N is the number of light probes. In this embodiment, the number of light probes is set to 8, the number of fully connected layers is 3, each layer has 48 neurons, and the activation function of the network is set to Softmax.

[0068] (1.2) Set the state of the reinforcement learning network as the coordinates of each light probe position, and the action as the movement of the light probe in the six directions of up, down, left, right, front, and back; the position of each light probe is a vector of length 3, so the state of the model can be represented by a vector of length 3n. In the scene, the action of each probe can be moving in the four directions of front, back, up, down, left, and right by one unit length, so the choice of action for the entire scene at each step is 6n.

[0069] (1.3) Set the initial exploration rate of the reinforcement learning network to 1 and the exploration rate at the end of exploration to 0.01, and randomly obtain the initial position coordinates of the light probe.

[0070] (2) Input the number of light probes and the initial position coordinates of each light probe to the real-time renderer.

[0071] (3) The real-time renderer divides the space into a number of tetrahedral meshes according to the position coordinates of the light probes, and stores the divided space structure information, and generates a light probe storage structure to store indirect lighting information and depth information.

[0072] This step specifically includes:

[0073] (3.1) The real-time renderer performs Delaunay tetrahedral division with the light probe position as the vertex to obtain a number of tetrahedral meshes; specifically, the API provided by Tetgen can be used to implement it;

[0074] (3.2) Store the space structure information of each tetrahedral mesh, including the vertex index and coordinates of the tetrahedron, the index of the contained light probe, the contained face index, and the corresponding four neighbor tetrahedral mesh indexes;

[0075] (3.3) Generate a light probe storage structure to store the indirect lighting irradiance map, the depth average map, and the square mean of the depth map.

[0076] The indirect lighting irradiance map is obtained by the following method:

[0077] A map is constructed, and the light probe is sampled in a Fibonacci spherical sampling manner, emitting light in 512 directions around it, recording the color information at the collision points with the scene, and using these collision points as secondary light sources. Calculate the irradiance in each direction, that is, integrate the radiance provided by the secondary light source, to obtain the indirect lighting irradiance provided by the light probe in each direction, and use octahedral mapping to store it in the constructed map to obtain the indirect lighting irradiance map.

[0078] The Fibonacci spherical sampling manner is specifically:

[0079] Sampling is performed according to the following formula:

[0080] P i =(cos(φ i )sin(θ i ),sin(φ i )sin(θ i ),cos(θ i ))

[0081]

[0082]

[0083]

[0084] In the formula, P i is the i-th point of sampling, and n is the number of sampling points.

[0085] (4) The real-time renderer combines the indirect light provided by the light probe with the direct light to form a final light rendering image.

[0086] This step specifically includes:

[0087] (4.1) For a shading point in the scene, an initial tetrahedral mesh is randomly selected based on the stored spatial structure information to calculate the barycentric coordinates thereof, if the barycentric coordinates have negative components, the barycentric coordinates are moved to the neighbor tetrahedral mesh in the direction corresponding to the smallest negative component until all components of the barycentric coordinates are positive values, at this time it is indicated that the tetrahedral mesh where the shading point is located is found;

[0088] (4.2) The light probes on the four vertices of the tetrahedral mesh where the shading point is located are obtained, the indirect light irradiance map of the four light probes is sampled, and the barycentric coordinates are used to interpolate the sampled indirect light irradiance; the specific interpolation scheme comes from: [CUPISZ R. Light probe interpolation using tetrahedral tessellations [C] / / Game Developers Conference (GDC). 2012.];

[0089] (4.3) According to the depth mean map and the square mean map of the depth stored by the light probe storage structure, the mean and variance of the depth of the light probe in the direction of the shading point are obtained, and then the visibility value of the shading point to the probe is calculated by using Chebyshev inequality in combination with the depth of the shading point relative to the light probe.

[0090] (4.4) The indirect light irradiance after interpolation is weighted according to the visibility value to obtain the indirect light irradiance of the shading point, which is added to the direct light irradiance to obtain the final light rendering result.

[0091] Specifically, the method for calculating the barycentric coordinates of the internal space of the tetrahedral mesh is as follows: if the shading point P and the internal space tetrahedron (P1, P2, P3, P4) are known as shown in Figure 3 The three-dimensional barycentric coordinates can be derived from the two-dimensional barycentric coordinates as follows:

[0092] P(x, y, z) = aP0 + bP1 + cP2 + dP3

[0093] a + b + c + d = 1

[0094] After being arranged in matrix form, we have:

[0095]

[0096] d = 1 - a - b - c

[0097] Since only P is real-time input data, the matrix can be pre-computed to improve the efficiency of real-time rendering.

[0098] The method for calculating the barycentric coordinates of the external space of the tetrahedral mesh is as follows:

[0099] As shown in Figure 4 , let the position of the Shading point be point P, and the positions of the three Probes corresponding to the external space be P0, P1, P2, be the surface normal of the triangle P0P1P2, be the boundary ray of the space, and satisfy the projection in direction is 1. Then the triangle P0P1P2 can be extrapolated to the triangle Q0Q1Q2 along , and P is on the plane where Q0Q1Q2 is located:

[0100]

[0101]

[0102]

[0103]

[0104]

[0105] Since P is on the plane where Q0Q1Q2 is located, we have:

[0106] P = aQ0 + bQ1 + cQ2

[0107]

[0108]

[0109]

[0110] A = P0 - P1

[0111]

[0112] B = P1 - P2

[0113]

[0114] C = P2 - P

[0115]

[0116]

[0117] In order to make homogeneous equation There is a non-zero solution, then the determinant of T must be 0:

[0118] det(T) = 0

[0119] In det(T), only t is unknown, so the equation is finally simplified to a cubic equation about t.

[0120] Suppose the equation is finally:

[0121] t 3 + pt 2 + qt + r = 0

[0122] The coefficients p, q, r of the equation will be related to P0, P1, P2, and P. Since the coordinates of point P are input in real time, the equation coefficients can be expressed in the following form:

[0123]

[0124] Where the matrix m 3×4 is related to , which can be pre-computed to improve the efficiency of real-time calculation.

[0125] According to the above equation, the coefficients are obtained, and the cubic equation is solved.

[0126] After solving t, the coordinates of Q0, Q1, Q2 can be obtained, and the barycentric coordinates of point P in triangle Q0Q1Q2 can be calculated.

[0127] ​(5) calculating a feedback value according to a preset feedback function based on the illumination rendered image.

[0128] The implementation of step (5) can be various, and the present embodiment gives two implementation manners A and B. The manner A includes:

[0129] (5.1A) generating a reference standard Ground-truth by using Pathtracer;

[0130] (5.2A) calculating a feedback value according to the following feedback function based on the illumination rendered image and the reference standard Ground-truth:

[0131]

[0132] In the formula, R represents the feedback value, width and height respectively represent the length and width of the illumination rendered image, r i,j , g i,j , b i,j respectively represent the color values of the r, g and b channels of the i-th row and j-th column pixel point of the illumination rendered image, r' i,j , g' i,j , b' i,j respectively represent the color values of the r, g and b channels of the i-th row and j-th column pixel point of the reference standard. The generation of Ground-truth in the manner A is time-consuming, and the time cost required by the probe adaptive arrangement is high.

[0133] The manner B includes:

[0134] (5.1B) rendering the entire scene onto a map to generate a scene map;

[0135] (5.2B) calculating the visibility value of each illumination probe to each pixel on the scene map, and performing a union set on all the pixel sets covered by the illumination probes based on the visibility value, and calculating the number I N of pixels contained in the union set; the more the number of pixels, the higher the reward given to the model.

[0136] (5.3B) calculating the distance of each illumination probe to all the visible pixels on the scene map, and assigning a positive value or a negative value to the distance according to the following manner to obtain a final distance value d k : if the dot product of the vector from the illumination probe to the coloring point on the surface of the scene and the vector of the normal of the surface of the scene is positive, that is, the included angle is acute, it indicates that the probe is inside the object, and the distance is recorded as a negative value; otherwise, the distance is recorded as a positive value; mistakenly placing the probe inside the object is one of the common causes of light leakage, and this problem can be avoided by the distance index.

[0137] (5.4B) For each light probe, calculate the dot product of the vector from each shading point visible on the scene map to the light probe and the surface normal, and accumulate the dot product results of all light probes, denoted as F. The closer the vector from the probe to the surface shading point to the surface is to 0° (Grazing Angle), the smaller the light probe's radiance contribution to the shading point, and smaller contribution is prone to numerical precision errors, which cannot achieve the expected effect on surface shading, so the model can try to avoid Grazing Angle when learning.

[0138] (5.5B) Calculate the feedback value according to the following feedback function:

[0139]

[0140] In the formula, R represents the feedback value, a, β, γ represent the corresponding weights, and G represents the set of visible pixels on the scene map. The weights of each index can be learned autonomously by the model.

[0141] (6) The reinforcement learning network updates the position coordinates of the light probe and its own strategy based on the feedback value.

[0142] During the training of the reinforcement learning network, the Batchsize is set to 32, the learning rate is set to 0.001, the exploration rate in the initial stage is 1, and the exploration rate at the end of exploration is 0.01. During training, the exploration rate is adjusted to gradually decrease. Then, it is determined by random whether to explore. If the determination result is to explore, a random behavior is selected for exploration, otherwise the current state is input into the previous training network for behavior selection, and the four-tuple of the current state, the selected action, the Reward and the next state is stored in the experience pool as an experience. When the experience pool stores enough information, a number of experiences are taken out from the experience pool for training. During training, the real-time renderer is interacted with to obtain feedback to guide the next action of the model.

[0143] (7) Input the updated position coordinates of the light probe into the real-time renderer.

[0144] (8) Repeat (3) to (7) until the light rendering image generated by the real-time renderer reaches the preset effect.

[0145] The renderer in this embodiment is built using the G3D renderer in the C++ environment, and the training network is built using Tensorflow in Python. The communication between the two can be realized using shared memory. Specifically, the mmap library can be used in Python, and the CreateFileMapping and MapViewOfFile functions can be used in C++.

[0146] The embodiment also provides a real-time rendering light probe adaptive arrangement device, which comprises:

[0147] The real-time renderer is configured to divide the space into a plurality of tetrahedral meshes according to the position coordinates of the light probes, store the divided space structure information, generate a light probe storage structure, store indirect light information and depth information, and combine the indirect light provided by the light probes with direct light to form a final light rendering image.

[0148] The reinforcement learning network is configured to calculate a feedback value according to a preset feedback function based on the light rendering image, update the position coordinates of the light probes and the policy of the light probes based on the feedback value, and input the updated position coordinates of the light probes to the real-time renderer, wherein the reinforcement learning network takes the position of the light probes as a state and takes a moving direction as an action.

[0149] The embodiment corresponds to the above method one by one, and details not described in the method are not described again.

[0150] The simulation verification of the present application is described below.

[0151] 64 probes are placed in the scene, the sampling number of each probe is 256, the probe texture size is 256*256, a weighting index unrelated to the ground truth is used to guide the model training, and after 20 Episides and 1600 iterations for each, the generated comparison results are as shown in Figure 5 The comparison of the default position of the light probe and the position learned by the network is as shown in Figure 6 In this example, it can be seen that the light probes arranged by the method tend to be in areas with more intensive light changes. Figure 7 8 probes are trained for 2500 rounds, and the comparison chart before and after training can observe more obvious indirect light under the left pool, which proves the effectiveness of the method.

[0152] The above only discloses a preferred embodiment of the present application and cannot limit the scope of the right of the present application, so equivalent changes made according to the claims of the present application still fall within the scope of the present application.

Claims

1. A method for adaptive placement of light probes in real-time rendering, characterized in that The method comprises the following steps: (1) constructing and initializing a reinforcement learning network with the coordinates of the light probe positions as the state and the moving direction as the action; (2) inputting the number of light probes and the initial position coordinates of each light probe into the real-time renderer; (3) dividing the space into a plurality of tetrahedral meshes according to the position coordinates of the light probes by the real-time renderer, and storing the divided space structure information, and generating a light probe storage structure to store the indirect light information and the depth information; (4) combining the indirect light provided by the light probe with the direct light to form a final light rendering image by the real-time renderer; (5) calculating a feedback value according to a preset feedback function based on the light rendering image; (6) updating the position coordinates of the light probes and the policy of the reinforcement learning network based on the feedback value; (7) inputting the updated position coordinates of the light probes into the real-time renderer; (8) repeating steps (3) to (7) until the light rendering image generated by the real-time renderer reaches a preset effect.

2. The method of claim 1, wherein: Step (1) specifically comprises: (1.1) constructing a reinforcement learning network, which is specifically a DQN network, comprising a plurality of sequentially connected fully connected layers, and the last fully connected layer is an output layer with a dimension of 6N, wherein N is the number of light probes; (1.2) setting the state of the reinforcement learning network as the position coordinates of each light probe, and setting the action as the movement of the light probe in the up, down, left, right, front and back six directions; (1.3) setting the initial exploration rate of the reinforcement learning network and the exploration rate when the exploration is terminated, and randomly obtaining the initial position coordinates of the light probe.

3. The method of claim 1, wherein: Step (3) specifically comprises: (3.1) performing Delaunay tetrahedral division with the light probe positions as the vertices to obtain a plurality of tetrahedral meshes; (3.2) storing the space structure information of each tetrahedral mesh, including the vertex index and coordinates of the tetrahedron, the index of the contained light probe, the contained face index, and the corresponding four neighbor tetrahedral mesh indexes; (3.3) generating a light probe storage structure to store the indirect light irradiance map, the depth average map and the square average map of the depth.

4. The method of claim 3, wherein: The indirect light irradiance map is obtained by the following method: A map is constructed, and the light probe is sampled in a Fibonacci spherical sampling manner, and light rays are emitted in the four directions, and the color information at the scene collision points is recorded, and these collision points are used as secondary light sources, and the irradiance in each direction is calculated, that is, the irradiance provided by the secondary light source is integrated to obtain the indirect light irradiance provided by the light probe in each direction, and the indirect light irradiance map is stored in the constructed map by using octahedral mapping.

5. The method of claim 4, wherein: The Fibonacci spherical sampling manner specifically comprises: Sampling is performed according to the following formula: P i = (cos(φ i )sin(θ i ), sin(φ i )sin(θ i ), cos(θ i )) In the formula, P i is the i-th point of the sampling, and n is the number of sampling points.

6. The method of claim 1, wherein: Step (4) specifically comprises: (4.1) selecting an initial tetrahedral mesh to calculate the barycentric coordinates of a coloring point in the scene based on the stored space structure information, and if the barycentric coordinates have negative components, moving to the neighbor tetrahedral mesh in the direction corresponding to the smallest negative component until all components of the barycentric coordinates are positive, at which time it is indicated that the tetrahedral mesh where the coloring point is located is found; (4.2) obtaining the light probes on four vertices of the tetrahedral mesh where the shading point is located, sampling the indirect light irradiance map of the four light probes, and interpolating the sampled indirect light irradiance by using the barycentric coordinates; (4.3) obtaining the mean and variance of the depth of the light probe in the direction of the shading point according to the mean depth map and the mean square depth map stored in the light probe storage structure, and then combining the depth of the shading point relative to the light probe to calculate the visibility value of the shading point to the probe by using the Chebyshev inequality; (4.4) weighting the interpolated indirect light irradiance according to the visibility value to obtain the indirect light irradiance of the shading point, and then adding the direct light irradiance to obtain the final light rendering result.

7. The method of claim 1, wherein: Step (5) specifically includes: (5.1A) generating a reference standard Ground-truth; (5.2A) calculating a feedback value according to the following feedback function based on the light rendering image and the reference standard Ground-truth: where R represents a feedback value, width and height represent the length and width of the light rendering image respectively, r i,j , g i,j , b i,j are color values of the r, g, b channels of the i-th row and j-th column pixel point of the light rendering image, r′ i,j , g′ i,j , b′ i,j are color values of the r, g, b channels of the i-th row and j-th column pixel point of the reference standard.

8. The method of claim 1, wherein: Step (5) specifically includes: (5.1B) rendering the entire scene onto a map to generate a scene map; (5.2B) For each light probe, compute its visibility value for each pixel on the scene map, union all the pixels that can be covered by all the light probes based on the visibility value, and compute the number of pixels I contained in the union N ; (5.3B) For each light probe, calculate its distance to all visible pixels on the scene map, and assign a positive or negative value to the distance in the following way to get the final distance value d k : If the dot product of the vector from the light probe to the shading point on the scene surface and the vector of the scene surface normal is positive, i.e. the angle is acute, it means the probe is inside the object, assign a negative value to the distance; otherwise, assign a positive value to the distance; (5.4B) for each light probe, calculating the dot product of the vector from each visible shading point on the scene map to the light probe and the surface normal, and accumulating the dot product results of all light probes as F; (5.5B) calculating a feedback value according to the following feedback function: In the formula, R represents the feedback value, a, β, and γ represent corresponding weights, and G represents a set of visible pixels on the scene map.

9. A device for adaptive arrangement of lighting probes in real-time rendering, characterized in that... including: a real-time renderer configured to divide a space into a plurality of tetrahedral meshes according to position coordinates of light probes, store spatial structure information of the divided space, and generate a light probe storage structure to store indirect light information and depth information; and combine the indirect light provided by the light probe with direct light to form a final light rendering image; a reinforcement learning network configured to calculate a feedback value based on the light rendering image according to a preset feedback function, update the position coordinates of the light probe and its own strategy based on the feedback value, and input the updated position coordinates of the light probe to the real-time renderer; wherein the reinforcement learning network takes the position coordinates of the light probe as a state and a moving direction as an action.

Images