Global illumination calculation method, graphic processing device, readable storage medium and program product

Abstract

The invention discloses a global illumination calculation method, a graphic processing device, a readable storage medium and a program product, which can avoid light leakage, mottling and the like under the condition that manual marking is not needed. The global illumination calculation method comprises the following steps: collecting a plurality of sampling points in a virtual scene and arranging a plurality of illumination probes; baking the plurality of sampling points based on light source information in the virtual scene to obtain illumination information of each sampling point; classifying the plurality of sampling points according to the illumination information and the position information of each sampling point; calculating illumination information and weight information of each illumination probe based on the illumination information and the position information of the sampling points of each category, wherein the weight information represents the contribution degree of the illumination information, corresponding to each category, of each illumination probe to illumination coloring of coloring points in the virtual scene; and performing illumination calculation on the virtual scene according to the illumination information and the weight information of the plurality of illumination probes.

Description

Technical Field

[0001] The present invention relates to the field of graphics rendering, and in particular to a global illumination calculation method, a graphics processing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] It is known that in the fields of film, animation and game production, the quality of production depends to a certain extent on whether the rendering technology used is high quality. Global illumination technology is a lighting technology widely used in the rendering environment link in current film and television and game production, which can effectively enhance the realism of the scene. At present, as far as the game industry is concerned, from the perspective of technical principles, the technology for realizing global illumination can generally be divided into two categories, one is a method based on three-dimensional space, and the other is a method based on screen space. However, both types of technology have their own advantages and disadvantages. Which type of technology to choose usually depends on the accuracy requirements of the specific application, real-time requirements, scene characteristics and computing resources. At present, as far as the three-dimensional space-based method is concerned, it includes rendering technology using Light Map, rendering technology based on Irradiance Volume, ray tracing / path tracing technology, voxel-based global illumination rendering technology, etc.

[0003] As a rendering technology using Light Map, the mesh is unfolded and mapped to a map, and the baked lighting results of the mesh surface are stored in the map, which becomes the Light Map. Then, when performing real-time rendering, the Light Map is sampled according to the Light Map UV of the mesh to obtain the baked lighting effect. However, this solution requires manual configuration of the Light Map UV for the mesh, and the configuration process is cumbersome, time-consuming and labor-intensive. In particular, in large map scenes, the workload will be very large.

[0004] As a rendering technology based on Irradiance Volume, multiple light probes are pre-set in the scene to record ambient light information and perform interpolation calculations during real-time rendering. Compared with the rendering solution using Light Map, this solution is suitable for large-scale scenes and has high calculation efficiency.

[0005] However, in some cases, two adjacent probes in a virtual scene such as a game scene may be located in spaces with different lighting. For ease of understanding, the following is an example of a two-dimensional case. Fig.11 Schematic diagram of a virtual scene containing a computation unit with light probes as vertices. Fig.11As shown, a building is constructed in the virtual scene, thereby dividing the virtual scene into an indoor area and an outdoor area. In addition, a square computing unit with a light probe as a vertex spans the building, a part of which is located in the indoor area, and the rest is located in the outdoor area. In particular, the vertex located in the lower left corner of the four vertices of the computing unit is located in the indoor area, and the other three vertices are located in the outdoor area. It is assumed that the indoor area and the outdoor area are completely isolated in an optical sense. In this way, the ambient lighting information recorded by the vertex located in the indoor area is the lighting information corresponding to the dark environment (in Fig.11 The ambient lighting information recorded by the three vertices located in the outdoor area is the lighting information corresponding to the bright environment (in Fig.11 In this case, if the lighting information of the shading point in the calculation unit is calculated by interpolation based on the lighting information recorded by the four lighting probes, the lighting calculation result of the shading point near the boundary between the indoor area and the outdoor area, that is, at the eaves, will deviate greatly from the actual result. As one result, gray will appear on the eaves, making the eaves look mottled. As another result, light leakage will occur in the indoor area that should have remained dark, and light will leak from the outdoor area into the indoor area. That is to say, in the face of such a virtual scene as described above, when the calculation unit spans two or more areas with large differences in ambient light information, the calculation result obtained by interpolating directly based on different types of lighting probes that record the lighting information corresponding to the ambient light information of each area is not accurate enough, which will cause the lighting of the shading point to be unrealistic and flawed. Summary of the invention

[0006] Technical problem to be solved by the invention

[0007] This application is formed to solve the above technical problems, and its purpose is to provide a global illumination calculation method that can avoid light leakage, mottle, etc. without the need for manual marking. On this basis, this application further provides a graphics processing device, a computer-readable storage medium, and a computer program product for implementing the above global illumination calculation method.

[0008] Technical solutions adopted to solve technical problems

[0009] The present application provides a global illumination calculation method, which is characterized by comprising:

[0010] Collect multiple sampling points and arrange multiple lighting probes in the virtual scene;

[0011] Baking the plurality of sampling points based on the light source information in the virtual scene to obtain lighting information of each of the sampling points;

[0012] Classifying the plurality of sampling points according to the illumination information and the position information of each of the sampling points;

[0013] Calculating lighting information and weight information of each lighting probe based on the lighting information and position information of the sampling points of each category, wherein the weight information represents the contribution of the lighting information corresponding to each category of each lighting probe to the lighting shading of the shading point in the virtual scene; and

[0014] Perform lighting calculation on the virtual scene according to the lighting information and weight information of the multiple lighting probes.

[0015] According to the global illumination calculation method described in the technical solution, unlike the prior art, by making the illumination probe have (probe) illumination information corresponding to each category of ambient light information, and at the same time making the illumination probe also have weight information for characterizing the contribution of each illumination information of the illumination probe to the illumination shading of the shading point in the virtual scene, it is possible to make each (probe) illumination information of each illumination probe participate in the calculation of the illumination information of the shading point in an appropriate combination according to the coordinate (position) information of the shading point in the virtual scene and the positional relationship between the shading point and each illumination probe used to calculate the illumination information of the shading point. Compared with the illumination shading of the shading point based on the illumination probe with only a single illumination information, the real illumination condition at the shading point can be accurately calculated, thereby avoiding the occurrence of light leakage, mottle, etc.

[0016] Optionally, in the above-mentioned global illumination calculation method, the virtual scene is divided into multiple calculation units with the multiple illumination probes as vertices, and the shading points in the virtual scene are illuminated and shaded according to the illumination information and weight information of each illumination probe of the calculation unit where the shading point is located.

[0017] According to the global illumination calculation method described in the technical solution, by illuminating and shading the target shading point based on the (probe) illumination information of multiple illumination probes closest to the target shading point, the amount of calculation data can be reduced while ensuring the accuracy of the illumination calculation.

[0018] Optionally, in the above-mentioned global illumination calculation method, the shading point is illuminated and shaded according to the distance of each illumination probe of the calculation unit where the shading point is located relative to the shading point and the illumination information and weight information of each illumination probe.

[0019] Optionally, in the above-mentioned global illumination calculation method, some of the plurality of calculation units are cuboid units with eight illumination probes as vertices. The shading point is illuminated by linearly interpolating the illumination information and weight information corresponding to each category of the eight illumination probes of the cuboid unit where the shading point is located relative to the shading point, and performing weighted summation of the interpolated illumination information corresponding to each category based on the interpolated weight information.

[0020] According to the global illumination calculation method described in the technical solution, the illumination shading of the shading points can be performed while fully considering different ambient light information, which can improve the accuracy of the illumination shading.

[0021] Optionally, in the above-mentioned global illumination calculation method, some of the multiple calculation units are tetrahedral units formed by performing Delaunay triangulation on some of the multiple illumination probes.

[0022] According to the global illumination calculation method described in the technical solution, the number of illumination probes participating in illumination shading calculation in a computing unit can be reduced, thereby improving the calculation speed and reducing the amount of calculation.

[0023] Optionally, in the above-mentioned global illumination calculation method, the number of the sampling points collected in the calculation unit that contains a grid of a virtual object in the virtual scene is greater than the number of the sampling points collected in the calculation unit that does not contain the grid.

[0024] According to the global illumination calculation method described in the technical solution, taking into account that areas with meshes are prone to light leakage and spotting, compared with areas without meshes, by collecting more sampling points in areas with meshes (i.e., a higher sampling density), the calculation accuracy of the (probe) illumination information and weight information of the illumination probes in the area can be improved.

[0025] Optionally, in the above global illumination calculation method, when the sampling point is located inside the grid space of the virtual object in the virtual scene, the sampling point is not baked.

[0026] According to the global illumination calculation method described in the technical solution, considering that the actual camera cannot enter the interior of the mesh, the sampling points located inside it are actually invalid samples. Therefore, by not sampling points located inside the mesh space, it is possible to avoid baking invalid samples while reducing unnecessary calculations.

[0027] Optionally, in the above-mentioned global illumination calculation method, the illumination information and weight information of each illumination probe are calculated based on the illumination information and position information of the sampling points of each category in at least a part of all the calculation units with the illumination probe as a common vertex.

[0028] According to the global illumination calculation method described in the technical solution, since a certain illumination probe will participate in the illumination shading of the shading points in each calculation unit with the illumination probe as the vertex, the calculation accuracy of the illumination information and weight information of the illumination probe can be improved by making the sampling points in all the calculation units with the illumination probe as the common vertex or at least a part of the calculation units participate in the determination of the illumination information and weight information of the illumination probe.

[0029] Optionally, in the above-mentioned global illumination calculation method, the sampling points include at least sampling points of a first category and sampling points of a second category. The illumination information of each of the illumination probes includes at least first illumination information and second illumination information. The first illumination information of each of the illumination probes is determined based on the illumination information and position information of the sampling points of the first category within the at least one portion of the calculation units with the illumination probe as a common vertex. The second illumination information of each of the illumination probes is determined based on the illumination information and position information of the sampling points of the second category within the at least one portion of the calculation units with the illumination probe as a common vertex. The weight information of each of the illumination probes is determined based on the illumination information and position information of all the sampling points within the at least one portion of the calculation units with the illumination probe as a common vertex.

[0030] Optionally, in the above-mentioned global illumination calculation method, when the sampling points in all the calculation units with the illumination probe as a common vertex belong to the same category, the illumination information corresponding to the category of the illumination probe is calculated only based on the illumination information and position information of the sampling points of the category.

[0031] According to the global illumination calculation method described in the technical solution, for a calculation unit in which all sampling points belong to the same category, considering that the ambient light in the range in which it is located is roughly in a state of little change, only the illumination information corresponding to the category is calculated. In this way, unnecessary calculation amount can be reduced.

[0032] In addition, the present application also provides a graphics processing device, a computer-readable storage medium, and a computer-readable product for implementing the global illumination calculation method described in any of the above technical solutions.

[0033] Effects of the Invention

[0034] According to the global illumination calculation method described in the present application, compared with the prior art in which the illumination probe only has a single illumination information, by making the illumination probe have illumination information corresponding to different categories of sampling points and weight information characterizing the contribution of each illumination information to the illumination shading of the shading point, the illumination information of the shading point near the boundary can be calculated more accurately, thereby avoiding the occurrence of light leakage, mottle and the like caused by inaccurate calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart showing a method for calculating global illumination according to one embodiment of the present application.

[0036] Figure 2 Schematic diagram showing the network composition of a point cloud processing network for sampling point classification.

[0037] Figure 3 It is a schematic diagram showing an example of a calculation unit used in a global illumination calculation method according to an embodiment of the present application.

[0038] Figure 4 A diagram showing all computational units with a light probe as a common vertex.

[0039] Figure 5 Table 4 Figure 4 A top view of the computing unit is shown.

[0040] Figure 6 It is a schematic diagram of a network model for preliminarily estimating first probe illumination information, second probe illumination information, and weight information corresponding to a first category and a second category, respectively, of an illumination probe.

[0041] Figure 7 is a graph showing how position proportional weights are calculated using trilinear interpolation.

[0042] Figure 8 It is a schematic diagram showing the lighting shading of a shading point within a computing unit based on the lighting information and weight information of the lighting probe of the computing unit.

[0043] Fig. 9 It is a schematic diagram showing the calculation method based on tetrahedron linear interpolation.

[0044] Fig.10 It is a schematic diagram showing the structure of a computer device used for the global illumination calculation method of the present application.

[0045] Fig.11 is a diagram showing a virtual scene containing a computational unit with light probes as vertices. DETAILED DESCRIPTION

[0046] Below, refer to Figures 1 to 8 , the main process of the global illumination calculation method of one embodiment of the present application is described.

[0047] However, before describing the following embodiments, some technical terms will be described first for ease of understanding and description.

[0048] The "virtual scene" mentioned in this application is, for example, a CG movie scene, an animation scene, or a game scene, and is not a real scene in a real three-dimensional space. The "virtual object (or virtual object)" mentioned in this application is a non-real element existing in a virtual scene, such as a character, building, vehicle, terrain, etc. in a game.

[0049] The "illumination information" mentioned in this application refers to the information required to achieve the rendering of the virtual scene, which mainly describes, including but not limited to, the intensity and color of light in different directions. It should be noted that the illumination information may have different representation forms (for example, SH coefficient, etc.), and the information described may also be slightly different.

[0050] The "global illumination (GI)" described in this application refers to an illumination technology that takes into account both the illumination directly from the light source in the virtual scene, i.e., direct illumination, and the illumination reflected by other virtual objects (or virtual objects) in the virtual scene, i.e., indirect illumination. In other words, global illumination is an illumination technology that takes into account both direct illumination and indirect illumination. Specifically, unlike the local illumination model, the global illumination model takes into account multiple reflections, refractions, and scattering of light in the virtual scene, simulating more complex and realistic illumination phenomena. Direct light is light from the light source, and indirect light is light emitted from the light source that is reflected and / or scattered from the surface of a virtual object to the surface of other virtual objects. The technical solution of this application discusses a global illumination technology that covers both direct illumination and indirect illumination, as well as a global illumination technology that only covers indirect illumination.

[0051] In addition, the “sampling point” described in the present application is a dummy point (Dummy Point) collected from a virtual scene according to a specific rule or randomly, and the lighting information at the point is recorded by baking.

[0052] In addition, the "Light Probe" described in this application is a data structure for storing scene lighting information. By pre-calculating and storing the lighting information at the probe, we can quickly obtain the global lighting effect of the scene at runtime.

[0053] Figure 1 A flow chart of a global illumination calculation method according to an embodiment of the present application is shown.

[0054] First, in step ST1, a plurality of sampling points are collected in the virtual scene. As described above, a "sampling point" is a virtual point (DummyPoint) collected from the virtual scene and recording the lighting information therein by baking. The sampling point can be considered as a virtual sphere with a small diameter. In this case, the lighting information in multiple directions of each sampling point is calculated, and a specific storage method is used to record the lighting information in multiple directions of each sampling point. For example, the lighting information in six directions of each sampling point can be calculated, and the six faces of the ambient light cube can be used to record the lighting information in the above six directions. Regarding the six directions, for example, six orthogonal directions (i.e., X, Y, Z) in a three-dimensional orthogonal coordinate system with the center of each spherical sampling point as the origin can be selected. + , X - , Y + , Y - , Z + , Z - ). In fact, the global illumination is decomposed into six faces and recorded separately. In addition, the sampling point can also be considered as a tiny imaginary cube with six faces. In this case, the illumination information irradiated to the face is precalculated based on the relative position relationship between the normal direction of each face and the irradiation direction of the light irradiated to the face. In addition, the sampling points can be collected (or arranged) in an individualized manner according to actual needs and spatial distribution. For example, if it is assumed that the virtual scene is a cubic indoor space, the sampling points can be evenly arranged in three orthogonal directions of the indoor space. However, the collection or arrangement form of the sampling points is not limited to this, and the sampling points can also be collected or arranged by an adaptive algorithm. On the other hand, generally speaking, there are often virtual objects such as characters and objects in the virtual scene, which are represented by specific geometric forms (for example, grids). Considering that the area where the grid is located is prone to light leakage and mottle, it is preferred to increase the collection frequency of the sampling points in the area where the grid is located. That is, compared with the area where there is no virtual object in the virtual scene, the sampling density in the area where the virtual object is located in the virtual scene is larger (in other words, the number of sampling points per unit space volume is larger).

[0055] In addition, in step ST1, a plurality of lighting probes are also arranged in the virtual scene. As described above, the lighting probe is a data structure for storing lighting information of the virtual scene. Similar to the sampling point, the lighting probe can be considered as an imaginary sphere with a small diameter, or as a small imaginary cube with six faces. In the prior art, the lighting information at the lighting probe is pre-calculated by each lighting probe, and the pre-calculated lighting information is stored in a specific form. It can be seen that in the prior art, each lighting probe only stores one set of lighting information. Different from this, in the present embodiment, each lighting probe can store multiple sets of lighting information according to the type of sampling point, which will be described in detail later. In addition, there is no particular restriction on the arrangement of the lighting probes, which can be evenly arranged in three orthogonal directions of the virtual scene, or arranged according to a certain rule. When the lighting probes are evenly arranged in the virtual scene in three orthogonal directions of the virtual scene, eight adjacent lighting probes in three orthogonal directions form a calculation unit (such as Figure 3 As shown in FIG. 1 , the shading points in the calculation unit are illuminated and shaded, which will be described in detail later. In addition, for the illumination probe, four adjacent illumination probes can be connected by Delaunay triangulation to form a tetrahedron as a calculation unit, so as to illuminate and shade the shading points inside the tetrahedron, which will also be described in detail later. After completing the action of step ST1, enter step ST2.

[0056] It should be noted that, although the collection of sampling points and the arrangement of illumination probes are both placed in step ST1 in this embodiment, this is only an example and does not mean that these two actions must be performed simultaneously. For example, sampling points can be collected in the virtual scene first, and then illumination probes can be arranged, or illumination probes can be arranged in the virtual scene first, and then sampling points can be collected, or sampling points can be collected and illumination probes can be arranged simultaneously, and any method can be adopted according to the actual situation.

[0057] In step ST2, multiple pre-collected sampling points are baked based on the light source information in the virtual scene, and the lighting information of each sampling point is recorded. The "light source information" mentioned here includes at least ambient light information, and on this basis, it may also include local light source information. In addition, the "baking" mentioned here is a lighting technical term, which refers to a process of pre-calculating any information related to lighting, that is, lighting information. Specifically, it is assumed that the sampling point is regarded as a sphere (point). In this case, the lighting information in six directions of the sampling point is pre-calculated. In other words, the sampling point is baked. After the lighting information in the six directions of the sampling point is obtained by baking, the lighting information is stored in a specific form. Specifically, considering that the lighting information of the sampling point is pre-calculated from six different directions, an ambient light cube (also called an ambient light box, ambient cube) can be used to store the lighting information. However, in this embodiment, the above-mentioned lighting information stored in the ambient light cube is not directly used for lighting and shading the shading points in the virtual scene, but is used to calculate the lighting information and weight information of each lighting probe information. The weight information will be described in detail later. In addition, it should be noted that the storage form of the illumination information of each sampling point is not limited to the ambient light cube, and other storage forms may be used according to actual conditions, such as octahedral mapping, spherical harmonic function, etc. After completing step ST2, proceed to step ST3.

[0058] In step ST3, the sampling points are classified according to the illumination information and position information of each of the multiple sampling points. Specifically, it is assumed that the virtual scene is an open field, and a building is configured in the field. In this way, there are both indoor space and outdoor space in the virtual scene. In this case, since the indoor space and the outdoor space are separated by the wall of the building, the light source information of the indoor space and the light source information of the outdoor space are different. As a result, there is a certain degree of difference between the illumination information (for example, light intensity or color) of the sampling points located in the indoor space and the illumination information (for example, light intensity or color) of the sampling points in the indoor space. Here, the sampling points with different illumination information are classified for the determination of illumination information and weight information of the illumination probe in the subsequent steps. As one of the methods for implementing classification, for example, the sampling points can be regarded as point clouds, thereby equivalently converting the classification problem of the sampling points into a point cloud segmentation problem. In this case, the sampling points can be classified using a point cloud processing network, and more specifically, each sampling point is marked to divide all the sampling points into different categories. Figure 2 Figure 2 shows a schematic diagram of the network structure of the point cloud processing network used for sampling point classification. Figure 2As shown, the position coordinate information of the sampling point and the pre-baked and stored illumination information are set as input data, and these data are sequentially input into the point cloud encoder (Point Encoder) and the Sigmoid function for operation, so as to obtain the label (Point Label) representing the category to which each sampling point belongs as the output result. Among them, the point cloud encoder is a point cloud feature extraction network, which includes a Point Transformer module, an upsampling module and a downsampling module. On the other hand, when classifying, a reference value P0 is set in advance (for example, P0=0.5). For each input sample point, the Sigmoid function returns a value in the range of 0 to 1, that is, a probability value P. By comparing the probability value P with the above-mentioned pre-set reference value P0, if P>P0, the sample corresponding to the probability value P is divided into the first category, otherwise it is divided into the second category. As a result, the sampling points are divided into two categories of sampling points, one is the sampling point belonging to the first category ( Figure 2 The other is the sampling point belonging to the second category ( Figure 2 In the figure, the right side is shown with hollow dots). In addition, the reference value P0 can also be an interval, for example, 0.45 to 0.54. When the probability value P of a certain sampling point is less than 0.45, the sample point is classified into the first category, and when the probability value P of a certain sampling point is greater than 0.54, the sample point is changed to the second category. However, when the reference value P0 is set as an interval, there may be a situation where the probability value P of a certain sampling point is between 0.45 and 0.54. At this time, it is impossible to classify the sampling point according to the above rules. For this reason, the sampling point can be divided into the first category or the second category in an alternative manner. It should be noted that in this embodiment, it is very important to classify the sampling points. Specifically, by calculating the illumination information of each illumination probe through sampling points of different categories, each illumination probe can have multiple sets of illumination information corresponding to each category, thereby solving the problems of light leakage and mottle. This point will be described in detail later. After completing the action of step ST3, enter step ST4.

[0059] In the above, a specific value or a specific interval is specified as a reference value P0 or a reference interval, and the categories of the sampling points are divided based on the reference value or the reference period, but the division method is not limited to this. For example, random sampling can also be used to divide the sampling points. Specifically, an open interval (0, 1) is assumed to satisfy a uniform random distribution (that is, within the interval, the probability of each value appearing is equal). Each time a sampling point is selected, a random number P' is generated from the above open interval. Since the probability value P of the sampling point has been determined and known by the Sigmoid function in the point cloud processing network, the probability value P of the sampling point is compared with the above random number P'. If the probability value P is less than the random number P', the sampling point is divided into the first category, otherwise the sampling point is divided into the second category. It should be emphasized again that when random sampling is used to classify the sampling points, a random value P' is randomly generated for each sampling point as a reference value, rather than specifying a fixed reference value for all sampling points.

[0060] In step ST4, the illumination information and weight information of each illumination probe are calculated based on the illumination information and position information of each category of sampling points, wherein the weight information represents the contribution of the illumination information corresponding to each category of each illumination probe to the illumination shading of each shading point in the virtual scene. For ease of understanding, the following description is made by taking the calculation unit as a cube as an example.

[0061] Figure 3 A schematic diagram showing an example of a computing unit, i.e., a cube, with the probe as a vertex, is shown. Figure 4 A schematic diagram of all (eight in total) computing units, i.e., a cube, with one probe as a common vertex is shown, Figure 5 Shows Figure 4 Top view of the . Figure 4 and Figure 5 An example of the distribution of sampling points with two categories is shown.

[0062] First, the illumination information and position information of each category of the illumination probe corresponding to each category are preliminarily estimated based on the illumination information and position information of each category of sampling points in all (eight in total) computing units or a part of the computing units with each illumination probe as a common vertex. For ease of understanding, the following is an example of the case where the sampling points include sampling points of the first category and sampling points of the second category. Figure 6 A schematic diagram of a network model for preliminarily estimating first probe illumination information, second probe illumination information, and weight information corresponding to a first category and a second category of an illumination probe is shown. Figure 6 The first half of the network model shown (i.e., the left half in the figure) is Figure 2The point cloud processing network shown in FIG. 1 is a network part that preliminarily estimates the first probe illumination information, the second probe illumination information, and the weight information of an illumination probe based on the output result of the point cloud processing network. Figure 4 The light probe in the center of the image is used as an example to illustrate this, and it is assumed that all sampling points in the eight computing units with the light probe as the common vertex participate in the estimation of the light information and weight information of the light probe. Figure 6 As shown, the input data used for preliminarily estimating the weight information of each lighting probe includes the categories and position information of all sampling points in the eight calculation units with the lighting probe as a common vertex. The input data used for preliminarily estimating the first probe lighting information of each lighting probe (in other words, the input data used for obtaining the estimated value of the first probe lighting information of each lighting probe) includes the lighting information and position information of the first category of sampling points in the eight calculation units with the lighting probe as a common vertex and the weighted average value V of the lighting information of all sampling points in the above eight calculation units. avg About the weighted average V avg , for example using Figure 7 The trilinear interpolation method shown in FIG. 1 calculates the position ratio weight of each light probe to each sampling point based on the position information of each light probe and the position information of all sampling points in eight calculation units with the light probe as a common vertex, and normalizes the position ratio weight. Based on the normalized position ratio weight of each sampling point and the light information, a weighted sum is performed on all sampling points to calculate the weighted average value V avg The input data used as the preliminary estimate of the second probe lighting information of each lighting probe includes the second category lighting information and position information of all sampling points in the eight computing units with the lighting probe as the common vertex and a randomly generated parameter r b (is a random vector). It should be noted that r b It is not regenerated at each network iteration, but is randomly initialized once when the network starts training. In other words, during the model inference phase, the parameter r b is already trained. Specifically, by inputting the respective categories and position information of all sampling points in the above eight computing units into the T prediction network, the weight information (weight value) estimation value T′ of the light probe is obtained as the output result. The T prediction network is, for example, a neural network including multiple layers (for example, three layers) of convolution, but is not limited thereto. As long as it can complete the above acquisition task, it can also be other types of neural networks. In addition, by taking the illumination information and position information of the first category of sampling points in the above eight computing units and the average illumination information V of all sampling points in the above eight computing units avgInput the Identity network to obtain the first probe illumination information estimation value A′ of the illumination probe as the output result. In addition, the illumination information and position information of the second category in the above eight computing units and the random vector r b Input the multi-layer perceptron network (MLP network) to obtain the second probe illumination information estimation value B′ of the illumination probe as the output result. In this way, the illumination information and weight information of each illumination probe can be preliminarily estimated based on the illumination information and position information of multiple sample points that have been classified. However, it should be noted that the above method is only a method for realizing the preliminary estimation of the illumination information and weight information of each illumination probe, and is not limited to this. The data input to each network and the type of network used to calculate the above information can also be modified according to actual needs, as long as the above purpose can be achieved. In addition, in the above implementation method, all sample points in all eight calculation units with the illumination probe as a common vertex participate in the above calculation as an example, but it is not limited to this. For example, all sample points in a part of the eight calculation units can also participate in the above calculation. In addition, even if the calculation unit is a cubic unit, the number of calculation units with a certain illumination probe as a common vertex is not necessarily eight. For example, at the boundary of the virtual scene, the number of calculation units with a certain illumination probe located near the boundary as a common vertex may be only four, three or even two. In summary, for each lighting probe, the lighting information and weight information of the lighting probe can be calculated using the lighting information and position information of each category of sampling points in all or part of the computing units taking the lighting probe as a common vertex.

[0063] In the above, the illumination information and position information of all sampling points of each category in all or part of the computation units based on a certain illumination probe as a common vertex are used to preliminarily estimate the illumination information and weight information of the illumination probe.

[0064] Next, for the sake of ease of understanding, we still take the two categories of sampling points, namely the first category and the second category, as an example to explain in detail how to fit the illumination information and weight information of each illumination probe based on the illumination information and position information of the sampling points and the illumination information estimation value and weight information estimation value of each illumination probe. First, for example, from Figure 4 and Figure 5 A sampling point is randomly selected from the sampling points shown in Figure 5 The eight illumination probes closest to the sampling point are determined by index, thereby determining the computation unit where the sampling point is located. Repeat the above operation until all sampling points in the computation unit are selected. Figure 3As shown, the eight illumination probes are set to P0 to P7 respectively. At this time, since the actual first probe illumination information, second probe illumination information and weight information of the eight illumination probes P0 to P7 are currently unknown, the first probe illumination information estimation value, second probe illumination information estimation value and weight information estimation value of the eight illumination probes P0 to P7 are used as the initial value A′ i , B i ′ and T i ' (where i=0, 1, 2, 3, 4, 5, 6, 7). For a sampling point v, the illumination information of the sampling point v can be estimated based on the initial information and by the following mathematical formula (in other words, the illumination information estimation value of the sampling point v is obtained).

[0065]

[0066] in:

[0067] w v,i Represents the light probe P i For the position proportion weight of the sampling point v, Figure 7 A diagram showing how the position scale weights are calculated, using a trilinear interpolation technique;

[0068] Represents the sum of the estimated values ​​of the first probe lighting information of the lighting probes P0 to P7 based on the position proportional weights;

[0069] Represents the sum of the second probe lighting information estimation values ​​of the lighting probes P0 to P7 based on the position proportional weights;

[0070] represents the weighted sum of the weight information estimates obtained by mapping the sum of the weight information estimates of the light probes P0 to P7 based on the position proportional weights through a Sigmoid function (sometimes also called a squeeze function);

[0071] Represents the calculated value of the lighting information at the sampling point v.

[0072] However, as explained above, the above mathematical formula is only for one sampling point. In the actual calculation process, the illumination information and position information of all sampling points in one calculation unit are usually input as input data. Here, for ease of understanding, the above mathematical formula is appropriately modified and shown in the following manner.

[0073]

[0074] Wherein, j represents the jth sampling point in a computing unit.

[0075] In this way, the estimated values ​​of the illumination information of all sampling points in the calculation unit with the eight illumination probes P0 to P7 as vertices can be calculated.

[0076] It should be noted that in the above calculation method, the Sigmoid activation function is used to calculate and But it is not limited to this. For example, a step function can also be used to calculate and At this time, the third mathematical formula is changed to the following mathematical formula.

[0077]

[0078] However, as explained above, the actual value of the illumination information of each sampling point in the above calculation unit has been pre-calculated in step ST2, so the actual value is compared with the estimated value, the error between the two is evaluated by, for example, MSE, and the error is back-propagated in the network model to correct the error. Figure 6 The model parameters of the network model shown are adjusted to minimize the error between the actual value and the estimated value, such as MSE. When the error is minimized, the calculation result A is output. i , B i and T i (wherein, i=0, 1, 2, 3, 4, 5, 6, 7) are used as the final value of the first probe lighting information, the final value of the second probe lighting information, and the final value of the weight information of each of the eight lighting probes P0 to P7.

[0079] It should be noted that, as Figure 4 Like the light probe in the center of the image, the light probe is the vertex of eight calculation units. Therefore, if the calculation range is set as one calculation unit, the lighting information and weight information of the light probe are actually fitted eight times. For this, for example, the average of the eight fitting results can be taken as the final calculation result.

[0080] In addition, the above description is based on the case where the illumination information and weight information of each illumination probe are fitted based on the sampling points in one calculation unit, but it is not limited to this. For example, the illumination information and weight information of each illumination probe constituting the vertices of each calculation unit can also be fitted based on the sampling points in multiple calculation units at the same time. In this way, the accuracy of fitting can be improved compared with the case where multiple calculations are performed separately and the average value is taken as the final result.

[0081] After completing the operation in step ST4, the process proceeds to step ST5.

[0082] In step ST5, the virtual scene is rendered according to the probe lighting information and weight information of the above-mentioned lighting probe. More specifically, the shading points in the virtual scene are illuminated and shaded according to the probe lighting information and weight information of the multiple lighting probes configured in the virtual scene. More specifically, for each shading point in the virtual scene, lighting and shading are performed according to the probe lighting information and weight information of the lighting probe as the vertex of the computing unit where the shading point is located. In the following, the example in which the computing unit is a cubic unit and each lighting probe has the first probe lighting information, the second probe lighting information and the weight information is still used for explanation.

[0083] First, based on the position information of each shading point in the virtual scene, it can be determined whether the shading point is a sampling point collected in step ST1. If the shading point is a sampling point collected before, lighting shading is performed directly based on the lighting information of the sampling point. If the shading point is not any sampling point collected before, the calculation unit where the shading point is located is determined. It should be noted that the calculation unit where the shading point is located can also be directly determined based on the position information of each shading point in the virtual scene, without determining whether it is a sampling point collected before. The choice of judgment logic can be determined according to actual needs. Assume that the shading point is Figure 3 The shading point s shown in the figure has eight vertices of the computation unit where the shading point s is located, namely, the illumination probes P0 to P7. i The first probe illumination information A i , the second probe illumination information B i And the weight information T i All have been calculated in step ST4 (where i=0, 1, 2, 3, 4, 5, 6, 7). Therefore, the lighting information at the shading point can be calculated by the following trilinear interpolation algorithm, thereby performing lighting shading and rendering on it.

[0084]

[0085] in:

[0086] w s,i Represents the light probe P i For the position proportion weight of the coloring point s, Figure 7 A diagram showing how the position scale weights are calculated, using a trilinear interpolation technique;

[0087] A represents the sum of the final values ​​of the first probe lighting information of the lighting probes P0 to P7 based on the position proportion weight;

[0088] B represents the sum of the final values ​​of the second probe lighting information of the lighting probes P0 to P7 based on the position proportion weight;

[0089] T represents the weighted sum of the final values ​​of the weight information obtained by mapping the sum of the final values ​​of the weight information of the light probes P0 to P7 based on the position proportional weights through a Sigmoid function (sometimes also called a squeeze function);

[0090] F represents the calculated value of the lighting information of the shading point s.

[0091] In this way, all shading points in the virtual scene can be illuminated and shaded, thereby further completing the rendering of the entire virtual scene.

[0092] Figure 8 A schematic diagram is shown of lighting shading a shading point within a computing unit based on lighting information and weight information of a lighting probe of the computing unit. Figure 8 An application scenario is shown, that is, the virtual scene includes indoor and outdoor areas. Here, for ease of understanding, an extreme assumption is made that the light sources are all located in the outdoor area, there is no light source in the indoor area, and the outdoor light does not enter the indoor area. Therefore, it can be considered that the sample points collected in the virtual scene are divided into two categories. The first category is the completely dark category (for example, corresponding to Figure 2 The second category is the fully lit category (e.g., corresponding to Figure 2 In this case, by executing the above step ST4, the first probe illumination information corresponding to the completely dark category of each illumination probe can be obtained ( Figure 8 The second probe illumination information corresponding to the full-bright category ( Figure 8 In order to facilitate explanation and understanding, a step function is used here to perform the weighted summation of the weight information of each light probe. Through calculation, it is found that the roof of the building just forms a dividing interface (see Figure 8 Specifically, when the lighting shading rendering is performed on the shading point located in the indoor area relative to the interface, the weighted sum of the weight information for the shading point is 1. That is, for each shading point located in the indoor area relative to the interface, Figure 8 As shown in the leftmost figure, linear interpolation is actually performed based on the first probe lighting information of each lighting probe, and for each shading point located in the outdoor area relative to the interface, Figure 8 As shown in the middle figure in , linear interpolation calculation is actually performed only based on the second probe lighting information of each lighting probe.

[0093] However, it should be noted that Figure 8 This is just an example. In fact, when the Sigmoid activation function is used instead of the step function to calculate the T value, when Figure 8 When lighting is performed on a shading point in the calculation unit shown, lighting is often not performed based on lighting information from only one of the lighting probes, but rather on a weighted sum of lighting information from each of the lighting probes. For example, when lighting is performed on a shading point located in an indoor area, the T value calculated based on the weight information of each lighting probe is 0.76, and the lighting information of the lighting probe of the shading point is determined by the following mathematical formula.

[0094] T=A·0.76+B·0.24

[0095] According to the global illumination rendering technology described in the above-mentioned embodiment, compared with the prior art, each illumination probe does not have only one kind of illumination information, but has multiple kinds of illumination information corresponding to the number of categories according to the number of categories of sampling points collected in the virtual scene. In addition, compared with the prior art, each illumination probe also has weight information, which characterizes the contribution of the illumination information corresponding to each category of each illumination probe to the illumination shading of the shading point in the virtual scene. More specifically, as described above, the position proportion weight of each illumination probe for the shading point is determined according to the position information of each illumination probe and the position information of the shading point, and the total weight (i.e., the overall contribution degree) of each illumination information of the illumination probe for the shading point is obtained by multiplying and adding the weight information of each illumination probe. Thus, the illumination information of the shading point is obtained by weighted summation, so as to perform shading and rendering. In this way, when the calculation unit where the shading point is located spans, for example, an outdoor area and an indoor area, these shading points can be illuminated and shaded by using an illumination probe that takes into account both the illumination information of the indoor area and the illumination information of the outdoor area, thereby avoiding light leakage or patchiness. In addition, compared with the prior art, the global illumination calculation method described in this embodiment does not require manual marking, can automatically solve problems such as light leakage and patchiness, and is still applicable when manual marking is impossible. In addition, if the prior art means such as marking meshes are used, only the light leakage problem of static scenes can be solved. At this time, if a player character walks into the virtual scene, light leakage will still occur and cannot be marked in advance. However, the global illumination calculation method described in this embodiment can solve the problem of lighting of static objects and dynamic objects at the same time, that is, it can solve the above-mentioned technical problems existing in the prior art using marked meshes.

[0096] In the above embodiment, the computing unit is described as a cubic unit, but the shape of the computing unit is not limited thereto, and may also be a rectangular parallelepiped. In addition, the computing unit is not limited to a hexahedron, for example, four adjacent illumination probes may be connected by Delaunay triangulation to form a tetrahedron as the computing unit. Fig. 9 A schematic diagram of a calculation method based on tetrahedron linear interpolation is shown, and repeated description is omitted here.

[0097] In addition, taking the field of electronic games as an example, the "virtual scene" referred to in the above embodiments includes the entire game scene or a part of the game scene. In particular, the global illumination calculation method described in the above embodiments is particularly effective for scenes with large differences in internal illumination distribution (for example, there are both indoor and outdoor areas).

[0098] In addition, it should be noted that in the above embodiment, the number of types of sampling points is two, but the present invention is not limited thereto. For example, the number of types of sampling points may be three or more.

[0099] Fig.10 is a schematic diagram of the structure of a computer device 500 used in the global illumination calculation method of the present application. It should be noted that: Fig.10 The computer device shown is merely an example of an electronic device and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0100] like Fig.10 As shown, the computer device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from the storage unit 108 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the computer device 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0101] The I / O interface 505 is connected to an I / O device, which may include an input unit 506 such as a keyboard and a mouse, an output unit 507 such as a liquid crystal display (LCD) and a speaker, a storage unit 508 such as a hard disk, and a communication unit 509 of a network interface card such as a modem. The communication unit 509 performs communication processing via a network such as the Internet. A drive 510 may also be connected to the I / O interface 505 as needed. In addition, a removable medium 511 may also be installed on the drive 510 as needed, so that a computer program read from the removable medium 511 can be installed in the storage unit 508 as needed.

[0102] In particular, the process described with reference to the various figures, i.e., the global illumination calculation method of the present application, can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the methods shown in the various figures. In such an embodiment, the computer program can be downloaded and installed from the network through the communication unit 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0103] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0104] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the computer device described in this specification, or may exist independently without being assembled into the computer device. The above-mentioned computer-readable storage medium carries one or more programs, and when the above-mentioned one or more programs are executed by a computer device, the computer device implements the method as described in the above-mentioned embodiment and its variant. For example, the above-mentioned computer device can implement each step shown in each figure.

[0105] According to one aspect of the present invention, a computer program product is provided, the computer program product comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in various optional implementations of the above-mentioned embodiments. In particular, the global illumination calculation method of the present application can be implemented in the form of a program plug-in in, for example, Unreal Engine, Unity, Maya or 3ds max.

[0106] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A global illumination calculation method, characterized in that: include: Collect multiple sampling points and arrange multiple lighting probes in the virtual scene; Baking the plurality of sampling points based on the light source information in the virtual scene to obtain lighting information of each of the sampling points; Classifying the plurality of sampling points according to the illumination information and the position information of each of the sampling points; Calculating lighting information and weight information of each lighting probe based on the lighting information and position information of the sampling points of each category, wherein the weight information represents the contribution of the lighting information corresponding to each category of each lighting probe to the lighting shading of the shading point in the virtual scene; as well as Perform lighting calculation on the virtual scene according to the lighting information and weight information of the multiple lighting probes.

2. The global illumination calculation method according to claim 1, characterized in that: Taking the plurality of illumination probes as vertices, dividing the virtual scene into a plurality of computing units, The shading point in the virtual scene is illuminated and shaded according to the illumination information and weight information of each illumination probe of the computing unit where the shading point is located.

3. The global illumination calculation method according to claim 2, characterized in that: The shading point is illuminated and shaded according to the distance of each illumination probe of the computing unit where the shading point is located relative to the shading point and the illumination information and weight information of each illumination probe.

4. The global illumination calculation method according to claim 3, characterized in that: A portion of the plurality of computing units is a cuboid unit with eight of the light probes as vertices, The shading point is illuminated by linearly interpolating the illumination information and weight information corresponding to each category of the eight illumination probes of the rectangular unit where the shading point is located relative to the shading point, and performing weighted summation of the interpolated illumination information corresponding to each category based on the interpolated weight information.

5. The global illumination calculation method according to claim 2, characterized in that: A portion of the computational units in the plurality of computational units are tetrahedral units formed by Delaunay triangulation of a portion of the plurality of illumination probes.

6. The global illumination calculation method according to claim 2, characterized in that: The number of the sampling points collected in the computing unit that contains a grid of virtual objects in the virtual scene is greater than the number of the sampling points collected in the computing unit that does not contain the grid.

7. The global illumination calculation method according to claim 1, characterized in that: In the case where the sampling point is located inside the grid space of the virtual object in the virtual scene, the sampling point is not baked.

8. The global illumination calculation method according to claim 2, characterized in that: The lighting information and weight information of each lighting probe are calculated based on the lighting information and position information of each category of the sampling points in at least a portion of all the computing units that have the lighting probe as a common vertex.

9. The global illumination calculation method according to claim 8, characterized in that: The sampling points at least include sampling points of a first category and sampling points of a second category. The lighting information of each lighting probe includes at least first lighting information and second lighting information. The first lighting information of each lighting probe is determined according to lighting information and position information of sampling points of the first category in at least a portion of computing units having the lighting probe as a common vertex, The second lighting information of each lighting probe is determined based on the lighting information and position information of the second category of sampling points in the at least a portion of the computing units having the lighting probe as a common vertex, The weight information of each of the illumination probes is determined according to illumination information and position information of all sampling points in the at least a portion of the computing units that have the illumination probe as a common vertex.

10. The global illumination calculation method according to claim 2, characterized in that: When the sampling points in all the computing units with the lighting probe as a common vertex belong to the same category, the lighting information corresponding to the category of the lighting probe is calculated only based on the lighting information and position information of the sampling points of the category.

11. A graphics processing device, comprising a memory, a processor, and a computer program stored in the memory, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images