Method, device, medium and product for compressing storage of spherical harmonic coefficients

Abstract

This application discloses a method, device, medium, and product for compressing and storing spherical harmonic coefficients. The method includes: updating the spherical harmonic probe in the spherical harmonic probe body to obtain the spherical harmonic coefficients of the current frame, wherein the current frame spherical harmonic coefficients include the zero-order spherical harmonic coefficients and first-order spherical harmonic coefficients corresponding to each color channel, and the first-order spherical harmonic coefficients include three directional components; decoding historical compressed spherical harmonic data and fusing it with the current frame spherical harmonic coefficients to obtain cumulative spherical harmonic coefficients; retaining the zero-order spherical harmonic coefficients of each color channel in the cumulative spherical harmonic coefficients, and relative encoding the three directional components of the corresponding first-order spherical harmonic coefficients based on the zero-order spherical harmonic coefficients of each color channel to obtain multiple first-order encoded values; storing the zero-order spherical harmonic coefficients of each color channel and one of the multiple first-order encoded values ​​in a first three-dimensional texture, and storing the remaining first-order encoded values ​​in one or more low-precision three-dimensional textures to obtain the current frame compressed spherical harmonic data.

Description

Technical Field

[0001] This application relates to the field of image rendering technology, and in particular to a method, device, medium and product for compressing and storing spherical harmonic coefficients. Background Technology

[0002] Real-time global illumination is used to simulate the multiple propagations of light within a scene to generate indirect lighting effects. To reduce the computational cost of calculating global illumination pixel by pixel, multiple probes can be set up in the 3D scene, and the incident radiation distribution in the corresponding spatial region of each probe can be characterized by spherical harmonic coefficients. When rendering a point in the scene, the spherical harmonic coefficients of neighboring probes can be read to determine the indirect lighting of that point.

[0003] Spherical harmonics typically include the zeroth-order spherical harmonics, which characterize the average radiant energy, and the first-order spherical harmonics, which characterize the change in radiation directionality. As the number of probes in a probe body increases, storing all spherical harmonics with the same high storage precision will consume more video memory and increase the data read / write bandwidth during probe updates and rendering sampling. This problem is more pronounced on mobile devices where storage resources and data transmission bandwidth are limited.

[0004] Meanwhile, probe spherical harmonic coefficients can be accumulated across consecutive frames to improve the stability of indirect illumination. Saving, retrieving, and fusing historical spherical harmonic data further increases storage requirements. Directly reducing the storage precision of all spherical harmonic coefficients easily compresses the high dynamic range radiation information carried by the zeroth-order spherical harmonic coefficients; conversely, saving only the zeroth-order spherical harmonic coefficients results in the loss of directional information represented by the first-order spherical harmonic coefficients. Therefore, a spherical harmonic coefficient storage scheme that balances storage requirements, data bandwidth, radiation dynamic range, and directional information is needed. Summary of the Invention

[0005] One objective of this application is to provide a method, device, medium, and product for compressed storage of spherical harmonic coefficients, at least to solve the problem in related technologies where the storage of spherical harmonic coefficients in spherical harmonic probes occupies a large amount of space and the data read / write bandwidth is large, making it difficult to simultaneously take into account the radiation dynamic range of the zero-order spherical harmonic coefficients and the directional information of the first-order spherical harmonic coefficients.

[0006] To achieve the above objectives, some embodiments of this application provide the following aspects: In a first aspect, some embodiments of this application provide a method for compressed storage of spherical harmonic coefficients, the method comprising: Perform probe update on the spherical harmonic probe in the spherical harmonic probe body to obtain the spherical harmonic coefficients of the current frame. The spherical harmonic coefficients of the current frame include the zero-order spherical harmonic coefficients and the first-order spherical harmonic coefficients corresponding to each color channel. The first-order spherical harmonic coefficients include three directional components. The historical compressed spherical harmonic data is decoded and then fused with the spherical harmonic coefficients of the current frame to obtain the cumulative spherical harmonic coefficients; The zero-order spherical harmonic coefficients of each color channel in the cumulative spherical harmonic coefficients are retained, and the three directional components of the corresponding first-order spherical harmonic coefficients are relatively encoded based on the zero-order spherical harmonic coefficients of each color channel to obtain multiple first-order encoded values. The zero-order spherical harmonic coefficients of each color channel and one of the multiple first-order encoded values ​​are stored in a first three-dimensional texture, and the remaining first-order encoded values ​​are stored in one or more low-precision three-dimensional textures. The storage precision of the low-precision three-dimensional textures is lower than that of the first three-dimensional textures, thus obtaining the compressed spherical harmonic data of the current frame, and using the compressed spherical harmonic data of the current frame as the historical compressed spherical harmonic data of the next frame.

[0007] Secondly, some embodiments of this application also provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method described above.

[0008] Thirdly, some embodiments of this application also provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the method described above.

[0009] Fourthly, some embodiments of this application also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0010] Compared with related technologies, the solution provided in this application, by retaining the zero-order spherical harmonic coefficients and relative encoding the first-order spherical harmonic coefficients based on the zero-order spherical harmonic coefficients, and storing the zero-order spherical harmonic coefficients and the first-order encoded values ​​in the three-dimensional texture according to different storage precisions, can reduce the storage volume and data read / write bandwidth of the spherical harmonic probe while preserving the radiation dynamic range and directional information. By using the compressed current frame spherical harmonic data as the historical compressed spherical harmonic data of the next frame and decoding it before fusion, the compressed storage structure can participate in the spherical harmonic coefficient update of consecutive frames. Through low-brightness neutral value processing, frame-by-frame switching of texture groups, and interpolation followed by decoding, the usability of compressed data in low-brightness areas, historical accumulation, and rendering sampling processes can be improved. Attached Figure Description

[0011] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0012] Figure 1 An exemplary flowchart of a method for compressing and storing spherical harmonic coefficients provided in some embodiments of this application; Figure 2 An exemplary structural diagram of the electronic device provided for some embodiments of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] Figure 1 This is an exemplary flowchart illustrating a method for compressing and storing spherical harmonic coefficients, provided in some embodiments of this application. The method can be applied to a real-time global illumination generation process based on a spherical harmonic probe. The spherical harmonic probe includes multiple spherical harmonic probes distributed according to a three-dimensional grid. Each spherical harmonic probe corresponds to a world spatial location and is used to characterize the incident radiation distribution in the corresponding spatial region.

[0015] Spatial transformation parameters, current frame spherical harmonic texture, and historical spherical harmonic texture can be maintained for the spherical harmonic probe. The spatial transformation parameters are used to realize the conversion between probe index, world space position, and texture position, and the current frame spherical harmonic texture and historical spherical harmonic texture are used to store the current frame spherical harmonic data and historical spherical harmonic data, respectively.

[0016] Before performing probe updates, multiple spherical harmonic probes can be filtered for validity based on the current rendered view to determine valid spherical harmonic probes. Distribution information is then generated based on the number of valid spherical harmonic probes and the 3D mesh parameters of the probe bodies. Probe updates are then performed on the valid spherical harmonic probes according to this distribution information. After completing the above processing, the following steps are executed.

[0017] S101. Perform probe update on the spherical harmonic probe in the spherical harmonic probe body to obtain the spherical harmonic coefficients of the current frame. The spherical harmonic coefficients of the current frame include the zero-order spherical harmonic coefficients and the first-order spherical harmonic coefficients corresponding to each color channel. The first-order spherical harmonic coefficients include three directional components.

[0018] Specifically, the effective spherical harmonic probe to be updated can be determined based on the dispatch information, and the world space position of the effective spherical harmonic probe can be used as the starting point of the initial ray. Multiple initial rays with different sampling directions can be generated around the world space position.

[0019] The sampling direction can be generated using a predetermined set of directions, or it can be generated using random sampling, low-difference sampling, or a sampling method that varies with the frame. By assigning different initial rays to different sampling directions, incident radiation information in multiple directions around the effective spherical harmonic probe can be obtained.

[0020] For each initial ray, the ray's origin and direction can be used as query inputs to retrieve the query results from the scene radiation data according to the radiation query rules. Scene radiation data can include one or more of the following: scene depth, scene color, scene normals, material information, mesh surface buffers, distance field data, voxel radiation data, or surface metadata.

[0021] Radiation data from different scenarios can be organized in different ways. Radiation query rules can convert query results into a unified output, which can include a hit marker, ray hit location, incident radiation information, and hit location normal.

[0022] When a ray of light achieves a valid hit, the cumulative radiation result of the corresponding ray path can be updated based on the incident radiation information, and a bounce hemisphere can be determined based on the normal of the hit position. Within this bounce hemisphere, the direction of the bounced ray is generated. Subsequently, the hit position of the ray, or the position offset along the normal of the hit position, can be used as the new starting point for the next hit query.

[0023] The decision to continue subsequent bounces can be based on the number of bounces, the hit result, the path contribution status, or preset update conditions. When a ray does not achieve a valid hit, the number of bounces meets the termination condition, or the path contribution is lower than the continuation query condition, the corresponding ray path can be terminated. For ray paths that do not achieve a valid hit, environmental radiation information can be used as supplementary radiation results.

[0024] After querying multiple light rays, the incident radiation result corresponding to the effective spherical harmonic probe can be determined based on the cumulative radiation result and sampling direction of each light ray path. The incident radiation result corresponding to different sampling directions is projected onto the spherical harmonic basis function, and the projection results of multiple sampling directions are accumulated, weighted, and reduced to obtain the spherical harmonic coefficients of the current frame.

[0025] Furthermore, an update end brightness limit can be set for the radiation results obtained from probe updates. When the brightness of the radiation result exceeds the update end brightness limit, it is restricted to the update end brightness limit before generating or writing the corresponding current frame spherical harmonic coefficients, thereby reducing spherical harmonic coefficient overshoot caused by individual high-brightness sampling results. In one specific embodiment, the update end brightness limit can be set to 30, which is an empirical parameter that can be adjusted according to the scene's radiation range.

[0026] The current frame's spherical harmonic coefficients include the zeroth-order and first-order spherical harmonic coefficients for each color channel. The zeroth-order spherical harmonic coefficients represent the average radiant energy of the corresponding color channel; the first-order spherical harmonic coefficients include three directional components, representing the variation of incident radiation relative to the three directions. The number and representation of color channels can be set according to the organization of the spherical harmonic data, and are not limited to the RGB color space.

[0027] S102. After decoding the historical compressed spherical harmonic data, fuse it with the spherical harmonic coefficients of the current frame to obtain the cumulative spherical harmonic coefficients.

[0028] Specifically, historical compressed spherical harmonic data can be compressed spherical harmonic data stored for the corresponding spatial region in the previous frame or preceding frame. Historical compressed spherical harmonic data can include historical zero-order spherical harmonic coefficients and historical first-order coded values ​​encoded relative to the historical zero-order spherical harmonic coefficients.

[0029] When the spherical harmonic probe remains fixed between consecutive frames, the corresponding historical compressed spherical harmonic data can be read based on the current probe position. When the position of the spherical harmonic probe changes, the corresponding historical sampling position can be determined based on the current world space position of the spherical harmonic probe and the spatial transformation parameters of the spherical harmonic probe in the previous frame, and the historical compressed spherical harmonic data can be read based on the historical sampling position.

[0030] After reading the historical compressed spherical harmonic data, the historical zero-order spherical harmonic coefficients are retained. Then, based on the historical zero-order spherical harmonic coefficients of each color channel, the corresponding historical first-order coded values ​​are decoded to recover the historical first-order spherical harmonic coefficients. The historical zero-order spherical harmonic coefficients and the recovered historical first-order spherical harmonic coefficients together constitute the historical spherical harmonic coefficients.

[0031] The corresponding components in the spherical harmonic coefficients of the current frame and the historical spherical harmonic coefficients can be fused according to the fusion weight. The fusion weight can be determined based on whether the historical data is valid, the cumulative state of the historical data, the motion state between the current viewpoint and the viewpoint of the previous frame, and the degree of spatial correspondence between the current spherical harmonic probe and the historical sampling position.

[0032] When historical data is valid and viewpoint changes are minimal, the fusion ratio of historical spherical harmonic coefficients can be increased; when historical data is invalid, spatial location is mismatched, or viewpoint changes are significant, the fusion ratio of historical spherical harmonic coefficients can be decreased. If no usable historical data exists, the spherical harmonic coefficients of the current frame can be used as the cumulative spherical harmonic coefficients.

[0033] S103. Retain the zero-order spherical harmonic coefficients of each color channel in the cumulative spherical harmonic coefficients, and relative encode the three directional components of the corresponding first-order spherical harmonic coefficients based on the zero-order spherical harmonic coefficients of each color channel to obtain multiple first-order coded values.

[0034] Specifically, for each color channel in the cumulative spherical harmonic coefficients, the zero-order spherical harmonic coefficient and the three directional components of the first-order spherical harmonic coefficient for that color channel are obtained. The fusion result of the zero-order spherical harmonic coefficient is retained, and this zero-order spherical harmonic coefficient is used as the encoding reference for the first-order spherical harmonic coefficient of the same color channel.

[0035] For the same color channel, the relative representations of the three directional components of the first-order spherical harmonic coefficients with respect to the zero-order spherical harmonic coefficients can be determined separately, and the relative representations of the three directional components can be converted into first-order coded values ​​respectively.

[0036] Different color channels are relatively encoded using their respective zeroth-order spherical harmonic coefficients to reduce the impact of differences in radiant energy between different color channels on the encoding results. After relative encoding, each color channel corresponds to one retained zeroth-order spherical harmonic coefficient and three first-order encoded values.

[0037] By using the zeroth-order spherical harmonic coefficients as the encoding reference, the first-order spherical harmonic coefficients with positive and negative values ​​can be converted into relative data suitable for lower-precision storage. The quantization error generated by relative encoding can be scaled with the magnitude of the zeroth-order spherical harmonic coefficients, thereby reducing the absolute error in the low-radiation-energy region.

[0038] S104. Store the zero-order spherical harmonic coefficients of each color channel and one of the multiple first-order encoded values ​​in a first three-dimensional texture, and store the remaining first-order encoded values ​​in one or more low-precision three-dimensional textures. The storage precision of the low-precision three-dimensional textures is lower than that of the first three-dimensional textures. This yields the compressed spherical harmonic data for the current frame, and the compressed spherical harmonic data for the current frame is used as the historical compressed spherical harmonic data for the next frame.

[0039] Specifically, the zero-order spherical harmonic coefficients of each color channel are written into the first three-dimensional texture. The first three-dimensional texture uses a storage precision capable of preserving the radiative dynamic range of the zero-order spherical harmonic coefficients, in order to retain high dynamic range indirect light information in different spatial regions.

[0040] In addition to the storage channel occupied by the zeroth-order spherical harmonic coefficients, a storage channel can also be set up in the first 3D texture to store a first-order encoded value. Storing a first-order encoded value and the zeroth-order spherical harmonic coefficients together in the first 3D texture can utilize the storage channel of the first 3D texture.

[0041] The remaining first-order encoded values ​​are written to one or more low-precision 3D textures. Since the first-order encoded values ​​represent directional changes relative to the zeroth-order spherical harmonic coefficients, they can be stored with a lower precision than the first 3D texture to reduce the storage requirements and data read / write bandwidth of the spherical harmonic probe.

[0042] Each 3D texture can be organized according to the 3D mesh structure of the spherical harmonic probe body. For any spherical harmonic probe, the corresponding texel position can be determined according to the mesh position of the spherical harmonic probe in the spherical harmonic probe body, and the zero-order spherical harmonic coefficient and the first-order encoding value of the spherical harmonic probe can be written into the corresponding texel position in each 3D texture.

[0043] The storage format of the first three-dimensional texture, the storage format of the low-precision three-dimensional texture, the number of low-precision three-dimensional textures, and the distribution method of the first-order encoding value among the three-dimensional textures can be set according to the number of color channels and the organization method of spherical harmonic data.

[0044] After texture writing is complete, the data corresponding to the same spherical harmonic probe in each 3D texture together constitute the compressed spherical harmonic data of the current frame. The compressed spherical harmonic data of the current frame participates in decoding and fusion as historical compressed spherical harmonic data in the next frame, thus forming a process of compression storage, decoding, fusion and re-encoding between consecutive frames.

[0045] In this embodiment, the current frame spherical harmonic coefficients are provided for compressed storage of spherical harmonic coefficients through the preceding spherical harmonic probe body construction, effective spherical harmonic probe screening, and probe update task dispatch. By decoding historical compressed spherical harmonic data and fusing it with the current frame spherical harmonic coefficients, the compressed data can participate in continuous frame accumulation. By relatively encoding the first-order spherical harmonic coefficients and saving the zero-order spherical harmonic coefficients and the first-order encoded values ​​according to different storage precisions, the storage occupation and data read / write bandwidth can be reduced while preserving the radiation dynamic range and directional information.

[0046] In one embodiment, the relative encoding includes: When the zeroth-order spherical harmonic coefficient of any color channel in the cumulative spherical harmonic coefficients is not lower than a preset threshold, the ratios between the three directional components of the first-order spherical harmonic coefficient of that color channel and the zeroth-order spherical harmonic coefficient are determined respectively, and the ratios are mapped to the normalization interval according to the preset encoding scaling parameters, so that the ratios with a value of zero are mapped to neutral values. When the zero-order spherical harmonic coefficient of any color channel in the cumulative spherical harmonic coefficients is lower than the preset threshold, the three first-order coding values ​​corresponding to that color channel are set to the neutral value. The decoding process includes inverse mapping of the first-order encoded value to be decoded and recovering the first-order spherical harmonics based on the zero-order spherical harmonics of the corresponding color channels.

[0047] Specifically, for any color channel in the cumulative spherical harmonics, the zeroth-order spherical harmonics of that color channel can be denoted as... The first-order spherical harmonic coefficient of this color channel is denoted as .in, It is a three-dimensional vector consisting of three directional components, all of which are signed.

[0048] The preset encoding scaling parameter can be denoted as S, and the first-order encoded value obtained by relative encoding can be denoted as e. Here, e is the value relative to... The corresponding three-dimensional vector, e, has three components that correspond to... The three directional components.

[0049] When the zeroth order spherical harmonic coefficient When the value is not lower than the preset threshold, relative encoding can be performed according to the following formula: The above formula is for The three directional components are executed separately. Specifically, for For any directional component in the equation, first determine the relationship between that directional component and the zeroth-order spherical harmonic coefficient. The ratio between the two components is then scaled using a preset encoding scaling parameter S, and finally an offset of 0.5 is added to obtain the first-order encoded value corresponding to that directional component.

[0050] First-order spherical harmonic coefficients The directional component can be positive, negative, or zero. When the directional component is zero, it is related to the zeroth-order spherical harmonic coefficient. The ratio between them is zero, and the first-order coding value obtained according to the above formula is 0.5. Therefore, 0.5 is used as a neutral value to represent no directional bias, and the positive and negative first-order spherical harmonic coefficient directional components are mapped to both sides of the neutral value 0.5.

[0051] Spherical harmonic coefficients are obtained by projecting the incident radiation onto the spherical harmonic basis functions. When the integrand corresponding to the spherical harmonic projection is non-negative radiation, the amplitude of the first-order spherical harmonic coefficient has a finite physical upper bound relative to the zeroth-order spherical harmonic coefficient. The zeroth-order spherical harmonic coefficient characterizes the average radiant energy of the corresponding color channel, while the first-order spherical harmonic coefficient characterizes the variation of this radiant energy in different directions. Therefore, the ratio between the first-order and zeroth-order spherical harmonic coefficients will not increase indefinitely.

[0052] By setting the preset encoding scaling parameter S, the ratio encoding range can cover the ratio between the first-order spherical harmonic coefficient and the zero-order spherical harmonic coefficient under normal lighting conditions, and retain encoding margin for radiation distributions with strong directionality.

[0053] When the zeroth order spherical harmonic coefficient When the coefficients are close to zero, the first-order spherical harmonic coefficients are calculated directly. With zeroth order spherical harmonic coefficient The ratio between them may become numerically unstable due to the small zeroth-order spherical harmonic coefficient. Therefore, the zeroth-order spherical harmonic coefficient can be... Compare with a preset threshold.

[0054] In one embodiment, the preset threshold can be set to When the zeroth order spherical harmonic coefficient When the value is below the preset threshold, no calculation is performed. and The ratio between them, but rather the ratio between them. The first-order coding values ​​corresponding to the three directional components are all set to a neutral value of 0.5, that is: Setting the three first-order coding values ​​to neutral values ​​indicates that directional bias is not recorded when the radiant energy of the corresponding color channel is close to zero. Since the zeroth-order spherical harmonic coefficients are small at this time, the first-order spherical harmonic coefficients obtained by scaling these zeroth-order spherical harmonic coefficients during decoding are also correspondingly small.

[0055] When decoding a first-order coded value, the first-order spherical harmonic coefficients can be recovered using the following formula: The above decoding formula is also applied separately to the three directional components. Specifically, first, the neutral value of 0.5 is subtracted from each first-order encoded value to restore its signed offset relative to the neutral value; then, it is scaled inversely according to the preset encoding scaling parameter S; finally, it is multiplied by the zero-order spherical harmonic coefficient of the corresponding color channel. To restore the first-order spherical harmonic coefficients of this color channel .

[0056] When the first-order coding value is a neutral value of 0.5, the directional components of the first-order spherical harmonic coefficients obtained according to the above decoding formula are zero. Therefore, regardless of whether the neutral value comes from the original directional components being zero or from the zero-order spherical harmonic coefficients being lower than the preset threshold, the decoding result will not produce any additional directional bias.

[0057] For historical compressed spherical harmonic data, the historical zero-order spherical harmonic coefficients and historical first-order encoded values ​​of each color channel can be read, and the historical first-order spherical harmonic coefficients can be recovered according to the above decoding formula. The recovered historical first-order spherical harmonic coefficients and the historical zero-order spherical harmonic coefficients together constitute the historical spherical harmonic coefficients, so as to be fused with the spherical harmonic coefficients of the current frame.

[0058] The quantization error of the relative encoding is determined by the zeroth-order spherical harmonic coefficients during decoding. Scaling is applied. As a result, in low-brightness regions where the zeroth-order spherical harmonic coefficient is small, the absolute error generated after decoding decreases synchronously with the zeroth-order spherical harmonic coefficient, which helps to reduce directional noise or color deviation generated by absolute quantization in low-brightness regions.

[0059] In this embodiment, by relative encoding the first-order spherical harmonic coefficients based on the zero-order spherical harmonic coefficients of each color channel, the directional changes of different color channels can be preserved; by setting the first-order encoded value to a neutral value when the zero-order spherical harmonic coefficients are below a preset threshold, the numerical instability caused by excessively small scales can be reduced; by using the inverse decoding formula corresponding to the relative encoding, the first-order spherical harmonic coefficients can be recovered.

[0060] In one embodiment, the normalization interval is [0,1], and the neutral value is 0.5; when the mapping result obtained according to the preset encoding scaling parameter exceeds the normalization interval, the mapping result is restricted to the normalization interval.

[0061] Specifically, all three directional components of the first-order spherical harmonic coefficients are signed. During relative encoding, 0.5 is used as the zero-center, and the signed ratio between the directional components of the first-order spherical harmonic coefficients and the zero-order spherical harmonic coefficients is mapped to the normalized interval, so that negative ratios correspond to the negative side of the neutral value 0.5, positive ratios correspond to the positive side of the neutral value 0.5, and ratios with a value of zero correspond to the neutral value 0.5.

[0062] When the preset encoding scaling parameter is S, the ratio between the directional components of the first-order spherical harmonic coefficients and the zero-order spherical harmonic coefficients within the range [-S, S] is linearly mapped to the normalization interval [0, 1]. Here, the negative boundary -S corresponds to the lower boundary of the normalization interval, a ratio with a value of zero corresponds to the neutral value 0.5, and the positive boundary S corresponds to the upper boundary of the normalization interval.

[0063] In one embodiment, the preset encoding scaling parameter S can be set to 2.0. In this case, the ratio between the first-order spherical harmonic coefficient directional component and the zero-order spherical harmonic coefficient in the range [-2,2] is linearly mapped to [0,1], thereby using unsigned normalized numerical representations to represent the first-order spherical harmonic coefficient directional component with positive and negative values.

[0064] After obtaining the mapping result, it is determined whether the mapping result is within the normalized interval [0,1]. When the mapping result is within [0,1], the mapping result is retained as the corresponding first-order encoded value; when the mapping result is less than 0, it is restricted to the lower boundary 0 of the normalized interval; when the mapping result is greater than 1, it is restricted to the upper boundary 1 of the normalized interval.

[0065] The above-mentioned restriction processing is performed on each first-order encoded value of each color channel. When any first-order encoded value exceeds the normalization interval, the first-order encoded value is only restricted to the corresponding interval boundary, without readjusting the first-order encoded values ​​of other directional components or other color channels.

[0066] In the implementation that uses unsigned normalized 3D textures to store first-order encoded values, mapping results exceeding [0,1] can be restricted to interval boundaries when writing to the 3D texture according to the writing rules of unsigned normalized 3D textures. Specifically, mapping results below 0 are stored as 0, and mapping results above 1 are stored as 1.

[0067] The limiting process described here is a saturation clipping, rather than a readjustment of the scaling of all first-order encoded values ​​based on values ​​that exceed the range. Therefore, first-order encoded values ​​that do not exceed the normalization interval will not be rescaled due to other directional components exceeding the limit, and different spherical harmonic probes and different color channels can continue to use the same preset encoding scaling parameters.

[0068] In an implementation where the preset encoding scaling parameter S is 2.0, when the ratio between the first-order spherical harmonic coefficient direction component and the zero-order spherical harmonic coefficient is less than -2, the mapping result obtained according to the relative encoding formula is less than 0, and is restricted to 0 when writing to an unsigned normalized 3D texture; when the ratio is greater than 2, the mapping result obtained according to the relative encoding formula is greater than 1, and is restricted to 1 when writing to the 3D texture. This process is equivalent to saturating and clipping ratios exceeding [-2, 2] to the corresponding encoding boundary.

[0069] When decoding a first-order coded value that has undergone saturation clipping, the first-order coded value 0 corresponds to the negative boundary -S of the ratio range, and the first-order coded value 1 corresponds to the positive boundary S of the ratio range. When S is 2.0, the ratio between the directional component of the first-order spherical harmonic coefficient and the zero-order spherical harmonic coefficient obtained by decoding is restricted to [-2, 2].

[0070] For color channels with zero-order spherical harmonic coefficients below a preset threshold, all three first-order coding values ​​are set to 0.5. Since 0.5 is located at the center of the normalization interval and corresponds to zero directional components of the first-order spherical harmonic coefficients, the low-radiance region will not experience additional directional bias due to the boundary values ​​of the normalization interval.

[0071] In this embodiment, by mapping the signed ratios within [-S,S] to [0,1] and using 0.5 as the zero center, the positive and negative directions of the first-order spherical harmonic coefficients can be represented using unsigned normalized values. By saturating and clipping the mapping results that exceed the normalization interval, the first-order encoded values ​​written to the 3D texture can be limited, and there is no need to store additional scaling parameters or out-of-bounds markers for out-of-bounds cases.

[0072] In one embodiment, the three-dimensional texture used to store the current frame compressed spherical harmonic data includes a first three-dimensional texture, a second three-dimensional texture, and a third three-dimensional texture, wherein the second three-dimensional texture and the third three-dimensional texture are the low-precision three-dimensional textures; The first three-dimensional texture is a semi-floating four-channel three-dimensional texture, and the second and third three-dimensional textures are both 8-bit unsigned normalized four-channel three-dimensional textures.

[0073] Specifically, the first, second, and third 3D textures have 3D texture dimensions corresponding to the spherical harmonic probe volume. Each spherical harmonic probe in the spherical harmonic probe volume corresponds to a texture unit in the 3D texture. The zero-order spherical harmonic coefficients and multiple first-order encoded values ​​corresponding to the same spherical harmonic probe are stored in different texture channels at the same texture coordinates in the three 3D textures according to a preset spherical harmonic coefficient allocation method.

[0074] The first 3D texture can use the R16G16B16A16_SFloat format. This format includes four half-precision floating-point channels, each represented by a 16-bit floating-point number. The first 3D texture is used to store the zero-order spherical harmonic coefficients of each color channel and one of multiple first-order encoded values.

[0075] The zeroth-order spherical harmonic coefficients are used to characterize the average radiant energy of the corresponding color channels. Their value range and dynamic range are typically greater than the normalized first-order encoded value. Using semi-floating-point channels to store the zeroth-order spherical harmonic coefficients can reduce the number of bits required to retain the corresponding radiant intensity information. Channels in the first 3D texture not occupied by the zeroth-order spherical harmonic coefficients can also store a first-order encoded value to improve the utilization rate of each channel in the first 3D texture.

[0076] The second and third 3D textures can each use the R8G8B8A8_UNorm format. This format includes four 8-bit unsigned normalized channels, each used to represent a value within the normalized interval [0,1]. After relative encoding and saturation clipping, the remaining first-order encoded values ​​are all located within [0,1], and therefore can be stored in the corresponding channels of the second and third 3D textures respectively.

[0077] When writing to the second and third 3D textures, the first-order encoded values ​​within [0,1] are quantized into corresponding 8-bit unsigned normalized values. During reading, these 8-bit unsigned normalized values ​​are converted back to first-order encoded values ​​within [0,1], and then decoded using the zero-order spherical harmonic coefficients of the corresponding color channels. Therefore, it is unnecessary to store all first-order spherical harmonic coefficients in a half-floating-point format.

[0078] The first, second, and third 3D textures together constitute a heterogeneous 3D texture group. The first 3D texture stores the zeroth-order spherical harmonic coefficients, which require high dynamic range and storage precision. The second and third 3D textures store normalized first-order encoded values. 3D textures with different storage precisions use the same 3D texture size and probe correspondence, enabling the same texture sampling coordinates to be used to read the compressed spherical harmonic data corresponding to the same spherical harmonic probe.

[0079] When using the above texture format, each texture unit of the first 3D texture occupies 8 bytes, while each texture unit of the second and third 3D textures occupies 4 bytes each. Therefore, the three 3D texture units corresponding to each spherical harmonic probe occupy a total of 16 bytes. Compared to the 24 bytes occupied by each spherical harmonic probe when all three 3D textures use a half-floating-point four-channel format, the above heterogeneous storage method can reduce the storage footprint of each spherical harmonic probe by 8 bytes, or one-third.

[0080] Taking a spherical harmonic probe body comprising 8000 spherical harmonic probes as an example, when using a three-and-a-half-float four-channel 3D texture, a single set of spherical harmonic data occupies 187.5 KiB; when using the heterogeneous 3D texture group of this embodiment, a single set of spherical harmonic data occupies 125 KiB. For two sets of textures that separately store the compressed spherical harmonic data of the current frame and the compressed spherical harmonic data of the past frame, the storage occupation can be reduced from 375 KiB to 250 KiB.

[0081] Since the texture data corresponding to each spherical harmonic probe is reduced from 24B to 16B, the amount of data written when writing the same number of spherical harmonic probes is reduced to two-thirds of that when using three full-and-half-float four-channel 3D textures. This reduces the amount of data written to the 3D texture during the spherical harmonic probe update process.

[0082] When performing indirect light sampling on the rendering points, 3D texture sampling is performed on the first, second, and third 3D textures respectively. For each texel group participating in the sampling, 24B is read when using three full-and-half-float four-channel 3D textures, and 16B is read when using a heterogeneous 3D texture group, thus reducing the amount of data read to two-thirds of the original. Actual data read / write performance is also related to texture cache hit rate and the storage access characteristics of the running device.

[0083] In this embodiment, by using a semi-floating-point four-channel three-dimensional texture to store the zero-order spherical harmonic coefficients and two 8-bit unsigned normalized four-channel three-dimensional textures to store the remaining first-order encoded values, the storage precision can be configured according to the value characteristics of different spherical harmonic data, thereby reducing the storage footprint of the first-order spherical harmonic coefficients while preserving the dynamic range of the zero-order spherical harmonic coefficients. Furthermore, by using the same probe correspondence among the three three-dimensional textures, the consistency of the compressed spherical harmonic data reading and sampling process can be maintained.

[0084] In one embodiment, the cumulative spherical harmonic coefficients include three color channels, namely a first color channel, a second color channel, and a third color channel, and the three directional components of the first-order spherical harmonic coefficients of each color channel are the first directional component, the second directional component, and the third directional component, respectively. The first channel of the first three-dimensional texture stores the zero-order spherical harmonic coefficient of the first color channel, the second channel stores the zero-order spherical harmonic coefficient of the second color channel, the third channel stores the zero-order spherical harmonic coefficient of the third color channel, and the fourth channel stores the first-order encoded value corresponding to the first direction component of the first color channel. The first, second, and third channels of the second three-dimensional texture sequentially store the first-order encoded values ​​corresponding to the first directional component, the second directional component, and the third directional component of the second color channel, while the fourth channel stores the first-order encoded value corresponding to the second directional component of the first color channel. The first, second, and third channels of the third 3D texture sequentially store the first-order encoded values ​​corresponding to the first, second, and third directional components of the third color channel, while the fourth channel stores the first-order encoded value corresponding to the third directional component of the first color channel.

[0085] Specifically, the first color channel, the second color channel, and the third color channel can correspond to the red color channel, the green color channel, and the blue color channel, respectively. The first directional component, the second directional component, and the third directional component can correspond to the three spatial directional components of the first-order spherical harmonic coefficients, respectively.

[0086] For each spherical harmonic probe, the three color channels each have one zero-order spherical harmonic coefficient and three first-order coded values, totaling three zero-order spherical harmonic coefficients and nine first-order coded values. These three zero-order spherical harmonic coefficients and nine first-order coded values ​​are allocated to the first, second, and third 3D textures according to different storage precisions, rather than storing all twelve values ​​with the same precision.

[0087] The first 3D texture can be denoted as an SHR texture and uses the R16G16B16A16_SFloat format. The first, second, third, and fourth channels of the SHR texture can correspond to the R, G, B, and A channels, respectively, with each channel stored using a 16-bit half-floating-point number. Therefore, each SHR texture unit corresponding to each spherical harmonic probe occupies 8 bytes. See the table below for details: Table 1 The first channel of the SHR texture stores the zero-order spherical harmonic coefficient of the first color channel. The .r channel stores the zeroth-order spherical harmonics of the second color channel. .g. The third channel stores the zeroth-order spherical harmonic coefficients of the third color channel. .b, the fourth channel stores the first-order encoded value corresponding to the first direction component of the first color channel. .

[0088] The second 3D texture can be denoted as an SHG texture and uses the R8G8B8A8_UNorm format. Each channel of the SHG texture is stored using an 8-bit unsigned normalized value. Therefore, each SHG texture unit corresponding to each spherical harmonic probe occupies 4 bytes.

[0089] The first, second, and third channels of the SHG texture sequentially store the first-order encoded values ​​corresponding to the first, second, and third directional components of the second color channel. , and The fourth channel stores the first-order encoded value corresponding to the second-direction component of the first color channel. .

[0090] The third-dimensional texture can be denoted as an SHB texture and uses the R8G8B8A8_UNorm format. Each channel of the SHB texture is also stored using an 8-bit unsigned normalized value. Therefore, each SHB texture unit corresponding to each spherical harmonic probe occupies 4 bytes.

[0091] The first, second, and third channels of the SHB texture sequentially store the first-order encoded values ​​corresponding to the first, second, and third directional components of the third color channel. , and The fourth channel stores the first-order encoded value corresponding to the third-direction component of the first color channel. .

[0092] According to the above allocation method, the three first-order coded values ​​of the first color channel , and The first three color channels are stored in the fourth channels of the SHR, SHG, and SHB textures respectively; the three first-order encoded values ​​of the second color channel are stored in the first three channels of the SHG texture; and the three first-order encoded values ​​of the third color channel are stored in the first three channels of the SHB texture.

[0093] Among them, the first-order encoded value corresponding to the first direction component of the first color channel Stored in the fourth channel of the SHR texture. Because the SHR texture uses a semi-floating four-channel format, It employs half-floating-point precision storage, which offers higher precision than the 8-bit unsigned normalized first-order encoded values ​​stored in SHG and SHB textures. This allocation method utilizes the fourth channel in the SHR texture that is not occupied by the three zero-order spherical harmonic coefficients, eliminating the need for... Set the 3D texture separately.

[0094] For each spherical harmonic probe, the SHR texture occupies 8 bytes, the SHG texture occupies 4 bytes, and the SHB texture occupies 4 bytes, totaling 16 bytes. In comparison, when the SHR, SHG, and SHB textures all use the R16G16B16A16_SFloat format, each texture occupies 8 bytes, and each spherical harmonic probe occupies a total of 24 bytes. Using the above heterogeneous packing method, the storage footprint of each spherical harmonic probe is reduced from 24 bytes to 16 bytes, a reduction of one-third.

[0095] When writing compressed spherical harmonic data for the current frame, the texture coordinates are determined based on the 3D index of the spherical harmonic probe within the probe body. The three zero-order spherical harmonic coefficients and nine first-order encoded values ​​are then written to the texture units corresponding to the same texture coordinates in the three 3D textures according to the aforementioned allocation relationship. During reading, the zero-order spherical harmonic coefficients and first-order encoded values ​​for each color channel are obtained according to the same channel correspondence, and then the first-order spherical harmonic coefficients for each color channel are recovered.

[0096] In this embodiment, by storing the zero-order spherical harmonic coefficients of the three color channels and a first-order encoded value in an 8-byte SHR texture, and storing the remaining first-order encoded values ​​in 4-byte SHG and SHB textures respectively, a heterogeneous texture layout occupying 16 bytes per probe can be formed. At the same time, all channels of the three four-channel textures are utilized, thereby reducing the storage footprint of the spherical harmonic probe while maintaining the storage accuracy of the zero-order spherical harmonic coefficients.

[0097] Furthermore, in one embodiment, configuration parameters can be set for selecting the spherical harmonic coefficient storage method, and a choice can be made between a full-semi-floating-point storage method and a heterogeneous texture storage method based on the configuration parameters. The full-semi-floating-point storage method uses three semi-floating-point four-channel 3D textures, with each spherical harmonic probe occupying 24 bytes, eliminating the need for 8-bit quantization of the first-order encoded value; the heterogeneous texture storage method uses one semi-floating-point four-channel 3D texture and two 8-bit unsigned normalized four-channel 3D textures, with each spherical harmonic probe occupying 16 bytes, and decodes the first-order encoded value during sampling. In one specific embodiment, the heterogeneous texture storage method can be permanently enabled through a preset configuration.

[0098] If all three 3D textures use an 8-bit unsigned normalized format, then the zero-order spherical harmonic coefficients also need to be limited to the normalized storage range. Since the zero-order spherical harmonic coefficients carry high dynamic range sky and hit radiation information, uniformly using an 8-bit unsigned normalized format could easily cause truncation of radiation energy or hue information. Therefore, a semi-floating-point format is used to store the zero-order spherical harmonic coefficients of each color channel.

[0099] RGBM (Red, Green, Blue Shared Multiplier Encoding) or RGBE (Red, Green, Blue Shared Exponent Encoding) encoding represents non-negative color data by sharing a scaling factor or exponent among multiple color components, while the first-order spherical harmonic coefficients are signed quantities. When the shared scaling factor or exponent changes with adjacent spherical harmonic probes, directly performing trilinear interpolation on the encoded data may produce discontinuous changes, thus making it unsuitable for sampling methods that use interpolation followed by decoding.

[0100] ASTC (Adaptive Scalable Texture Compression) or BC (Block Compression) methods use texture blocks as the encoding unit, making it difficult to independently write the texels corresponding to a single spherical harmonic probe during the spherical harmonic probe update process in each frame. Although saving only the zeroth-order spherical harmonic coefficients can further reduce the amount of stored data, it will lose the directional information represented by the first-order spherical harmonic coefficients.

[0101] In this embodiment, by choosing between full half-floating-point storage and heterogeneous texture storage, the storage method of spherical harmonic coefficients can be configured according to storage resources and sampling processing costs; by retaining the half-floating-point precision of the zero-order spherical harmonic coefficients and performing relative encoding on the first-order spherical harmonic coefficients, high dynamic range radiation information, directional information and storage occupancy can be taken into account.

[0102] In one embodiment, the current frame compressed spherical harmonic data and the historical compressed spherical harmonic data are stored in the current texture group and the historical texture group, respectively, and the current texture group and the historical texture group have the same texture format and spherical harmonic coefficient allocation method; After the compressed spherical harmonic data of the current frame is written, the current texture group is used as the historical texture group of the next frame, and the historical texture group is used as the current texture group of the next frame.

[0103] Specifically, the current texture group and the historical texture group include a first 3D texture, a second 3D texture, and a third 3D texture, respectively. The first 3D texture in each texture group adopts a half-floating-point four-channel format, while the second and third 3D textures adopt an 8-bit unsigned normalized four-channel format.

[0104] The corresponding 3D textures in the current texture group and the historical texture group have the same 3D texture size. The same texture coordinates correspond to the same spherical harmonic probe in the spherical harmonic probe body, enabling the current texture group and the historical texture group to be accessed according to the same probe index or texture coordinates.

[0105] Both texture groups also employ the same spherical harmonic coefficient allocation method. The zero-order spherical harmonic coefficients of each color channel and the first-order encoded values ​​corresponding to each directional component are stored in the same channels of the same type of 3D texture in both the current texture group and the historical texture group. Therefore, the same channel correspondence can be used when reading historical compressed spherical harmonic data from the historical texture group and writing current frame compressed spherical harmonic data to the current texture group.

[0106] During the probe update process of the current frame, based on the texture coordinates corresponding to the spherical harmonic probe to be updated, the historical zero-order spherical harmonic coefficients and historical first-order encoded values ​​of the spherical harmonic probe are read from the historical texture group, and the historical first-order encoded values ​​are decoded to obtain the historical spherical harmonic coefficients. The historical spherical harmonic coefficients are then fused with the current frame spherical harmonic coefficients obtained from the probe update to obtain the cumulative spherical harmonic coefficients.

[0107] The fusion is performed in the decoded spherical harmonic coefficient domain, rather than directly mixing the historical first-order coded values ​​with the first-order spherical harmonic coefficients of the current frame. Relative coding includes a scaling relationship based on the zero-order spherical harmonic coefficients of the corresponding color channels, and the historical first-order coded values ​​and the first-order spherical harmonic coefficients of the current frame are in different numerical representations. By restoring the historical first-order spherical harmonic coefficients before fusion, the historical and current frame spherical harmonic coefficients can be placed in the same spherical harmonic coefficient representation domain.

[0108] Subsequently, the cumulative spherical harmonic coefficients are relatively encoded, and according to a preset spherical harmonic coefficient allocation method, the zero-order spherical harmonic coefficients and their first-order encoded values ​​for each color channel are written into the texture coordinates corresponding to the spherical harmonic probe in the current texture group. Thus, during the processing of the current frame, the historical texture group provides historical compressed spherical harmonic data, and the current texture group receives the re-encoded compressed spherical harmonic data for the current frame.

[0109] The completion of writing compressed spherical harmonic data for the current frame means that the compressed spherical harmonic data corresponding to the spherical harmonic probes that need to be updated in the current frame has been written to the current texture group according to the spherical harmonic coefficient allocation method. After writing is completed, the compressed spherical harmonic data in the current texture group is not copied to the historical texture group; instead, the usage of the two texture groups in the next frame is changed.

[0110] Specifically, the current texture group that has completed writing the current frame is designated as the historical texture group for the next frame, and the compressed spherical harmonic data of the current frame stored in it is used as the historical compressed spherical harmonic data for processing in the next frame. At the same time, the historical texture group in the current frame that provides the historical compressed spherical harmonic data is designated as the current texture group for the next frame, and is used to receive the compressed spherical harmonic data re-encoded in the next frame.

[0111] For example, in the first frame, the first texture group can be used as the current texture group, and the second texture group as the historical texture group. After the first frame is written, in the second frame, the first texture group is used as the historical texture group, and the second texture group is used as the current texture group. After the second frame is written, in the third frame, the second texture group is again used as the historical texture group, and the first texture group is used as the current texture group. The two texture groups take turns handling the writing of current compressed spherical harmonic data and the reading of historical compressed spherical harmonic data according to the frame sequence.

[0112] Alternating the use of texture groups does not change the format, size, and channel allocation of the 3D texture itself. Therefore, regardless of whether any texture group is used for writing or reading in the current frame, compressed spherical harmonic data can be processed according to the same texture coordinates and channel correspondences.

[0113] Historical spherical harmonic data is always stored in compressed form within the texture group and is only decoded when participating in the current frame fusion. After fusing to obtain the cumulative spherical harmonic coefficients, the cumulative spherical harmonic coefficients are then relatively encoded and written to the current texture group. Therefore, there is no need to continuously store a set of uncompressed historical spherical harmonic coefficients between adjacent frames.

[0114] In the implementation that uses 8-bit unsigned normalized numerical storage for first-order encoded values, the first-order encoded values ​​are quantized to the corresponding discrete value level when written to a low-precision 3D texture. After the current texture group becomes the historical texture group for the next frame, the quantized first-order encoded values ​​are read and decoded, and thus the quantization error of the first-order encoded values ​​participates in the cumulative update between adjacent frames.

[0115] When the scene and lighting conditions remain stable, the fusion result of the spherical harmonic coefficients of the current frame and the historical spherical harmonic coefficients gradually stabilizes. Once the first-order coded value corresponding to the fusion result falls into a fixed quantization level, the results obtained by storing, reading, decoding, and re-encoding the first-order coded value in subsequent frames remain near the value determined by the corresponding quantization level, thereby reducing the continuous drift or oscillation caused by repeated encoding and decoding.

[0116] In this embodiment, by having the current texture group and the historical texture group alternately undertake the writing and reading purposes according to the frame, the compressed spherical harmonic data of the current frame can be directly used as the historical compressed spherical harmonic data of the next frame, and the copying of the entire group of compressed spherical harmonic data between frames can be avoided. By having the two texture groups adopt the same texture format and spherical harmonic coefficient allocation method, the consistency of data reading, decoding, fusion and rewriting relationships between frames can be maintained. By performing historical fusion in the decoded spherical harmonic coefficient domain and keeping the first-order encoded value in the stable state near the corresponding quantization level, the continuous drift or oscillation caused by the round trip of compressed data frame by frame can be reduced.

[0117] In one embodiment, the texture sampling coordinates of the point to be rendered in the spherical harmonic probe are determined based on the world space location of the point to be rendered. Based on the texture sampling coordinates, trilinear interpolation is performed on each three-dimensional texture storing the compressed spherical harmonic data of the current frame to obtain the interpolated zero-order spherical harmonic coefficients and first-order encoded values. Then, the interpolated first-order encoded values ​​are decoded based on the interpolated zero-order spherical harmonic coefficients.

[0118] Specifically, the spherical harmonic probes in the spherical harmonic probe body are arranged according to a three-dimensional grid, and each spherical harmonic probe corresponds to a world space position and a three-dimensional texture coordinate. The first three-dimensional texture, the second three-dimensional texture, and the third three-dimensional texture use the same three-dimensional texture size and probe correspondence, so that the same texture sampling coordinate can be used to read compressed spherical harmonic data in the three three-dimensional textures.

[0119] For any point to be rendered, obtain its world space position, and based on the position, coverage area, orientation, and probe spacing of the spherical harmonic probe in world space, convert the world space position to the local coordinates of the spherical harmonic probe. Then, based on the 3D mesh size of the spherical harmonic probe, convert the local coordinates into texture sampling coordinates in the 3D texture.

[0120] The texture sampling coordinates can represent the relative position of the point to be rendered between adjacent spherical harmonic probes. Based on these texture sampling coordinates, eight adjacent spherical harmonic probes surrounding the point to be rendered can be determined, and the trilinear interpolation weights of the eight adjacent spherical harmonic probes can be determined based on the relative positions of the point to be rendered along the three coordinate directions of the spherical harmonic probe body.

[0121] Using the same texture sampling coordinates, trilinear interpolation is performed on the first, second, and third 3D textures in the current texture group. The trilinear interpolation is applied to the four channels of each 3D texture, so that the values ​​provided by the eight adjacent spherical harmonic probes in each channel are fused according to the corresponding trilinear interpolation weights.

[0122] For the first 3D texture, trilinear interpolation is used to obtain the interpolated zero-order spherical harmonic coefficients of each of the three color channels, as well as the interpolated first-order encoded value stored in its fourth channel. For the second and third 3D textures, trilinear interpolation is used to obtain the remaining interpolated first-order encoded values ​​stored in the corresponding channels.

[0123] Subsequently, according to the spherical harmonic coefficient allocation method, the zero-order spherical harmonic coefficients and the first-order encoded values ​​corresponding to the three directional components of each color channel are obtained from the interpolation results of the three 3D textures. For each color channel, using the interpolated zero-order spherical harmonic coefficients of that color channel as a reference, the three interpolated first-order encoded values ​​belonging to that color channel are back-mapped to obtain the three directional components of the first-order spherical harmonic coefficients of that color channel.

[0124] Since the weights in trilinear interpolation are non-negative and the sum of the weights is 1, when all the first-order coded values ​​involved in the interpolation are within the normalized interval [0,1], the interpolated first-order coded values ​​will still be within [0,1]. Therefore, after completing trilinear interpolation, the zero-order spherical harmonic coefficients of the corresponding color channel can be directly used for decoding without performing normalization processing on the interpolated first-order coded values ​​again.

[0125] Furthermore, trilinear interpolation is first performed on each 3D texture under compressed encoding, and then the interpolation results are decoded. Compared with reading and decoding the compressed spherical harmonic data of eight adjacent spherical harmonic probes separately and then performing weighted fusion, this processing method can reduce the number of decoding times of the first-order encoded values ​​and obtain the spherical harmonic coefficients corresponding to the points to be rendered by utilizing the continuous sampling relationship of the 3D textures.

[0126] The zeroth and first-order spherical harmonic coefficients of each color channel obtained from decoding can be used to reconstruct the incident radiation distribution at the point to be rendered, and combined with the surface orientation of the point to be rendered to determine its indirect lighting result. For other points to be rendered within the coverage area of ​​the spherical harmonic probe, texture coordinate transformation, trilinear interpolation, and decoding can be performed according to their corresponding world space positions.

[0127] In this embodiment, by converting the world space position of the point to be rendered into the texture sampling coordinates in the spherical harmonic probe, a spatial correspondence between the point to be rendered and the surrounding spherical harmonic probes can be established. By first performing trilinear interpolation on the compressed spherical harmonic data, and then decoding the interpolated first-order encoded value based on the interpolated zero-order spherical harmonic coefficients, the number of decoding processes can be reduced, and the spherical harmonic coefficients used for indirect lighting calculation of the point to be rendered can be obtained with lower storage usage.

[0128] Furthermore, in one embodiment, compressed spherical harmonic data stored in a heterogeneous 3D texture group can be directly used for indirect lighting calculations of the points to be rendered through 3D texture sampling and spherical harmonic coefficient decoding, without setting up a separate spatial filtering processing stage. The spatial continuity between spherical harmonic probes can be obtained through trilinear interpolation or neighborhood weighted fusion, thus eliminating the need to generate additional spatially filtered textures or save spatial filtering results.

[0129] In this embodiment, by reusing the three-dimensional texture sampling process to complete the spatial fusion of adjacent spherical harmonic probe data, the intermediate data and additional storage resources required for independent spatial filtering processing can be reduced, enabling compressed spherical harmonic data to be directly accessed in the indirect lighting sampling process of the point to be rendered.

[0130] Furthermore, in one embodiment, a neighborhood sampling method that first decodes and then fuses can be used to obtain the spherical harmonic coefficients corresponding to the point to be rendered. This sampling method is set up in parallel with the sampling method that first performs trilinear interpolation on the 3D texture of the encoded domain and then decodes the interpolation result, and can be selected according to the processing capability of the running device or the requirements of indirect lighting quality.

[0131] Specifically, the texture sampling coordinates of the point to be rendered within the spherical harmonic probe volume are determined based on its world space location, and the eight adjacent spherical harmonic probes surrounding the point to be rendered are determined based on these texture sampling coordinates. The texture unit positions corresponding to the eight adjacent spherical harmonic probes in the first, second, and third 3D textures are then determined respectively.

[0132] For each adjacent spherical harmonic probe, the zero-order spherical harmonic coefficients and first-order encoded values ​​of the three color channels are read at their corresponding texture unit positions. Then, using the zero-order spherical harmonic coefficients of each color channel of the adjacent spherical harmonic probe as a reference, the first-order encoded values ​​belonging to the corresponding color channels are decoded to obtain the zero-order and first-order spherical harmonic coefficients of the adjacent spherical harmonic probe.

[0133] After the eight adjacent spherical harmonic probes have completed their decoding, the fusion weight of each adjacent spherical harmonic probe is determined based on the spatial relationship between the point to be rendered and each adjacent spherical harmonic probe. The fusion weight can be determined by combining trilinear interpolation weight and inverse distance weight, so that the spherical harmonic probes that are closer to the point to be rendered have relatively larger fusion weights.

[0134] The fusion weights of the eight adjacent spherical harmonic probes are normalized so that the sum of the normalized fusion weights is 1. Then, in the decoded spherical harmonic coefficient domain, the zero-order and first-order spherical harmonic coefficients of the eight adjacent spherical harmonic probes are weighted and fused according to the normalized fusion weights to obtain the target spherical harmonic coefficients corresponding to the point to be rendered.

[0135] Since the first-order coded values ​​of each adjacent spherical harmonic probe are decoded based on its own zero-order spherical harmonic coefficients, decoding before fusion can reduce the approximation deviation caused by direct interpolation in the coding domain in regions where the zero-order spherical harmonic coefficients of adjacent spherical harmonic probes differ significantly. This sampling method requires reading and decoding the compressed spherical harmonic data of eight adjacent spherical harmonic probes separately, thus it can be used in operating scenarios with high requirements for indirect lighting quality.

[0136] Furthermore, when the point to be rendered is located in the boundary region of the spherical harmonic probe, the boundary transition weight can be determined based on the positional relationship between the point to be rendered and the boundary of the spherical harmonic probe. Then, according to the boundary transition weight, the indirect lighting result obtained from the spherical harmonic probe and the preset indirect lighting result are gradually blended. For sampling coordinates that exceed the texture range of the spherical harmonic probe, they can be restricted to the boundary coordinates of the corresponding 3D texture.

[0137] Furthermore, a brightness upper limit can be set for the sampled indirect lighting results. When the brightness of the indirect lighting results exceeds the upper limit, it is restricted to that limit to reduce brightness flicker caused by overshoot in the encoding, decoding, or neighborhood fusion results of individual spherical harmonic probes. The upper limit can be set according to the radiation intensity range of the rendered scene.

[0138] In this embodiment, by decoding multiple spherical harmonic probes around the point to be rendered separately, and then performing weighted fusion in the decoded spherical harmonic coefficient domain, the approximate deviation caused by encoding domain interpolation when the zero-order spherical harmonic coefficients of adjacent spherical harmonic probes differ greatly can be reduced; by setting boundary transition, texture boundary limit and brightness upper limit, the continuity and stability of indirect lighting in the boundary region of the spherical harmonic probe and the local bright region can be improved.

[0139] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0140] Furthermore, some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as cellular phones, smartphones, wearable devices, and other similar computing devices.

[0141] The electronic device includes: one or more processors; and a memory storing computer program instructions that, when executed, cause the processor to perform the steps of the methods provided in any one or more of the above embodiments. Figure 2An exemplary structural diagram of the electronic device is disclosed. The electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0142] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103 and output device 1104 may be connected by a bus or other means, as shown in the figure, which is connected by a bus 1105.

[0143] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display, a light-emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0144] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback); and input from the user can be received in any form (e.g., voice input or tactile input).

[0145] In this embodiment, a computer-readable medium stores a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in any one or more of the above embodiments. This computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The aforementioned computer-readable medium carries one or more computer-readable instructions.

[0146] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0147] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0148] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0149] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, read-only optical discs, digital versatile optical discs or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0150] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0151] In the above embodiments, all or part of the implementation can be achieved through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. In addition, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0152] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0153] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0154] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device in software or hardware. Terms such as "first," "second," etc., are used only for distinguishing descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0155] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method for compressed storage of spherical harmonic coefficients, characterized in that, The method includes: Perform probe update on the spherical harmonic probe in the spherical harmonic probe body to obtain the spherical harmonic coefficients of the current frame. The spherical harmonic coefficients of the current frame include the zero-order spherical harmonic coefficients and the first-order spherical harmonic coefficients corresponding to each color channel. The first-order spherical harmonic coefficients include three directional components. The historical compressed spherical harmonic data is decoded and then fused with the spherical harmonic coefficients of the current frame to obtain the cumulative spherical harmonic coefficients; The zero-order spherical harmonic coefficients of each color channel in the cumulative spherical harmonic coefficients are retained, and the three directional components of the corresponding first-order spherical harmonic coefficients are relatively encoded based on the zero-order spherical harmonic coefficients of each color channel to obtain multiple first-order encoded values. The zero-order spherical harmonic coefficients of each color channel and one of the multiple first-order encoded values ​​are stored in a first three-dimensional texture, and the remaining first-order encoded values ​​are stored in one or more low-precision three-dimensional textures. The storage precision of the low-precision three-dimensional textures is lower than that of the first three-dimensional textures, thus obtaining the compressed spherical harmonic data of the current frame, and using the compressed spherical harmonic data of the current frame as the historical compressed spherical harmonic data of the next frame.

2. The method for compressing and storing spherical harmonic coefficients according to claim 1, characterized in that, The relative encoding includes: When the zeroth-order spherical harmonic coefficient of any color channel in the cumulative spherical harmonic coefficients is not lower than a preset threshold, the ratios between the three directional components of the first-order spherical harmonic coefficient of that color channel and the zeroth-order spherical harmonic coefficient are determined respectively, and the ratios are mapped to the normalization interval according to the preset encoding scaling parameters, so that the ratios with a value of zero are mapped to neutral values. When the zero-order spherical harmonic coefficient of any color channel in the cumulative spherical harmonic coefficients is lower than the preset threshold, the three first-order coding values ​​corresponding to that color channel are set to the neutral value. The decoding process includes inverse mapping of the first-order encoded value to be decoded and recovering the first-order spherical harmonics based on the zero-order spherical harmonics of the corresponding color channels.

3. The method for compressing and storing spherical harmonic coefficients according to claim 2, characterized in that, The normalization interval is [0,1], and the neutral value is 0.

5. When the mapping result obtained according to the preset encoding scaling parameters exceeds the normalization interval, the mapping result is restricted to the normalization interval.

4. The method for compressing and storing spherical harmonic coefficients according to claim 1, characterized in that, The three-dimensional texture used to store the compressed spherical harmonic data of the current frame includes a first three-dimensional texture, a second three-dimensional texture, and a third three-dimensional texture, wherein the second three-dimensional texture and the third three-dimensional texture are the low-precision three-dimensional textures; The first three-dimensional texture is a semi-floating four-channel three-dimensional texture, and the second and third three-dimensional textures are both 8-bit unsigned normalized four-channel three-dimensional textures.

5. The method for compressing and storing spherical harmonic coefficients according to claim 4, characterized in that, The cumulative spherical harmonic coefficients include three color channels, namely the first color channel, the second color channel, and the third color channel. The three directional components of the first-order spherical harmonic coefficients of each color channel are the first directional component, the second directional component, and the third directional component, respectively. The first channel of the first three-dimensional texture stores the zero-order spherical harmonic coefficient of the first color channel, the second channel stores the zero-order spherical harmonic coefficient of the second color channel, the third channel stores the zero-order spherical harmonic coefficient of the third color channel, and the fourth channel stores the first-order encoded value corresponding to the first direction component of the first color channel. The first, second, and third channels of the second three-dimensional texture sequentially store the first-order encoded values ​​corresponding to the first directional component, the second directional component, and the third directional component of the second color channel, while the fourth channel stores the first-order encoded value corresponding to the second directional component of the first color channel. The first, second, and third channels of the third 3D texture sequentially store the first-order encoded values ​​corresponding to the first, second, and third directional components of the third color channel, while the fourth channel stores the first-order encoded value corresponding to the third directional component of the first color channel.

6. The method for compressing and storing spherical harmonic coefficients according to claim 1, characterized in that, The current frame compressed spherical harmonic data and the historical compressed spherical harmonic data are stored in the current texture group and the historical texture group, respectively. The current texture group and the historical texture group have the same texture format and spherical harmonic coefficient allocation method. After the compressed spherical harmonic data of the current frame is written, the current texture group is used as the historical texture group of the next frame, and the historical texture group is used as the current texture group of the next frame.

7. The method for compressing and storing spherical harmonic coefficients according to claim 1, characterized in that, The texture sampling coordinates of the point to be rendered in the spherical harmonic probe are determined based on the world space location of the point to be rendered. Based on the texture sampling coordinates, trilinear interpolation is performed on each three-dimensional texture storing the compressed spherical harmonic data of the current frame to obtain the interpolated zero-order spherical harmonic coefficients and first-order encoded values. Then, the interpolated first-order encoded values ​​are decoded based on the interpolated zero-order spherical harmonic coefficients.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images