Method and device for ray tracing and computer readable medium

Abstract

The invention relates to a ray tracing method and device and a computer readable medium. The method comprises the steps that an acceleration structure in a ray tracing scene is updated so that the acceleration structure can be matched with the performance of mobile terminal equipment, and the acceleration structure is used for storing geometrical shapes and position information of all objects in the ray tracing scene; performing light detection on an object in the light tracing scene based on an acceleration structure, and rendering a first rendering result with a shadow effect, an ambient light shielding effect and noise according to a detection result; and performing noise reduction on the first rendering result by using a target noise reducer for optimizing the mobile terminal equipment to obtain a second rendering result. The mobile terminal is optimized from the updating stage of the acceleration structure, the rendering stage of shadow and ambient light shielding and the noise reduction stage, the performance and efficiency can be improved, and the technical problem that an existing shadow and ambient light shielding technology for mobile terminal light tracking is low in adaptation degree with the mobile terminal is solved.

Description

Technical Field

[0001] The present application relates to the field of ray tracing technology, and in particular to a method, device and computer-readable medium for performing ray tracing. Background Art

[0002] Ray tracing is a method to display objects realistically. The ray tracing method tracks the light in the opposite direction of the light reaching the viewpoint, passing through each pixel on the screen, finding the point P0 on the surface of the object that intersects the line of sight, and continues to track to find all light sources that affect the light intensity at point P0, thereby calculating the precise light intensity at point P0. It is often used to express mirror effects in material editing. Ray tracing technology has been maturely applied on desktops, but there is still a lot of room for optimization on mobile devices.

[0003] In the related technologies, mobile ray tracing has room for optimization in both shadows and ambient occlusion. Currently, the implementation of mobile ray tracing shadows can usually be achieved through hardware-supported Ray Tracing Inline technology to perform collision detection on the scene Acceleration Structure, thereby obtaining a completely physically correct occlusion result, or using ShadowMap technology, according to the position and direction of the light source that needs to generate the shadow, set the corresponding observation matrix and projection matrix, render all the geometric bodies in the scene from the perspective of the light source, store the depth information in the Shadow Map, and when rendering the scene, project the current pixel from the camera's perspective, and find out whether the pixel is occluded in the Shadow Map, and use PCF and other technologies to optimize sampling, thereby generating shadows. Currently, mobile ambient occlusion can use post-processed screen space ambient occlusion, that is, by detecting the depth information around the pixel to calculate the occlusion rate of the pixel, thereby simulating ambient occlusion and improving the realism of the scene. However, all of the above solutions have the problem of low compatibility with mobile terminals, such as no optimization of frustum clipping or occlusion culling for mobile terminals, no support for masked materials for mobile terminals, and no maximum optimization of performance for mobile terminals.

[0004] There is currently no effective solution to the problem that the existing mobile ray tracing shadow and ambient occlusion technologies are not well adapted to mobile devices. Summary of the invention

[0005] The present application provides a method, device and computer-readable medium for performing ray tracing to solve the technical problem that the shadow and ambient light occlusion technology of existing mobile ray tracing has low compatibility with mobile terminals.

[0006] According to one aspect of an embodiment of the present application, the present application provides a method for performing ray tracing, including: updating an acceleration structure in a ray tracing scene so that the acceleration structure matches the performance of a mobile device, wherein the acceleration structure is used to save the geometric shape and position information of all objects in the ray tracing scene; performing light detection on objects in the ray tracing scene based on the acceleration structure, and rendering a first rendering result with shadow effects and ambient occlusion effects and noise based on the detection results; and denoising the first rendering result using a target denoiser optimized for mobile devices to obtain a second rendering result.

[0007] Optionally, updating the acceleration structure in the ray tracing scene includes: for a first object of the vegetation class, using a first solid angle to perform solid angle culling on the top-level acceleration structure of the first object to reduce the number of instances constituting the top-level acceleration structure; for a second object of the standard model class, when the second object is in front of the virtual camera, using a second solid angle to perform solid angle culling on the top-level acceleration structure of the second object to reduce the number of instances constituting the top-level acceleration structure, wherein the second solid angle is smaller than the first solid angle; when the second object is in the back of the virtual camera and the distance between the second object and the virtual camera is greater than or equal to a first distance threshold, using a third solid angle to perform solid angle culling on the top-level acceleration structure of the second object to reduce the number of instances constituting the top-level acceleration structure, wherein the third solid angle is greater than the first solid angle; for a third object of the skeletal animation class, when updating the base-level acceleration structure of the third object in each frame, selecting a target object whose distance from the virtual camera is less than or equal to the second distance threshold from all third objects in each frame, and updating the base-level acceleration structure of the target object.

[0008] Optionally, light detection is performed on objects in a ray tracing scene based on an acceleration structure, and based on the detection result, a first rendering result with shadow effects and ambient occlusion effects and noise is rendered, including: inputting the frame index of the object in the acceleration structure and the pixel coordinates of the object in the corresponding frame into a micro-encryption model to generate a pseudo-random seed value through the micro-encryption model; inputting the pseudo-random seed value into a linear congruential generator to generate random floating-point numbers uniformly distributed in a target square through the linear congruential generator, wherein the random floating-point numbers are used to represent the importance sampling values ​​of the objects; calculating the concentric mapping of the random floating-point numbers to generate sampling points within a unit circle; determining the direction of the light source toward the sampling point as the light source direction, and generating an offset light direction based on the sampling point and the radius of the light source; calculating the shadow light direction through the light source direction and the offset light direction, and calculating the ambient occlusion light direction based on the sampling point; rendering the shadow effect according to the shadow light direction, and rendering the ambient occlusion effect according to the ambient occlusion light direction, to obtain a first rendering result.

[0009] Optionally, the method also includes rendering the shadow effect in the following manner: determining the material type of the target pixel hit by the query ray, wherein the query ray is used to select pixels that require secondary shading during shadow calculation of ray tracing; when the material type is a mask, using a shadow map to calculate the shadow of the target pixel, or using a shadow map to clamp the starting point and ending point of the shadow rendered by ray detection; when the material type is a non-mask, performing ray detection on the target pixel, and rendering the shadow effect of the target pixel based on the detection result.

[0010] Optionally, the first rendering result is denoised using a target denoiser optimized for mobile devices to obtain a second rendering result including: performing time domain filtering on the first rendering result to calculate the time domain variance of the first rendering result; performing iterative spatial domain filtering on the first rendering result, and using the time domain variance to adjust the filtering weights for each round of iteration, wherein the filtering weights include shadow weights and ambient occlusion weights; performing weighted summation on the shadows output from each round, and taking the average of the sum as the shadow of the second rendering result, and performing weighted summation on the ambient occlusion output from each round, and taking the average of the sum as the ambient occlusion of the second rendering result.

[0011] Optionally, performing time domain filtering on the first rendering result to calculate the time domain variance of the first rendering result includes: obtaining depth information of the first rendering result and a preset ray tracing result, wherein the first rendering result is the ray tracing result of the first scene in the latest frame, and the preset ray tracing result is the ray tracing result of the second scene in the historical frame, and the preset ray tracing result is pre-cached in the on-chip local shared memory; when the similarity of the depth information of the two reaches a target threshold, replacing the shadow of the first rendering result with the shadow of the preset ray tracing result; performing pooling on the first rendering result to obtain a third rendering result; obtaining a neighborhood rendering result of each sampling point in the third rendering result; determining the variance of the neighborhood rendering result as the time domain variance of the sampling point; obtaining the time domain variance of each sampling point to obtain the time domain variance of the first rendering result.

[0012] Optionally, pooling is performed on the first rendering result to obtain a third rendering result, including: dividing the first rendering result into multiple sub-areas, and initializing the shadow sum and ambient occlusion sum of each sub-area; traversing the sampling points in each sub-area, calculating the shadow weight, ambient occlusion weight, shadow intensity and ambient occlusion intensity of each sampling point, and adding the product of the shadow intensity and the shadow weight to the shadow sum of the corresponding sub-area, and adding the product of the ambient occlusion intensity and the ambient occlusion weight to the ambient occlusion sum of the corresponding sub-area; calculating the average value of the shadow sum for each sub-area to obtain the shadow average value, and calculating the average value of the ambient occlusion sum for each sub-area to obtain the ambient occlusion average value; using the shadow average value as the shadow value of the corresponding sub-area, and using the ambient occlusion average value as the ambient occlusion value of the corresponding sub-area to obtain the third rendering result.

[0013] According to another aspect of an embodiment of the present application, the present application provides a device for performing ray tracing, including: an update module, used to update an acceleration structure in a ray tracing scene so that the acceleration structure matches the performance of a mobile device, wherein the acceleration structure is used to save the geometric shape and position information of all objects in the ray tracing scene; a rendering module, used to perform light detection on objects in the ray tracing scene based on the acceleration structure, and render a first rendering result with shadow effects and ambient occlusion effects and noise based on the detection results; a denoising module, used to perform denoising on the first rendering result using a target denoiser optimized for mobile devices to obtain a second rendering result.

[0014] According to another aspect of an embodiment of the present application, the present application also provides a computer-readable medium having a non-volatile program code executable by a processor, and the program code enables the processor to execute the above method.

[0015] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:

[0016] The present application provides a method for performing ray tracing, including: updating an acceleration structure in a ray tracing scene so that the acceleration structure matches the performance of a mobile device, wherein the acceleration structure is used to store the geometric shape and position information of all objects in the ray tracing scene; performing light detection on objects in the ray tracing scene based on the acceleration structure, and rendering a first rendering result with shadow effects and ambient occlusion effects and noise based on the detection result; and denoising the first rendering result using a target denoiser optimized for mobile devices to obtain a second rendering result. The present application is optimized for mobile devices from the update stage of the acceleration structure, the rendering stage of shadows and ambient occlusion, and the denoising stage, which can improve performance and efficiency, thereby realizing real-time ray tracing rendering on mobile devices, and solving the technical problem that the existing mobile ray tracing shadow and ambient occlusion technologies have low adaptability to mobile devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A schematic flow chart of an optional method for performing ray tracing according to an embodiment of the present application;

[0020] Figure 2 A schematic diagram of an optional architecture for performing ray tracing according to an embodiment of the present application;

[0021] Figure 3 A schematic diagram of an optional light detection process provided according to an embodiment of the present application;

[0022] Figure 4 A schematic diagram of an optional rendering process provided according to an embodiment of the present application;

[0023] Figure 5 A schematic diagram of a preparation phase flow of an optional time domain filtering provided according to an embodiment of the present application;

[0024] Figure 6 A schematic diagram of a flow chart of a pooling phase of an optional time domain filtering according to an embodiment of the present application;

[0025] Figure 7A schematic diagram of a filtering phase flow of an optional time domain filtering provided according to an embodiment of the present application;

[0026] Figure 8 A schematic diagram of an optional spatial domain filtering process provided according to an embodiment of the present application;

[0027] Fig. 9 A comparison diagram of ray tracing effects on a mobile terminal provided according to an embodiment of the present application;

[0028] Fig.10 This is another comparison diagram of ray tracing effects on a mobile terminal provided according to an embodiment of the present application;

[0029] Fig.11 A block diagram of an optional device for performing ray tracing according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0031] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0032] First, the abbreviations and key terms involved in this application are explained:

[0033] AO: Ambient Occlusion is a technology used to improve the realism of game scenes. It simulates the occlusion relationship between light on the surface of objects, shades the scene during the rendering process, and achieves a natural gradient of shadows and brightness between objects, thereby enhancing the sense of detail of the scene.

[0034] Ray Tracing Inline: Ray Tracing Inline is a ray tracing technology that allows the use of hardware-accelerated ray tracing directly in shader programs. This means that developers can embed ray tracing code in shader programs.

[0035] AS: Acceleration Structure, is a data structure used to accelerate the intersection calculation between light and objects in the scene. It is created during the scene initialization phase and stores the geometry and position information of all objects in the scene.

[0036] TLAS: TLAS refers to the Top-Level Acceleration Structure, a data structure used to accelerate ray tracing in the DirectXRaytracing API. It is a hierarchical structure composed of multiple instances, which is used to describe the position, posture, shape and other information of all objects in the scene.

[0037] BLAS: BLAS refers to the Basic Level Acceleration Structure, which is used to accelerate ray tracing calculations. It can be seen as a collection of one or more geometric figures, each of which is composed of triangular meshes or other basic geometric elements. BLAS compresses and optimizes this geometric information to improve the efficiency and speed of ray tracing. When performing ray tracing in a scene, the application can directly use the BLAS data structure without recalculating the geometric information. In addition, BLAS also supports dynamic updates and reconstructions, which means that applications can add, delete, and transform objects in the scene without rebuilding the acceleration structure of the entire scene, which can greatly improve efficiency and flexibility.

[0038] Masked:

[0039] Materials usually use a black and white map as a mask to define the opaque and transparent areas of the model.

[0040] LCG:

[0041] The full name is Linear congruential generator. It is a pseudo-random number generation algorithm. Its basic principle is to generate a sequence of numbers through modular operations and linear functions. Due to its simplicity, fast calculation speed, and small memory usage, it is often used in various random number generation modules in game development.

[0042] TEA:

[0043] The full name is Tiny Encryption Algorithm, which is a symmetric encryption algorithm that uses a key and a plaintext block as input, generates ciphertext through multiple rounds of iterative operations, and can be used to generate pseudo-random number sequences. Its characteristics are simple algorithm, fast speed and small memory usage, and because the TEA algorithm has high security, the generated pseudo-random number sequence also has good randomness.

[0044] PDF:

[0045] The probability density function (PDF) is a function used to describe the probability distribution of a random variable. It is mathematically defined as the probability distribution function of a continuous random variable. The probability within a certain interval can be obtained by integrating the value range within the interval.

[0046] DiskConcentric:

[0047] DiskConcentric is an algorithm for two-dimensional image processing that maps a square area onto a disk and keeps the image texture unchanged. The algorithm uses the inscribed circle with the center point as the center point and the radius as the border width. By using trigonometric functions and polar coordinate conversion, each pixel is mapped from a square grid to a disk in the polar coordinate system, which can be used for texture mapping and sampling area conversion.

[0048] In the related technologies, mobile ray tracing has room for optimization in both shadows and ambient occlusion. Currently, the implementation of mobile ray tracing shadows can usually be achieved through hardware-supported Ray Tracing Inline technology to perform collision detection on the scene Acceleration Structure, thereby obtaining a completely physically correct occlusion result, or using ShadowMap technology, according to the position and direction of the light source that needs to generate the shadow, set the corresponding observation matrix and projection matrix, render all the geometric bodies in the scene from the perspective of the light source, store the depth information in the Shadow Map, and when rendering the scene, project the current pixel from the camera's perspective, and find out whether the pixel is occluded in the Shadow Map, and use PCF and other technologies to optimize sampling, thereby generating shadows. Currently, mobile ambient occlusion can use post-processed screen space ambient occlusion, that is, by detecting the depth information around the pixel to calculate the occlusion rate of the pixel, thereby simulating ambient occlusion and improving the realism of the scene. However, all of the above solutions have the problem of low compatibility with mobile terminals, such as no optimization of frustum clipping or occlusion culling for mobile terminals, no support for masked materials for mobile terminals, and no maximum optimization of performance for mobile terminals.

[0049] In order to solve the problem mentioned in the background technology, according to one aspect of the embodiment of the present application, an embodiment of a method for performing ray tracing is provided, which is executed by a mobile device, such as Figure 1 As shown, the method may include the following steps:

[0050] Step S102, updating the acceleration structure in the ray tracing scene so that the acceleration structure matches the performance of the mobile device, wherein the acceleration structure is used to store the geometric shape and position information of all objects in the ray tracing scene;

[0051] Step S104, performing light detection on the object in the ray tracing scene based on the acceleration structure, and rendering a first rendering result with shadow effect and ambient occlusion effect and noise according to the detection result;

[0052] Step S106: Use a target denoiser optimized for mobile devices to denoise the first rendering result to obtain a second rendering result.

[0053] Through steps S102 to S106, the present application is optimized for mobile terminals from the acceleration structure update stage, the shadow and ambient occlusion rendering stage, and the noise reduction stage, which can improve performance and efficiency, thereby realizing real-time ray tracing rendering on mobile devices, and solving the technical problem that the existing mobile ray tracing shadow and ambient occlusion technology has low adaptability to the mobile terminal.

[0054] The present application provides a method for ray tracing based on Figure 2 The architecture shown in this application is targeted at mobile platforms, such as Qualcomm Snapdragon 8gen2 CPU, with the goal of running ray tracing shadows and ambient occlusion smoothly at 60 frames. To achieve this goal, this application has made a series of optimization measures for the update stage of the acceleration structure, the rendering stage of shadows and ambient occlusion, and the denoising stage. Figure 2 It shows the main contents of the rendering stage and the denoising stage. The rendering stage is Figure 2 The hybrid ray tracing part, the denoising stage is the time domain filtering kernel spatial domain filtering part.

[0055] In the hybrid ray tracing stage, this application uses lower rendering resolution and Inline ray tracing with one light per shading point (1spp) to reduce the amount of calculation, uses the depth map to restore the geometric normals and world space coordinates of the screen space shading point, uses the HitGroupIndex (BindingTable) information to determine the material type of the occluder, and calculates the depth-Relative Bias information of the ray tracing based on the depth to avoid self-intersection. For shadows, shadowmap and ray tracing shadows are used in combination. The low-frequency light source perspective HZB is used to exclude shading points that do not require ray tracing (all in light / shadow), and the starting point (Tmin) and ending point (Tmax) of ray tracing are clamped to improve the efficiency of ray intersection. In terms of ambient occlusion, a single Cosine Normal Ray is emitted from the shading point, the distance between the hit point and the current shading point is obtained, the intensity of the ambient occlusion is calculated, and the 0-1 occlusion value of the ambient occlusion is output.

[0056] In the time domain filtering stage, according to the real machine test results, the half-precision depth and extended neighborhood ray tracing results are pre-cached into the on-chip local shared memory (LDS) for time domain calculation. First, the historical frame sampling data is clamped based on the Variance-based bounds (7x7 kernel) of the current frame ray tracing neighborhood results to balance the impact of time domain ghosting and time domain jitter caused by downsampling. Then, the time domain variance information is calculated based on the clamped historical frames, and this information is used to adjust the kernel weights of subsequent spatial filtering.

[0057] In the spatial domain filtering stage, the Edge-Avoiding A-Trous Wavelet (EAW) filtering method based on wavelet transform is used, with three filtering passes, 9 samples per pass, and a total of 27 samples to approximate a 9x9 bilateral filter convolution kernel. To ensure edge quality, the Edge Stopping function is used to input screen space depth, temporal variance, and shading point geometric normal information to calculate the filtering weight, and the temporal variance is used to detect whether the shading point needs to be filtered to save performance.

[0058] The following is a detailed description of steps 102 to 106 of the method for performing ray tracing.

[0059] In step S102, updating the acceleration structure in the ray tracing scene includes:

[0060] For a first object of the vegetation class, solid angle culling is performed on a top-level acceleration structure of the first object using a first solid angle to reduce the number of instances constituting the top-level acceleration structure;

[0061] For a second object of the standard model class, when the second object is in front of the virtual camera, using a second solid angle to perform solid angle culling on the top acceleration structure of the second object to reduce the number of instances constituting the top acceleration structure, wherein the second solid angle is smaller than the first solid angle;

[0062] When the second object is at a position behind the virtual camera and the distance between the second object and the virtual camera is greater than or equal to the first distance threshold, use a third solid angle to perform solid angle culling on the top acceleration structure of the second object to reduce the number of instances constituting the top acceleration structure, wherein the third solid angle is greater than the first solid angle;

[0063] For the third object of the skeletal animation class, when the basic hierarchical acceleration structure of the third object in each frame is updated, a target object whose distance from the virtual camera is less than or equal to a second distance threshold is selected from all the third objects in each frame, and the basic hierarchical acceleration structure of the target object is updated.

[0064] In the embodiment of the present application, in order to obtain high performance in complex scenes, ray tracing performs ray intersection on an optimized data structure (called an acceleration structure (AS)) built on scene information. The acceleration structure is divided into two levels, namely the top-level acceleration structure (Top-Level Acceleration Structure) and the basic level acceleration structure (Basic Level Acceleration Structure). For acceleration structures of different levels, the present application provides corresponding update methods.

[0065] In the embodiment of the present application, objects can be divided into three categories, namely vegetation, standard model and skeletal animation. The standard model is a model obtained by combining basic geometric bodies such as triangles and rectangles. The top-level acceleration structure exists in vegetation and standard models, and the basic-level acceleration structure exists in vegetation and skeletal animation.

[0066] In an embodiment of the present application, vegetation can be culled using the first solid angle, and for standard models, those in front of the camera can be culled using the second solid angle, and those on the back of the camera and whose distance from the virtual camera exceeds the first distance threshold can be culled according to the third solid angle, wherein the second solid angle is smaller than the first solid angle, and the third solid angle is larger than the first solid angle. The above-mentioned distance thresholds and solid angles can be set according to actual needs, such as a culling solid angle of 4 degrees for vegetation, a culling solid angle of 2 degrees for the front of the standard model, a culling solid angle of 10 degrees for the standard model on the back of the camera and more than 10 meters away, and so on. By optimizing the update of the top-level acceleration structure, the number of instances in the top-level acceleration structure that need to be updated per frame can be reduced, which can ensure culling efficiency and improve rendering accuracy.

[0067] In an embodiment of the present application, since the skeletal animation needs to update the basic hierarchical acceleration structure every frame, but is limited by the functional limitations of the mobile terminal, when a large number of basic hierarchical acceleration structures are updated at the same time, it will lead to increased energy consumption and greatly reduced efficiency. Therefore, the present application proposes a strategy for frame-by-frame updating of the basic hierarchical acceleration structure, that is, only the target objects whose distance from the virtual camera is less than or equal to the second distance threshold are updated in each frame, such as only updating the skeletal animation of the characters within 30 meters from the camera per frame, and the number of simultaneous updates can also be limited, such as updating the skeletal animation of a maximum of 10 characters within 30 meters per frame, so as to further control energy consumption and ensure update efficiency.

[0068] Existing rendering schemes based on ray tracing require complex calculations for each pixel in real-time processing, including multiple staggered lighting reflections, refractions, and occlusion relationships. This scheme requires that multiple rays be emitted for each pixel in each frame, and each ray needs to undergo multiple intersection detections with objects in the scene, and calculate and store the relevant shading information of the hit point. It can be seen that this highly computationally intensive technology has extremely high performance requirements and requires more powerful graphics card performance to ensure smooth presentation. On the desktop side, usually only high-end graphics cards can handle this task. However, due to the performance limitations of the mobile target platform device, the existing scheme cannot be successfully implemented, so a new solution is needed to meet the needs. Based on this, the present application proposes a mobile ray tracing importance sampling algorithm, which is described in detail below, wherein the target platform can be a Qualcomm Snapdragon 8gen2 CPU platform.

[0069] In step S104, light detection is performed on the object in the ray tracing scene based on the acceleration structure, and according to the detection result, a first rendering result with shadow effect and ambient occlusion effect and noise is rendered, including:

[0070] Step 1, inputting the frame index of the object in the acceleration structure and the pixel coordinates of the object in the corresponding frame into the micro encryption model to generate a pseudo-random seed value through the micro encryption model;

[0071] Step 2, inputting the pseudo-random seed value into the linear congruential generator to generate random floating point numbers uniformly distributed in the target square through the linear congruential generator, wherein the random floating point numbers are used to represent the importance sampling values ​​of the objects;

[0072] Step 3, calculating the concentric mapping of random floating point numbers to generate sampling points within the unit circle;

[0073] Step 4, determining the direction of the light source toward the sampling point as the light source direction, and generating an offset light direction based on the sampling point and the radius of the light source;

[0074] Step 5, calculate the shadow light direction through the light source direction and the offset light direction, and calculate the ambient light occlusion light direction based on the sampling point;

[0075] Step 6: Render the shadow effect according to the direction of the shadow light, and render the ambient occlusion effect according to the direction of the ambient occlusion light to obtain a first rendering result.

[0076] In the embodiment of the present application, the micro encryption model may be a micro encryption algorithm (TEA). The target square may be a square whose horizontal and vertical coordinates are in the range of [-1, 1]. Figure 3 As shown, during light detection, first, based on the pixel coordinates and frame index, a pseudo-random seed value PseudoRandomSeed is generated using the TEA algorithm. Then, this value is input into the LCG linear congruential generator to obtain a uniformly distributed random floating point number float2RandSample in the range [-1,1]. The concentric mapping (DiskConcentric) of the random floating point number float2RandSample is calculated to generate the UV value float2DiskUV of the sampling point within the unit circle. Where UV satisfies u 2 +v 2 ≤1, the corresponding probability density function is:

[0077]

[0078] set up As the distance from the center of the unit circle where the current sampling point is located, the value range is [0,1], and the corresponding probability density function is:

[0079]

[0080] Next, the direction of the light source toward the sampling point is determined as the light source direction, and an offset light direction is generated based on the sampling point and the radius of the light source. The shadow light direction is calculated using the light source direction and the offset light direction. The offset light direction is used to detect shadows to see if there are other objects blocking the path of the light, thereby determining whether the pixel is in the shadow. Finally, the ambient occlusion light direction is calculated based on the sampling point to determine whether the lighting of the pixel should be affected by the occlusion of the surrounding geometry. The ambient occlusion light direction is a float3TBN space normal generated by the UV value of the sampling point, where T=U, B=V, The probability density function of the random variable N is:

[0081]

[0082] Similarly, the probability density function of the distribution of the zenith angle θ in the polar coordinate system of the corresponding ambient light shielding ray angle is:

[0083]

[0084] From the probability density function, we can get that the current distribution of θ is concentrated in Based on the performance test results of ray tracing of 1spp (one ray per shading point) on the mobile target platform device, the ray tracing hit point distance HitDistance can be obtained. Assume that the relationship between the hit point occlusion area and the hit point distance is f(r) =

[0085] 2π(r 2 -r 2.5 ), use the following formula to estimate the intensity of ambient occlusion in the current frame:

[0086]

[0087] Among them, HitDistance identifies the distance from the ray hitting point to the camera, MaxRayDistance tests the adjustable maximum ray distance of the current scene, and Intensity indicates the intensity of the occlusion.

[0088] Compared with the traditional ray tracing AO algorithm, in which each ray can only obtain whether it is blocked, the present application can calculate the occlusion ratio by emitting multiple tracing rays per pixel, which can slightly improve the convergence speed of the time domain denoiser when the performance budget is limited.

[0089] In an optional embodiment, since the mobile terminal cannot fully use all the features of the PC terminal, during ray tracing calculation, if the occluder is vegetation, it needs to be re-colored after the query ray hits it. At this time, the material information needs to be rebuilt to index the masked map corresponding to the material, so as to determine whether the point needs to be calculated. A large amount of secondary shading in the shader not only wastes performance, but also requires that the transparent test maps of all masked materials in the scene be passed into the rendering stage, which consumes a lot of bandwidth. From the perspective of material production, the mask result may be calculated by a series of expressions, not a simple index map. That is to say, when calculating the mobile terminal ray tracing shadow, it is impossible to index the map at all. The ray query is random and may hit any material in the scene. It is impossible to write any masked expression in the scene into a material, so the artist needs to modify the material and bake the expression into a map in advance for storage. Vegetation generally has animations, and the corresponding vertices will be moved through the World position offset. However, the current functional bottleneck of the mobile terminal cannot support the update of its BLAS for a large number of vegetation in the scene.

[0090] Therefore, in order to solve the problems of secondary shading and low BLAS update and traversal efficiency of Masked materials, the present application proposes a rendering scheme that adds mixed shadows. Specifically, the method also includes rendering the shadow effect in the following manner: determining the material type of the target pixel hit by the query ray, wherein the query ray is used to select the pixels that need to be secondary shaded during the shadow calculation of ray tracing; when the material type is masked, using a shadow map to calculate the shadow of the target pixel, or using a shadow map to clamp the starting and ending points of the shadow rendered through light detection; when the material type is non-masked, performing light detection on the target pixel, and rendering the shadow effect of the target pixel based on the detection result.

[0091] In the embodiment of the present application, HitGroupIndex can be used in ray tracing rendering to determine whether the pixel hit by RayQuery (query ray) is a Masked material. If it is a Masked material, ShadowMap will be used to calculate the shadow, or the ShadowMap sampling result can be used to clamp the ray tracing range and exclude invalid pixels, thereby improving rendering efficiency and correctness. Since Masked materials are generally vegetation, the use of hybrid shadow technology can save a lot of BLAS update calculations and ray tracing scene BVH traversal calculations, while avoiding the impact of secondary shading on the rendering results.

[0092] The following describes the execution process of the ray tracing entry function RaygenPS to further illustrate the rendering of ray tracing shadows and ambient occlusion. Figure 4As shown, at the beginning, the current screen pixel coordinates are passed, and the depth value of the point in the device coordinate system is calculated. Next, the depth value is checked to eliminate invalid pixels and sky boxes. On this basis, because the forward rendering of the target platform mobile terminal cannot obtain GBuffer information, this application restores the geometric normal through depth information, and further calculates the normal vector and spatial position in the world space corresponding to the pixel point as the initial parameters of the light vector. Among them, if mixed shadow rendering is turned on, the shadow position is calculated by sampling ShadowMap according to the current pixel coordinates, and then the PCF1x1 filter is executed, and the shadow vector ShadowFactor is stored as an alternative to the Mask material projection shadow. At the same time, the distance of the hit point in the ShadowMap neighborhood of the current pixel will be used as a reference for the range of ray tracing shadow light to increase the BVH traversal speed of the ray tracing scene and reduce the impact of the long tail effect on the execution speed of the current Pass. Finally, shadow ray tracing is performed to check whether there is a hit on the triangle surface. If yes, get the HitGroup index and determine whether the hit target is a Masked material. If it is a Masked material, use the stored ShadowFactor value as the shadow result of the current frame, otherwise use the final ray tracing result. If you need to enable ray tracing ambient occlusion, you need to check the current ray tracing AO setting Flag information, such as the current scene AO ​​intensity, maximum ray tracing distance, whether to enable AO brightness-based optimization, etc. Use the above ambient occlusion algorithm to calculate the corresponding occlusion value according to different parameters. Finally, return the shadow and ambient occlusion values ​​of the pixel based on the calculation results.

[0093] Existing ray tracing real-time denoisers, such as UE's SSD, AMD's FidelityFx-Denoiser, Nvidia's SVGF and ReSTIR, are designed for high-end desktop graphics cards and next-generation game consoles that have upgraded their video memory bandwidth and general computing power in recent years. These denoisers eliminate noise in real time during the rendering process by using a variety of complex algorithms and techniques, making the image clearer and more realistic. However, due to the limitations of mobile platform architecture and power consumption control, existing ray tracing real-time denoising solutions cannot meet the requirements of real-time operation on mobile platforms. Therefore, this application designs a denoiser specialized for mobile hardware. In the time domain filtering stage, the algorithm uses real machine test results to pre-cache half-precision depth and extended neighborhood ray tracing results into the on-chip local shared memory (LDS) for time domain calculations, balancing the effects of time domain ghosting and time domain jitter caused by downsampling. In the spatial filtering stage, the Edge-Avoiding based on wavelet transform is adopted. -Trous Wavelet (EAW) filtering method, through three filtering passes, each pass 9 samples a total of 27 samples to approximate a 9x9 bilateral filter convolution kernel. The algorithm uses temporal variance information to detect the shading points that need to be filtered, and calculates the filtering weights based on the screen space depth, temporal variance and shading point geometric normal information to ensure edge quality, thereby achieving efficient spatial filtering processing. This is explained below.

[0094] In step S106, the first rendering result is denoised using a target denoiser optimized for mobile devices, and the second rendering result obtained includes:

[0095] Step 1, performing time domain filtering on the first rendering result to calculate the time domain variance of the first rendering result;

[0096] Step 2, performing iterative spatial domain filtering on the first rendering result, and using the temporal domain variance to adjust the filtering weight of each round of iteration, wherein the filtering weight includes a shadow weight and an ambient occlusion weight;

[0097] Step 3, perform weighted summation on the shadows output in each round, and then take the average value of the sum as the shadow of the second rendering result, perform weighted summation on the ambient occlusion output in each round, and then take the average value of the sum as the ambient occlusion of the second rendering result.

[0098] Wherein, step 1 performs temporal filtering on the first rendering result to calculate the temporal variance of the first rendering result specifically includes:

[0099] Step 11, obtaining depth information of a first rendering result and a preset ray tracing result, wherein the first rendering result is a ray tracing result of a first scene in a latest frame, and the preset ray tracing result is a ray tracing result of a second scene in a historical frame, and the preset ray tracing result is pre-cached in an on-chip local shared memory;

[0100] Step 12: When the similarity of the depth information of the two reaches a target threshold, the shadow of the first rendering result is replaced with the shadow of the preset ray tracing result;

[0101] Step 13, performing pooling processing on the first rendering result to obtain a third rendering result;

[0102] Step 14, obtaining a neighborhood rendering result of each sampling point in the third rendering result;

[0103] Step 15, determining the variance of the neighborhood rendering result as the time domain variance of the sampling point;

[0104] Step 16, obtaining the time domain variance of each sampling point to obtain the time domain variance of the first rendering result.

[0105] In an embodiment of the present application, in the time domain filtering stage, according to the real machine test results, the half-precision depth and extended neighborhood ray tracing results (i.e., the ray tracing results of the second scene in the historical frame) are pre-cached into the on-chip local shared memory (LDS) for time domain calculations. First, based on the Variance-based bounds (7x7 kernel) of the current frame ray tracing neighborhood results, the historical frame sampling data is clamped to balance the effects of time domain ghosting and time domain jitter caused by downsampling. Then, the time domain variance information is calculated based on the clamped historical frames, and this information can be used to adjust the filtering weights of subsequent spatial filtering.

[0106] The temporal filtering stage includes the preparation stage, the pooling stage, and the filtering stage.

[0107] In the preparation stage, the depth information of the first rendering result and the preset ray tracing result is obtained, and when the similarity of the depth information of the two reaches the target threshold, the shadow of the first rendering result is replaced with the shadow of the preset ray tracing result. That is, if the first scene and the second scene to be rendered in the current frame and the historical frame do not change much, the shadow obtained by rendering the second scene in the historical frame can be directly used.

[0108] Optionally, before obtaining the depth information of the first rendering result and the preset ray tracing result, speed information may also be obtained, and the projection of the historical frame may be calculated according to the speed information, and the depth information may be obtained according to the projection. Figure 5 As shown, first you need to get the speed information and check whether the speed is greater than 0.0. If the speed is greater than 0.0, decode the speed information and calculate the previous projection (the projection of the historical frame); otherwise, directly calculate the previous projection. Then get the depth information of the previous scene (i.e., the second scene of the historical frame) and compare the similarity between the current depth information and the depth information of the previous scene. If the depths of the two are similar, return the historical shadow; otherwise, return the newly generated shadow.

[0109] The pooling phase specifically includes the following steps:

[0110] Step 1, dividing the first rendering result into multiple sub-areas, and initializing the shadow sum and ambient occlusion sum of each sub-area;

[0111] Step 2, traverse the sampling points in each sub-area, calculate the shadow weight, ambient occlusion weight, shadow intensity and ambient occlusion intensity of each sampling point, and add the product of the shadow intensity and the shadow weight to the shadow sum of the corresponding sub-area, and add the product of the ambient occlusion intensity and the ambient occlusion weight to the ambient occlusion sum of the corresponding sub-area;

[0112] Step 3, calculating the average value of the sum of shadows for each sub-area to obtain the shadow average value, and calculating the average value of the sum of ambient occlusion for each sub-area to obtain the ambient occlusion average value;

[0113] Step 4: Use the shadow average value as the shadow value of the corresponding sub-area, and use the ambient occlusion average value as the ambient occlusion value of the corresponding sub-area to obtain a third rendering result.

[0114] In the embodiment of the present application, in the pooling stage, the first rendering result is first divided into multiple sub-areas, and the sum of the shadows and the sum of the ambient occlusion of each sub-area are initialized to 0. At this time, the sampling points in each sub-area are traversed. Each time the traversal is performed, the UV coordinates of the sampling points are first obtained (calculated according to the current coordinates x, y and the buffer size), and then the weights of the shadows and ambient occlusions are calculated according to the distance of the x, y from the current position. Next, the shadow and ambient occlusion intensity of the current sampling point are calculated, and multiplied by the corresponding weights and then added to the sum of the shadows and the sum of the ambient occlusions of the corresponding sub-areas. When all the sampling points in a sub-area are traversed, the shadow average value and the ambient occlusion average value are calculated, and used as the unified shadow value and ambient occlusion value of the sub-area. After all sub-areas are calculated, the third rendering result is obtained.

[0115] The execution process of the program is as follows Figure 6 As shown, initialize SumAO, SumShadow and SumWeight to 0, and loop to obtain the shadow and ambient occlusion information in the local area. In each loop, first get the UV coordinates of the sampling point (calculated according to the current coordinates x, y and Buffer size), and then calculate the weights of shadow and ambient occlusion according to the distance of x, y from the current position. Next, calculate the shadow and ambient occlusion intensity of the current sampling point, multiply it with the corresponding weight and add it to SumShadow and SumAO. When all sampling in the local area is completed, calculate the average value and set the output OutShadowAndAO.x and OutShadowAndAO.y.

[0116] In the filtering stage, the neighborhood rendering result of each sampling point in the third rendering result is obtained, the variance of the neighborhood rendering result is determined as the time domain variance of the sampling point, the time domain variance of each sampling point is obtained, and the time domain variance of the first rendering result is obtained.

[0117] In addition, if Figure 7As shown, the filtering stage can also obtain the local neighborhood and reproject it, then calculate the neighborhood mean and variance, and then calculate the standard deviation and clamp range (forced range). The clamp range is used to remove invalid historical frames to reduce ghosting. Next, the weight of the historical frame is considered and the variance value is adjusted accordingly, especially when there are fewer historical frames (such as less than 16 frames). This method can increase the blurring strength of the subsequent spatial filtering on the new pixel points, thereby effectively reducing the image jitter.

[0118] In the embodiment of the present application, in the spatial filtering stage, the Edge-Avoiding -TrousWavelet (EAW) filtering method, three filtering passes, 9 samples per pass, a total of 27 samples to approximate a 9x9 bilateral filter convolution kernel. To ensure edge quality, the Edge Stopping function is used to input screen space depth, temporal variance, and shading point geometric normal information to calculate the filtering weights, and the temporal variance is used to detect whether the shading point needs to be filtered to save performance. The steps of spatial filtering mainly include calculating the weights based on the depth, normal, and shadow values ​​of the input center sample and surrounding samples, and weighted averaging to generate the output. Specifically, Figure 8 As shown in the figure, first, the center sample is loaded, and then the weight and shadow sum are initialized. Next, the current EAW Pass is iterated, and data is processed according to the sample's depth, shadow value, and optional geometric normal. After that, the skybox pixels and invalid pixels are excluded, and the weight is multiplied by the Edge-Stopping similarity of the shadow value between the center sample and the neighboring samples as an adjustment coefficient, and finally the shadow value and ambient occlusion value are obtained by weighted average.

[0119] like Fig. 9 and Fig.10 As shown, based on the mobile terminal ray tracing method provided in the present application, real-time ray tracing rendering can be achieved on the mobile terminal, and the display effects of shadows and ambient occlusion can be further optimized, solving the technical problem that the existing mobile terminal ray tracing shadow and ambient occlusion technology has low adaptability to the mobile terminal.

[0120] The present application provides a method for performing ray tracing, including: updating an acceleration structure in a ray tracing scene so that the acceleration structure matches the performance of a mobile device, wherein the acceleration structure is used to store the geometric shape and position information of all objects in the ray tracing scene; performing light detection on objects in the ray tracing scene based on the acceleration structure, and rendering a first rendering result with shadow effects and ambient occlusion effects and noise based on the detection result; and denoising the first rendering result using a target denoiser optimized for mobile devices to obtain a second rendering result. The present application is optimized for mobile devices from the update stage of the acceleration structure, the rendering stage of shadows and ambient occlusion, and the denoising stage, which can improve performance and efficiency, thereby realizing real-time ray tracing rendering on mobile devices, and solving the technical problem that the existing mobile ray tracing shadow and ambient occlusion technologies have low adaptability to mobile devices.

[0121] According to another aspect of the embodiment of the present application, Fig.11 As shown, a device for performing ray tracing is provided, comprising:

[0122] An updating module 1101 is used to update an acceleration structure in a ray tracing scene so that the acceleration structure matches the performance of the mobile device, wherein the acceleration structure is used to store geometric shapes and position information of all objects in the ray tracing scene;

[0123] A rendering module 1103 is used to perform light detection on an object in a ray tracing scene based on an acceleration structure, and render a first rendering result with shadow effects and ambient occlusion effects and noise according to the detection result;

[0124] The denoising module 1105 is configured to denoise the first rendering result using a target denoiser optimized for the mobile device to obtain a second rendering result.

[0125] It should be noted that the update module 1101 in this embodiment can be used to execute step S102 in the embodiment of the present application, the rendering module 1103 in this embodiment can be used to execute step S104 in the embodiment of the present application, and the denoising module 1105 in this embodiment can be used to execute step S106 in the embodiment of the present application.

[0126] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the above modules, as part of the device, can be implemented by software or hardware.

[0127] Optionally, the update module is specifically used to: for a first object of the vegetation class, use a first solid angle to perform solid angle culling on the top-level acceleration structure of the first object to reduce the number of instances constituting the top-level acceleration structure; for a second object of the standard model class, when the second object is in the front position of the virtual camera, use a second solid angle to perform solid angle culling on the top-level acceleration structure of the second object to reduce the number of instances constituting the top-level acceleration structure, wherein the second solid angle is smaller than the first solid angle; when the second object is in the back position of the virtual camera and the distance between the second object and the virtual camera is greater than or equal to a first distance threshold, use a third solid angle to perform solid angle culling on the top-level acceleration structure of the second object to reduce the number of instances constituting the top-level acceleration structure, wherein the third solid angle is greater than the first solid angle; for a third object of the skeletal animation class, when updating the base-level acceleration structure of the third object in each frame, select a target object whose distance from the virtual camera is less than or equal to the second distance threshold from all third objects in each frame, and update the base-level acceleration structure of the target object.

[0128] Optionally, the rendering module is specifically used to: input the frame index of the object in the acceleration structure and the pixel coordinates of the object in the corresponding frame into a micro-encryption model to generate a pseudo-random seed value through the micro-encryption model; input the pseudo-random seed value into a linear congruential generator to generate a random floating-point number uniformly distributed in a target square through the linear congruential generator, wherein the random floating-point number is used to represent the importance sampling value of the object; calculate the concentric mapping of the random floating-point number to generate a sampling point in a unit circle; determine the direction of the light source toward the sampling point as the light source direction, and generate an offset light direction based on the sampling point and the radius of the light source; calculate the shadow light direction through the light source direction and the offset light direction, and calculate the ambient occlusion light direction based on the sampling point; render the shadow effect according to the shadow light direction, and render the ambient occlusion effect according to the ambient occlusion light direction to obtain a first rendering result.

[0129] Optionally, the rendering module is also used to: determine the material type of the target pixel hit by the query ray, wherein the query ray is used to select pixels that require secondary shading during shadow calculation of ray tracing; when the material type is masked, use a shadow map to calculate the shadow of the target pixel, or use the shadow map to clamp the starting and ending points of the shadow rendered by ray detection; when the material type is non-masked, perform ray detection on the target pixel, and render the shadow effect of the target pixel based on the detection result.

[0130] Optionally, the denoising module is specifically used to: perform time domain filtering on the first rendering result to calculate the time domain variance of the first rendering result; perform iterative spatial domain filtering on the first rendering result, and use the time domain variance to adjust the filtering weights of each round of iteration, wherein the filtering weights include shadow weights and ambient occlusion weights; perform weighted summation on the shadows output in each round, and then take the average of the sum as the shadow of the second rendering result, perform weighted summation on the ambient occlusion output in each round, and then take the average of the sum as the ambient occlusion of the second rendering result.

[0131] Optionally, the denoising module is also used to: obtain depth information of a first rendering result and a preset ray tracing result, wherein the first rendering result is the ray tracing result of the first scene in the latest frame, the preset ray tracing result is the ray tracing result of the second scene in the historical frame, and the preset ray tracing result is pre-cached in the on-chip local shared memory; when the similarity of the depth information of the two reaches a target threshold, replace the shadow of the first rendering result with the shadow of the preset ray tracing result; perform pooling on the first rendering result to obtain a third rendering result; obtain a neighborhood rendering result of each sampling point in the third rendering result; determine the variance of the neighborhood rendering result as the time domain variance of the sampling point; obtain the time domain variance of each sampling point to obtain the time domain variance of the first rendering result.

[0132] Optionally, the denoising module is also used to: divide the first rendering result into multiple sub-areas, and initialize the shadow sum and ambient occlusion sum of each sub-area; traverse the sampling points in each sub-area, calculate the shadow weight, ambient occlusion weight, shadow intensity and ambient occlusion intensity of each sampling point, and add the product of the shadow intensity and the shadow weight to the shadow sum of the corresponding sub-area, and add the product of the ambient occlusion intensity and the ambient occlusion weight to the ambient occlusion sum of the corresponding sub-area; calculate the average value of the shadow sum for each sub-area to obtain the shadow average value, calculate the average value of the ambient occlusion sum for each sub-area to obtain the ambient occlusion average value; use the shadow average value as the shadow value of the corresponding sub-area, and use the ambient occlusion average value as the ambient occlusion value of the corresponding sub-area to obtain a third rendering result.

[0133] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above embodiments.

[0134] Optionally, in an embodiment of the present application, the computer-readable medium is configured to store program codes for the processor to perform the following steps:

[0135] Update the acceleration structure in the ray tracing scene to match the performance of the mobile device. The acceleration structure is used to store the geometry and position information of all objects in the ray tracing scene.

[0136] Performing light detection on the object in the ray tracing scene based on the acceleration structure, and rendering a first rendering result with shadow effect and ambient occlusion effect and noise according to the detection result;

[0137] The first rendering result is denoised using a target denoiser optimized for mobile devices to obtain a second rendering result.

[0138] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0139] When the embodiments of the present application are specifically implemented, reference may be made to the above-mentioned embodiments, which have corresponding technical effects.

[0140] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0141] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0142] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0143] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0144] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0147] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0148] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for performing ray tracing, characterized in that: include: Updating an acceleration structure in a ray tracing scene so that the acceleration structure matches the performance of the mobile device, wherein the acceleration structure is used to store geometric shapes and position information of all objects in the ray tracing scene; Performing light detection on the object in the ray tracing scene based on the acceleration structure, and rendering a first rendering result with shadow effect and ambient occlusion effect and noise according to the detection result; The first rendering result is denoised using a target denoiser optimized for mobile devices to obtain a second rendering result.

2. The method according to claim 1, characterized in that The updating of the acceleration structure in the ray tracing scene includes: For a first object of the vegetation class, using a first solid angle to perform solid angle culling on a top-level acceleration structure of the first object to reduce the number of instances constituting the top-level acceleration structure; For a second object of the standard model class, when the second object is in front of the virtual camera, using a second solid angle to perform solid angle culling on the top-level acceleration structure of the second object to reduce the number of instances constituting the top-level acceleration structure, wherein the second solid angle is smaller than the first solid angle; When the second object is at a position behind the virtual camera and the distance between the second object and the virtual camera is greater than or equal to a first distance threshold, use a third solid angle to perform solid angle culling on the top-level acceleration structure of the second object to reduce the number of instances constituting the top-level acceleration structure, wherein the third solid angle is greater than the first solid angle; For the third object of the skeletal animation class, when the basic hierarchical acceleration structure of the third object in each frame is updated, a target object whose distance from the virtual camera is less than or equal to a second distance threshold is selected from all the third objects in each frame, and the basic hierarchical acceleration structure of the target object is updated.

3. The method according to claim 1, characterized in that The performing of light detection on the object in the ray tracing scene based on the acceleration structure, and rendering a first rendering result with shadow effect and ambient occlusion effect and noise according to the detection result includes: Inputting the frame index of the object in the acceleration structure and the pixel coordinates of the object in the corresponding frame into the micro encryption model to generate a pseudo-random seed value through the micro encryption model; Inputting the pseudo-random seed value into a linear congruential generator to generate random floating point numbers uniformly distributed in a target square through the linear congruential generator, wherein the random floating point numbers are used to represent the importance sampling values ​​of the object; Calculating a concentric map of the random floating point numbers to generate sampling points within a unit circle; Determine the direction of the light source toward the sampling point as the light source direction, and generate an offset light direction based on the sampling point and the radius of the light source; Calculating the shadow light direction by using the light source direction and the offset light direction, and calculating the ambient occlusion light direction based on the sampling point; The shadow effect is rendered according to the direction of the shadow light, and the ambient occlusion effect is rendered according to the direction of the ambient occlusion light to obtain the first rendering result.

4. The method according to claim 3, characterized in that The method also includes rendering the shadow effect as follows: Determine the material type of the target pixel hit by the query ray, wherein the query ray is used to select pixels that need to be re-colored during shadow calculation of ray tracing; In the case where the material type is a mask, a shadow map is used to calculate the shadow of the target pixel, or the shadow map is used to clamp the starting point and the ending point of the shadow rendered by light detection; When the material type is non-masked, light detection is performed on the target pixel, and a shadow effect of the target pixel is rendered according to the detection result.

5. The method according to claim 1, characterized in that The denoising of the first rendering result by using a target denoiser optimized for the mobile terminal device to obtain a second rendering result includes: Performing time-domain filtering on the first rendering result to calculate the time-domain variance of the first rendering result; Iteratively perform spatial domain filtering on the first rendering result, and use the temporal variance to adjust the filtering weight of each round of iteration, wherein the filtering weight includes a shadow weight and an ambient occlusion weight; A weighted sum is performed on the shadows output in each round, and the average value of the sum is taken as the shadow of the second rendering result; a weighted sum is performed on the ambient occlusion output in each round, and the average value of the sum is taken as the ambient occlusion of the second rendering result.

6. The method according to claim 5, characterized in that The performing time-domain filtering on the first rendering result to calculate the time-domain variance of the first rendering result includes: Obtaining depth information of the first rendering result and a preset ray tracing result, wherein the first rendering result is a ray tracing result of a first scene in a latest frame, the preset ray tracing result is a ray tracing result of a second scene in a historical frame, and the preset ray tracing result is pre-cached in an on-chip local shared memory; When the similarity of the depth information of the two reaches a target threshold, replacing the shadow of the first rendering result with the shadow of the preset ray tracing result; Performing pooling processing on the first rendering result to obtain a third rendering result; Obtaining a neighborhood rendering result of each sampling point in the third rendering result; Determine the variance of the neighborhood rendering result as the time domain variance of the sampling point; The time domain variance of each sampling point is obtained to obtain the time domain variance of the first rendering result.

7. The method according to claim 6, characterized in that The performing pooling processing on the first rendering result to obtain a third rendering result comprises: Divide the first rendering result into a plurality of sub-areas, and initialize the shadow sum and ambient occlusion sum of each sub-area; Traversing the sampling points in each sub-area, calculating the shadow weight, ambient occlusion weight, shadow intensity and ambient occlusion intensity of each sampling point, and adding the product of the shadow intensity and the shadow weight to the shadow sum of the corresponding sub-area, and adding the product of the ambient occlusion intensity and the ambient occlusion weight to the ambient occlusion sum of the corresponding sub-area; Calculating the average value of the sum of the shadows for each sub-area to obtain a shadow average value, and calculating the average value of the sum of the ambient occlusion for each sub-area to obtain an ambient occlusion average value; The shadow average value is used as the shadow value of the corresponding sub-area, and the ambient occlusion average value is used as the ambient occlusion value of the corresponding sub-area to obtain the third rendering result.

8. A device for performing ray tracing, characterized in that: include: An updating module, used for updating an acceleration structure in a ray tracing scene so that the acceleration structure matches the performance of a mobile device, wherein the acceleration structure is used for storing geometric shapes and position information of all objects in the ray tracing scene; A rendering module, configured to perform light detection on an object in the ray tracing scene based on the acceleration structure, and render a first rendering result having a shadow effect and an ambient light occlusion effect and having noise according to the detection result; The denoising module is used to denoise the first rendering result using a target denoiser optimized for the mobile device to obtain a second rendering result.

9. A computer-readable medium having a non-volatile program code executable by a processor, characterized in that: The program code enables the processor to execute the method according to any one of claims 1 to 7.

Images