A game image generation method and system based on a 3D rendering engine

Abstract

The application relates to a game image generation method and system based on a 3D rendering engine, and belongs to the field of computer graphics processing. The method comprises the following steps: obtaining the material identity code, the depth value and the normal vector of each pixel through deferred rendering, querying a material sensitivity mapping table and combining the degree of dynamic light source spread to generate a pixel-level light and shadow sensitivity level; based on the level, combining the motion intensity and the depth gradient to perform multi-dimensional non-uniform sampling judgment to divide the pixels into key areas and non-key areas; performing high-frequency ray tracing sampling on the key areas, and through motion vector reprojection, reusing historical data on the non-key areas, and when reprojection is invalid, performing time domain interpolation compensation; and fusing the two types of radiance data to generate a game view frame. The application realizes accurate allocation of rendering computing power from the macro object level to the microscopic pixel level, and balances the rendering quality and real-time performance under limited resources.

Description

Technical Field

[0001] This application belongs to the field of computer graphics processing, specifically relating to a method and system for generating game images based on a 3D rendering engine. Background Technology

[0002] With the rapid development of 3D rendering engine technology, real-time ray tracing has become a core driver for enhancing the realism and immersion of video games. Modern game graphics generation processes simulate the propagation paths of physical light rays, achieving highly realistic global illumination, soft shadows, and complex material reflection effects. This process is extremely resource-intensive, typically requiring the emission of massive amounts of sampled rays to perform real-time intersection calculations with scene geometry to ensure smooth transitions between light and shadow and accurate physical representation.

[0003] Among these, sampling control strategies based on real-time ray tracing are a key element in balancing rendering efficiency and visual quality. This technological direction aims to maximize image information density within a limited rendering time through refined photon distribution management. As game scene complexity continues to increase, developers are increasingly focused on how to dynamically adjust the sampling frequency based on material properties, lighting environment, and motion state, thereby ensuring high frame rate operation while accurately reproducing the visual details of highly sensitive surfaces such as metal and water surfaces under complex interactive conditions.

[0004] However, traditional rendering sampling models typically apply a uniform sampling frequency to all pixels of an image, resulting in a large amount of unnecessary computation in static areas or rough-textured areas insensitive to lighting and shadows, leading to significant computational redundancy. Furthermore, because existing methods lack in-depth mining of spatiotemporal correlation information between multiple frames, frame rates can only be maintained by reducing the overall sampling rate when computational resources are limited. This directly results in noticeable image noise and aliasing in highly reflective areas. In addition, a single computational logic cannot differentiate between different semantic blocks, making it difficult to accurately allocate computational resources in complex scenes, resulting in an irreconcilable contradiction between high-fidelity image output and real-time performance maintenance. Summary of the Invention

[0005] The purpose of this invention is to provide a game image generation method and system based on a 3D rendering engine, which can effectively solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A game image generation method based on a 3D rendering engine, implemented through an environment semantic recognition module, a sampling and judgment logic unit, a ray casting execution engine, and an image fusion post-processing unit, includes:

[0008] The geometric description of each pixel in the screen space of the current frame is obtained by deferred rendering. The geometric description includes material identity code, depth value and world space normal vector.

[0009] Based on the material identity code, a preset material sensitivity mapping table is queried for each pixel to obtain the basic sensitivity value;

[0010] Based on the world space coordinates corresponding to each pixel and the attributes of active light sources in the scene, the basic sensitivity values ​​of pixels falling within the influence volume of the light source are offset and corrected to obtain the light and shadow sensitivity level of the pixel. The light and shadow sensitivity level of each pixel is then stored in the L1 level real-time frame buffer of the spatiotemporal buffer pool.

[0011] The sampling and judgment logic unit reads the light and shadow sensitivity level of each pixel from the L1 level real-time frame buffer, and combines the pixel's motion intensity across multiple consecutive frames and the depth gradient in the depth map to perform multi-dimensional non-consistent sampling and judgment on all pixels in the frame, dividing all pixels in the frame into critical and non-critical regions.

[0012] The ray casting execution engine uses an importance sampling strategy that performs non-linear adjustments based on the pixel's light and shadow sensitivity level for pixels in key areas to guide the sampling direction of ray tracing and generate the first radiance data;

[0013] For pixels in non-critical areas, motion vectors are extracted from the spatiotemporal buffer pool, second radiance data is obtained from the radiance data of the previous frame through reprojection, and temporal interpolation compensation is performed when reprojection is invalid.

[0014] The image fusion post-processing unit performs image fusion processing on the first radiance data and the second radiance data to generate a complete game view frame.

[0015] Preferably, the step of constructing the spatiotemporal cache pool includes:

[0016] The video memory is configured with a storage structure that includes an L1-level real-time frame buffer and an L2-level history buffer. The L1-level real-time frame buffer is used to store the light and shadow sensitivity level, first radiance data or second radiance data, and motion vector of each pixel in the current frame. The L2-level history buffer adopts a first-in-first-out circular queue structure to manage the historical radiance data, historical material identification code, and historical depth value of multiple consecutive frames.

[0017] When writing luminance data and normal data to the spatiotemporal buffer pool, differential pulse coding compression is performed using the spatial correlation of adjacent pixels within the frame to generate a compressed data stream.

[0018] Based on the head pointer of the circular queue structure, the compressed data stream of the new frame is overwritten into the address space corresponding to the oldest frame with the longest retention time.

[0019] Preferably, the multi-dimensional non-consistency sampling determination includes:

[0020] The first dimension of the determination is to compare the light and shadow sensitivity level of a pixel with a preset first sensitivity threshold. If the light and shadow sensitivity level of a pixel is greater than or equal to the first sensitivity threshold, it is marked as a critical area.

[0021] The second dimension determination involves calculating the average value of the motion vector magnitude of a pixel over multiple consecutive frames. If the average value is greater than or equal to a preset motion intensity threshold, it is marked as a critical region.

[0022] The third dimension determination involves performing edge detection on the depth map composed of the depth values, calculating the depth gradient magnitude of the pixels, and marking the region as a critical area if the depth gradient magnitude is greater than or equal to a preset depth gradient threshold.

[0023] Pixels that are not marked as critical regions in any dimension are classified as non-critical regions.

[0024] Preferably, it also includes a closed-loop computing power adaptive adjustment step based on the rendering time of a single frame:

[0025] Obtain the single-frame rendering time of the graphics processor;

[0026] When the rendering time of a single frame exceeds the preset real-time threshold, a step-by-step response strategy is initiated; the step-by-step response strategy includes: a first-level response, which adjusts the low-frequency light sampling performed when the reprojection of non-critical areas is invalid to frame skipping.

[0027] If the rendering time of the next frame still does not fall below the real-time threshold, the second-level response is initiated, and the first sensitivity threshold in the non-consistent sampling determination is increased to reduce the total number of pixels marked as critical areas.

[0028] Once the rendering time of subsequent consecutive frames stabilizes below the real-time threshold, the response strategy is gradually revoked in the order of first restoring the first sensitivity threshold and then restoring the low-frequency light sampling frequency.

[0029] Preferably, the image fusion processing step includes:

[0030] A multiplexing counter is maintained for each pixel in the spatiotemporal buffer pool, and the multiplexing counter is incremented when the pixel is successfully reprojected and reused in the current frame;

[0031] A historical contribution weighting factor is introduced, which is an exponential decay function of the reuse counter;

[0032] When performing weighted fusion on pixels in non-critical areas, the contribution of historical radiance data that has been reused for multiple consecutive frames is reduced according to the historical contribution weighting factor.

[0033] When the value of the multiplexing counter of any pixel exceeds the preset multiplexing count threshold, high-frequency ray tracing sampling is forced to be performed on any pixel to refresh the radiance data, and the multiplexing counter of any pixel is cleared to zero.

[0034] Preferred importance sampling strategies based on non-linear adjustment of pixel light and shadow sensitivity levels include:

[0035] The light and shadow sensitivity level of the pixel is obtained, and the light and shadow sensitivity level is converted into a bias coefficient based on a preset monotonically increasing mapping function;

[0036] Using the aforementioned bias coefficient, a power transformation is performed on the joint probability density function generated by the light source brightness distribution and the material bidirectional reflection distribution function to adjust its distribution concentration.

[0037] Importance sampling is performed based on the transformed joint probability density function to determine the sampling direction for ray tracing.

[0038] Preferably, a reprojection validity check is performed before reusing the radiance data from the previous frame; the conditions for the reprojection validity check include:

[0039] Compare the material identity code of the current frame pixel with the material identity code of the corresponding position in the previous frame obtained through reprojection coordinates to see if they are consistent;

[0040] Compare the absolute value of the difference between the depth value of the current frame pixel and the depth value of the corresponding position in the previous frame to see if it is less than or equal to the preset depth tolerance.

[0041] If both conditions are met, the reprojection is deemed valid; otherwise, it is deemed invalid.

[0042] Preferably, the step of performing the time-domain interpolation compensation includes:

[0043] For non-critical region pixels that are invalid for reprojection, perform low-frequency light sampling with a sampling number lower than that of the critical region to obtain the center sampling value;

[0044] Read the radiance values ​​of other pixels within a 3×3 spatial neighborhood centered on the current pixel that have already had their radiance data obtained through reprojection validity checks or low-frequency light sampling;

[0045] The central sampled value is weighted and averaged with the radiance values ​​of other pixels in the neighborhood. The weights are allocated according to the source type of the radiance data of other pixels in the neighborhood to fill the hole areas caused by invalid reprojection.

[0046] Preferably, the process of offsetting and correcting the basic sensitivity value of pixels falling within the influence volume of the light source based on the world space coordinates corresponding to each pixel and the attributes of active light sources in the scene to obtain the light and shadow sensitivity level of the pixel further includes:

[0047] For a surface light source, calculate the dot product between the world space normal vector of the pixel and the unit direction vector pointing from the pixel to the geometric center of the surface light source;

[0048] When the dot product falls within a preset dot product threshold range, the pixel is determined to be in the irradiance gradient sensitive region caused by the surface light source, and the basic sensitivity value of the pixel is adjusted by a secondary offset to make the irradiance gradient sensitive region obtain a higher light and shadow sensitivity level than the smooth region directly illuminated by the light source.

[0049] Secondly, a game image generation system based on a 3D rendering engine, used to perform the above method, includes:

[0050] An environmental semantic recognition module is used to generate the light and shadow sensitivity level of each pixel in the current frame. The light and shadow sensitivity level is calculated based on the material identity code, depth value and world space normal vector obtained from the geometric attribute buffer of deferred rendering, combined with the material sensitivity mapping table, dynamic light source sweep rate and surface light source irradiance change information.

[0051] The spatiotemporal cache management module is used to build a spatiotemporal cache pool in the video memory, which includes an L1-level real-time frame cache and an L2-level historical record cache, to store and manage the light and shadow sensitivity level, as well as the motion vector, radiance data, material identity code and depth value of each pixel across multiple frames.

[0052] The sampling and determination logic unit is used to divide the entire frame of pixels into key regions and non-key regions based on the light and shadow sensitivity level and the motion vector and depth gradient obtained from the spatiotemporal buffer management module.

[0053] A ray casting execution engine is used to perform high-frequency ray tracing sampling on the critical region and low-frequency ray sampling on pixels in the non-critical region that are invalid for reprojection.

[0054] The image fusion post-processing unit is used to fuse high-frequency ray tracing sampling and spatiotemporally multiplexed radiance data, and apply a historical contribution weighting factor based on the multiplexing counter to generate the final game view frame.

[0055] In summary, this application includes at least one of the following beneficial technical effects:

[0056] 1. This application constructs a pixel-level environmental semantic recognition mechanism. Based on material sensitivity, dynamic light source sweep rate, and area light source irradiance change information, it generates a fine-grained light and shadow sensitivity level for each pixel in the screen space. Based on this, it performs non-consistent sampling judgment that integrates three dimensions: material sensitivity, motion intensity, and depth gradient. This divides the entire frame of pixels into critical and non-critical regions, achieving a precise leap in rendering computing power allocation from the macro-object level to the micro-pixel level. Furthermore, by non-linearly coupling the light and shadow sensitivity level with the probability density function of ray tracing importance sampling, the sensitivity level not only regulates the number of samples but also deeply participates in the regulation of the sampling direction distribution. Under the same sampling budget, this increases the proportion of effective light paths, achieving a more refined pixel-level sampling strategy control than simply adjusting the number of samples or simply classifying materials.

[0057] 2. This application constructs a hierarchical spatiotemporal cache pool architecture in video memory, including an L1-level real-time frame cache and an L2-level historical record cache, and performs differential pulse coding compression based on the spatial correlation of adjacent pixels within the frame on the written data. This provides efficient underlying data support for motion vector reprojection and spatiotemporal reuse in non-critical areas. At the same time, by introducing a reuse counter and a historical contribution weighting factor based on an exponential decay function, and by forcibly triggering high-frequency ray tracing sampling to refresh radiance data when a pixel is reused continuously for more than a preset number of times, it effectively suppresses image motion artifacts caused by the accumulation of long-term reprojection errors, and achieves synergistic optimization of rendering efficiency and spatiotemporal consistency of the image.

[0058] 3. This application establishes a global computing power monitoring mechanism that operates in parallel with the rendering pipeline to obtain the rendering time of a single frame in real time. When the rendering time exceeds a preset real-time threshold, a closed-loop computing power adaptive adjustment mechanism with a two-level progressive response strategy is activated. Priority is given to performing frame skipping sparsity processing on the temporal interpolation compensation frequency of non-critical areas. When computing power pressure continues, the judgment threshold of critical areas is dynamically increased to reduce the total number of high-frequency sampling pixels. This provides a rigid guarantee for the frame rate of real-time interaction under the premise of controllable image quality loss. The degradation strategy is automatically revoked after computing power recovers, enhancing the robustness and adaptability of the game image generation system under the environment of hardware resource fluctuations. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the overall architecture of the game image generation method based on a 3D rendering engine according to the present invention;

[0060] Figure 2 This is a core principle framework diagram of the present invention based on the non-consistent sampling determination of material sensitivity and motion vectors;

[0061] Figure 3 This is a logical flowchart of the environmental semantic recognition and multi-level spatiotemporal cache pool construction of the present invention;

[0062] Figure 4 This is a multi-level interaction diagram of the high-frequency light projection in the key area and the reprojection compensation in the non-key area of ​​the present invention. Detailed Implementation

[0063] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description of specific embodiments based on the present invention is provided in conjunction with the accompanying drawings and preferred embodiments.

[0064] refer to Figures 1 to 4 As shown, this embodiment provides a game image generation method based on a 3D rendering engine, including the following steps:

[0065] Step S1, environmental semantic recognition, is implemented as follows:

[0066] Step S101, capture of geometric attributes based on deferred rendering.

[0067] After the 3D rendering engine enters the geometry processing stage, the GPU's vertex shader and pixel shader transform and project the 3D geometry in the scene onto the 2D screen space. During the projection stage, the final color and lighting calculations are not performed. Instead, the geometric attribute information corresponding to each pixel is written to the geometry attribute buffer in real time using multi-rendering target technology.

[0068] The geometry attribute buffer is configured with at least three independent data channels: the first channel stores the depth value of the current frame, which is the depth of the pixel in normalized device coordinates at the corresponding point on the geometry surface in screen space; the second channel stores the world space normal vector, which is taken from the normal direction of the geometry surface at the corresponding point and is used for subsequent lighting direction sensitivity calculations; the third channel stores the material identification code, which is a unique identifier of the material resource bound when rendering the geometry. The data from these three channels together constitute the underlying geometric description of each pixel in the current frame.

[0069] Step S102: Static lookup and assignment of material sensitivity values.

[0070] After acquiring the geometric attribute buffer, the environmental semantic recognition module begins to analyze each pixel point by point. The material identity code extracted from the material channel is directly used as an address index to query a material sensitivity mapping table stored in the graphics processor's constant storage area.

[0071] The material sensitivity mapping table predefines 256 different material categories, where each material identification code uniquely corresponds to an 8-bit unsigned integer light and shadow sensitivity value, all of which fall within the range of 0 to 255.

[0072] Specifically, materials exhibiting specular reflection or complex refraction characteristics, such as metal, glass, and water surfaces, are classified as high-sensitivity materials with light and shadow sensitivity values ​​set between 200 and 255. Rough materials, such as stone, fabric, and wood, which are primarily diffuse reflective, are classified as low-sensitivity materials with light and shadow sensitivity values ​​set between 0 and 100. Values ​​between 100 and 200 are assigned to materials with mixed reflective properties, such as smooth plastic or wet surfaces. The specific values ​​are specified by the artists during the material creation stage within the range of 0 to 255 based on the material's response to changes in lighting, and are pre-defined in the mapping table.

[0073] Step S103: Initial correction of sensitivity under the influence of dynamic light source.

[0074] After generating the basic sensitivity, the environmental semantic recognition module will dynamically adjust this sensitivity value for the first time based on the lighting complexity of the current scene. The environmental semantic recognition module uses the depth value of the current pixel, combined with the camera's projection matrix and view matrix, to calculate the world space coordinates corresponding to the pixel from the pixel coordinates in screen space.

[0075] Subsequently, the environmental semantic recognition module traverses all active surface light sources, volumetric light sources, and dynamic point light sources in the scene. For each light source, the environmental semantic recognition module reads its spatial position, effective illumination radius or bounding box size, and light source attenuation parameters one by one.

[0076] The influence volume of a light source is defined by its read spatial location and the corresponding illumination radius or bounding box size. The criterion is whether the world coordinates of a pixel are located within the influence volume. For example, for a point light source, the influence volume is a sphere centered on the light source location and with the illumination radius as its radius; for a surface light source, the influence volume is a box-shaped region defined by its bounding box size.

[0077] If the world coordinates of a pixel fall within the illumination volume of a complex light source, the environmental semantic recognition module will increase the sensitivity value of this pixel by a preset offset, which is 40 by default.

[0078] Through the aforementioned dynamic detection and offset correction, areas with rich lighting variations and complex light and shadow structures will immediately obtain a higher sensitivity response level, guiding subsequent sampling logic to allocate more computing resources to these visually prominent locations.

[0079] Step S104: Identification and secondary correction of the sensitive area of ​​the surface light source irradiance gradient.

[0080] For surface light sources in the scene, the environmental semantic recognition module will further identify areas where the light and shadow changes drastically due to the high probability of geometric occlusion. The purpose is to increase the sensitivity of such visually important areas so as to better preserve the continuous grayscale levels of soft shadows in subsequent processing and suppress possible strip-like color level breaks.

[0081] As a low-computational-cost heuristic calibration method, this step utilizes the geometric relationship between surface normals and illumination directions to quickly approximate the location of these regions. The environmental semantic recognition module first calculates the dot product between the pixel geometric normal vector and a unit direction vector. The unit direction vector is obtained by subtracting the pixel's world space coordinates from the geometric center world coordinates of the surface light source and then normalizing the result. The environmental semantic recognition module introduces a preset dot product threshold range to define the regions with drastic changes in light and shadow. The typical setting for the dot product threshold range is 0.2 to 0.8.

[0082] If the calculated dot product value falls within the range of 0.2 to 0.8, it indicates that from the perspective of that pixel, the angle between it and the surface light source is relatively tilted. This area is often where the geometric coverage relationship changes most frequently and the illuminance gradient is largest when an occluder casts a soft shadow under a surface light source, i.e., the high probability area of ​​penumbra. For pixels falling within this range, the environmental semantic recognition module adds a correction amount to the sensitivity value, with the correction amount fluctuating within the range of 32 to 64 steps. The specific value of the correction amount is determined by the linear position interpolation from the dot product value to the boundary of the range, i.e., 0.2 or 0.8. The closer to the 0.2 side (i.e., closer to the geometric edge of the light source), the higher the added value, so as to fully preserve the potential soft shadow details.

[0083] If the dot product value is higher than 0.8, it indicates that the pixel is basically in the direct illumination area of ​​the area light source, and the lighting is relatively smooth; if the dot product value is lower than 0.2, it indicates that the pixel is in the extreme grazing angle of the light. For these two cases, no correction is applied.

[0084] At this point, the environmental semantic recognition step defines the light and shadow sensitivity level of each pixel in the current frame. The light and shadow sensitivity level value takes into account multiple factors such as the material's own properties, the degree of influence of complex light sources, and the degree of drastic changes in illumination under surface light sources. As a dense and physically reliable priority basis, it is directly stored in the spatiotemporal cache pool and associated with the corresponding pixel record.

[0085] Step S2, construct the spatiotemporal cache pool, the specific implementation process is as follows:

[0086] Step S201: Cache space initialization and hierarchical structure configuration.

[0087] A contiguous, protected buffer is allocated within the video memory space as a spatiotemporal cache pool. The total capacity of the buffer is dynamically calculated and reserved based on the current rendering resolution. The capacity is calculated by multiplying the number of horizontal pixels of the rendering resolution by the number of vertical pixels, and then multiplying by the number of bytes occupied by a single pixel record. The number of bytes occupied by a single pixel record is the sum of the storage bit widths of each field in a single record in the L1-level real-time frame buffer and the L2-level history buffer, divided by 8.

[0088] When the rendering resolution is 3840×2160 and each pixel occupies 48 bytes, the required capacity of the L1 level real-time frame buffer is approximately 3840×2160×48 bytes, and the L2 level history buffer is multiplied by the number of history frames. The video memory allocation is completed by the GPU driver in one go during the scene loading stage, and the allocated buffer address space remains fixed throughout the entire rendering cycle.

[0089] The spatiotemporal cache pool adopts a multi-level cache structure, which is divided into at least two levels: L1 level real-time frame cache and L2 level history cache. The physical address spaces of the two caches are kept contiguous in the video memory, and logically they are addressed independently through their respective base addresses and offsets.

[0090] The L1 level real-time frame buffer stores intermediate data during the current frame processing, including the light and shadow sensitivity level generated in step S1, the initial radiance estimate obtained by differential sampling in step S4, and the pixel motion vector in screen space. The data in the L1 level real-time frame buffer is only valid in the current frame and is overwritten by the corresponding data of the new frame at the beginning of the next frame.

[0091] The L2-level history cache manages recently generated multi-frame historical image data, with no fewer than 8 historical frames, and adopts a first-in-first-out replacement strategy. The L2-level history cache is implemented through a circular queue structure, with the head of the queue always pointing to the oldest frame with the longest retention time, and the tail of the queue pointing to the most recently written frame. When the L2-level cache space is full and a new frame of data is waiting to be written, the memory address space of the old frame pointed to by the head of the queue is marked as writable. After the new frame of data is written to the corresponding address space, the head pointer of the queue moves one position to the right.

[0092] Each pixel record in the L2-level historical record cache synchronously stores historical radiance data, historical material identification code, historical depth value, and world space coordinates calculated in step S103.

[0093] Step S202: Fill the cached record with pixel-by-pixel data.

[0094] Each record in the spatiotemporal cache pool corresponds to a pixel position in the screen space, and synchronously stores brightness information, color information, motion vectors, and material identity codes; records in the L1 level real-time frame cache additionally store the light and shadow sensitivity level generated in step S1, while records in the L2 level historical record cache do not contain this field.

[0095] The luminance information uses a 16-bit wide floating-point format, i.e., a half-precision floating-point number, conforming to the binary16 format of the IEEE 754-2008 standard, including 1 sign bit, 5 exponent bits, and 10 mantissa bits; the luminance range that can be represented by the above floating-point format is approximately 6.10 × 10⁻⁶. -5 Up to 6.55×10 4 It can preserve complete lighting details from extremely dark to extremely bright in high dynamic range scenes.

[0096] Color information is stored in RGBA multi-channel format, with 8 bits per channel and a total of 32 bits. The four channels are red component R, green component G, blue component B, and transparency component A, respectively. The value of each component is an integer ranging from 0 to 255, which is linearly mapped to a floating-point number range of 0.0 to 1.0. The color data originates from the radiance value generated by differential sampling in step S4, which is then converted into an 8-bit unsigned integer after hue mapping.

[0097] Motion vectors are obtained by calculating the screen space projection offset of the current frame's pixels between the previous and next frames. The specific process is as follows:

[0098] Using the world space coordinates of the current frame pixels calculated in step S103, these world space coordinates are projected using the view projection matrix of the current frame and the view projection matrix of the previous frame respectively to obtain the screen space coordinates of the current frame. and the screen space coordinates of the previous frame The view projection matrix of the previous frame is read from the camera parameters associated with the L2-level history cache.

[0099] The difference between two screen space coordinates is used as the two components of the motion vector, that is:

[0100]

[0101]

[0102] The motion vectors described above represent the displacement of a pixel in the current frame relative to the corresponding pixel in the previous frame within screen space. In the screen space coordinate system, positive values ​​are... This indicates that the corresponding position in the previous frame is to the left of the current frame position, and is a positive value. This indicates that the corresponding position in the previous frame is above the position in the current frame.

[0103] The material identity code is directly reused in step S102, where an 8-bit unsigned integer is extracted from the material channel, maintaining consistency with the index relationship of the material sensitivity mapping table.

[0104] Step S203: Compress the written data based on spatiotemporal correlation.

[0105] To reduce the bandwidth pressure when writing data to the video memory via the high-speed bus, lossless compression is performed on the luminance and normal channels respectively before the data is written to the space buffer pool.

[0106] The compression algorithm utilizes the spatial correlation between adjacent pixels within a frame for differential pulse coding. For each row of pixels in the current frame, starting from the beginning of the row, the luminance value or normal component value of the pixel to be written is subtracted from the corresponding value after decompression and reconstruction of the left-hand adjacent pixel in the same row, and the resulting difference is written as the encoded value into the compressed data stream; the luminance and normal data of the first pixel in each row are not differentially processed, and are directly written as the original value as the starting reference value for decoding the corresponding row.

[0107] During decompression, the decoding end reads the starting reference value and then adds the differential encoding value of each subsequent pixel to the value of the reconstructed pixel on the left to recover the complete luminance map and normal map. The compressed data blocks are organized by row, with each row's encoding value stored continuously with the starting reference value, and written to the video memory by the video memory controller in burst transfer mode.

[0108] After a new frame's compressed data stream is written to the video memory via the high-speed bus, the video memory controller automatically detects the oldest frame with the longest remaining time in the L2 history cache and marks the memory address space occupied by the corresponding old frame as writable. The address space marking method is to record the start position and length of the corresponding address range in the head pointer of the circular queue maintained by the cache management module; when a new frame is written next time, the video memory controller will preferentially allocate physical pages from the address space that has been marked as writable.

[0109] Thus, step S2 constructs a structured spatiotemporal data warehouse within the video memory. The L1 cache carries the real-time intermediate results of the current frame, and the L2 cache retains the history of multiple frames in the form of a sliding window, in conjunction with differential pulse coding compression and a first-in-first-out (FIFO) replacement strategy.

[0110] Step S3, Non-consistent sampling determination, the specific implementation logic is as follows:

[0111] Step S301, make a determination based on the first dimension of material sensitivity.

[0112] The non-consistent sampling determination module performs a three-dimensional composite dimension attribute evaluation on each pixel in the screen space. The non-consistent sampling determination module first reads the light and shadow sensitivity level value of the target pixel from the L1 level real-time frame buffer of the spatiotemporal buffer pool. The light and shadow sensitivity level value is output by step S104 and written to the buffer. The non-consistent sampling determination module compares the read light and shadow sensitivity level value with the preset first sensitivity threshold. The default value of the first sensitivity threshold is 128. The first sensitivity threshold corresponds to the boundary between the high sensitivity range and the low sensitivity range in the material sensitivity mapping table.

[0113] If the light and shadow sensitivity level value of the target pixel is greater than or equal to 128, it indicates that the material corresponding to the target pixel is a highly sensitive material, or that the target pixel is in a light-sensitive area such as a complex light source range or an irradiance gradient sensitive area, and the target pixel is initially marked as a potential key area; if the light and shadow sensitivity level value is less than 128, the target pixel will not be judged by the first dimension for the time being, and will be left to be evaluated by the second and third dimensions.

[0114] Step S302: Determine the second dimension based on the intensity of the exercise.

[0115] Based on the first-dimensional judgment, a motion intensity factor is introduced for a second-dimensional evaluation. The non-uniform sampling judgment module extracts the motion vector of the target pixel within three consecutive frames from the spatiotemporal buffer. The horizontal and vertical components of the motion vector are denoted as... and The subscript i represents the frame number that is traced back from the current frame. i takes the values ​​0, 1, and 2, which correspond to the current frame, the previous frame, and the two frames before that, respectively.

[0116] The formula for calculating the magnitude of the motion vector in the i-th frame out of three consecutive frames is:

[0117]

[0118] The arithmetic mean of the motion vector magnitudes of the three frames is used as the average single-frame offset, calculated as follows:

[0119]

[0120] The average single-frame offset is compared with a preset motion intensity threshold, the default value of which is 2 pixels. If the average single-frame offset is greater than or equal to 2 pixels, it is confirmed that the corresponding area is affected by the rapid movement of the object or the high-speed rotation of the camera, resulting in drastic changes in light and shadow. The target pixel is marked as a critical area. Even if the target pixel fails the first dimension judgment in step S301 because the light and shadow sensitivity level value is lower than 128, the target pixel is still forcibly marked as a critical area as long as the motion intensity judgment condition of this step is met.

[0121] Step S303: Determine the third dimension based on the depth gradient.

[0122] The non-uniform sampling determination module performs edge detection operations on the depth map to identify object edges and occlusion boundaries in the scene. The depth map is a two-dimensional array composed of depth values ​​stored in the depth channel of the geometric attribute buffer in step S101. The edge detection operation performs convolution operations on each pixel in the depth map and its 8 neighboring pixels, using the horizontal and vertical kernels of the Sobel operator.

[0123] The weight matrix of the horizontal kernel is:

[0124]

[0125] The weight matrix of the vertical kernel is:

[0126]

[0127] The horizontal gradient is obtained by element-wise multiplying the 3×3 neighborhood depth value of the target pixel with the horizontal and vertical kernels respectively and then summing the results. and vertical gradient The depth gradient magnitude G at a pixel location is given by the formula Calculated.

[0128] The depth gradient magnitude G is compared with the preset depth gradient threshold, which is set according to the normalization of the scene depth range. The default value is 0.05. The default value of 0.05 means that when the change in the depth value of adjacent pixels accounts for 5% of the total depth range of the scene, a geometric edge is considered to exist.

[0129] If the depth gradient magnitude G is greater than or equal to the depth gradient threshold, it indicates that the target pixel is located near the geometric edge of the object or the occlusion boundary. To prevent jagged edges or aliasing at the object edge, the target pixel is forcibly classified as a critical region.

[0130] The depth gradient determination in this step is parallel and independent of the determination in the first two dimensions. If the conditions in any dimension are met, it is classified as a critical region.

[0131] Step S304, comprehensive determination of non-critical areas.

[0132] After pixel-by-pixel evaluation in the above three dimensions, pixels that do not trigger critical region marking in any dimension are comprehensively checked. If a pixel meets the following three conditions at the same time: the light and shadow sensitivity level value is less than 128, the average single-frame motion vector offset is less than 2 pixels for 3 consecutive frames, and the depth gradient magnitude G is less than the depth gradient threshold, then the corresponding pixel is determined to be a non-critical region.

[0133] Non-critical areas in a scene are characterized by rough textures and a flat response to lighting, relatively static or slightly shifted position between consecutive frames, and flat surfaces rather than geometric edges. Typical distributions of non-critical areas include static backgrounds in the distance, skyboxes, and rough ground and walls with flat surfaces made of stone or wood.

[0134] Through steps S301 to S304, the three-dimensional composite dimension determination logic assigns a clear binary classification label to each pixel in the screen space, namely, a key region or a non-key region. The binary classification label is directly used as the control input for the differential sampling strategy in step S4.

[0135] At this point, step S3 completes the sampling priority classification of all pixels in the frame. By combining the conditions of material sensitivity, motion intensity, and depth gradient in parallel, the focus of computational resource budget is marked on each pixel within a single frame.

[0136] Step S4, differentiated execution, the specific process is as follows:

[0137] Step S401: High-frequency ray tracing sampling of key areas.

[0138] For pixels identified as critical regions in step S3, the high-frequency ray casting engine is invoked to perform dense sampling. The high-frequency ray casting engine directly schedules the dedicated hardware ray tracing core inside the GPU to initiate real-time ray casting calculations for each critical region pixel, with a basic sampling number of no less than 16 times per pixel.

[0139] To obtain the optical path that contributes most to the final radiance within a limited number of samplings, a hierarchical importance sampling strategy is used to guide the selection of the light direction. The execution process of the hierarchical importance sampling strategy consists of two steps:

[0140] The first step is to read the brightness power distribution data of all active light sources in the scene. The brightness power distribution data includes the light source type identifier, luminous power value, light source geometry, and the spatial position and orientation of the light source in the scene. Based on the brightness power distribution data, a discretized brightness distribution probability function is constructed. The construction method is as follows: the luminous power values ​​of all active light sources in the scene are summed to obtain the total power, and the sampling probability of each light source is equal to the luminous power of the corresponding light source divided by the total power.

[0141] The second step is to read the bidirectional reflectance distribution function of the micro-surface for the material corresponding to the current pixel. The bidirectional reflectance distribution functions of different materials are pre-stored in the material parameter table. The metal material adopts the GGX micro-surface model, and the diffuse reflectance material adopts the Lambertian model. The discretized brightness distribution probability function and the bidirectional reflectance distribution function are combined in the form of a product to generate a joint probability density function.

[0142] After generating the joint probability density function, a nonlinear coupling correction step based on the light and shadow sensitivity level is further introduced. This allows the sensitivity level to not only affect the number of samples but also to deeply participate in the control of the sampling direction. The specific method is as follows:

[0143] Read the light and shadow sensitivity level S of the current pixel (value range 0 to 255), and normalize it to the [0,1] interval to obtain the normalized sensitivity. ;

[0144] For the joint probability density function To adjust its distribution concentration, a power transformation of the following form is performed:

[0145]

[0146] in, For a monotonically increasing function based on normalization sensitivity, the default value is... ( The preset bias coefficient has a default value of 3. This results in higher sensitivity, and the sampling distribution becomes more concentrated in the direction of higher contribution.

[0147] Subsequent importance random sampling steps are based on Execute, replacing the original joint probability density function .

[0148] The effect of the above mechanism is that, for high-sensitivity pixels, the sampling direction is more concentrated in the direction of the light source and the direction of the specular reflection lobe, which greatly increases the proportion of effective optical paths with the same number of samplings; for low-sensitivity pixels, the sampling distribution remains relatively uniform to preserve the basic accuracy of diffuse reflection. This deep coupling through sensitivity level, sampling distribution shape and sampling efficiency makes it impossible to achieve continuous and adaptive adjustment of pixel-level sampling strategies by using material classification or importance sampling alone.

[0149] After completing the above coupling correction, the domain of the joint probability density function covers the hemispherical incident direction space centered on the surface point corresponding to the pixel. Before each ray projection, importance random sampling is performed based on the joint probability density function. The specific method of importance random sampling is as follows: first, a candidate direction sample is generated in a uniform distribution within the domain of the joint probability density function, the joint probability density function value corresponding to the candidate direction is calculated, and then a uniform random number between 0 and 1 is generated. If the uniform random number is less than the ratio of the joint probability density function value corresponding to the candidate direction to the maximum value of the joint probability density function in the domain, then the candidate direction is accepted as the ray emission direction; otherwise, a candidate direction sample is regenerated until it is accepted.

[0150] Using the above sampling method, the emitted light is preferentially directed towards the light source direction and the reflection lobe direction that contribute the most to the final radiance. The number of samples per pixel is not fixed. The high-frequency light projection engine reads the current available computing power reserve from the global computing power monitoring module. The computing power reserve is expressed as a percentage of GPU utilization or the number of milliseconds of available single frame time. When the computing power reserve is sufficient, the number of samples is gradually increased from 16 times, with an upper limit of 64 times. When the computing power reserve becomes tight, the number of samples is maintained at the lower limit of 16 times.

[0151] When the global computing power monitoring module initiates the second-level response and raises the first sensitivity threshold, pixels reclassified as non-critical regions not only have fewer sampling times, but also, because they are no longer marked as critical regions, the aforementioned sampling distribution bias coefficient based on sensitivity level... This also reduces the number of samples and the sampling strategy, ensuring that limited computing power is concentrated on the most critical visual area of ​​the current frame.

[0152] Step S402: Reprojection and spatiotemporal reuse of motion vectors in non-critical areas.

[0153] For pixels identified as non-critical regions, instead of initiating new ray tracing sampling directly, an attempt is made to reuse existing radiance data from the previous frame in the spatiotemporal buffer pool constructed in step S2. First, the motion vector of the current pixel is extracted from the L2-level historical record cache of the spatiotemporal buffer pool. The motion vector includes a horizontal component. and vertical components The motion vector is used to calculate the corresponding coordinates of the reprojection point in the screen space of the previous frame.

[0154] The calculation method is to subtract the component values ​​of the motion vector from the screen coordinates of the current pixel, that is...

[0155]

[0156]

[0157] In the L1-level buffer or L2-level history buffer of the previous frame, the radiance data of the corresponding pixel is searched according to the reprojection coordinates. If the reprojection coordinates fall on the sub-pixel position, the radiance value at that position is obtained by bilinear interpolation. The sampling point of bilinear interpolation is the four nearest integer coordinate pixels around the reprojection coordinates. Before reusing the radiance data, a reprojection validity check is performed. The validity check formula is shown in step S403.

[0158] Step S403: Reprojection validity check.

[0159] The validity of a reprojection is determined by the following conditions:

[0160]

[0161] In the formula:

[0162] The result of the validity determination is either true or false, where true indicates that the reprojection is valid and false indicates that the reprojection is invalid.

[0163] The material identity code for the target pixel in the current frame is read from the current pixel record in the L1 level real-time frame buffer of the spatiotemporal buffer pool. The data type is an 8-bit unsigned integer.

[0164] The material identification code for the reprojection point at the corresponding position in the previous frame is read from the L2-level history cache by addressing the reprojection coordinates, and the data type is an 8-bit unsigned integer.

[0165] The depth value of the target pixel in the current frame is read from the depth channel in step S101, and the data type is a 32-bit floating-point number.

[0166] The depth value of the reprojection point at the corresponding position in the previous frame is read from the L2 history cache by addressing the reprojection coordinates, and the data type is a 32-bit floating-point number;

[0167] The preset depth tolerance is set to 0.05 times the standard deviation of the depth values ​​of all pixels in the current frame by default. It is calculated once by the global control unit and written to the constant register when the scene is first loaded.

[0168] Operator " The operator "" indicates whether two operands are equal. "Represents the logical AND operation, operator" " indicates less than or equal to".

[0169] The execution logic for validity determination is as follows: first compare... and If they are equal, the reprojection is deemed invalid, and no further depth difference calculation is needed. If the material identity code of the target pixel in the current frame is the same as the material identity code of the corresponding position of the reprojection point in the previous frame, the absolute value of the depth difference is calculated and compared with the depth tolerance. If true, the reprojection is effective, and the interpolated radiance data is directly used as the radiance output of the current pixel without initiating a new ray tracing sample.

[0170] Step S404: Temporal interpolation compensation after reprojection failure.

[0171] If step S403 determines that reprojection is invalid, it indicates that the current pixel cannot directly reuse historical data due to object occlusion, changes in material identity encoding caused by object movement in the scene, or sudden changes in camera perspective. In this case, temporal interpolation compensation logic is initiated. The temporal interpolation compensation logic performs low-frequency light sampling on the current pixel, with a fixed sampling number of once per pixel. The radiance value obtained from the low-frequency light sampling is recorded as the center sampling value of the current pixel. At the same time, the radiance values ​​of each point in the 3×3 neighborhood of the current pixel that has passed the validity check or obtained radiance data through low-frequency sampling are read. The neighborhood range extends outward by 1 pixel from the current pixel, for a total of 9 pixel positions.

[0172] A weighted average is calculated between the central sample value and the known radiance values ​​in its neighborhood, with the weights allocated as follows:

[0173] If the radiance data of neighboring pixels comes from reprojection multiplexing, the weight is set to 1.0; if the radiance data of neighboring pixels also comes from low-frequency sampling, the weight is set to 0.5; the weight of the low-frequency sampling result of the center pixel itself is set to 1.0.

[0174] The weighted average is calculated as follows: the final radiance is equal to the sum of the radiance values ​​of each effective pixel multiplied by their corresponding weights, and then divided by the sum of all weights.

[0175] By using a weighted average, the hole regions generated by reprojection are filled in a way that allows for a smooth transition with the surrounding known pixels.

[0176] At this point, step S4 uses the pixel binary classification labels output in step S3 to execute completely different sampling paths for critical and non-critical regions. Within the limited ray tracing hardware resources and single-frame time budget, the main computing power is concentrated on areas with high material sensitivity, geometric edges, and dynamic motion.

[0177] Step S5, image fusion, processing details are as follows:

[0178] Step S501: Bilateral filtering and smoothing process.

[0179] After the differential execution in step S4, the high-frequency sampling results of the key area and the spatiotemporal multiplexing or interpolation compensation results of the non-key area belong to different pixels in space. Visual seams may appear at the boundary between the two types of areas due to the difference in sampling density.

[0180] Step S5 uses a bilateral filtering algorithm to smooth the entire frame image to eliminate such seams. The weights of the bilateral filter are determined by three factors: spatial distance, normal difference, and depth difference. The spatial distance weight factor adopts a Gaussian function. The spatial distance is the Euclidean distance from the neighboring pixels to the center pixel within the filtering window, in pixels.

[0181] The normal difference weight factor is determined by the dot product of the normal vector of the center pixel and the normal vector of the neighboring pixels. The depth difference weight factor is determined by the normalized absolute difference between the depth value of the center pixel and the depth value of the neighboring pixels. The normalized absolute difference of the depth value is calculated as follows: the absolute value of the depth value of the center pixel minus the absolute value of the depth value of the neighboring pixels, divided by the difference between the depth of the far clipping plane and the depth of the near clipping plane in the scene. The radius of the filter kernel is set in the range of 3 to 5 pixels, that is, taking 3 to 5 pixels in each of the four directions (up, down, left, and right) centered on the currently processed pixel to form a square filter window.

[0182] Step S502, geometric similarity-driven filter weight allocation.

[0183] During the filtering process, the geometric similarity between each pair of center pixels and neighboring pixels within the filtering window is calculated in real time. The geometric similarity considers both the normal direction and the depth value. First, the world space normal vector of the center pixel is read from the normal channel of the geometric attribute buffer, and the depth value of the center pixel is read from the depth channel. The normal vector and depth value are also read for each neighboring pixel within the filtering window.

[0184] When the dot product of the normal vectors of the center pixel and a neighboring pixel is greater than or equal to 0.95, and the normalized absolute difference of the depth value is less than or equal to 0.01, the neighboring pixel is determined to belong to the same continuous surface as the center pixel. A higher weight value is assigned to this neighboring pixel. The specific calculation method for the higher weight value is to multiply the spatial distance Gaussian weight, the normal consistency weight, and the depth consistency weight, where the normal consistency weight is 1.0 and the depth consistency weight is 1.0.

[0185] When the dot product of the normal vector is less than 0.8, or the absolute difference of the depth normalization exceeds 0.01, it is determined that the neighboring pixel is on both sides of the geometric edge or belongs to a different object from the center pixel. The weight is reduced to approach 0. The specific way to reduce the weight to approach 0 is to set the item that does not meet the threshold condition in the normal consistency weight or depth consistency weight to 0.0, and then multiply it with the other weight items to normalize all weights in the filtering window. The normalization method is that the final weight of each neighboring pixel is equal to the original weight divided by the sum of the original weights of all neighboring pixels in the window. The normalized weight is used to calculate the weighted average of the radiance values ​​of each pixel in the window, and the calculation result is used as the smoothed radiance output of the center pixel.

[0186] Through the above weight allocation, the filtering operation smooths out sampling noise while preserving the radiance transitions on both sides of the geometric hard edges in the scene, thus maintaining the clarity of texture details.

[0187] Step S503: Nonlinear decay of historical contribution weighting factor.

[0188] In the image fusion process, a historical contribution weight factor is introduced. The historical contribution weight factor decays non-linearly based on the number of times the cached data is reused consecutively. A reuse counter is maintained for each pixel record in the spatiotemporal cache pool. The reuse counter is an 8-bit unsigned integer with an initial value of 0.

[0189] Whenever a pixel passes the reprojection validity check and is successfully reused in step S402, the reuse counter is incremented by 1; whenever a pixel is determined to be a critical region in a frame and high-frequency sampling is performed, the reuse counter is reset to zero.

[0190] In the weighted average of step S502, if the radiance of a neighboring pixel comes from reprojection multiplexing, its contribution coefficient is multiplied by the historical contribution weighting factor. The historical contribution weighting factor is defined as the exponential decay function of the multiplexing counter, specifically in the following form:

[0191]

[0192] In the formula, The historical contribution weighting factor has a value range of 0 to 1. This is the reuse counter value, which is the number of frames that the pixel has been reused consecutively. The data type is an 8-bit unsigned integer. This is the decay constant, with a default value of 10, used to control the decay rate of historical contribution as the number of reuses increases.

[0193] Multiplexing counter value The larger the historical contribution weighting factor, the greater the contribution weighting factor. The closer the value is to 0, the lower the contribution of historical pixel data to the fusion result of the current frame; if the multiplexing counter value of a certain pixel is... If the preset reuse count threshold is exceeded (the default reuse count threshold is set to 15 frames), then after step S502 is completed, an additional full-frequency high-frequency sampling is forcibly performed on this pixel. The number of light projections for full-frequency high-frequency sampling is consistent with the lower limit of the number of samplings for the key area in step S401, which is 16 times per pixel.

[0194] The newly sampled radiance value is overwritten into the corresponding record in the spatiotemporal buffer pool, and the reuse counter is simultaneously updated. Zeroing out; by introducing a forced refresh mechanism with a historical contribution weight factor and a reuse count threshold, image ghosting caused by long-term reprojection error accumulation is suppressed.

[0195] At this point, step S5 completes the fusion of the two sets of radiance images with different sampling densities output in step S4, and outputs a complete radiance image that is seamless in intra-frame space and has controlled motion blur in inter-frame temporal sequence. The radiance image is directly passed to the post-processing anti-aliasing unit as input, and is also written to the L2 level history buffer for use in the spatiotemporal multiplexing stage of step S4 in the next frame, thus realizing the closed loop of the entire method from geometric input to final image output.

[0196] The central control unit uniformly schedules the working timing and data flow of the environmental semantic recognition module, spatiotemporal cache management module, sampling and judgment logic unit, ray casting execution engine, and image fusion post-processing unit. The modules do not directly access each other's local caches; all data exchange is completed through a high-speed video memory bus architecture. The high-speed video memory bus architecture allocates independent read and write channels for each type of data stream. The channel allocation method is bound to the number of physical channels of the GPU video memory controller. When the video memory controller has an 8-channel structure, the depth data channel and color data channel each occupy 2 channels, the motion vector channel and material identity encoding channel each occupy 1 channel, and the remaining 2 channels are reserved for the transmission of control signals and synchronization primitives.

[0197] Within a single frame processing cycle, the environmental semantic recognition module first pushes the sensitivity map generated in step S1 to the L1 level real-time frame buffer of the spatiotemporal buffer management module. Then, the sampling and judgment logic unit retrieves the sensitivity map and motion vector from the same buffer, completes the pixel binary classification in step S3, and the classification label is then sent to the ray projection execution engine to drive the differential sampling in step S4. After the initial radiance map obtained by sampling is processed by the image fusion post-processing unit in step S5, it is written back to the L2 level historical record buffer of the spatiotemporal buffer management module. The above process forms a pipeline closed chain in which the modules are connected end to end within a frame.

[0198] Running in parallel with the aforementioned pipeline is a global computing power monitoring mechanism. The global computing power monitoring module obtains the GPU load rate and single-frame rendering time in each frame cycle. The GPU load rate is read from the shader core utilization rate in the GPU performance counter and expressed as a percentage. The single-frame rendering time is obtained by reading the GPU hardware timestamp counter twice, at the beginning and end of the frame, taking the difference and dividing it by the GPU clock frequency, in milliseconds. Both indicators are pushed to the central control unit via the video memory bus once per frame.

[0199] The central control unit compares the rendering time of a single frame with the preset real-time threshold of 33 milliseconds. The real-time threshold corresponds to a real-time rendering requirement of 30 frames per second. Once the rendering time of a single frame exceeds 33 milliseconds, the central control unit determines that the current computing power has entered the over-limit range and then starts a two-level step-by-step response strategy.

[0200] The first-level response locks the temporal interpolation compensation frequency of non-critical areas and adjusts the low-frequency ray sampling in step S404 from frame-by-frame execution to frame-skipping execution. The frame-skipping execution method is to perform low-frequency ray sampling once for non-critical pixels that are invalid for reprojection only every 1 frame. The corresponding pixels in the skipped frames directly use the radiance value after temporal interpolation compensation in the previous frame, thereby releasing a part of the ray tracing computing power in non-critical areas.

[0201] If the frame time of the next frame after the implementation of the first-level response still does not fall back to within 33 milliseconds, the central control unit continues to start the second-level response. The second-level response increases the first sensitivity threshold for key area determination in step S301 from the default 128 to 160, with an increase step of 32 units. Pixels with light and shadow sensitivity values ​​between 128 and 159 are no longer marked as key areas, and the total number of pixels that need to be sampled at high frequency is reduced accordingly.

[0202] Once the rendering time for each of the five consecutive frames has stabilized below 33 milliseconds, the central control unit gradually cancels the above adjustments in the order of first restoring the sensitivity threshold and then restoring the interpolation compensation frequency. That is, the first sensitivity threshold is first restored from 160 to 128, and then the low-frequency light sampling is restored from frame skipping to frame-by-frame execution.

[0203] The two-level response strategy leverages the sparsity of sampling in non-critical areas and the dynamic upward shift of the entry threshold in critical areas to achieve a rigid guarantee of frame rate while keeping image quality loss under control.

[0204] After the fusion process in step S5 is completed, the radiance image is not directly sent to the display, but enters the post-processing anti-aliasing unit to receive further image quality enhancement. The post-processing anti-aliasing unit uses runtime super-resolution technology to extract the radiance images of the previous four consecutive frames from the L2-level historical record cache. Motion compensation is performed on each historical image according to the motion vector of its corresponding pixel. The pixel position of the historical frame is reverse-mapped to the current frame view to achieve inter-frame alignment. The reverse mapping method of motion compensation is to add the screen coordinates of the historical frame pixels to the motion vector stored in the historical frame pixels to obtain the corresponding coordinates in the current frame view.

[0205] The aligned 5-frame image, i.e. the current frame plus the compensated 4 frames, is weighted and averaged using exponential decay weights with a decay factor of 0.8. The weights of each frame are distributed as follows: the current frame has a weight of 1.0, the previous frame has a weight of 0.8, the previous 2 frames have a weight of 0.64, the previous 3 frames have a weight of 0.512, and the previous 4 frames have a weight of 0.4096. The weighted average is calculated by multiplying the corresponding pixel values ​​of each frame by their weights and then dividing by the sum of all weights. This aggregates the sub-pixel-level offset information between multiple frames into the same output frame, thereby improving the edge smoothness and visual clarity of the image.

[0206] The entire processing flow starts from the geometric input stage, runs through steps S1 to S5, and then outputs the final pixel in the post-processing anti-aliasing unit. The total delay is strictly constrained to within 2 frames. The specific timing arrangement of the total delay is as follows: the geometric attribute capture in step S1 is completed in the current frame; all data operations and temporal cumulative reconstruction in steps S2 to S5 are completed in the first half of the next frame; and the final output is sent to the display buffer at the end of the next frame.

[0207] The global control architecture takes the central control unit as the scheduling core, frame-by-frame computing power monitoring as the decision-making basis, and post-processing anti-aliasing as the end-quality compensation. Together, these three form a closed-loop runtime guarantee mechanism. This method can simultaneously meet the rigid constraints of real-time interaction on frame rate and the smooth requirements of visual continuity for multi-frame continuous images on resource-constrained hardware platforms.

[0208] This embodiment also provides a game image generation system, including:

[0209] The environmental semantic recognition module is used to obtain the material identity code, depth value and world space normal vector of each pixel in the current frame screen space through deferred rendering, query the material sensitivity mapping table to obtain the basic sensitivity value, combine the dynamic light source scattering degree and surface light source irradiance change information to perform offset correction, generate the light and shadow sensitivity level of each pixel, and store it in the L1 level real-time frame buffer of the spatiotemporal cache management module.

[0210] The spatiotemporal cache management module is used to construct a spatiotemporal cache pool in the video memory, which includes an L1-level real-time frame cache and an L2-level history cache. The L1-level real-time frame cache stores the light and shadow sensitivity level, radiance data, and motion vector of each pixel in the current frame. The L2-level history cache adopts a first-in-first-out circular queue structure to manage the historical radiance data, historical material identification codes, and historical depth values ​​of multiple consecutive frames. When writing radiance data and normal data, differential pulse coding compression is performed using the spatial correlation of adjacent pixels within the frame.

[0211] The sampling and judgment logic unit is used to read the light and shadow sensitivity level of each pixel from the L1 level real-time frame buffer, and combine it with the motion vector and depth gradient obtained from the spatiotemporal buffer management module to perform multi-dimensional non-consistent sampling and judgment on the pixels of the whole frame, dividing the pixels of the whole frame into key regions and non-key regions.

[0212] The ray casting execution engine is used to perform high-frequency ray tracing sampling on pixels in critical areas using an importance sampling strategy to generate the first radiance data; for pixels in non-critical areas, historical radiance data is reused from the spatiotemporal buffer through motion vector reprojection to obtain the second radiance data; low-frequency ray sampling is performed on pixels in non-critical areas where reprojection is invalid, and temporal interpolation compensation is performed.

[0213] The image fusion post-processing unit is used to fuse the first radiance data and the second radiance data, apply a historical contribution weight factor based on the multiplexing counter to reduce the contribution of historical data that has been continuously reused for multiple frames, and force a refresh of sampling when the number of pixel reuses exceeds a preset threshold to generate the final game view frame.

[0214] Under the unified scheduling of the central control unit, the above modules exchange data through a high-speed video memory bus architecture, forming a closed pipeline chain that connects the beginning and end of a frame.

[0215] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0216] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment includes only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for generating game images based on a 3D rendering engine, characterized in that, This is achieved through an environmental semantic recognition module, a sampling and judgment logic unit, a ray casting execution engine, and an image fusion post-processing unit, including: The geometric description of each pixel in the screen space of the current frame is obtained by deferred rendering. The geometric description includes material identity code, depth value and world space normal vector. Based on the material identity code, a preset material sensitivity mapping table is queried for each pixel to obtain the basic sensitivity value; Based on the world space coordinates corresponding to each pixel and the attributes of active light sources in the scene, the basic sensitivity values ​​of pixels falling within the influence volume of the light source are offset and corrected to obtain the light and shadow sensitivity level of the pixel. The light and shadow sensitivity level of each pixel is then stored in the L1 level real-time frame buffer of the spatiotemporal buffer pool. The sampling and judgment logic unit reads the light and shadow sensitivity level of each pixel from the L1 level real-time frame buffer, and combines the pixel's motion intensity across multiple consecutive frames and the depth gradient in the depth map to perform multi-dimensional non-consistent sampling and judgment on all pixels in the frame, dividing all pixels in the frame into critical and non-critical regions. The ray casting execution engine uses an importance sampling strategy that performs non-linear adjustments based on the pixel's light and shadow sensitivity level for pixels in key areas to guide the sampling direction of ray tracing and generate the first radiance data; For pixels in non-critical areas, motion vectors are extracted from the spatiotemporal buffer pool, second radiance data is obtained from the radiance data of the previous frame through reprojection, and temporal interpolation compensation is performed when reprojection is invalid. The image fusion post-processing unit performs image fusion processing on the first radiance data and the second radiance data to generate a complete game view frame.

2. The game image generation method based on a 3D rendering engine according to claim 1, characterized in that, The steps for constructing the spatiotemporal cache pool include: The video memory is configured with a storage structure that includes an L1-level real-time frame buffer and an L2-level history buffer. The L1-level real-time frame buffer is used to store the light and shadow sensitivity level, first radiance data or second radiance data, and motion vector of each pixel in the current frame. The L2-level history buffer adopts a first-in-first-out circular queue structure to manage the historical radiance data, historical material identification code, and historical depth value of multiple consecutive frames. When writing luminance data and normal data to the spatiotemporal buffer pool, differential pulse coding compression is performed using the spatial correlation of adjacent pixels within the frame to generate a compressed data stream. Based on the head pointer of the circular queue structure, the compressed data stream of the new frame is overwritten into the address space corresponding to the oldest frame with the longest retention time.

3. The game image generation method based on a 3D rendering engine according to claim 1, characterized in that, The multi-dimensional non-consistent sampling determination includes: The first dimension of the determination is to compare the light and shadow sensitivity level of a pixel with a preset first sensitivity threshold. If the light and shadow sensitivity level of a pixel is greater than or equal to the first sensitivity threshold, it is marked as a critical area. The second dimension determination involves calculating the average value of the motion vector magnitude of a pixel over multiple consecutive frames. If the average value is greater than or equal to a preset motion intensity threshold, it is marked as a critical region. The third dimension determination involves performing edge detection on the depth map composed of the depth values, calculating the depth gradient magnitude of the pixels, and marking the region as a critical area if the depth gradient magnitude is greater than or equal to a preset depth gradient threshold. Pixels that are not marked as critical regions in any dimension are classified as non-critical regions.

4. The game image generation method based on a 3D rendering engine according to claim 3, characterized in that, It also includes a closed-loop computing power adaptive adjustment step based on the rendering time of a single frame: Obtain the single-frame rendering time of the graphics processor; When the rendering time of a single frame exceeds the preset real-time threshold, a step-by-step response strategy is initiated. The step-by-step response strategy includes: a first-level response, which adjusts the low-frequency light sampling performed when the reprojection of non-critical areas is invalid to frame skipping; If the rendering time of the next frame still does not fall below the real-time threshold, the second-level response is initiated, and the first sensitivity threshold in the non-consistent sampling determination is increased to reduce the total number of pixels marked as critical regions. Once the rendering time of subsequent consecutive frames stabilizes below the real-time threshold, the response strategy is gradually revoked in the order of first restoring the first sensitivity threshold and then restoring the low-frequency light sampling frequency.

5. The game image generation method based on a 3D rendering engine according to claim 2, characterized in that, The image fusion processing step includes: A multiplexing counter is maintained for each pixel in the spatiotemporal buffer pool, and the multiplexing counter is incremented when the pixel is successfully reprojected and reused in the current frame; A historical contribution weighting factor is introduced, which is an exponential decay function of the reuse counter; When performing weighted fusion on pixels in non-critical areas, the contribution of historical radiance data that has been reused for multiple consecutive frames is reduced according to the historical contribution weighting factor. When the value of the multiplexing counter of any pixel exceeds the preset multiplexing count threshold, high-frequency ray tracing sampling is forced to be performed on any pixel to refresh the radiance data, and the multiplexing counter of any pixel is cleared to zero.

6. The game image generation method based on a 3D rendering engine according to claim 1, characterized in that, Importance sampling strategies based on pixel-level light and shadow sensitivity levels for non-linear adjustment include: The light and shadow sensitivity level of the pixel is obtained, and the light and shadow sensitivity level is converted into a bias coefficient based on a preset monotonically increasing mapping function; Using the aforementioned bias coefficient, a power transformation is performed on the joint probability density function generated by the light source brightness distribution and the material bidirectional reflection distribution function to adjust its distribution concentration. Importance sampling is performed based on the transformed joint probability density function to determine the sampling direction for ray tracing.

7. The game image generation method based on a 3D rendering engine according to claim 1, characterized in that, Before reusing the radiance data from the previous frame, a reprojection validity check is performed; the conditions for the reprojection validity check include: Compare the material identity code of the current frame pixel with the material identity code of the corresponding position in the previous frame obtained through reprojection coordinates to see if they are consistent; Compare the absolute value of the difference between the depth value of the current frame pixel and the depth value of the corresponding position in the previous frame to see if it is less than or equal to the preset depth tolerance. If both conditions are met, the reprojection is deemed valid; otherwise, it is deemed invalid.

8. The game image generation method based on a 3D rendering engine according to claim 7, characterized in that, The steps for performing the time-domain interpolation compensation include: For non-critical region pixels that are invalid for reprojection, perform low-frequency light sampling with a sampling number lower than that of the critical region to obtain the center sampling value; Read the radiance values ​​of other pixels within a 3×3 spatial neighborhood centered on the current pixel that have already had their radiance data obtained through reprojection validity checks or low-frequency light sampling; The central sampled value is weighted and averaged with the radiance values ​​of other pixels in the neighborhood. The weights are allocated according to the source type of the radiance data of other pixels in the neighborhood to fill the hole areas caused by invalid reprojection.

9. The game image generation method based on a 3D rendering engine according to claim 1, characterized in that, The process of offsetting and correcting the basic sensitivity value of pixels falling within the influence volume of the light source based on the world space coordinates corresponding to each pixel and the attributes of active light sources in the scene to obtain the light and shadow sensitivity level of the pixel further includes: For a surface light source, calculate the dot product between the world space normal vector of the pixel and the unit direction vector pointing from the pixel to the geometric center of the surface light source; When the dot product falls within a preset dot product threshold range, the pixel is determined to be in the irradiance gradient sensitive region caused by the surface light source, and the basic sensitivity value of the pixel is adjusted by a secondary offset to make the irradiance gradient sensitive region obtain a higher light and shadow sensitivity level than the smooth region directly illuminated by the light source.

10. A game image generation system based on a 3D rendering engine, used to execute the game image generation method based on a 3D rendering engine as described in any one of claims 1-9, characterized in that, include: The environmental semantic recognition module is used to generate the light and shadow sensitivity level of each pixel in the current frame; The light and shadow sensitivity level is calculated based on the material identity code, depth value, and world space normal vector obtained from the geometric attribute buffer of deferred rendering, combined with the material sensitivity map, dynamic light source sweep rate, and area light source irradiance change information. The spatiotemporal cache management module is used to build a spatiotemporal cache pool in the video memory, which includes an L1-level real-time frame cache and an L2-level historical record cache, to store and manage the light and shadow sensitivity level, as well as the motion vector, radiance data, material identity code and depth value of each pixel across multiple frames. The sampling and determination logic unit is used to divide the entire frame of pixels into key regions and non-key regions based on the light and shadow sensitivity level and the motion vector and depth gradient obtained from the spatiotemporal buffer management module. A ray casting execution engine is used to perform high-frequency ray tracing sampling on the critical areas and low-frequency ray sampling on pixels in the non-critical areas that are invalid for reprojection. The image fusion post-processing unit is used to fuse high-frequency ray tracing sampling and spatiotemporally multiplexed radiance data, and apply a historical contribution weighting factor based on the multiplexing counter to generate the final game view frame.

Images