Three-dimensional scene rendering processing method based on real-time ray tracing

Abstract

The invention discloses a three-dimensional scene rendering processing method based on real-time ray tracing. The three-dimensional scene rendering processing method comprises the following steps that semantic category information is labeled on objects in a scene in advance through an object label module; the scene analysis module collects the semantic category information and illumination information in a scene and judges the category of an object; based on the object category, a mixed rendering module calls a rasterization rendering strategy or a ray tracing rendering strategy to carry out scene rendering, and rendered image data is formed; and the image processing module carries out de-noising processing on the image data to generate a high-resolution image. The problem that an existing scene rendering method cannot give consideration to application requirements of high rendering quality and rendering efficiency, low rendering cost and hardware requirements and is small in application range can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of game rendering, in particular to a three-dimensional scene rendering processing method based on real-time ray tracing. BACKGROUND

[0002] With the rapid development of science and technology, for cultural exhibition enterprises, 3D game display and experience projects in exhibition halls are increasing. High-quality 3D games can better carry cultural content, spread cultural value through the interactive medium of games, and promote the development of the cultural industry.

[0003] In recent years, with the rapid improvement of computer hardware performance, real-time ray tracing technology has been increasingly widely used in the game field. Ray tracing technology can create extremely realistic images by simulating the interaction between light and objects, presenting complex light and shadow effects such as reflection, refraction, shadow, and global illumination, which is beyond the reach of traditional rasterization technology. Abroad, NVIDIA and other companies actively promote the development of real-time ray tracing technology, and the RTX series of graphics cards specially provide hardware acceleration (such as the RT core) for ray tracing calculations, greatly improving the efficiency of ray tracing calculations. At the same time, Microsoft's DirectX Raytracing (DXR) API allows developers to use ray tracing functions in DirectX 12, further promoting the application of real-time ray tracing technology in game development. Many 3A game works have adopted real-time ray tracing technology to render scenes and special effects, greatly improving the realism and immersion of game pictures, and bringing players an unprecedented visual experience. The domestic game industry is also following the development trend of real-time ray tracing technology. Many game development companies have begun to try to introduce ray tracing technology into games, although there is a certain gap in the depth and breadth of technology application compared with foreign countries, but the development momentum is good. With the development of the domestic hardware industry and the improvement of the technical level of game developers, the application prospect of real-time ray tracing technology in the domestic game market is broad. From the development trend, real-time ray tracing technology will gradually become a standard for 3D game development, and future games will pay more attention to the realism and delicacy of light and shadow effects, creating a more immersive game environment for players.

[0004] At present, scene picture rendering in 3D game production mainly relies on real-time ray tracing technology, rasterization technology, etc. The rendering quality of the former is high, but the rendering cost is high due to high computational complexity and large amount of calculation, which requires high hardware, which also makes the audience range small. In addition, the use of rendering strategies and rendering technologies is single, which makes the game rendering have obvious disadvantages, and cannot balance the application requirements of high rendering quality and rendering efficiency, low rendering cost and hardware requirements, making its application range small. SUMMARY

[0005] The application aims at solving the problem that the existing scene rendering method cannot balance the application requirements of high rendering quality and rendering efficiency, low rendering cost and hardware requirements, and has a small application range.

[0006] In order to achieve the above-mentioned purpose, the application adopts the following technical scheme: a three-dimensional scene rendering processing method based on real-time ray tracing, comprising the following steps: The object label module is used to pre-label the semantic category information of the objects in the scene; The scene analysis module collects the semantic category information and the lighting information in the scene and determines the object category; Based on the object category, the hybrid rendering module calls the rasterization rendering strategy or the ray tracing rendering strategy to perform scene rendering and form image data after rendering; The image processing module performs denoising processing on the image data to generate high-resolution images.

[0007] As a further description of the above technical scheme: The semantic category information includes object dynamic / static information and object density information, when the object is static and has low density, the rasterization rendering strategy is called to perform scene rendering, otherwise, the ray tracing rendering strategy is called.

[0008] As a further description of the above technical scheme: The semantic category information also includes light effect requirement information, when the light effect requirement of the object is low, the rasterization rendering strategy is called to perform scene rendering, otherwise, the ray tracing rendering strategy is called.

[0009] As a further description of the above technical scheme: The rendering strategy calling of the hybrid rendering module is based on the calling rules in the rule library, the rendering simulation module is used to construct test cases of different scenes, perform rendering effect simulation on the test cases, and determine the priority of the rendering effect and performance of the rasterization rendering strategy and the ray tracing rendering strategy in the corresponding scene through a machine learning model, so as to form the corresponding calling rules and store them in the rule library.

[0010] As a further description of the above technical scheme: The ray tracing rendering strategy is based on the BVH algorithm, in the construction stage, the division mode of the BVH node is dynamically adjusted according to the semantic category information and the lighting information, and the ray-triangle intersection test is performed through GPU parallel calculation.

[0011] As a further description of the above technical scheme: The image processing module fuses the image data with multi-modal data to form fused data and inputs the fused data into a denoising model of a deep learning module for model training.

[0012] As a further description of the above technical solution: The multi-modal data is composed of object geometric information and material information in the semantic category information.

[0013] As a further description of the above technical solution: According to the game frame rate and image parameter requirements, the DLSS parameters are dynamically adjusted by the DLSS module of the image processing module to balance the frame rate and the image quality.

[0014] As described above, due to the adoption of the above technical solution, the present application has the following beneficial effects compared with the prior art: 1. The three-dimensional scene rendering processing method of the present application can be applied to 3D game scene picture rendering. The intelligent switching algorithm based on scene semantic analysis combines traditional rasterization technology with ray tracing. The object label module pre-labels the object with semantic categories, and the combination of "rule base + machine learning model" maps the semantic feature vector to a specific rendering strategy, updates the semantic analysis result in real time, and optimizes according to the actual rendering effect feedback. It is realized that in most scenes, rasterization rendering is used, and only in local areas or special cases where accurate light and shadow effects are needed, ray tracing is used. The algorithm can analyze the semantic information such as element composition, light effect demand and lighting conditions of the game scene in real time and use corresponding efficient rendering strategy. Compared with the traditional single rendering method, it is more intelligent and accurate, and can realize the optimal combination of traditional rasterization technology and ray tracing technology in different scenes, and maximize the rendering efficiency and picture quality.

[0015] 2. Through the improvement of adaptive BVH construction and traversal algorithm, the construction method and traversal strategy of BVH can be dynamically adjusted according to the distribution characteristics of objects in the scene and the propagation characteristics of light rays. Compared with the traditional fixed BVH structure, the calculation amount of ray tracing can be significantly reduced in complex scenes, and the calculation efficiency can be improved.

[0016] 3. Through the deep learning denoising and super-resolution algorithm based on multi-modal data fusion, not only the information of the image itself is used, but also multi-modal data such as geometric information and material information of the scene is fused. Compared with the traditional image denoising method, it can more accurately remove noise and generate higher quality super-resolution images, further improving the image quality and rendering efficiency after ray tracing. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0018] Figure 1 A flow chart of a three-dimensional scene rendering processing method based on real-time ray tracing. DETAILED DESCRIPTION

[0019] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0021] It should be noted that: similar reference numbers and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0022] In the description of the embodiments of the present application, it should be noted that the orientation or position relationship indicated by the terms "upper", "inner" and the like is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship commonly used when the product of the present application is placed, and is only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0023] In the description of the present application, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set", "mount", "connected", "connected" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances. Embodiment one:

[0024] Please see Figure 1 This invention provides a technical solution: a 3D scene rendering method based on real-time ray tracing, comprising the following steps: The object labeling module pre-labels objects in the scene with semantic category information; The scene analysis module collects the semantic category information and the lighting information in the scene and determines the object category; Based on the object category, the hybrid rendering module calls the rasterization rendering strategy or the ray tracing rendering strategy to perform scene rendering and form the rendered image data; The image processing module performs noise reduction on the image data to generate a high-resolution image.

[0025] The three-dimensional scene rendering processing method of the present invention can be applied to the rendering of 3D game scene images. Through the design of hybrid rendering strategy, based on the type labeling and identification of objects in the scene, a corresponding higher cost-effective rendering technology can be selected. While ensuring the image quality, the rendering cost is reduced, the rendering efficiency is significantly improved, and noise reduction processing is performed to generate high-resolution images.

[0026] The semantic category information includes object dynamic / static information and object density information. When the object is static and has low density, the rasterization rendering strategy is invoked for scene rendering; otherwise, the ray tracing rendering strategy is invoked. Through the rendering mode switching design based on the hybrid rendering strategy, it is possible to automatically and accurately switch between traditional rasterization technology and ray tracing technology according to factors such as the complexity of the game scene.

[0027] The hybrid rendering module invokes rendering strategies based on rules in a rule base. The rendering simulation module constructs test cases for different scenarios, simulates rendering effects on these test cases, and uses a machine learning model to determine the priority of the rasterization rendering strategy and ray tracing rendering strategy in the corresponding scenarios, forming corresponding invocation rules and storing them in the rule base. Under these rules, during actual rendering, the rendering strategy with higher priority for the corresponding scenario type is used first. In the experimental verification phase, a dedicated experimental platform is built. Using mainstream game development engines (such as Unity and Unreal Engine), combined with NVIDIA's RTX technology and DirectX Raytracing (DXR) tools, optimization experiments of real-time ray tracing technology are conducted. For the hybrid rendering strategy, test cases for different scenarios are designed to compare the rendering effects and performance of traditional rasterization and ray tracing technologies in different scenarios, verifying the effectiveness of the intelligent switching algorithm. For ray tracing algorithm optimization, complex scenes are constructed on the experimental platform to test the performance of different BVH construction and traversal algorithms, recording metrics such as ray tracing calculation time and memory usage to evaluate the optimization effect of the algorithm.

[0028] This intelligent switching algorithm, based on scene semantic analysis, combines traditional rasterization technology with ray tracing. Leveraging the game engine's "object labeling system," it pre-labels objects with semantic categories and collects semantic feature vectors in real time. Through a combination of a "rule base" and a machine learning model, it maps these semantic feature vectors to specific rendering strategies, updates the semantic analysis results in real time, and optimizes based on actual rendering effects. This allows for rasterization rendering in most scenes, while ray tracing is used only in localized areas requiring precise lighting and shadow effects or in special cases, thus reducing rendering costs.

[0029] The described ray tracing rendering strategy is based on the BVH (Browser-Video Heights) algorithm. During the construction phase, the partitioning method of BVH nodes is dynamically adjusted according to the semantic category information and lighting information. Ray-triangle intersection tests are performed using GPU parallel computing. By optimizing the BVH construction algorithm, construction time is reduced. Simultaneously, the BVH traversal algorithm is optimized. The algorithm can dynamically adjust the BVH construction method and traversal strategy based on the distribution characteristics of objects in the scene and the propagation characteristics of light. Compared with the traditional fixed BVH structure, this significantly reduces the computational load of ray tracing in complex scenes and improves the computational efficiency of light-scene geometry interaction. In terms of hardware acceleration, the parallel computing capabilities of the GPU are fully utilized. Deep optimization is performed on hardware-accelerated BVH traversal and ray-triangle intersection tests to ensure efficient processing of large numbers of rays in real-time ray tracing. Example 2:

[0030] Please see Figure 1The figure illustrates a 3D scene rendering method based on real-time ray tracing provided in Embodiment 2 of the present invention. This embodiment further improves upon the previous embodiment by incorporating the following technical solutions: The semantic category information also includes lighting effect requirement information. When the lighting effect requirement of an object is low, the rasterization rendering strategy is invoked for scene rendering; otherwise, the ray tracing rendering strategy is invoked. In this embodiment, the selection of the rendering strategy is also based on the lighting effect requirement. This parameter is pre-labeled using an object tagging module to guide the selection of the rendering strategy and improve scene rendering efficiency. For example, in game scenes, traditional rasterization technology is used for rendering simple environmental backgrounds and non-critical objects, while ray tracing technology is used for parts requiring precise lighting effects, such as the reflection of metal equipment on characters and complex water surface refraction. This automatically identifies objects and scenes requiring refined lighting effect rendering, achieving efficient rendering and saving the cost and time of rendering most objects and scenes that do not require refined lighting effect rendering, greatly improving rendering efficiency. Example 3:

[0031] Please see Figure 1 The figure illustrates a 3D scene rendering method based on real-time ray tracing provided in Embodiment 3 of the present invention. This embodiment further improves upon the previous embodiments by implementing the following technical solutions: The image processing module fuses the image data with multimodal data to form fused data, and inputs the fused data into the denoising model of the deep learning module for model training. After training, the denoising model denoises the image data. The multimodal data consists of object geometric information and material information from the semantic category information. This algorithm not only utilizes the information of the image itself but also fuses multimodal data such as scene geometric information and material information. Compared with traditional deep learning algorithms based solely on images, it can more accurately remove noise, generate higher-quality super-resolution images, and further improve the image quality and rendering efficiency after ray tracing.

[0032] Based on the game's frame rate and image parameter requirements, the DLSS module of the image processing module dynamically adjusts the DLSS parameters to balance frame rate and image quality. In the AI-assisted optimization experiment, a denoising and super-resolution model based on multimodal data fusion was trained using deep learning frameworks (such as TensorFlow and PyTorch). By comparing the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) of the images before and after denoising, as well as the game's frame rate and image resolution before and after the application of DLSS technology, the performance improvement effect of the AI-assisted optimization technology was verified.

[0033] This research delves into the application of deep learning algorithms in ray tracing, developing a deep learning-based denoising algorithm to effectively remove noise generated during ray tracing and improve image quality. Simultaneously, it further optimizes the DLSS model technology to better integrate with real-time ray tracing techniques, significantly improving image resolution while maintaining game frame rates.

[0034] The main technical approach and method flow of the above embodiments are as follows: First, we developed a game scene analysis module. This module utilizes computer vision technologies such as semantic segmentation and object detection to perform real-time analysis of the game scene, extracting information such as object types, distribution, and lighting conditions, providing data support for intelligent switching of hybrid rendering strategies. Regarding the implementation of the hybrid rendering strategy, based on the output of the scene analysis module, when the scene is relatively simple and the requirements for lighting and shadow effects are not high, the traditional rasterization rendering pipeline is used for rendering; when there are objects or areas in the scene that require precise lighting and shadow effects, the ray tracing rendering pipeline is activated. Simultaneously, a rendering resource management system is developed to ensure the reasonable allocation and efficient utilization of resources during the switching between the two rendering methods. For ray tracing algorithm optimization, an adaptive BVH (Browser-Video Hierarchy) construction technique is adopted. During the scene construction phase, the partitioning method of BVH nodes is dynamically adjusted based on the spatial distribution of objects and the probability model of ray propagation, reducing the number of nodes and improving ray traversal efficiency. During ray tracing, the ray-triangle intersection testing algorithm is optimized, leveraging the parallel computing capabilities of the GPU to accelerate the intersection calculation process. In the AI-assisted optimization phase, the multimodal data output from the scene analysis module is fused with the ray-traced image data and input into a deep learning-based denoising and super-resolution model. After training, the model denoises the ray-traced images and generates high-resolution images. Simultaneously, the collaborative workflow between DLSS and ray tracing technologies is optimized. Based on game frame rate and image quality requirements, DLSS parameters are dynamically adjusted to achieve a balance between frame rate and image quality. Finally, the performance monitoring and feedback module monitors the game's frame rate, image quality, and other performance metrics in real time, and feeds the monitoring results back to various optimization modules so that optimization strategies can be dynamically adjusted and optimized to ensure that the entire real-time ray tracing system can maintain efficient and stable operation in different game scenarios.

[0035] Key technical specifications: Under mainstream hardware configurations, it achieves a stable frame rate of 60fps or higher for 3D games supporting real-time ray tracing; in terms of image quality, the lighting and shadow effects after ray tracing reach industry-leading levels, such as more realistic and natural reflection and refraction effects, more delicate and accurate shadow edges, and global illumination effects that are more in line with real-world physical laws; the noise in the denoised image is reduced by more than 90%, and the image resolution is improved by at least 2 times after the application of DLSS technology.

[0036] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A 3D scene rendering method based on real-time ray tracing, characterized in that, Includes the following steps: The object labeling module pre-labels objects in the scene with semantic category information; The scene analysis module collects the semantic category information and the lighting information in the scene and determines the object category; Based on the object category, the hybrid rendering module calls the rasterization rendering strategy or the ray tracing rendering strategy to perform scene rendering and form the rendered image data; The image processing module performs noise reduction on the image data to generate a high-resolution image.

2. The three-dimensional scene rendering method based on real-time ray tracing according to claim 1, characterized in that, The semantic category information includes object dynamic / static information and object density information. When the object is static and has low density, the rasterization rendering strategy is invoked to render the scene; otherwise, the ray tracing rendering strategy is invoked.

3. The three-dimensional scene rendering method based on real-time ray tracing according to claim 1, characterized in that, The semantic category information also includes lighting effect requirement information. When the lighting effect requirement of an object is low, the rasterization rendering strategy is invoked to render the scene; otherwise, the ray tracing rendering strategy is invoked.

4. The three-dimensional scene rendering method based on real-time ray tracing according to claim 1, characterized in that, The rendering strategy of the hybrid rendering module is called based on the calling rules in the rule base. The rendering simulation module is used to build test cases for different scenarios, simulate the rendering effect of the test cases, and use a machine learning model to determine the priority of the rendering effect and performance of the rasterization rendering strategy and the ray tracing rendering strategy in the corresponding scenario, so as to form the corresponding calling rules and store them in the rule base.

5. The three-dimensional scene rendering method based on real-time ray tracing according to claim 1, characterized in that, The ray tracing rendering strategy is based on the BVH algorithm. During the construction phase, the partitioning method of BVH nodes is dynamically adjusted according to the semantic category information and lighting information. Ray-triangle intersection tests are performed through GPU parallel computing.

6. The three-dimensional scene rendering method based on real-time ray tracing according to claim 1, characterized in that, The image processing module fuses the image data with multimodal data to form fused data, and inputs the fused data into the denoising model of the deep learning module for model training. After training, the denoising model performs denoising processing on the image data.

7. A three-dimensional scene rendering method based on real-time ray tracing according to claim 6, characterized in that, The multimodal data consists of the object's geometric information and material information from the semantic category information.

8. A three-dimensional scene rendering method based on real-time ray tracing according to claim 6, characterized in that, Based on the game's frame rate and image parameter requirements, the DLSS module of the image processing module dynamically adjusts the DLSS parameters to balance frame rate and image quality.

Images