Image rendering method, device, equipment and storage medium based on importance sampling

Abstract

The present application relates to the field of image rendering technology, and specifically discloses an image rendering method, device, equipment and storage medium based on importance sampling. Based on the scene file, the image to be rendered is preliminarily rendered to obtain the noise evaluation results of each area in the image to be rendered; based on the noise evaluation results, the sampling strategy of each area is determined; sampling is performed according to the sampling strategy to obtain sampling data, and image rendering is performed based on the sampling data to obtain the target rendered image. The present application can determine the sampling strategy of each area according to the noise evaluation results of each area in the image to be rendered, and sample each area according to the sampling strategy, so as to set the sampling strategy that meets the specific conditions of each area in the image to be rendered, avoid oversampling and insufficient sampling, and ensure the rendering quality. At the same time, the sampling strategy that is more adapted to the specific conditions of each area improves the sampling efficiency of each area, thereby improving the rendering efficiency.

Description

Technical Field

[0001] The present application relates to the field of image rendering technology, and in particular to an image rendering method, apparatus, device and storage medium based on importance sampling. Background Art

[0002] In the field of offline rendering, a large number of ray tracing calculations must be performed in order to generate high-quality images. This process usually involves simulating the propagation of light in the scene, including the interaction between light and objects, such as reflection, refraction, and scattering. However, this rendering method requires huge computing resources and time, especially for complex scenes. Existing rendering technologies have improved ray tracing calculations, which has reduced resource requirements and improved rendering efficiency, but the rendered images have high noise, resulting in low image quality. In order to reduce rendering noise while reducing resource costs and improving rendering efficiency to ensure image quality, existing rendering technologies use various sampling techniques to reduce rendering noise, such as the Monte Carlo method and its variants. These methods reduce noise by increasing the number of samples, but this also increases rendering time accordingly, resulting in low rendering efficiency.

[0003] Therefore, how to improve rendering efficiency while ensuring rendering quality has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides an image rendering method, apparatus, device and storage medium based on importance sampling to improve rendering efficiency while ensuring rendering quality.

[0005] In a first aspect, the present application provides an image rendering method based on importance sampling, the method comprising:

[0006] Based on the scene file, the image to be rendered is preliminarily rendered to obtain a noise evaluation result of each area in the image to be rendered;

[0007] Based on the noise assessment results, determining a sampling strategy for each area;

[0008] Sampling is performed according to the sampling strategy to obtain sampling data, and image rendering is performed based on the sampling data to obtain a target rendered image.

[0009] In a second aspect, the present application further provides an image rendering device based on importance sampling, the device comprising:

[0010] A noise evaluation result obtaining module is used to perform preliminary rendering on the image to be rendered based on the scene file, and obtain the noise evaluation result of each area in the image to be rendered;

[0011] A sampling priority determination module, used to determine the sampling strategy for each area based on the noise assessment result;

[0012] The target rendered image acquisition module is used to perform sampling according to the sampling strategy to obtain sampling data, and perform image rendering based on the sampling data to obtain a target rendered image.

[0013] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the image rendering method based on importance sampling as described above when executing the computer program.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image rendering method based on importance sampling as described above.

[0015] The present application discloses an image rendering method, apparatus, device and storage medium based on importance sampling. Based on a scene file, a preliminary rendering is performed on an image to be rendered to obtain a noise evaluation result of each region in the image to be rendered; based on the noise evaluation result, a sampling strategy for each region is determined; sampling is performed according to the sampling strategy to obtain sampling data, and image rendering is performed based on the sampling data to obtain a target rendered image. The present application can determine the sampling strategy for each region based on the noise evaluation result of each region in the image to be rendered, and sample each region according to the sampling strategy, thereby setting a sampling strategy that meets the specific conditions of each region in the image to be rendered, avoiding oversampling and insufficient sampling, and ensuring rendering quality. At the same time, a sampling strategy that is more adapted to the specific conditions of each region improves the sampling efficiency of each region, thereby improving rendering efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 is a schematic flow chart of an image rendering method based on importance sampling provided by the first embodiment of the present application;

[0018] Figure 2 is a schematic flow chart of an image rendering method based on importance sampling provided by the second embodiment of the present application;

[0019] Figure 3 is a schematic flow chart of an image rendering method based on importance sampling provided by the third embodiment of the present application;

[0020] Figure 4 A schematic block diagram of an image rendering device based on importance sampling provided in an embodiment of the present application;

[0021] Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0024] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0025] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0026] The embodiments of the present application provide an image rendering method, apparatus, device and storage medium based on importance sampling. Among them, the image rendering method based on importance sampling can be applied to a server, and the sampling strategy of each area is determined according to the noise evaluation results of each area in the image to be rendered, and each area is sampled according to the sampling strategy, so as to set a sampling strategy that meets the specific conditions of each area in the image to be rendered, avoid oversampling and insufficient sampling, and ensure the rendering quality. At the same time, the sampling strategy that is more adapted to the specific conditions of each area improves the sampling efficiency of each area, thereby improving the rendering efficiency. Among them, the server can be an independent server or a server cluster.

[0027] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0028] See also Figure 1 , Figure 1 This is a schematic flow chart of an image rendering method based on importance sampling provided by an embodiment of the present application. The image rendering method based on importance sampling can be applied to a server, and is used to determine the sampling strategy of each area according to the noise evaluation results of each area in the image to be rendered, and sample each area according to the sampling strategy, so as to set a sampling strategy that meets the specific conditions of each area in the image to be rendered, avoid oversampling and insufficient sampling, and ensure the rendering quality. At the same time, the sampling strategy that is more adapted to the specific conditions of each area improves the sampling efficiency of each area, thereby improving the rendering efficiency.

[0029] like Figure 1 As shown, the image rendering method based on importance sampling specifically includes steps S101 to S103.

[0030] S101, performing preliminary rendering on the image to be rendered based on the scene file to obtain a noise evaluation result of each area in the image to be rendered;

[0031] In one embodiment, the imported scene file is read and parsed to extract all necessary information in the scene, including geometric data of objects, material properties, light source settings, etc.

[0032] Use a standard rendering algorithm (such as ray tracing or rasterization) to perform preliminary rendering on the image to be rendered. Perform noise evaluation on the image obtained by the preliminary rendering, and analyze the noise level of each area in the image. Specifically, image processing techniques (such as calculating local variance) can be used to evaluate the noise in different areas.

[0033] Furthermore, based on the scene file, the image to be rendered is preliminarily rendered to obtain a noise evaluation result of each area in the image to be rendered, including: analyzing the scene file to obtain light source information and object information in the scene; performing preliminarily rendering based on the light source information and the object information to obtain a preliminarily rendered image; analyzing the noise level of each area of ​​the preliminarily rendered image, and based on preset image quality evaluation indicators, predicting the impact of the noise level of each area on the image quality, to obtain the noise evaluation result.

[0034] In one embodiment, the position, type (point light source, parallel light, spotlight, etc.), intensity, color, etc. of all light sources are extracted from the scene file. The geometric data, material properties (such as reflectivity, transparency, roughness) and spatial position of all objects are extracted from the scene file.

[0035] In one embodiment, rendering parameters set by the user are obtained, such as resolution, field of view, sampling rate, etc. Specifically, the rendering parameters can be collected from the user through a user interface.

[0036] The rendering engine is initialized with rendering parameters to prepare for ray tracing. Specifically, the Monte Carlo method is used for ray tracing, virtual rays are emitted from the camera, and intersection points are randomly selected for sampling. For each pixel, multiple ray paths are randomly generated to calculate the interaction between light and objects, such as reflection, refraction, and shadow.

[0037] For each sampling point, the color contribution is calculated based on the material and light source information of the object. The color contributions of all sampling points are accumulated to determine the preliminary color value of each pixel. The color values ​​of all pixels are combined into an image to obtain a preliminary rendered image.

[0038] In one embodiment, a statistical method (such as calculating a standard deviation) is used to quantify the noise level of each region in the preliminary rendered image, and the noise distribution and noise pattern of different regions in the image are analyzed to determine the source and characteristics of the noise. Specifically, the type of noise (such as Gaussian noise, salt and pepper noise, etc.) can be identified by observing the statistical distribution of the noise and analyzing the distribution pattern of the noise in the image.

[0039] The impact of noise level on image quality is predicted based on preset image quality assessment indicators (such as peak signal-to-noise ratio, structural similarity index, etc.). The noise assessment results of each area are obtained by combining the above noise analysis and quality assessment results.

[0040] S102, determining a sampling strategy for each area based on the noise evaluation result;

[0041] In one embodiment, the regions of the image to be rendered are classified according to the noise evaluation result, for example, the image regions are classified into high noise regions, medium noise regions and low noise regions.

[0042] The sampling density and sampling mode (such as random sampling, regular sampling or importance sampling) corresponding to each category of area are obtained to generate a sampling strategy for each area. The sampling density and sampling mode of different categories of areas can be pre-stored in a mapping table.

[0043] For example, for high-noise areas, a higher sampling density corresponds to reduce noise, and importance sampling or adaptive sampling technology can be used to concentrate more samples in key areas; for medium-noise areas, a moderate sampling density is maintained to balance image quality and rendering time; for low-noise areas, since low-noise areas have less impact on the final image quality, the sampling density can be reduced to save computing resources.

[0044] In another embodiment, a pre-trained sampling strategy generation model can also be used to determine the specific sampling values ​​of each area according to the noise evaluation results, and accurately generate a sampling strategy that meets the requirements of each area. The sampling strategy generation model is obtained by training a sampling strategy sample data set, and the sampling strategy sample data set includes at least one sampling strategy and a rendered image corresponding to the sampling strategy.

[0045] S103 . Perform sampling according to the sampling strategy to obtain sampling data, and perform image rendering based on the sampling data to obtain a target rendered image.

[0046] In one embodiment, sampling is performed according to the sampling strategy of each area, and data generated during the sampling process is collected to ensure the quality and integrity of the sampled data. The collected sampled data is used to apply a rendering algorithm to generate a target rendered image.

[0047] In one embodiment, the image is rendered using the sampled data obtained from the scene. During the rendering process, the image changes during the rendering process are monitored in real time, and the sampling strategy is dynamically adjusted according to the noise level and content change information of the image. For example, for areas with high noise levels or large content change information, the sampling density is increased to improve the image quality. For areas with low noise levels and small content change information, the sampling density is reduced to save computing resources.

[0048] Use the adjusted sampling strategy to resample the scene and obtain new sampling data. Re-render the image based on the new sampling data. After the image sampling and rendering are completed, the rendered image is obtained, and the image is denoised as required to obtain the target rendered image.

[0049] Furthermore, performing image rendering based on the sampling data to obtain a target rendered image includes: performing image rendering based on the sampling data to obtain an image to be denoised; and performing denoising processing on the image to be denoised based on a preset deep learning denoising algorithm to obtain the target rendered image.

[0050] In a specific embodiment, the sampling data of each region obtained from the scene is integrated, a preliminary rendering image is constructed using the integrated sampling data, and the image to be denoised obtained by rendering is output.

[0051] The image to be denoised is input into a deep learning denoising algorithm (such as a convolutional neural network or other types of neural networks). The deep learning denoising algorithm will automatically learn the noise pattern in the image and remove the noise to achieve denoising. The denoised image is output to obtain the target rendered image.

[0052] In another embodiment, after the deep learning denoising algorithm outputs an image, the denoising effect of the output image can be evaluated. When the evaluation result reaches a preset quality standard, the output image is used as the target rendering image. Otherwise, the image is input into the deep learning denoising algorithm for another denoising process to ensure that the image quality meets the preset standard.

[0053] Among them, the deep learning denoising algorithm is obtained by training a sample data set, and the sample data set contains at least one image that needs to be denoised and related information (such as color values, depth information, normal information, etc.) and the denoised image corresponding to the image.

[0054] In one embodiment, a bias correction technique is applied to adjust the denoised image to reduce visual bias. Specifically, the bias correction can be implemented by additional neural network layers, traditional image processing techniques (such as histogram equalization), or statistical methods.

[0055] The above-mentioned embodiments provide an image rendering method, apparatus, device and storage medium based on importance sampling, which performs preliminary rendering on the image to be rendered based on the scene file to obtain the noise evaluation results of each area in the image to be rendered; determines the sampling strategy of each area based on the noise evaluation results; performs sampling according to the sampling strategy to obtain sampling data, and performs image rendering based on the sampling data to obtain the target rendered image. The present application can determine the sampling strategy of each area based on the noise evaluation results of each area in the image to be rendered, and samples each area according to the sampling strategy, thereby setting a sampling strategy that meets the specific conditions of each area in the image to be rendered, avoiding oversampling and insufficient sampling, and ensuring the rendering quality. At the same time, the sampling strategy that is more adapted to the specific conditions of each area improves the sampling efficiency of each area, thereby improving the rendering efficiency.

[0056] See also Figure 2 , Figure 2 This is a schematic flow chart of an image rendering method based on importance sampling provided by an embodiment of the present application. The image rendering method based on importance sampling can be applied to a server to dynamically adjust the sampling strategy according to the image change data to optimize the rendering result, thereby improving the rendering efficiency and image quality.

[0057] like Figure 2 As shown, the image rendering method based on importance sampling specifically includes steps S201 to S202.

[0058] S201, performing image rendering based on the sampling data, and obtaining change data of the image to be rendered, analyzing the change data, and obtaining noise level change information and content change information of each area in the image to be rendered;

[0059] S202: Adjust the sampling strategy of each area based on the noise level change information and the content change information, and perform sampling and rendering based on the adjusted sampling strategy to obtain the target rendered image.

[0060] In one embodiment, image rendering is performed using sampling data obtained from a scene. During the rendering process, image changes in the rendering process are monitored in real time, and the sampling strategy is dynamically adjusted according to the noise level and content change information of the image.

[0061] In a specific embodiment, when rendering is performed, change data of the image to be rendered is collected, which may include changes in pixel color values, changes in local contrast of the image, etc. Specifically, image processing techniques may be used, such as calculating the difference between adjacent pixels or using a more complex feature extraction algorithm to obtain the change data.

[0062] For each area of ​​the image to be rendered, analyze the change in noise level. Identify the noise level change information of each area. Areas with greater noise level change information require more sampling to reduce noise. Analyze the changes in image content in each area, such as the movement of objects, changes in lighting conditions, or the appearance of new objects. Areas with greater content change information require more sampling to capture details.

[0063] In one embodiment, the sampling strategy of each region is adjusted according to the analysis results of the noise level change information and the content change information. For example, for regions with high noise levels or large content change information, the sampling density is increased to improve image quality. For regions with low noise levels and small content change information, the sampling density is reduced to save computing resources.

[0064] Use the adjusted sampling strategy to resample the scene and obtain new sampling data. Re-render the image based on the new sampling data. After the image sampling and rendering are completed, the rendered image is obtained, and the image is denoised as required to obtain the target rendered image.

[0065] Furthermore, the image rendering is performed based on the sampling data, and change data of the image to be rendered is obtained, and the change data is analyzed to obtain noise level change information and content change information of each area in the image to be rendered, including: image rendering is performed based on the sampling data, and at least two consecutive image frames are obtained according to a preset period; image detection is performed on the consecutive image frames to obtain the change data, wherein the change data includes pixel color value change data and feature change data; the pixel color value change data is analyzed to obtain the noise level change information, and the feature change data is analyzed to obtain the content change information.

[0066] In one embodiment, during the image rendering process, at least two consecutive image frames are acquired according to a preset period (eg, every certain number of frames or time interval). The consecutive image frames represent a continuous state during the rendering process.

[0067] The acquired consecutive image frames are compared pixel by pixel to detect and record the changes in pixel color values. Specifically, image processing techniques, such as difference methods, can be used to identify the difference between two frames.

[0068] Detect feature change data in the image, such as edges, textures, corners, etc. Specifically, feature detection algorithms such as Scale Invariant Feature Transform (SIFT), SURF (Speeded Up Robust Features), or ORB (Oriented FAST and Rotated BRIEF, a feature point extraction and description algorithm) can be used to identify and track key feature points in the image.

[0069] Analyze the changes of feature points in the image, including the movement of feature points, and the addition or disappearance of feature points.

[0070] In one embodiment, a statistical method (such as calculating a standard deviation) is used to quantify the change in pixel color values ​​to evaluate the noise level. The noise levels between consecutive frames are compared to identify noise level change information of each region.

[0071] Analyze the changes in feature points and identify changes in the image content of each area, such as the movement of objects, changes in lighting conditions, or the appearance of new objects.

[0072] In the above embodiment, the sampling strategy can be dynamically adjusted according to the change data of the image to optimize the rendering result, thereby improving the rendering efficiency and image quality.

[0073] See also Figure 3 , Figure 3This is a schematic flow chart of an image rendering method based on importance sampling provided by an embodiment of the present application. The image rendering method based on importance sampling can be applied to a server to dynamically adjust resources based on task complexity and computing node load status, and can realize parallel processing of each subtask, ensuring that rendering tasks are completed efficiently with limited computing resources, thereby improving resource utilization and rendering speed.

[0074] like Figure 3 As shown, the image rendering method based on importance sampling specifically includes steps S301 to S304.

[0075] S301, obtaining the corresponding scene file and task description information in the rendering task;

[0076] S302, analyzing the task description information to obtain rendering requirements, and analyzing the scene file to obtain object information and light source information in the scene;

[0077] S303, analyzing the rendering requirement, the object information, and the light source information to determine the task complexity of each area of ​​the image to be rendered;

[0078] S304: Adjust computing resources based on the task complexity of each area, so as to perform sampling and rendering based on the computing resources.

[0079] In one embodiment, when a rendering task is received, the scene file related to the rendering task and the task description information of the rendering task are read. The task description information may include rendering resolution, output format, specific rendering effect requirements, etc.

[0080] In one embodiment, the task description information is analyzed to extract specific rendering requirements, such as required image quality, rendering speed, special effects, etc. The scene file is analyzed to extract object information, including geometric shape, material properties, position, etc.; light source information is extracted, including light source type, position, intensity, and color, etc.

[0081] Based on rendering requirements, object information, and light source information, the scene is divided to obtain at least one area, and the task complexity of each area is evaluated based on factors such as the number of objects in each area, the complexity of materials, the number and type of light sources, etc.

[0082] In one embodiment, initial computing resources are allocated to each region according to the complexity of the task. For regions with high complexity, more computing resources can be allocated. For regions with low complexity, fewer computing resources can be allocated. The specific situation of the initial computing resources can be found in a pre-generated resource mapping table according to the task complexity. The resource mapping table can be set by the user based on historical experience, or the initial computing resources corresponding to each task complexity can be determined by a resource demand model. Among them, the resource demand model is obtained by the task complexity of historical tasks and computing resource training.

[0083] In one embodiment, during the processing of the rendering task, resource allocation is adjusted according to the real-time resource usage and task progress, and sampling and rendering are performed based on the adjusted computing resources.

[0084] Furthermore, adjusting computing resources based on the task complexity includes: decomposing the rendering task into at least one rendering subtask based on the task complexity of each area, and obtaining resource requirements of each rendering subtask; allocating each rendering subtask to a target computing node based on the load status of each computing node and the resource requirements of each rendering subtask.

[0085] In one embodiment, the entire rendering task is decomposed into multiple subtasks according to the task complexity of each area. Each subtask contains a part of the scene and can be rendered independently. For example, each area corresponds to a subtask or multiple areas are combined to generate a subtask, or a complex scene can be divided into different subtasks such as foreground, middle ground and background, or divided according to geographical location, object grouping, etc.

[0086] Perform resource requirement analysis on each subtask to determine the computing resources required to complete each subtask, including CPU (Central Processing Unit), GPU (Graphics Processing Unit), memory and storage, etc. Specifically, analyze the characteristics of the subtask, such as material complexity, number of light sources, scene details, etc., comprehensively analyze the task complexity of the subtask based on all the characteristics of the subtask, and determine the resource requirements of the subtask based on the resource requirement model.

[0087] In one embodiment, the load status of each computing node is monitored in real time, including CPU and GPU usage, memory usage, storage I / O (Input / Output) performance, etc. The computing node with the lowest current load is identified. A resource matching algorithm, such as minimum load allocation or resource priority allocation, is used to allocate the rendering subtask to the most suitable computing node. The algorithm will combine the resource requirements of the subtask and the current status of the computing node to achieve the optimal allocation.

[0088] In one embodiment, a task scheduling strategy, such as round-robin, priority scheduling, or dynamic scheduling, is implemented to ensure that all subtasks can be processed in a timely manner.

[0089] In one embodiment, each rendering subtask is assigned to one or more target computing nodes to ensure that the assigned tasks match the computing capabilities of the nodes, avoid resource waste, achieve load balancing, and avoid overloading some nodes while idling other node resources.

[0090] In the above embodiment, resources can be dynamically adjusted based on task complexity and computing node load status, and parallel processing of subtasks can be achieved to ensure that rendering tasks are completed efficiently with limited computing resources, thereby improving resource utilization and rendering speed.

[0091] See also Figure 4 , Figure 4 The embodiment of the present application provides a schematic block diagram of an image rendering device based on importance sampling, and the image rendering device based on importance sampling is used to execute the aforementioned image rendering method based on importance sampling. The image rendering device based on importance sampling can be configured on a server.

[0092] like Figure 4 As shown, the image rendering device 400 based on importance sampling includes:

[0093] A noise evaluation result obtaining module 401 is used to perform preliminary rendering on the image to be rendered based on the scene file, and obtain noise evaluation results of each area in the image to be rendered;

[0094] A sampling priority determination module 402, for determining a sampling strategy for each area based on the noise assessment result;

[0095] The target rendered image obtaining module 403 is used to perform sampling according to the sampling strategy to obtain sampling data, and perform image rendering based on the sampling data to obtain a target rendered image.

[0096] Furthermore, the target rendered image obtaining module 403 includes:

[0097] a change data analysis unit, configured to perform image rendering based on the sampling data, obtain change data of the image to be rendered, analyze the change data, and obtain noise level change information and content change information of each area in the image to be rendered;

[0098] The sampling strategy adjustment unit is used to adjust the sampling strategy of each area based on the noise level change information and the content change information, and perform sampling and rendering based on the adjusted sampling strategy to obtain the target rendered image.

[0099] Furthermore, the change data analysis unit includes:

[0100] An image frame acquisition subunit, configured to perform image rendering based on the sampled data and acquire at least two consecutive image frames according to a preset period;

[0101] a change data acquisition subunit, configured to perform image detection on the continuous image frames to obtain the change data, wherein the change data includes pixel color value change data and feature change data;

[0102] The change data analysis subunit is used to analyze the pixel color value change data to obtain the noise level change information, and to analyze the feature change data to obtain the content change information.

[0103] Furthermore, the image rendering device 400 based on importance sampling further includes a computing resource adjustment module, and the computing resource adjustment module includes:

[0104] An information acquisition unit, used to acquire the corresponding scene file and task description information in the rendering task;

[0105] A rendering requirement obtaining unit, used to analyze the task description information to obtain rendering requirements, and to analyze the scene file to obtain object information and light source information in the scene;

[0106] A task complexity determination unit, used to analyze the rendering requirement, the object information and the light source information, and determine the task complexity of each area of ​​the image to be rendered;

[0107] The computing resource adjustment unit is used to adjust the computing resources based on the task complexity of each area, so as to perform sampling and rendering based on the computing resources.

[0108] Furthermore, the computing resource adjustment unit includes:

[0109] A task decomposition subunit, used to decompose the rendering task into at least one rendering subtask based on the task complexity of each area, and obtain resource requirements of each rendering subtask;

[0110] The subtask allocation subunit is used to allocate each rendering subtask to a target computing node based on the resource status of each computing node and the resource requirement of each rendering subtask.

[0111] Furthermore, the noise assessment result obtaining module 401 includes:

[0112] A scene file analysis unit, used to analyze the scene file to obtain light source information and object information in the scene;

[0113] A preliminary rendering image obtaining unit, configured to perform preliminary rendering based on the light source information and the object information to obtain a preliminary rendering image;

[0114] The noise evaluation result obtaining unit is used to analyze the noise level of each area of ​​the preliminary rendered image, and based on a preset image quality evaluation index, predict the impact of the noise level of each area on the image quality to obtain the noise evaluation result.

[0115] Furthermore, the target rendered image obtaining module 403 includes:

[0116] A unit for obtaining an image to be denoised, used for performing image rendering based on the sampling data to obtain an image to be denoised;

[0117] The image denoising processing unit is used to perform denoising on the image to be denoised based on a preset deep learning denoising algorithm to obtain the target rendered image.

[0118] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0119] The above-mentioned device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.

[0120] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0121] See also Figure 5 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0122] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any image rendering method based on importance sampling.

[0123] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0124] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any image rendering method based on importance sampling.

[0125] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0126] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0127] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0128] Based on the scene file, the image to be rendered is preliminarily rendered to obtain a noise evaluation result of each area in the image to be rendered;

[0129] Based on the noise assessment results, determining a sampling strategy for each area;

[0130] Sampling is performed according to the sampling strategy to obtain sampling data, and image rendering is performed based on the sampling data to obtain a target rendered image.

[0131] In one embodiment, when the processor performs sampling according to the sampling strategy to obtain sampling data, and performs image rendering based on the sampling data to obtain a target rendered image, it is configured to implement:

[0132] Performing image rendering based on the sampling data, acquiring change data of the image to be rendered, analyzing the change data, and obtaining noise level change information and content change information of each area in the image to be rendered;

[0133] Based on the noise level change information and the content change information, the sampling strategy of each area is adjusted, and sampling and rendering are performed based on the adjusted sampling strategy to obtain the target rendered image.

[0134] In one embodiment, when the processor performs image rendering based on the sampling data, obtains change data of the image to be rendered, analyzes the change data, and obtains noise level change information and content change information of each area in the image to be rendered, it is used to implement:

[0135] Performing image rendering based on the sampled data, and acquiring at least two consecutive image frames according to a preset period;

[0136] Performing image detection on the continuous image frames to obtain the change data, wherein the change data includes pixel color value change data and feature change data;

[0137] The pixel color value change data is analyzed to obtain the noise level change information, and the feature change data is analyzed to obtain the content change information.

[0138] In one embodiment, before the processor implements sampling according to the sampling strategy to obtain sampling data, and performs image rendering based on the sampling data to obtain a target rendered image, it is further configured to implement:

[0139] Get the corresponding scene file and task description information in the rendering task;

[0140] Analyze the task description information to obtain rendering requirements, and analyze the scene file to obtain object information and light source information in the scene;

[0141] Analyzing the rendering requirements, the object information, and the light source information to determine the task complexity of each area of ​​the image to be rendered;

[0142] Based on the task complexity of each region, the computing resources are adjusted to perform sampling and rendering based on the computing resources.

[0143] In one embodiment, when adjusting computing resources based on the task complexity, the processor is used to implement:

[0144] Based on the task complexity of each area, decompose the rendering task into at least one rendering subtask, and obtain resource requirements of each rendering subtask;

[0145] Based on the resource status of each computing node and the resource requirements of each rendering subtask, each rendering subtask is allocated to a target computing node.

[0146] In one embodiment, when the processor performs preliminary rendering of the image to be rendered based on the scene file and obtains the noise evaluation result of each area in the image to be rendered, it is used to implement:

[0147] Analyze the scene file to obtain light source information and object information in the scene;

[0148] Perform preliminary rendering based on the light source information and the object information to obtain a preliminary rendered image;

[0149] The noise level of each area of ​​the preliminary rendered image is analyzed, and based on a preset image quality evaluation index, the influence of the noise level of each area on the image quality is predicted to obtain the noise evaluation result.

[0150] In one embodiment, when the processor performs image rendering based on the sampling data to obtain a target rendered image, the processor is used to implement:

[0151] Perform image rendering based on the sampling data to obtain an image to be denoised;

[0152] Based on a preset deep learning denoising algorithm, denoising is performed on the image to be denoised to obtain the target rendered image.

[0153] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the importance sampling-based image rendering methods provided in the embodiments of the present application.

[0154] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Medi a Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc., equipped on the computer device.

[0155] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An image rendering method based on importance sampling, characterized in that: include: Based on the scene file, the image to be rendered is preliminarily rendered to obtain a noise evaluation result of each area in the image to be rendered; Based on the noise assessment results, determining a sampling strategy for each area; Sampling is performed according to the sampling strategy to obtain sampling data, and image rendering is performed based on the sampling data to obtain a target rendered image.

2. The image rendering method based on importance sampling according to claim 1, characterized in that: The sampling is performed according to the sampling strategy to obtain sampling data, and image rendering is performed based on the sampling data to obtain a target rendered image, including: Performing image rendering based on the sampling data, acquiring change data of the image to be rendered, analyzing the change data, and obtaining noise level change information and content change information of each area in the image to be rendered; Based on the noise level change information and the content change information, the sampling strategy of each area is adjusted, and sampling and rendering are performed based on the adjusted sampling strategy to obtain the target rendered image.

3. The image rendering method based on importance sampling according to claim 2, characterized in that: The performing image rendering based on the sampling data, obtaining change data of the image to be rendered, analyzing the change data, and obtaining noise level change information and content change information of each area in the image to be rendered includes: Performing image rendering based on the sampled data, and acquiring at least two consecutive image frames according to a preset period; Performing image detection on the continuous image frames to obtain the change data, wherein the change data includes pixel color value change data and feature change data; The pixel color value change data is analyzed to obtain the noise level change information, and the feature change data is analyzed to obtain the content change information.

4. The image rendering method based on importance sampling according to claim 1, characterized in that: The step of sampling according to the sampling strategy to obtain sampling data, and performing image rendering based on the sampling data to obtain a target rendered image further includes: Get the corresponding scene file and task description information in the rendering task; Analyze the task description information to obtain rendering requirements, and analyze the scene file to obtain object information and light source information in the scene; Analyzing the rendering requirements, the object information, and the light source information to determine the task complexity of each area of ​​the image to be rendered; Based on the task complexity of each region, the computing resources are adjusted to perform sampling and rendering based on the computing resources.

5. The image rendering method based on importance sampling according to claim 4, characterized in that: The adjusting computing resources based on the task complexity includes: Based on the task complexity of each area, decompose the rendering task into at least one rendering subtask, and obtain resource requirements of each rendering subtask; Based on the resource status of each computing node and the resource requirements of each rendering subtask, each rendering subtask is allocated to a target computing node.

6. The image rendering method based on importance sampling according to claim 1, characterized in that: The performing preliminary rendering of the image to be rendered based on the scene file to obtain the noise evaluation result of each area in the image to be rendered includes: Analyze the scene file to obtain light source information and object information in the scene; Perform preliminary rendering based on the light source information and the object information to obtain a preliminary rendered image; The noise level of each area of ​​the preliminary rendered image is analyzed, and based on a preset image quality evaluation index, the influence of the noise level of each area on the image quality is predicted to obtain the noise evaluation result.

7. The image rendering method based on importance sampling according to any one of claims 1 to 6, characterized in that: The performing image rendering based on the sampling data to obtain a target rendered image includes: Perform image rendering based on the sampling data to obtain an image to be denoised; Based on a preset deep learning denoising algorithm, denoising is performed on the image to be denoised to obtain the target rendered image.

8. An image rendering device based on importance sampling, characterized in that: include: A noise evaluation result obtaining module is used to perform preliminary rendering on the image to be rendered based on the scene file, and obtain the noise evaluation result of each area in the image to be rendered; A sampling priority determination module, used to determine the sampling strategy for each area based on the noise assessment result; The target rendered image acquisition module is used to perform sampling according to the sampling strategy to obtain sampling data, and perform image rendering based on the sampling data to obtain a target rendered image.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the image rendering method based on importance sampling according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image rendering method based on importance sampling according to any one of claims 1 to 7.

Images