Distributed rendering processing method, system, device and equipment and storage medium

Abstract

The invention provides a distributed rendering processing method, system and device, equipment and a storage medium. The method comprises the following steps: determining N sub-rendering tasks obtained by splitting a target rendering task; wherein N is an integer greater than or equal to 2; calling M1 rendering nodes, and rendering the N sub-rendering tasks in parallel by using the M1 rendering nodes to obtain N sub-rendering results; m1 is an integer smaller than or equal to N; calling M2 noise reduction nodes, and using the M2 noise reduction nodes to carry out noise reduction on the N sub rendering results in parallel to obtain N noise reduction processing results; m2 is an integer smaller than or equal to M1; and obtaining a target rendering result for the target rendering task based on the N noise reduction processing results.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a distributed rendering processing method, apparatus, system, device and storage medium. Background Art

[0002] In the field of computer graphics, in order to improve the rendering solution quality of offline ray tracing, a combination of a renderer (mainly used for rendering) + a denoiser (mainly used for denoising) can be used. Here, both the renderer and the denoiser can be implemented using a central processing unit (CPU) or a graphics processing unit (GPU); the technical solution based on the CPU is mature and suitable for processing complex scenes, and can complete accurate global illumination calculations; while the technical solution based on the GPU developed later, thanks to the GPU having more parallel processing units, so the GPU is faster than the CPU in solving, but is limited by the processing unit's capabilities, and the calculation accuracy is limited, and the GPU's video memory space is limited, for example, the space of the video random access memory (VRAM) is limited, making it difficult to cope with complex scenes with large amounts of data; in addition, the hardware cost of the GPU is much higher than that of the CPU. Summary of the invention

[0003] The present disclosure provides a distributed rendering processing method, system, apparatus, device and storage medium to solve or alleviate one or more technical problems in the prior art.

[0004] In a first aspect, the present disclosure provides a distributed rendering processing method, comprising: Determine N sub-rendering tasks obtained by splitting the target rendering task; where N is an integer greater than or equal to 2; Calling M1 rendering nodes to render the N sub-rendering tasks in parallel using the M1 rendering nodes to obtain N sub-rendering results; M1 is an integer less than or equal to N; Calling M2 denoising nodes to perform denoising on the N sub-rendering results in parallel using the M2 denoising nodes to obtain N denoising processing results; M2 is an integer less than or equal to M1; A target rendering result for the target rendering task is obtained based on the N denoising processing results.

[0005] In a second aspect, the present disclosure provides a distributed rendering processing system, including: A scheduling node, used to determine N sub-rendering tasks obtained after splitting the target rendering task; wherein N is an integer greater than or equal to 2; and used to call M1 rendering nodes and M2 denoising nodes; M1 is an integer less than or equal to N; M2 is an integer less than or equal to M1; M1 rendering nodes, used to render the N sub-rendering tasks in parallel to obtain N sub-rendering results; M2 denoising nodes, used to perform denoising on the N sub-rendering results in parallel to obtain N denoising processing results; The rendering nodes among the M1 rendering nodes are further used to obtain a target rendering result for the target rendering task based on the N noise reduction processing results.

[0006] In a third aspect, the present disclosure provides a distributed rendering processing device, including: A splitting module, used to determine N sub-rendering tasks obtained by splitting the target rendering task; wherein N is an integer greater than or equal to 2; A processing module is used to call M1 rendering nodes to use the M1 rendering nodes to render the N sub-rendering tasks in parallel to obtain N sub-rendering results; M1 is an integer less than or equal to N; call M2 denoising nodes to use the M2 denoising nodes to perform denoising on the N sub-rendering results in parallel to obtain N denoising processing results; M2 is an integer less than or equal to M1; and obtain a target rendering result for the target rendering task based on the N denoising processing results.

[0007] In a fourth aspect, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0008] In a fifth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute any method according to the embodiments of the present disclosure.

[0009] In a sixth aspect, a computer program product is provided, comprising a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0010] The beneficial effects of the technical solution provided by the present disclosure include at least: The disclosed solution provides a new distributed heterogeneous rendering-denoising architecture, which can be deployed in a high-performance computing cluster. In this way, image rendering and denoising processing can be efficiently implemented and can be adapted to any rendering workflow. At the same time, it also effectively improves the throughput of the entire system.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments provided according to the present disclosure and should not be regarded as limiting the scope of the present disclosure.

[0013] Figure 1 is a structural diagram of a distributed rendering processing system according to an embodiment of the present application; Figure 2 This is a schematic flow chart of a distributed rendering processing method according to an embodiment of the present application. Figure 1 ; Figure 3 This is a schematic diagram of a distributed rendering processing method in a specific example according to an embodiment of the present application. Figure 1 ; Figure 4 This is a schematic diagram of a distributed rendering processing method in a specific example according to an embodiment of the present application. Figure 2 ; Figure 5 is a schematic diagram of the segmentation effect of a target image according to an embodiment of the present application; Figure 6 is a comparison diagram of the effect of noise reduction processing of sub-images according to an embodiment of the present application; Figure 7 is a schematic diagram of the effect of a rendered image after the results are merged according to an embodiment of the present application; Figure 8 is a schematic diagram of the structure of a distributed rendering device according to an embodiment of the present application; Fig. 9 It is a block diagram of an electronic device used to implement the distributed rendering processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0014] The present disclosure will be further described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0015] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0016] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0017] In the field of computer graphics, in order to improve the rendering solution quality of offline ray tracing, a combination of a renderer (mainly used for rendering) + a denoiser (mainly used for denoising) can be used. Here, both the renderer and the denoiser can be implemented using a CPU or a GPU; the technical solution based on the CPU is mature and suitable for processing complex scenes, and can complete accurate global illumination calculations; while the technical solution based on the GPU developed later, thanks to the GPU having more parallel processing units, so the GPU is faster than the CPU in solving, but is limited by the processing unit's capabilities, and the calculation accuracy is limited, and the GPU's video memory space is limited, for example, the space of the Video Random Access Memory (VRAM) is limited, making it difficult to cope with complex scenes with large amounts of data; in addition, the hardware cost of the GPU is much higher than that of the CPU.

[0018] In order to take into account both the rendering quality and the speed of rendering delivery, in one example, a heterogeneous combination device can also be used, namely a CPU renderer (i.e., a renderer implemented by a CPU) + a GPU denoiser (i.e., a denoiser implemented by a GPU). Here, the CPU renderer can make full use of computer memory to complete the precise calculation of complex scenes; while the GPU denoiser can load a pre-trained deep learning model, use the results of the CPU renderer as algorithm input, and quickly solve. For example, in one example, the workflow can be specifically as follows: input scene data to the CPU renderer, calculate and solve to obtain a set of rendering channel components, such as RGB channels, diffuse channels (diffuse), refraction channels (refraction), reflection channels (reflection) and specular reflection channels (specular), etc.; further, the obtained channel components are used as input to the deep learning model to infer the denoised image after the synthesis channel.

[0019] If the above heterogeneous combination device is installed and deployed on a single machine, the following problems will occur: In the scenario of completing batch rendering tasks, all computers need to be installed with high-performance CPUs and at least one GPU, which results in high hardware costs for the entire machine. Moreover, due to the serial structure of the workflow, the utilization rate of the GPU will also be reduced. For example, during the calculation and solution phase of the CPU renderer, the GPU is in an idle state. At this time, the overall throughput of the system is low.

[0020] Based on this, in order to solve the above problems, the disclosed solution provides a distributed rendering processing method, which can efficiently implement image rendering and noise reduction processing through a heterogeneous acceleration architecture, thereby effectively improving the throughput of the entire system.

[0021] The disclosed solution provides a distributed rendering processing system, such as Figure 1 As shown, the system specifically includes: The scheduling node 101 is used to determine N sub-rendering tasks obtained by splitting the target rendering task; wherein N is an integer greater than or equal to 2; and to call M1 rendering nodes and M2 denoising nodes; M1 is an integer less than or equal to N; and M2 is an integer less than or equal to M1; M1 rendering nodes 102, used to render the N sub-rendering tasks in parallel to obtain N sub-rendering results; M2 denoising nodes 103, used to perform denoising on the N sub-rendering results in parallel to obtain N denoising processing results; The rendering nodes among the M1 rendering nodes are further used to obtain a target rendering result for the target rendering task based on the N noise reduction processing results.

[0022] For example, in one example, the scheduling node can be mainly used to receive rendering requests from outside the system (for requesting to execute a target rendering task), and perform segmentation of the target rendering task, schedule rendering nodes or denoising nodes, and return results to the outside.

[0023] Furthermore, the rendering node can be mainly used to receive the sub-rendering tasks assigned by the scheduling node, and after completing the rendering, initiate a noise reduction request to the scheduling node; further, it can also be used to merge the final results through the coordination of the scheduling node to obtain the target rendering result, and return it to the scheduling node.

[0024] Furthermore, the denoising node is mainly used to receive the denoising task assigned by the scheduling node, and after completing the denoising process, return the denoising result to the rendering node.

[0025] That is to say, the disclosed solution provides a new distributed heterogeneous rendering-denoising architecture, which can be deployed in a high-performance computing cluster. For example, there are a certain number of GPU servers and a certain number of CPU servers in the cluster. In this way, it can efficiently adapt to the image rendering workflow, and at the same time, it can also efficiently adapt to the video (multi-frame image) rendering workflow.

[0026] Specifically, Figure 2 This is a schematic flow chart of a distributed rendering processing method according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0027] Further, the method includes at least part of the following contents. Figure 2 As shown, including: Step S201: Determine, through a scheduling node, N sub-rendering tasks obtained by splitting a target rendering task.

[0028] Here, N is an integer greater than or equal to 2.

[0029] Step S202: calling M1 rendering nodes through the scheduling node, so as to use the M1 rendering nodes to render the N sub-rendering tasks in parallel, and obtain N sub-rendering results.

[0030] Here, M1 is an integer less than or equal to N.

[0031] Furthermore, in one example, the number of rendering nodes that need to be called is related to the value of N, for example, M1 is equal to N.

[0032] Furthermore, in another example, the value of N can also be determined based on the number of currently available rendering nodes. In this case, N can be equal to M1. In this case, one sub-rendering task can correspond to one rendering node and one denoising node, thus laying the foundation for quickly completing the rendering task.

[0033] It is understandable that, in actual applications, the value of N can also be determined based on the total task volume of the target rendering task, or N can be a preset empirical value, etc. The disclosed solution does not impose specific limitations on this.

[0034] Step S203: calling M2 denoising nodes through the scheduling node, so as to utilize the M2 denoising nodes to perform denoising on the N sub-rendering results in parallel, and obtain N denoising processing results.

[0035] Here, M2 is an integer less than or equal to M1.

[0036] Further, in one example, the number of denoising nodes to be called is related to at least one of the following: the value of N, the number of rendering nodes. For example, in one example, M2 is equal to M1. Or, in another example, the values ​​of N, M1 and M2 are all equal.

[0037] Or, further, in another example, the number of denoising nodes that need to be called may also be related to the number of denoising nodes currently available in the rendering system, or the number of denoising nodes that can currently perform denoising tasks.

[0038] Step S204: Obtain a target rendering result for the target rendering task based on the N denoising results. For example, in one example, the scheduling node schedules a rendering node among the M1 rendering nodes to merge the results, that is, to obtain N denoising results, and obtain the target rendering result for the target rendering task based on the N denoising results.

[0039] That is to say, the disclosed solution provides a distributed rendering system that can call multiple rendering nodes and multiple denoising nodes. In this way, image rendering and denoising processing can be efficiently implemented through a heterogeneous acceleration architecture, thereby effectively improving the throughput of the entire system and reducing the rendering cost.

[0040] Here, in a specific example, the rendering system includes a plurality of processing nodes, and accordingly, the M1 rendering nodes and the M2 denoising nodes are at least part of the plurality of processing nodes included in the rendering system.

[0041] Further, in a specific example, the plurality of processing nodes include at least a plurality of CPU nodes and a plurality of GPU nodes. Further, the M1 rendering nodes are at least some of the plurality of CPU nodes and the plurality of GPU nodes; that is, in this example, some rendering nodes are GPU nodes, and some rendering nodes are CPU nodes.

[0042] Further, in another example, the M2 denoising nodes are at least some nodes in a plurality of GPUs. That is, in this example, the denoising nodes are GPU nodes.

[0043] Here, it should be noted that the CPU node mentioned above may specifically be a CPU server, and correspondingly, the GPU node may specifically be a GPU server.

[0044] Further, in a specific example, different task splitting (or segmentation) methods may be used for different rendering tasks, so as to facilitate subsequent rapid image rendering and noise reduction processing; specifically, the above step S201 (i.e., determining the N sub-rendering tasks obtained after splitting the target rendering task) may specifically include: Step S201 - 1 : when the target rendering task is an image rendering task, based on the image features of the target image targeted by the image rendering task, split the target image into N sub-images to obtain N sub-rendering tasks. For example, each sub-image corresponds to one sub-rendering task.

[0045] or, Step S201 - 2: when the target rendering task is a video rendering task, the video frame sequence is split into N subsequences based on the number of frames of the video frame sequence targeted by the video rendering task to obtain N sub-rendering tasks. For example, each subsequence corresponds to a sub-rendering task.

[0046] It should be noted that in the disclosed solution, in order to speed up the sub-workflow of denoising processing, denoising processing can be directly performed on the subspace rendering result (that is, the sub-rendering result described above). Furthermore, directly executing the denoising algorithm on the segmented subspace may cause the obtained result to be prone to boundary noise, affecting the final stitching effect. Based on this, in a specific example, before using M1 rendering nodes to render the N sub-rendering tasks in parallel, the rendering nodes can be used to perform boundary compensation processing on the sub-rendering tasks that they need to process, so as to effectively avoid boundary noise.

[0047] Further, in a specific example, the above-mentioned using the rendering node to perform boundary compensation processing on the sub-rendering task that needs to be processed by itself may specifically include: When the target rendering task is an image rendering task, the rendering node can be used to perform height compensation and / or width compensation on the boundary area of ​​the sub-image based on the image features of the sub-image targeted by the sub-rendering task to be processed. Here, the sub-image is a partial area in the target image targeted by the image rendering task, for example, multiple sub-images are obtained after the target image is segmented.

[0048] or, When the target rendering task is a video rendering task, the rendering node can be used to perform frame sequence compensation on the subsequence based on the number of frames of the subsequence targeted by the sub-rendering task to be processed by itself, where the subsequence is a subsequence in the video frame sequence targeted by the video rendering task. For example, a video frame sequence is segmented to obtain multiple subsequences.

[0049] Here, in a specific example, for an image rendering task, the above-mentioned height compensation and / or width compensation for the boundary area of ​​the sub-image may specifically include: The sub-image is converted based on the target perspective, wherein the target perspective is related to the acquisition perspective of the target image, for example, the target perspective is the camera perspective, so that the problem of reduced rendering effect due to perspective offset can be effectively avoided. Furthermore, the boundary area of ​​the sub-image after the perspective conversion is height compensated and / or width compensated.

[0050] Here, in another specific example, for a video rendering task, the frame sequence compensation for the subsequence described above may specifically include: extending a preset number of video frames before and / or after the subsequence. It should be noted that the preset number may be determined based on actual needs, and the disclosed solution does not limit this.

[0051] Furthermore, in order to further improve the rendering effect, in one example, after obtaining N denoising results, the boundary compensation part in the denoising results may be removed.

[0052] Furthermore, the above-described obtaining the target rendering result for the target rendering task based on the N denoising results may specifically include: merging the N denoising results after removing the boundary compensation part to obtain the target rendering result for the target rendering task.

[0053] The present disclosure is further described in detail below with reference to specific examples, specifically: This example provides a new distributed heterogeneous rendering and denoising system, which can be deployed in a high-performance computing cluster. For example, the cluster has a certain number of GPU servers and a certain number of CPU servers. The system can adapt to both image rendering workflows and video (multi-frame image) rendering workflows.

[0054] Furthermore, in this example, the input of the workflow is a rendering request, and the output thereof is a target rendering result.

[0055] Here, the rendering request encapsulates the data entity of the "target rendering task", for example, including complete rendering scene data and constraint information of the rendering result.

[0056] For example, in one example, the rendering scene data includes unstructured data such as the 3D mesh model (Mesh) and textures in the 3D scene, as well as the transformation matrix of the model in the scene coordinate system; further, for video rendering tasks, the rendering scene data can be further divided, for example, into frame sequences. Furthermore, the rendering request can also include camera data in the rendering scene; further, for video rendering tasks, the camera data can also be stored independently in each frame.

[0057] In addition, in another example, the rendering request may also include constraint information of the rendering result, such as resolution (height and width pixel values), encoding method of the result, etc.; further, for video rendering, it may further include information such as frame rate and bit rate.

[0058] It should be noted that the above content related to rendering requests is only an exemplary description. In actual applications, it can also be set according to actual needs, and the present disclosure does not limit this.

[0059] Furthermore, the workflow of this example is deployed in a distributed computing cluster. Furthermore, according to the functional roles, the nodes in the distributed computing cluster can be divided into: 1) core nodes, also known as business function nodes, for example, they can include scheduling nodes, rendering nodes and noise reduction nodes; 2) auxiliary nodes, also known as infrastructure nodes, for example, they can include queue nodes and storage nodes.

[0060] Specifically, the functions of each node are briefly described below: The scheduling node can be mainly used to receive rendering requests from outside the system, perform segmentation of target rendering tasks, schedule rendering nodes or denoising nodes, and return results to the outside.

[0061] The rendering node can be mainly used to receive the sub-rendering tasks assigned by the scheduling node, and after completing the rendering, initiate a denoising request to the scheduling node; further, it can also be used to merge the final rendering results through the coordination of the scheduling node, for example, merging the denoising results obtained after the denoising nodes have been denoised to obtain the target rendering task, and returning it to the scheduling node so that the scheduling node can feed back the target rendering task to the external system.

[0062] The denoising node is mainly used to receive the denoising tasks assigned by the scheduling node, and after completing the denoising process, return the denoising results to the rendering node.

[0063] Here, in one example, the distributed computing cluster includes at least one scheduling node, N rendering nodes, and N denoising nodes. Furthermore, in one example, the rendering nodes correspond one to one with the scheduling nodes. For example, for the i-th denoising node, the scheduling node can schedule the i-th denoising node to perform denoising on the sub-rendering result (also referred to as the i-th sub-rendering result) obtained by the i-th rendering node; further, after the i-th denoising node performs denoising on the sub-rendering result of the i-th rendering node and obtains the denoising processing result (also referred to as the i-th denoising processing result), the obtained i-th denoising processing result can be transmitted back to the i-th rendering node.

[0064] Queue nodes are mainly used to carry the responsibilities of message queues and assist scheduling nodes in implementing rendering and noise reduction task queues; Storage nodes mainly constitute storage clusters, providing large-scale storage of unstructured data such as rendering models and textures. Furthermore, they can also support data storage of queue nodes.

[0065] Furthermore, the workflow of this example may include multiple processing units, which may be deployed in the above core nodes (business function nodes). Figure 3 As shown, it may specifically include a rendering request segmentation unit, a segmentation fault tolerance compensation unit, a rendering processing unit, a noise reduction processing unit and a result merging unit.

[0066] Furthermore, the following combination Figure 3 and Figure 4 The specific processing flow of each unit is described in detail: Rendering request splitting unit: deployed on the scheduling node. The rendering request splitting unit can split a target rendering task into several subtasks (corresponding to the above sub-rendering tasks); further, additional splitting metadata can be attached to each subtask.

[0067] Furthermore, in an example, the rendering request segmentation unit may adopt different segmentation strategies according to the type of rendering task (image rendering or video rendering): For image rendering tasks, the rendering area can be divided into rectangular segments according to the resolution (for example, the resolution of the target image to be rendered). For example, a set of segmentation values ​​is preset according to the image resolution; the rendering request segmentation unit divides the target image for the target rendering task into sub-images, that is Subtasks can be scheduled to a rendering node for rendering. Based on this, the subtasks can be numbered in the order of upper left to lower right to ensure that the target image can be obtained by splicing according to the numbers. Rendering nodes, at this time, the solution area of ​​each rendering node is the target image Furthermore, the target total rendering result can be finally obtained based on the number.

[0068] For video rendering tasks, the video frame sequence to be rendered can be divided into several subsequences according to the number of frames (for example, the number of frames in the video frame sequence to be rendered). For example, a set of segmentation values ​​is preset according to the video resolution; the rendering request segmentation unit divides the video frame sequence targeted by the target rendering task into N subsequences, that is, N subtasks, according to the preset values. Furthermore, in one example, each subtask can be scheduled to a rendering node for parallel rendering. Based on this, a total of N rendering nodes are scheduled. At this time, the number of rendering frames of each rendering node is times the length of the original frame sequence (the sequence length of the video frame sequence to be rendered). .

[0069] It should be noted that if integer division cannot be achieved, the remainder can be appended to the last subtask. For example, for a video rendering task with an original frame sequence length of L, the subsequence length of the first N-1 subtasks is L DIVN', and the subsequence length of the Nth subtask is L DIV N + L MOD N; finally, the video frame sequence can be obtained by splicing according to the frame sequence number.

[0070] Segmentation fault-tolerant compensation unit: deployed on the rendering node. After being processed by the aforementioned units, the subtasks dispatched to each rendering node result in a subspace of the target solution (image rendering task: sub-pixel interval; video rendering task: sub-sequence interval). The amount of computation required by the renderer to solve the problem has been reduced, which is a parallel acceleration of the rendering sub-workflow. In order to further accelerate the denoising sub-workflow, denoising will be performed directly on the subspace rendering result. However, when the denoising algorithm is executed on the segmented subspace, the result obtained is prone to boundary noise, which affects the final stitching effect (image: visible segmentation boundary; video: visible flickering frame). This unit executes the following fault-tolerant compensation strategies based on the rendering task type: For the regions segmented by the image rendering task, region padding is performed. For example, for each sub-image, the offset and the viewing cone of the scene camera of the sub-image can be modified based on the viewing angle of the center point of the target image.

[0071] For example, in one example, the transformation matrix may be: Here, W refers to the width of the target image to be rendered; H refers to the height of the target image to be rendered.

[0072] For example, in one example, the pixel matrix of the sub-image may be multiplied by the above transformation matrix to obtain the sub-image after the perspective transformation. Furthermore, the sub-image after the perspective transformation may be regionally compensated.

[0073] Furthermore, let the resolution of the target image be pixels, the segmentation interval of each subtask (that is, the segmentation interval of each sub-image) is Pixel.

[0074] Furthermore, the height compensation and width compensation of the subtask are and , then according to the resolution of the subtask (that is, the resolution of the sub-image), the actual rendering width of the subtask (that is, the actual rendering width of the sub-image) is pixels, accordingly, the actual rendering height of the subtask (that is, the actual rendering height of the sub-image) is Pixel.

[0075] here, The value range is . Further, in one example, the value may be For example, in one example, the value is 2.

[0076] Accordingly, The value range is . Further, in one example, the value may be For example, in one example, the value is 2.

[0077] For the subsequences obtained by the video rendering task, frame padding is performed. For example, for each subsequence, n frames are extended forward and backward respectively as the actual rendering subsequence of the current rendering node. It should be noted that, for the first and last subsequences, only one direction of expansion is required. For example, for the first subsequence, backward expansion is performed, and for the last subsequence, forward expansion is performed.

[0078] Here, it should be noted that, in the merging stage of the final results, the above compensated boundaries may be clipped and discarded.

[0079] Rendering processing unit: deployed on the rendering node, mainly used to call the renderer to complete the rendering task. Depending on the workload, the rendering node can be a high-performance CPU server or a GPU server; it should be noted that the GPU server as a rendering node can also be used as the processing logic for executing the subsequent noise reduction processing unit. For example, in a scene, the currently idle rendering node (the rendering node implemented by the GPU server) can be used as a noise reduction node.

[0080] Denoising processing unit: deployed on the denoising node. Each rendering node can solve a set of rendering channel components for the subtask, for example, in one example, it can include RGB channels, diffuse channels, refraction channels, reflection channels, and specular channels. The components of these channels can be stored in a preset format, such as the OpenEXR 2.0 format, which supports multi-part data storage. For example, the metadata of each channel component can be stored separately in the file header.

[0081] Furthermore, after the aforementioned rendering processing unit completes the solution of the subtask, it encapsulates the relevant data and initiates a denoising request to the scheduling node. Accordingly, the scheduling node selects an idle denoising node to forward the denoising request. Furthermore, after the denoising processing unit responds to the denoising request, it pulls the rendering channel component from the corresponding rendering node through point-to-point (P2P) transmission, and calls the denoiser to complete the denoising. Furthermore, the denoising processing result is transmitted back to the corresponding rendering node.

[0082] For example, if Figure 5 As shown, according to the resolution of the target image to be rendered (such as ),according to The target image is divided into three sub-images (i.e., sub-image 1, sub-image 2, and sub-image 3). The original pixels of each sub-image are , at this time, the tolerance threshold of regional boundary compensation can be set as , then the actual pixels of each sub-image are , the specific contrast effect after noise reduction is as follows Figure 6 shown.

[0083] Result merging unit: deployed on the rendering node. The result merging unit waits for all rendering and denoising sub-workflows of a task to complete before merging the final results. Its steps include: Discard error tolerance compensation, for example, trimming redundant intervals and removing redundant frames; The results are merged and encoded, for example, according to the result constraints in the rendering request, the encoded images are merged or the encoded frame sequences are merged, and the final product is a complete rendered image or video (corresponding to the target rendering result described above).

[0084] For example, continue with Figure 5 Take the noise reduction process as an example, and merge the sub-images to get Figure 7 The complete rendered image is shown.

[0085] The disclosed solution can be applied to complex rendering tasks involving large scenes, multiple objects, and a large number of texture maps, and can obtain rendering results such as ultra-high-definition images (8K and above) and ultra-high-definition videos (4K and above).

[0086] The disclosed solution makes full use of the heterogeneous acceleration architecture and realizes efficient image rendering and noise reduction processing workflow through the collaborative work of CPU servers and GPU servers. Moreover, the disclosed solution can process both single image rendering and video rendering.

[0087] Furthermore, the disclosed solution can also reduce the pre-rendering sampling time and compensate the rendering image quality through the heterogeneous accelerated denoising algorithm, which is of great significance for improving rendering efficiency and overall visual performance: First, the overall system cost is low; the disclosed solution introduces a distributed batch processing architecture, which can control the overall system cost to the minimum by adjusting the ratio of CPU servers and GPU servers.

[0088] Second, strong adaptability: The disclosed solution can adapt to video rendering workflows and image rendering workflows, and can adaptively select the sharding method. Furthermore, the disclosed solution achieves decoupling from the renderer and denoiser, so that it can be adjusted according to actual scene requirements, further improving adaptability. For example, the renderer of the disclosed solution can achieve complete isolation or mixed deployment of rendering nodes and denoising nodes according to different implementation routes (CPU / GPU).

[0089] Third, high execution efficiency; the disclosed solution can give full play to the maximum throughput of the heterogeneous acceleration system.

[0090] The disclosed solution also provides a distributed rendering processing device, such as Figure 8 As shown, including: A splitting module 801 is used to determine N sub-rendering tasks obtained by splitting the target rendering task; wherein N is an integer greater than or equal to 2; The processing module 802 is used to call M1 rendering nodes to use the M1 rendering nodes to render the N sub-rendering tasks in parallel to obtain N sub-rendering results; M1 is an integer less than or equal to N; call M2 denoising nodes to use the M2 denoising nodes to perform denoising on the N sub-rendering results in parallel to obtain N denoising processing results; M2 is an integer less than or equal to M1; and obtain a target rendering result for the target rendering task based on the N denoising processing results.

[0091] In a specific example of the disclosed solution, the M1 rendering nodes and the M2 denoising nodes are at least part of a plurality of processing nodes included in the rendering system.

[0092] In a specific example of the disclosed solution, the plurality of processing nodes include at least a plurality of CPU nodes and a plurality of GPU nodes; the M1 rendering nodes are at least some of the plurality of CPU nodes and the plurality of GPU nodes; and / or, The M2 denoising nodes are at least some of the nodes in a plurality of GPUs.

[0093] In a specific example of the disclosed solution, the segmentation module 801 is specifically used to: When the target rendering task is an image rendering task, based on image features of the target image targeted by the image rendering task, split the target image into N sub-images to obtain N sub-rendering tasks; or, When the target rendering task is a video rendering task, the video frame sequence is split into N sub-sequences based on the number of frames of the video frame sequence targeted by the video rendering task to obtain N sub-rendering tasks.

[0094] In a specific example of the disclosed solution, the processing module 802 is further used to: Before using M1 rendering nodes to render the N sub-rendering tasks in parallel, the rendering nodes are used to perform boundary compensation processing on the sub-rendering tasks that need to be processed by themselves.

[0095] In a specific example of the disclosed solution, the processing module 802 is specifically used to: When the target rendering task is an image rendering task, using the rendering node, and based on the image features of the sub-image targeted by the sub-rendering task to be processed by itself, height compensation and / or width compensation are performed on the boundary area of ​​the sub-image; wherein the sub-image is a partial area in the target image targeted by the image rendering task; or, When the target rendering task is a video rendering task, the rendering node is used to perform frame sequence compensation on the subsequence based on the number of frames of the subsequence targeted by the sub-rendering task that needs to be processed by itself, wherein the subsequence is a subsequence in the video frame sequence targeted by the video rendering task.

[0096] In a specific example of the disclosed solution, the processing module 802 is specifically used to: Performing a perspective conversion on the sub-image based on a target perspective; wherein the target perspective is related to a capture perspective of the target image; Height compensation and / or width compensation are performed on the boundary area of ​​the sub-image after the viewing angle conversion.

[0097] In a specific example of the disclosed solution, the processing module 802 is specifically used to: Extend a preset number of video frames before and / or after the subsequence.

[0098] In a specific example of the disclosed solution, the processing module 802 is further used to: after obtaining N noise reduction processing results, remove the boundary compensation part in the noise reduction processing results.

[0099] In a specific example of the disclosed solution, the processing module 802 is specifically used to: The N denoising results after removing the boundary compensation part are merged to obtain a target rendering result for the target rendering task.

[0100] For the description of specific functions and examples of each module of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0101] Fig. 9 FIG. 1 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Fig. 9 As shown, the electronic device includes: a memory 910 and a processor 920, and the memory 910 stores a computer program that can be run on the processor 920. The number of memories 910 and processors 920 can be one or more. The memory 910 can store one or more computer programs. When the one or more computer programs are executed by the electronic device, the electronic device executes the method provided by the above method embodiment. The electronic device may also include: a communication interface 930, which is used to communicate with external devices and perform data exchange transmission.

[0102] If the memory 910, the processor 920 and the communication interface 930 are implemented independently, the memory 910, the processor 920 and the communication interface 930 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0103] Optionally, in a specific implementation, if the memory 910, the processor 920 and the communication interface 930 are integrated on a chip, the memory 910, the processor 920 and the communication interface 930 can communicate with each other through an internal interface.

[0104] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0105] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0106] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (for example: coaxial cable, optical fiber, data subscriber line (Digital Subscriber Line, DSL)) or wireless (for example: infrared, Bluetooth, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc. It is worth noting that the computer-readable storage medium mentioned in the present disclosure may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0107] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0108] In the description of the embodiments of the present disclosure, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0109] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means or, for example, A / B can mean A or B. "And / or" in this article is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0110] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0111] The above description is only an exemplary embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A distributed rendering processing method, comprising: Determine N sub-rendering tasks obtained by splitting the target rendering task; where N is an integer greater than or equal to 2; Calling M1 rendering nodes to render the N sub-rendering tasks in parallel using the M1 rendering nodes to obtain N sub-rendering results; M1 is an integer less than or equal to N; Calling M2 denoising nodes to perform denoising on the N sub-rendering results in parallel using the M2 denoising nodes to obtain N denoising processing results; M2 is an integer less than or equal to M1; A target rendering result for the target rendering task is obtained based on the N denoising processing results.

2. The method according to claim 1, wherein: The M1 rendering nodes and the M2 denoising nodes are at least part of a plurality of processing nodes included in the rendering system.

3. The method according to claim 2, wherein: The multiple processing nodes include at least multiple CPU nodes and multiple GPU nodes; the M1 rendering nodes are at least part of the multiple CPU nodes and multiple GPU nodes; and / or, The M2 denoising nodes are at least some of the nodes in a plurality of GPUs.

4. The method according to claim 1, wherein: The determining of N sub-rendering tasks obtained by splitting the target rendering task includes: When the target rendering task is an image rendering task, based on image features of the target image targeted by the image rendering task, split the target image into N sub-images to obtain N sub-rendering tasks; or, When the target rendering task is a video rendering task, the video frame sequence is split into N sub-sequences based on the number of frames of the video frame sequence targeted by the video rendering task to obtain N sub-rendering tasks.

5. The method according to any one of claims 1 to 4, wherein: Before using M1 rendering nodes to render the N sub-rendering tasks in parallel, the method further includes: The rendering node is used to perform boundary compensation processing on the sub-rendering tasks that need to be processed by itself.

6. The method according to claim 5, wherein: The using of the rendering node to perform boundary compensation processing on the sub-rendering task that needs to be processed by the rendering node itself includes: When the target rendering task is an image rendering task, using the rendering node, and based on the image features of the sub-image targeted by the sub-rendering task to be processed by itself, height compensation and / or width compensation are performed on the boundary area of ​​the sub-image; wherein the sub-image is a partial area in the target image targeted by the image rendering task; or, When the target rendering task is a video rendering task, the rendering node is used to perform frame sequence compensation on the subsequence based on the number of frames of the subsequence targeted by the sub-rendering task that needs to be processed by itself, wherein the subsequence is a subsequence in the video frame sequence targeted by the video rendering task.

7. The method according to claim 6, wherein: The performing height compensation and / or width compensation on the boundary area of ​​the sub-image includes: Performing a perspective conversion on the sub-image based on a target perspective; wherein the target perspective is related to a capture perspective of the target image; Height compensation and / or width compensation are performed on the boundary area of ​​the sub-image after the viewing angle conversion.

8. The method according to claim 6, wherein: The performing frame sequence compensation on the subsequence includes: A preset number of video frames is extended before and / or after the subsequence.

9. The method according to claim 5, wherein: After obtaining N noise reduction processing results, the method further includes: Remove the boundary compensation part in the noise reduction result.

10. The method according to claim 9, wherein: The obtaining a target rendering result for the target rendering task based on the N denoising processing results includes: The N denoising results after removing the boundary compensation part are merged to obtain a target rendering result for the target rendering task.

11. A distributed rendering processing system, comprising: A scheduling node, used to determine N sub-rendering tasks obtained after splitting the target rendering task; wherein N is an integer greater than or equal to 2; and used to call M1 rendering nodes and M2 denoising nodes; M1 is an integer less than or equal to N; M2 is an integer less than or equal to M1; M1 rendering nodes, used to render the N sub-rendering tasks in parallel to obtain N sub-rendering results; M2 denoising nodes, used to perform denoising on the N sub-rendering results in parallel to obtain N denoising processing results; The rendering nodes among the M1 rendering nodes are further used to obtain a target rendering result for the target rendering task based on the N noise reduction processing results.

12. A distributed rendering processing device, comprising: A splitting module, used to determine N sub-rendering tasks obtained by splitting the target rendering task; wherein N is an integer greater than or equal to 2; A processing module is used to call M1 rendering nodes to use the M1 rendering nodes to render the N sub-rendering tasks in parallel to obtain N sub-rendering results; M1 is an integer less than or equal to N; call M2 denoising nodes to use the M2 denoising nodes to perform denoising on the N sub-rendering results in parallel to obtain N denoising processing results; M2 is an integer less than or equal to M1; and obtain a target rendering result for the target rendering task based on the N denoising processing results.

13. The device according to claim 12, wherein: The processing module is further used for: Before using M1 rendering nodes to render the N sub-rendering tasks in parallel, the rendering nodes are used to perform boundary compensation processing on the sub-rendering tasks that need to be processed by themselves.

14. The device according to claim 13, wherein: The processing module is specifically used for: When the target rendering task is an image rendering task, using the rendering node, and based on the image features of the sub-image targeted by the sub-rendering task to be processed by itself, height compensation and / or width compensation are performed on the boundary area of ​​the sub-image; wherein the sub-image is a partial area in the target image targeted by the image rendering task; or, When the target rendering task is a video rendering task, the rendering node is used to perform frame sequence compensation on the subsequence based on the number of frames of the subsequence targeted by the sub-rendering task that needs to be processed by itself, wherein the subsequence is a subsequence in the video frame sequence targeted by the video rendering task.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images