Block rendering method and system based on video stream

Abstract

The invention provides a block rendering method and system based on a video stream. The method comprises the following steps: acquiring a task; performing task preview by adopting a dependency decoupling model, and judging rendering elements and dependency among the elements; selecting a blocking mode according to the complexity of the rendering elements and the dependency relationship among the elements; splitting task blocks according to the blocking mode; rearranging the task blocks, rendering the task blocks without the dependency relationship in parallel, and rendering the task blocks with the dependency relationship according to a sequence or a time sequence; and combining the rendered task blocks according to an initial task sequence, and carrying out smooth transition processing on a connection part. By constructing the dependency decoupling model and splitting the task blocks, the rendering process is simplified, the number of parallel rendering task blocks is increased, and the rendering time of a single frame is remarkably reduced, so that the overall rendering efficiency and the computing resource utilization rate are improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud rendering technology, and in particular to a block rendering method and system based on video stream. Background Art

[0002] In the field of visual effects and animation production, rendering large scenes or complex special effects may lead to a significant increase in single-frame rendering time. Large scenes usually contain a large amount of geometry, textures, and lighting calculations, and complex special effects include fluid dynamics simulations, particle systems, smoke and fire effects, and complex ray tracing calculations, which require a lot of computing resources to process, greatly increasing the complexity of rendering. Not only does it consume a lot of computing resources, but it may also cause delays in the progress of the entire project and affect the final delivery time. In addition, as the audience's requirements for visual effects become higher and higher, video producers need to maintain high work efficiency while ensuring high-quality visual effects; while optimizing the rendering process and reducing rendering time, they need to improve output quality.

[0003] Therefore, it is urgent to design a complex special effects rendering method with short single-frame rendering time, high computing resource utilization and high rendering quality. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and provide a block rendering method and system based on video stream to solve the problems of complex rendering process, large consumption of computing resources and long rendering time in the prior art when rendering large scenes or complex special effects.

[0005] In order to achieve the above objectives and other objectives, the present invention is implemented by including the following technical solutions: As a first aspect, the present invention proposes a block rendering method based on a video stream, the method comprising the steps of:

[0006] Get the task;

[0007] Use the dependency decoupling model to preview tasks and determine the rendering elements and dependencies between them;

[0008] Selecting a block division method according to the complexity of the rendering elements and the dependencies between the elements;

[0009] Splitting the task blocks according to the block division method;

[0010] Rearrange the task blocks, render the task blocks without dependencies in parallel, and render the task blocks with dependencies in sequence or time order;

[0011] Combine the rendered task blocks according to the initial task order and perform smooth transition on the connected parts.

[0012] Furthermore, the method for constructing the dependency decoupling model comprises the steps of:

[0013] Learn the processing order of historical rendering tasks;

[0014] Analyze the characteristics of different rendering elements in the rendering process and determine the dependencies between elements;

[0015] A dependency decoupling model is constructed according to the features of the rendering elements and the dependencies between the elements.

[0016] Furthermore, the dependency decoupling model is specifically as follows: the rendering elements are taken as nodes, each node contains the information, attributes and status of the element, the dependencies between the elements are taken as directed edges, and a directed acyclic graph is constructed.

[0017] Furthermore, the segmentation method includes image segmentation, layer slicing, element segmentation, and video stream time segmentation; the image segmentation is for tasks with simpler elements; the layer slicing is for tasks with fewer layers; the element segmentation is for tasks with more complex elements and weaker correlation between elements; the video stream time segmentation is for tasks with element special effects that change over time.

[0018] Furthermore, the selection of the block division method is determined based on task characteristic information, and the task characteristic information includes the number of special effect elements, layer complexity, element dependency and time variation characteristics.

[0019] Furthermore, the method for selecting the block mode specifically includes the steps of:

[0020] Build and train a decision tree model using the CART algorithm;

[0021] The feature information of the rendering task for which the partitioning method is to be selected is input into the decision tree model, reasoning is performed based on the feature information, decisions are made along the branches of the tree, and finally a leaf node is reached. The partitioning method corresponding to the node is the partitioning method predicted by the model.

[0022] Furthermore, when the task blocks that have been rendered by splitting the image into blocks are combined, edge smoothing transition processing is performed based on the complete element parts.

[0023] Furthermore, when the task blocks that are split and rendered by using the layer slices are combined, the edges of each layer are adapted based on the overall background tone of the rendering task to make the layers compatible and the overall image transition naturally.

[0024] As a second aspect, the present invention provides a block rendering system based on video stream, the system is applied to the method as described in any one of the first aspects, the system comprises

[0025] Acquisition module, used to acquire tasks;

[0026] The judgment module is used to use the dependency decoupling model to perform task preview and judge the rendering elements and the dependencies between the elements;

[0027] A selection module, used to select a block division method according to the complexity of the rendering elements and the dependencies between the elements;

[0028] A splitting module, used for splitting the task blocks according to the block division method;

[0029] A rendering module, used for rearranging the task blocks, rendering the task blocks without dependencies in parallel, and rendering the task blocks with dependencies in sequence or time order;

[0030] The combination module is used to combine the rendered task blocks according to the initial task order and perform smooth transition processing on the connected parts.

[0031] As a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the methods described in the first aspect.

[0032] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

[0033] This invention constructs a dependency decoupling model to determine the characteristics of rendering elements in the rendering task and the dependencies between elements, so as to select and split the task block method, split the task blocks without dependencies, simplify the rendering process, increase the number of task blocks for parallel rendering, and significantly reduce the rendering time of a single frame, thereby improving the overall rendering efficiency and computing resource utilization; at the same time, by performing smooth transition processing when combining task blocks, the quality of video special effects rendering production is improved. In addition, the method provided by the present invention is highly flexible, allowing users to select the most appropriate rendering strategy according to project requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Shown is a flow chart of a video stream-based block rendering method of the present invention. DETAILED DESCRIPTION

[0035] The embodiments of the present invention provide a video stream-based block rendering method and system, thereby solving the problems of complex rendering process, large computing resource consumption and long rendering time in the prior art when rendering large scenes or complex special effects.

[0036] Below, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments of the present invention. It should be understood that the present invention is not limited to the example embodiments described herein. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. It should also be noted that, for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all of them.

[0037] The overall idea of ​​the technical solution provided by the present invention is as follows: a block rendering method based on a video stream, the method comprising:

[0038] Get the task;

[0039] Use the dependency decoupling model to preview tasks and determine the rendering elements and dependencies between them;

[0040] Selecting a block division method according to the complexity of the rendering elements and the dependencies between the elements;

[0041] Splitting the task blocks according to the block division method;

[0042] Rearrange the task blocks, render the task blocks without dependencies in parallel, and render the task blocks with dependencies in sequence or time order;

[0043] Combine the rendered task blocks according to the initial task order and perform smooth transition on the connected parts.

[0044] The main idea is to determine the characteristics of rendering elements and the dependencies between elements in the rendering task by constructing a dependency decoupling model, so as to select and split the task block method, split the task blocks without dependencies, and increase the number of task blocks for parallel rendering, which can significantly reduce the rendering time of a single frame, thereby improving the overall rendering efficiency and computing resource utilization; at the same time, by performing smooth transition processing when combining task blocks, the quality of video special effects rendering is improved. In addition, the method provided by the present invention is highly flexible, allowing users to select the most appropriate rendering strategy according to project requirements.

[0045] After introducing the basic principles of the present invention, various non-limiting implementation methods of the present invention will be specifically described below in conjunction with the accompanying drawings.

[0046] Embodiment 1

[0047] Please refer to the attached Figure 1 The embodiment of the present invention provides a block rendering method based on a video stream, the method comprising the steps of:

[0048] S1. Get the task;

[0049] S2. Use the dependency decoupling model to preview the task and determine the rendering elements and the dependencies between them;

[0050] Specifically, the method for constructing the dependency decoupling model includes the following steps:

[0051] S21, learning the processing order of historical rendering tasks;

[0052] S22, analyzing the characteristics of different rendering elements in the rendering process, wherein the rendering elements include textures, buffers, rendering channels, shaders, etc., and each rendering element may be assigned a unique identifier; and determining dependencies between the elements according to the rendering process and the interactions between the elements;

[0053] S23. According to the features of the rendering elements and the dependencies between the elements, a dependency decoupling model is constructed using an image model algorithm. Specifically, the rendering elements are taken as nodes, each node contains information, attributes and status of the element, and the dependencies between the elements are taken as directed edges to construct a directed acyclic graph.

[0054] The dependency decoupling model is constructed, learned, trained and corrected to obtain a model that can autonomously judge the dependency between elements in the task. The trained model can achieve the effect of autonomously judging the dependency between elements after the task is input. The dependency between elements includes the existence of dependency and the non-existence of dependency. For example, the task includes three elements: wind, snow and people. The special effect of wind blowing is required before the snow flutters. Therefore, the dependency is to first produce the special effect of wind element, and then produce the special effect of snow element according to the special effect of wind. The movement of the character is not affected by other elements, so there is no dependency and it can be processed in parallel with other elements.

[0055] S3, selecting a block division method according to the complexity of the rendering elements and the dependencies between the elements;

[0056] Specifically, by constructing task block rules, judging according to characteristic information such as the number of special effect elements, layer complexity, element dependency and time change characteristics in the task, a targeted and most efficient block method is selected for block rendering. Among them, the characteristic factors for selecting the block method include but are not limited to: the number of special effect elements, layer complexity, element dependency and time change characteristics. The number of special effect elements can be obtained by counting the number of special effect elements in the task. The layer complexity can be quantified by evaluating the number of layers and analyzing the dependency and interaction between layers. The element dependency can be obtained by analyzing the dependency between elements, such as whether the elements are related to each other, whether the change of one element will affect other elements, etc. The time change feature is to extract the characteristics of the element special effects changing over time for the video stream task, such as the frequency and duration of the special effect changes. If the frequency of the element special effects changes is high, that is, multiple changes will occur in a short time, then time segmentation can capture these changes more carefully, ensuring that the video image of each time period can accurately reflect the state of the special effects.

[0057] Correspondingly, the block division methods include image block division, layer slicing, element segmentation, and video stream time segmentation. Image block division is for tasks with relatively simple elements; layer slicing is for tasks with fewer layers; element segmentation is for tasks with relatively complex elements and weak correlation between elements; and video stream time segmentation is for tasks with element effects that change over time.

[0058] Specifically, the method for selecting the block mode comprises the following steps:

[0059] S31. Build and train a decision tree model using the CART (classification and regression tree) algorithm;

[0060] Specifically, the decision tree model is trained using historical task processing data, and the data set includes feature information of different tasks and corresponding information on the selection results of the rendering block method. Through training, the decision tree model learns the rules of selecting the block method under different feature combinations.

[0061] S32, input the feature information of the rendering task of the block mode to be selected into the decision tree model, perform reasoning based on the input features, make decisions along the branches of the tree, and finally reach a leaf node. The block mode corresponding to the node is the predicted block mode selection of the model.

[0062] S4, splitting the task blocks according to the block division method;

[0063] Specifically, the image segmentation is as follows: the image is segmented evenly. For example, if the image segmentation ratio is set to 4×4, the image is divided into 16 blocks. However, the segmentation must keep the elements intact. When an element is segmented, the segmentation line is segmented along the edge of the element to form a new task block. The task block with complete elements is responsible for processing the rendering of the segment edge.

[0064] The layer slicing specifically includes: extracting task layer information, slicing the tasks according to the layers, and performing rendering operations on the layers respectively.

[0065] The element segmentation is specifically as follows: by decoupling the dependencies between elements, the elements are split to form task blocks with the elements as the core, and the processing order of the task blocks is rearranged. Task blocks without dependencies can be rendered at the same time to improve the element rendering efficiency.

[0066] The video stream time segmentation is specifically as follows: taking the switching of element special effects as nodes, the video stream is segmented by time. For example, the smoke dissipation changes with the change of time, and the special effect disappears after a certain period of time, so it can be segmented by time.

[0067] S5, rearrange the task blocks, render the task blocks without dependency in parallel, and render the task blocks with dependency in sequence or time order;

[0068] S6. Combine the rendered task blocks according to the initial task order, perform smooth transition processing on the connected parts, and make the edges of the task blocks compatible and adaptable to make the image more coordinated.

[0069] Specifically, when combining the task blocks (i.e., image blocks) that are split and rendered using the image blocks, smooth edge transition processing is performed based on the complete part of the element (element segmentation is similar to a jigsaw puzzle, with the element edge as the dividing line, and different task blocks can be spliced ​​and matched).

[0070] When combining the task blocks that are split and rendered using the layer slices, it is necessary to perform smooth transition processing on the edges of each layer, and adapt the edges of the layers based on the overall background tone of the rendering task to make the layers compatible and the overall image transition naturally. For example, if a character layer is added to the background image, it is necessary to perform transition processing on the edge of the character layer and the background image to blend the character with the background.

[0071] Embodiment 2

[0072] Based on the same inventive concept as the video stream-based block rendering method in the aforementioned embodiment, the present invention further provides a video stream-based block rendering system, comprising:

[0073] Acquisition module, used to acquire tasks;

[0074] The judgment module is used to use the dependency decoupling model to perform task preview and judge the rendering elements and the dependencies between the elements;

[0075] A selection module, used to select a block division method according to the complexity of the rendering elements and the dependencies between the elements;

[0076] A splitting module, used for splitting the task blocks according to the block division method;

[0077] A rendering module, used for rearranging the task blocks, rendering the task blocks without dependencies in parallel, and rendering the task blocks with dependencies in sequence or time order;

[0078] The combination module is used to combine the rendered task blocks according to the initial task order and perform smooth transition processing on the connected parts.

[0079] The foregoing Figure 1 The various variations and specific examples of the video stream-based block rendering method in Example 1 are also applicable to the video stream-based block rendering system in this embodiment. Through the above detailed description of the video stream-based block rendering method, those skilled in the art can clearly know the implementation method of the video stream-based block rendering system in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here.

[0080] Embodiment 3

[0081] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method of the first embodiment is implemented.

[0082] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of the same technology of the present invention, the present invention is also intended to include these modifications and variations.

Claims

1. A block rendering method based on video stream, characterized in that: The method comprises the steps of: Get the task; Use the dependency decoupling model to preview tasks and determine the rendering elements and dependencies between elements; Selecting a block division method according to the complexity of the rendering elements and the dependencies between the elements; Splitting the task blocks according to the block division method; Rearrange the task blocks, render the task blocks without dependencies in parallel, and render the task blocks with dependencies in sequence or time order; Combine the rendered task blocks according to the initial task order and perform smooth transition on the connected parts.

2. The video stream-based block rendering method according to claim 1, characterized in that: The method for constructing the dependency decoupling model comprises the steps of: Learn the processing order of historical rendering tasks; Analyze the characteristics of different rendering elements during the rendering process and determine the dependencies between elements; A dependency decoupling model is constructed according to the features of the rendering elements and the dependencies between the elements.

3. The video stream-based block rendering method according to claim 2, characterized in that: The dependency decoupling model is specifically as follows: the rendering elements are taken as nodes, each node contains the information, attributes and status of the element, the dependencies between the elements are taken as directed edges, and a directed acyclic graph is constructed.

4. The video stream-based block rendering method according to claim 1, characterized in that: The block division methods include image block division, layer slicing, element segmentation, and video stream time segmentation; the image block division is for tasks with relatively simple elements; the layer slicing is for tasks with fewer layers; the element segmentation is for tasks with relatively complex elements and weak correlation between elements; the video stream time segmentation is for tasks where element special effects change over time.

5. The video stream-based block rendering method according to claim 4, characterized in that: The selection of the block division method is determined based on task feature information, which includes the number of special effect elements, layer complexity, element dependency, and time variation characteristics.

6. The video stream-based block rendering method according to claim 5, characterized in that: The method for selecting the block mode specifically comprises the steps of: Build and train a decision tree model using the CART algorithm; The feature information of the rendering task for which the partitioning method is to be selected is input into the decision tree model, reasoning is performed based on the feature information, decisions are made along the branches of the tree, and finally a leaf node is reached. The partitioning method corresponding to the node is the partitioning method predicted by the model.

7. The video stream-based block rendering method according to claim 1, characterized in that: When combining the task blocks that have been rendered by splitting the image into blocks, edge smoothing transition processing is performed based on the complete element parts.

8. The video stream-based block rendering method according to claim 1, characterized in that: When combining the task blocks split and rendered using the layer slices, the edge of each layer is adapted based on the overall background tone of the rendering task to make the layers compatible and the overall image transition naturally.

9. A block rendering system based on video stream, characterized in that: The system is applied to the method according to any one of claims 1 to 8, and the system comprises Acquisition module, used to acquire tasks; The judgment module is used to use the dependency decoupling model to perform task preview and judge the rendering elements and the dependencies between the elements; A selection module, used to select a block division method according to the complexity of the rendering elements and the dependencies between the elements; A splitting module, used for splitting the task blocks according to the block division method; A rendering module, used for rearranging the task blocks, rendering the task blocks without dependencies in parallel, and rendering the task blocks with dependencies in sequence or time order; The combination module is used to combine the rendered task blocks according to the initial task order and perform smooth transition processing on the connected parts.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images