COMPUTER-IMPLEMENTED METHOD AND PROGRAMMABLE SYSTEM FOR RENDERING 2D / 3D MODELS - Patent application

The programmable system enhances rendering engine flexibility by allowing users to add and modify units, detect data changes, and generate multiple results, resulting in efficient and dynamic rendering pipelines.

JP7595884B2Active Publication Date: 2024-12-09GRITWORLD GMBH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2021575420
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-06-20
Filing Date
2020-06-19
Publication Date
2024-12-09
Estimated Expiration
2040-06-19

AI Technical Summary

Technical Problem

Existing rendering engines limit user flexibility and access to GPU functionality, restricting the ability to modify and expand rendering pipelines dynamically.

Method used

A programmable system with memory and effects units allows users to freely add and modify rendering units, detect data changes, and generate multiple rendering results, enabling flexible design and dynamic topology changes.

Benefits of technology

Enables users to create fully functional rendering pipelines with enhanced flexibility and efficiency, allowing for improved development processes and access to full GPU capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007595884000001
    Figure 0007595884000001
  • Figure 0007595884000002
    Figure 0007595884000002
Patent Text Reader

Abstract

The present invention relates to a computer-implemented method for rendering a 2D / 3D model, the method comprising: providing a memory unit and a first effects unit, the memory unit configured to store data related to a model and transmit the data to at least the first effects unit, the first effects unit configured to receive the transmitted data and render the model based on the received data to generate a first rendering result, the method further comprising: generating a second effects unit for performing the rendering process; configuring the second effects unit such that the second effects unit is configured to receive at least one of the first rendering result and data from the memory unit; detecting changes in the data stored in the memory unit; and rendering the 2D / 3D model by the first effects unit and the second effects unit.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a computer-implemented method for rendering 2D / 3D models, a programmable system for rendering 2D / 3D models, a computer program comprising instructions for causing a computer to perform said method, and a computer readable medium comprising instructions for causing a computer to perform the methods according to claims 1, 8, 9 and 10, respectively. [Background technology]

[0002] In computer vision and computer graphics, 3D reconstruction achieves the reproduction of the shape and appearance of real objects. To this end, digital three-dimensional (3D) models of real objects are created. Two-dimensional (2D) geometric models are also useful to describe certain types of artificial images, such as technical drawings, logos, and glyphs of fonts. They are essential tools in 2D computer graphics and are often used as building blocks for 3D geometric models.

[0003] Rendering is the automated process of generating photorealistic or non-photorealistic images from 2D or 3D models by computer programs. These computer programs for rendering are often called "renderers" or "rendering engines" and define rendering effects that together form a so-called "rendering pipeline". The rendering pipeline defines the order in which effects are executed to render the final image. Each effect is defined by a set of "shaders" (in the form of small programs) that can be executed on the GPU in the form of "passes", which may contain a list of render state setup commands.

[0004] Such a rendering pipeline can become very large and complex to manage, in part because it grows into a large number of passes, many of which need to be reconfigured, replaced, or removed entirely. This can happen statically at compile time when different hardware is targeted, or dynamically during runtime when quality settings are changed.

[0005] Additionally, it is always beneficial to improve the development process for a rendering pipeline.

[0006] Some commercial real-time rendering engines, such as Unity, only support a static rendering pipeline that offers certain degrees of freedom by providing entry points into the rendering pipeline. Users can add their own specific effects and / or passes to the pipeline using such entry points, if the engine developer allows it. Furthermore, the data that users can access at the entry points is also limited. Furthermore, users are only allowed to use a subset of the available GPU capabilities due to the high level of abstraction provided by those engines. Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention is therefore based on the objective of providing a flexible design for rendering and improving the development process for rendering pipelines.

[0008] This object is inventively solved by claims 1, 8, 9 and 10. Advantageous further developments can be taken from the dependent claims. [Means for solving the problem]

[0009] According to a first aspect, the present invention provides a computer-implemented method for rendering a 2D / 3D model, the method comprising the steps of providing a memory unit and a first effects unit, the memory unit configured to store data relating to a model and transmit the data to at least the first effects unit, the first effects unit configured to receive the transmitted data and render the model based on the received data to generate a first rendering result, the method further comprising the steps of generating a second effects unit for performing the rendering process, configuring the second effects unit such that the second effects unit is configured to receive the first rendering result or the second effects unit is configured to receive data from the memory unit, detecting changes in the data stored in the memory unit, and rendering the 2D / 3D model by the first effects unit and the second effects unit.

[0010] The method of the present invention is particularly advantageous in that it allows a user to freely and easily add (effect) units to an existing pipeline, such as adding a second effect unit to an existing pipeline that includes a first effect unit and a memory unit. Thus, a user can flexibly design a rendering process or rendering pipeline, without being limited to using only a subset of shaders provided by the rendering engine, and can only have access to rendering engine entry points provided by the rendering engine developer, and can only have access to specific data / data types at entry points defined by the rendering engine developer.

[0011] For example, inventively, a user may create and program a second effects unit to implement a new shadow mapping algorithm to achieve shadow rendering.

[0012] It would be further advantageous if the user were to continually modify the input data during the rendering process with the aid of automatic detection of changes in the data, e.g., to compare rendering results, thereby improving the development process of the rendering system.

[0013] In a preferred embodiment, when the second effects unit is configured to receive the first rendering result, the method further includes a step executed by the second effects unit of generating a second rendering result based on the received first rendering result, or generating a third rendering result based on the received first rendering result and data regarding the model from the memory unit.

[0014] When the first alternative of the above steps is performed, the output of the first effect unit is used as the only input of the second effect unit. When the second alternative of the above steps is performed, not only the output of the first effect unit but also the data stored in the memory unit are used as input of the second effect unit for inputting different data including different specific parameters to the second effect unit.

[0015] In another preferred embodiment, when the second effects unit is configured to receive data from the memory unit, the method further includes a step, executed by the second effects unit, of generating a fourth rendering result based on the received data from the memory unit.

[0016] In this embodiment, the data stored in the memory unit may be input to a first effects unit and a second effects unit, respectively, to generate different rendering effects, which may then be used separately as input to some other effects unit and / or together as input to another effects unit.

[0017] In another preferred embodiment, the method further comprises a step of transforming, in particular batching and / or instantiating, data relating to the model stored in the memory unit, and a step of transmitting the transformed data to the first effects unit and / or the second effects unit.

[0018] The data stored in the memory unit is preferably transformed and / or generally edited before being sent to the effects unit so that the user can concentrate on developing the rendering system without having to worry about reconstructing data from, for example, a scene file.

[0019] Preferably, data from, for example, a scene file is inventively batched to be organized into sets or groups. Data such as geometric data may also, or alternatively, be instantiated to render multiple copies of the same mesh (a collection of vertices and faces that define the shape of a polyhedral object in 3D computer graphics) in a scene at once.

[0020] In another preferred embodiment, the method further comprises removing the second effect unit, and the generation or removal of the second effect unit is performed during the rendering of the model by the first effect unit and / or the second effect unit, which further simplifies the development process.

[0021] In another preferred embodiment, the method further comprises the step of providing connection elements for connecting the second effect unit to the first effect unit and / or the memory unit, particularly during the rendering of the model by the first effect unit, wherein the first effect unit and the second effect unit and the memory unit are connected as a directed acyclic graph, which is a graph directly connecting the units without any cycles.

[0022] This allows the inventive creation of fully functional rendering pipelines whose topologies can change dynamically, enabling flexible design of rendering engines.

[0023] In another preferred embodiment, the first effect unit and the second effect unit each comprise at least a shader subunit, the shader subunit being generated from at least a shader file that is in particular stored in the memory unit, and the shader subunit being automatically updated during rendering of the model by the first effect unit and / or the second effect unit if the shader file is modified.

[0024] The shader file includes shader code that can be specialized during run-time. Depending on the changed input data stored in the memory unit, at least one corresponding code path in the shader file can be automatically selected to generate a new specialized shader during run-time. This can further enhance the development process.

[0025] According to a second aspect, the present invention further provides a programmable system for rendering a 2D / 3D model, comprising a memory unit, a first effects unit and a second effects unit, the memory unit configured to store data relating to the model and transmit the data to at least the first effects unit, the first input unit configured to receive the transmitted data and render the model based on the received data to generate a first rendering result, and the second effects unit configured to receive one of the first rendering result and the data from the memory unit, the first effects unit and the second effects unit configured to detect changes in the data stored in the memory unit and render the 2D / 3D model.

[0026] The present invention further provides, according to a third aspect, a computer program comprising instructions which, when executed by a computer, cause the computer to carry out a method according to the first aspect.

[0027] Any computer language may be used to write the code for the units and other elements of the rendering system of the present invention.

[0028] The present invention further provides, according to a fourth aspect, a computer readable medium comprising instructions which, when executed by a computer, cause the computer to carry out a method according to the first aspect.

[0029] Further advantageous details and features can be obtained from the following description of exemplary embodiments of the invention in conjunction with the drawings. [Brief description of the drawings]

[0030] [Figure 1] FIG. 1 shows a schematic diagram of one embodiment of a programmable system of the present invention for rendering 2D / 3D models. [Diagram 2] 2 shows a schematic diagram of new pass instance / shader generation according to the embodiment shown in FIG. 1; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] 1, an embodiment of the inventive programmable system 100 for rendering 2D / 3D models comprises five effects units 6, 10, 20, 30, 40 and four memory units 2, 12, 14, 16. The effects unit 6 is for transforming data stored in the memory unit 2 or generally editing them to fit the system.

[0032] Data from scene files stored in memory unit 2, which may include scene geometry, is input to effects unit 6. In the illustrated embodiment, the input data for meshes is batched by effects unit 6 for organization into groups.

[0033] The batched data is forwarded to a rendering depth effects unit, effects unit 20, which renders geometry without writing colors to a frame buffer, for example. The rendering depth effects unit 20 includes a rendering depth sub-unit 21 for executing the pre-depth effects unit 20.

[0034] Some additional rendering parameters related to the camera are stored in memory unit 16 and input to the rendering depth effects unit 20 and the forward add unit 30, which calculates the lighting and renders the graphics.

[0035] The batched data is also input to a shadow mapping effect unit 10, which allows adding shadows to 3D computer graphics. The shadow mapping effect unit 10 receives various light parameters stored in memory units 12, 14, respectively. The shadow mapping effect unit 10 includes a shadow mapping sub-unit 11 for executing the shadow mapping effect unit 10.

[0036] The forward plus unit 30 also receives the rendering results of the rendering depth effects unit 20 and the shadow mapping effects unit 10 as well as batched data. The forward plus unit 30 comprises a culling subunit 31 which culls relevant geometric objects to reduce the number of lights that must be considered during shading, a light binning subunit 32 to classify the lights, and a shading subunit 33 which iterates over the list of lights in the scene to determine how the geometric objects should be lit.

[0037] The rendering results of the forward plus effects unit 30 are forwarded to a post-processing effects unit 40, which allows effects that require recognition of the entire image to be used since typically each 3D object is rendered in isolation. The post-processing unit 40 in the illustrated embodiment comprises two post-processing sub-units 41, 42 for implementing the post-processing effects unit 40.

[0038] The rendering results of post-processing unit 40 may then be transferred to another effects unit (not shown) or stored for further use.

[0039] Consider as an example the shading sub-unit 33 of the forward plus effects unit 30 (see FIG. 2), which is generated from a shader file containing shader code 330 that can be specialized during runtime. Depending on the changed input data, the corresponding code path from the shader code 330 can be automatically selected to generate new path instances / shaders 331, 332 during runtime. This can further enhance the development process.

[0040] With the aid of the depicted inventive programmable system 100, flexible design for rendering of 2D / 3D models and simple modification of the rendering pipeline is achieved.

[0041] The present invention has been described and illustrated in detail by the above preferred embodiments. However, the present invention is not limited to the disclosed examples, and other variations can be derived therefrom while still remaining within the protection scope of the present invention.

Claims

1. 1. A computer-implemented method for rendering a 2D / 3D model, comprising: providing a memory unit and a first effects unit; the memory unit is configured to store data relating to the model and to transmit the data to at least the first effects unit; The first effects unit is configured to receive the transmitted data and render the model based on the received data to generate a first rendering result, the method further comprising: generating a second effects unit for performing a rendering process; configuring the second effects unit such that the second effects unit is configured to receive the first rendering result directly from the first effects unit or such that the second effects unit is configured to receive the data from the memory unit; detecting a change in the data stored in the memory unit; and rendering the 2D / 3D model by the first effects unit and the second effects unit.

2. When the second effects unit is configured to receive the first rendering result, the method is further performed by the second effects unit, generating a second rendering result based on the received first rendering result; or 2. The method of claim 1, further comprising the step of: generating a third rendering result based on the received first rendering result and the data regarding the model from the memory unit.

3. 2. The method of claim 1, wherein when the second effects unit is configured to receive the data from the memory unit, the method further comprises a step, performed by the second effects unit, of generating a fourth rendering result based on the received data from the memory unit.

4. The method further comprises: - transforming, in particular batching and / or instantiating, the data relating to the model stored in the memory unit; The method according to any one of claims 1 to 3, further comprising the step of: transmitting the converted data to the first effects unit and / or the second effects unit.

5. The method according to any one of claims 1 to 4, further comprising the step of removing the second effects unit, and wherein the generation or removal of the second effects unit is performed during the rendering of the model by the first effects unit and / or the second effects unit.

6. The method according to any one of claims 1 to 5, further comprising the step of providing connection elements for connecting the second effect unit to the first effect unit and / or the memory unit, in particular during rendering of the model by the first effect unit, wherein the first effect unit and the second effect unit and the memory unit are connected as a directed acyclic graph.

7. The method according to any one of claims 1 to 6, wherein the first effect unit and the second effect unit each include at least a shader subunit, the shader subunit being generated from at least a shader file that is in particular stored in the memory unit, and the shader subunit being automatically updated during rendering of the model by the first effect unit and / or the second effect unit if the shader file is modified.

8. 1. A programmable system for rendering a 2D / 3D model, comprising: a memory unit; a first effects unit; and a second effects unit; the memory unit is configured to store data relating to the model and to transmit the data to at least the first effects unit; the first effects unit is configured to receive the transmitted data and render the model based on the received data to generate a first rendering result; the second effects unit is configured to receive at least one of the first rendering result directly from the first effects unit or the data from the memory unit; A programmable system, wherein the first effects unit and the second effects unit are configured to detect changes in the data stored in the memory unit and render the 2D / 3D model.

9. A computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of claims 1 to 7.

10. A computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7.

Citation Information

Patent Citations

  • Geometry pipeline for computer graphics

    JP2001184500A

  • Three-dimensional graphic drawing device

    JP2003271986A

  • Image processing apparatus including cip and image processing method

    JP2017123034A

  • Automated graphics and compute tile interleave

    US20160358307A1