Virtual scene rendering method and device, electronic equipment and storage medium

By identifying and multiplexing the same instance rendering unit in multiple scene modules in a virtual scene and performing batch rendering, the problem of inefficient rendering in the prior art is solved, and efficient rendering processing is achieved.

CN119075304APending Publication Date: 2024-12-06SHANGHAI MIHA YOUHAIYUANCHENG TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411135089.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In virtual scenarios, existing batch-batch technology can only perform batch-batch processing for exactly the same rendering objects, resulting in low rendering efficiency and lag in large-scale virtual scenarios.

Method used

By obtaining scene instantiation data of a virtual scene, identifying and multiplexing instance rendering units of the same type in multiple scene modules, performing batch rendering, including obtaining instance transformation data of the instance rendering unit and performing transformation processing, to realize multiple Combined rendering of scene modules.

Benefits of technology

Significantly increase the batch range, improve rendering efficiency, reduce the number of rendering requests, reduce system overhead, and reduce memory overhead and loading time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119075304A_ABST
    Figure CN119075304A_ABST
Patent Text Reader

Abstract

The invention relates to the field of data processing, and particularly discloses a virtual scene rendering method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring scene instantiation data corresponding to a virtual scene; wherein the scene instantiation data is used for representing a plurality of scene modules contained in the virtual scene, and the type and the number of instance rendering units contained in each scene module; the plurality of scene modules reuse at least two instance rendering units; and performing batch rendering on the multiple instance rendering units of the same type included in the multiple scene modules. According to the embodiment of the invention, the composition elements of the virtual scene are split into reusable instance rendering units with smaller granularity, so that the batching range can be greatly increased, and the rendering efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for rendering a virtual scene. Background Art

[0002] Virtual scenes are realistic scenes formed by computer technology and can be widely used in games, virtual reality and other fields. In the rendering process of virtual scenes, the central processing unit (CPU) needs to send drawing instructions (Draw Call, DC) to the graphics processing unit (GPU) so that rendering operations can be performed through drawing instructions. Among them, the number of drawing instructions is an important indicator affecting the running performance. Too many drawing instructions will cause the GPU to have too much rendering burden, resulting in problems such as page freezes.

[0003] In related technologies, batching technology can be used to reduce the number of drawing instructions. However, the current batching technology has very high requirements for rendering objects, and can only batch completely identical rendering objects. In the case of a large amount of scene information in a virtual scene, the possibility of two completely identical rendering objects appearing is very small. Therefore, in large-scale virtual scenes, batching operations are usually not applied, which leads to a series of problems such as low rendering efficiency and page jams. Summary of the invention

[0004] In view of the above problems, the present disclosure is proposed to provide a method, device, electronic device and storage medium for rendering a virtual scene that overcome the above problems or at least partially solve the above problems.

[0005] According to one aspect of an embodiment of the present disclosure, a method for rendering a virtual scene is provided, comprising:

[0006] Acquire scene instantiation data corresponding to the virtual scene; wherein the scene instantiation data is used to characterize a plurality of scene modules included in the virtual scene, and the type and quantity of instance rendering units included in each scene module; the plurality of scene modules reuse at least two instance rendering units;

[0007] Multiple instance rendering units of the same type contained in the multiple scene modules are batch rendered.

[0008] In an optional implementation, the multiple instance rendering units of the same type correspond to the same instance basic information;

[0009] The batch rendering of multiple instance rendering units of the same type contained in the multiple scene modules includes:

[0010] For any instance rendering unit, the instance transformation data of the instance rendering unit is obtained; according to the instance transformation data, the instance basic information of the instance rendering unit is transformed, and the instance rendering unit after the transformation is rendered.

[0011] In an optional implementation, the instance basic information includes: mesh information associated with the instance shape, and / or texture information associated with the instance material; the instance transformation data includes: first transformation data for transforming the mesh information, and / or second transformation data for transforming the texture information;

[0012] The transforming of the instance basic information of the instance rendering unit according to the instance transformation data comprises:

[0013] According to the first transformation data, the mesh information of the instance rendering unit is transformed by a vertex shader; and / or,

[0014] According to the second transformation data, the texture information of the instance rendering unit is transformed by a pixel shader.

[0015] In an optional implementation manner, the acquiring the instance transformation data of the instance rendering unit includes:

[0016] Acquire instance transformation data of the instance rendering unit from an instance attribute matrix of the instance rendering unit;

[0017] The instance attribute matrix is ​​an N*N matrix, and includes: a first matrix area and a second matrix area, the instance transformation data is stored in the first matrix area, and the first matrix area at least includes: an area corresponding to the Nth row and the Nth column of the instance attribute matrix; wherein N is a natural number.

[0018] In an optional implementation, the second matrix area of ​​the instance attribute matrix is ​​used to store instance orientation data of the instance rendering unit;

[0019] Wherein, the instance orientation data includes: instance position data, instance rotation data, and / or instance scaling data;

[0020] The instance scaling data is used to represent a scaling direction of the instance rendering unit; wherein the scaling direction includes: a first direction and / or a second direction perpendicular to the first direction.

[0021] In an optional implementation, the scene instantiation data includes: a plurality of module instantiation data corresponding to the plurality of scene modules; wherein the module instantiation data includes: an instance attribute data set generated according to the type and number of instance rendering units included in the corresponding scene module;

[0022] Wherein, the number of instance attribute data sets contained in the module instantiation data corresponding to each scene module matches the number of types of instance rendering units contained in the scene module;

[0023] Furthermore, each instance attribute data set has a type identifier, and the type identifier is used to indicate the type of the instance rendering unit corresponding to the instance attribute data set.

[0024] In an optional implementation, each instance attribute data set is used to store instance transformation data of a plurality of instance rendering units of the same type and corresponding to the type identifier contained in the scene module;

[0025] Before batch rendering of the multiple instance rendering units of the same type contained in the multiple scene modules, the method further includes:

[0026] Determine instance attribute data sets having the same category identifier among the plurality of module instantiation data;

[0027] A plurality of instance rendering units of the same type included in the plurality of scene modules are determined according to the instance attribute data set having the same type identifier.

[0028] In an optional implementation, the number of rendering times of the batch rendering is the same as the number of types of instance rendering units included in the multiple scene modules.

[0029] In an optional implementation, the virtual scene is a game scene in a game application, and obtaining scene instantiation data corresponding to the virtual scene includes:

[0030] Determining a current visible area in the virtual scene according to a camera position in the game scene;

[0031] Dynamically obtain scene instantiation data corresponding to the current visible area, and load the scene instantiation data into the memory space.

[0032] In an optional implementation, after acquiring the scene instantiation data corresponding to the virtual scene, the method further includes:

[0033] Determining, according to the occlusion relationship, invisible instance rendering units included in the scene instantiation data;

[0034] The invisible instance rendering unit is cropped to obtain processed scene instantiation data, so as to perform batch rendering on the processed scene instantiation data.

[0035] In an optional implementation, the scenario module is determined in the following manner:

[0036] Determining a structure included in the virtual scene and a reference surface included in the structure according to the structural information of the virtual scene;

[0037] According to the size and / or shape of the reference plane, a reference line contained in the reference plane and a scene module corresponding to the reference line are determined.

[0038] According to another aspect of the present disclosure, a virtual scene rendering device is provided, comprising:

[0039] An acquisition module, adapted to acquire scene instantiation data corresponding to a virtual scene; wherein the scene instantiation data is used to characterize a plurality of scene modules included in the virtual scene, and the type and quantity of instance rendering units included in each scene module; the plurality of scene modules reuse at least two instance rendering units;

[0040] The rendering module is adapted to batch render multiple instance rendering units of the same type contained in the multiple scene modules.

[0041] According to another aspect of the present disclosure, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0042] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the virtual scene rendering method as described above.

[0043] According to another aspect of the present disclosure, a computer storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute the virtual scene rendering method as described above.

[0044] In an embodiment of the present disclosure, first, scene instantiation data corresponding to the virtual scene is obtained; the types and quantities of multiple scene modules contained in the virtual scene and the instance rendering units contained in each scene module are determined through the scene instantiation data; then, multiple instance rendering units of the same type contained in the multiple scene modules are batch rendered. It can be seen that in this method, multiple scene modules reuse at least two instance rendering units, so that multiple different scene modules can be spliced ​​with the help of at least two instance rendering units, and in the rendering process, the instance rendering units of the same type contained in multiple scene modules can be batch processed, so that one rendering process can simultaneously draw instance rendering units in multiple different scene modules. It can be seen that by splitting the constituent elements of the virtual scene into reusable, smaller-granularity instance rendering units, the batching range can be greatly increased and the rendering efficiency can be improved.

[0045] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present disclosure. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0047] Figure 1 A flowchart of a method for rendering a virtual scene provided by an embodiment of the present disclosure is shown;

[0048] Figure 2 A schematic diagram of the art production process in this example is shown;

[0049] Figure 3 A possible structural diagram of a scene management device is shown;

[0050] Figure 4 Describes the specific implementation process of scene rendering;

[0051] Figure 5 A schematic diagram showing the hierarchical structure of IRI data is shown;

[0052] Figure 6 Shows the rendering process of IRI in the shader;

[0053] Figure 7 A structural schematic diagram of a building is shown;

[0054] Figure 8 Shown according to Figure 7 Schematic diagram of three types of instance rendering units obtained by splitting the building in;

[0055] Fig. 9 A schematic diagram of a virtual scene rendering device provided by another embodiment of the present disclosure is shown;

[0056] Fig.10 A schematic structural diagram of an electronic device provided by yet another embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0057] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0058] Figure 1 FIG. 1 is a flowchart of a method for rendering a virtual scene provided by an embodiment of the present disclosure. Figure 1 As shown, the method specifically includes:

[0059] Step S110: Acquire scene instantiation data corresponding to the virtual scene; wherein the scene instantiation data is used to characterize multiple scene modules included in the virtual scene, and the type and quantity of instance rendering units included in each scene module; multiple scene modules reuse at least two instance rendering units.

[0060] Among them, the virtual scene can be various scenes such as game scenes and virtual reality scenes. Scene instantiation data refers to: data obtained after instantiation processing of the target scene object to be rendered in the virtual scene. Among them, the target scene object can be represented by a virtual structure. For example, the target scene object can be a composite structure object, a single structure object, a virtual plane object, and a virtual baseline object in the virtual structure. It can be seen that the present application does not limit the spatial size and granularity of the target scene object. Among them, a composite structure object can include multiple single structure objects, a single structure object can include multiple virtual plane objects, and a virtual plane object can include multiple virtual baseline objects.

[0061] Regardless of the spatial size and granularity of the target scene object, a target scene object includes multiple scene modules. It can be seen that a scene module is a spatial unit with a smaller granularity than the target scene object. For example, when the target scene object is a composite structure object, the scene module can be at least one of a single structure object, a virtual plane object, and a virtual baseline object; when the target scene object is a single structure object, the scene module can be at least one of a virtual plane object and a virtual baseline object; when the target scene object is a virtual plane object, the scene module can be a virtual baseline.

[0062] Among them, multiple scene modules reuse at least two instance rendering units means that the types of instance rendering units used in multiple scene modules are the same. In other words, multiple scene modules share at least two instance rendering units of the same type. For example, assuming that S different types of instance rendering units are pre-configured, the types of instance rendering units used in multiple scene modules are limited to the above S types. Usually, a scene module is spliced ​​by at least two different types of instance rendering units.

[0063] It can be seen that the scene instantiation data stores the following information: information about multiple scene modules contained in the virtual scene, and the types and quantities of instance rendering units contained in each scene module. For example, assuming that a scene module contains three types of instance rendering units, the scene instantiation data needs to store the quantity and specific information of each instance rendering unit.

[0064] Step S120: batch rendering multiple instance rendering units of the same type contained in multiple scene modules.

[0065] In this step, multiple instance rendering units of the same type contained in multiple scene modules are extracted respectively according to the scene instantiation data. That is, for any type of instance rendering unit, multiple instance rendering units of the same type contained in multiple scene modules are obtained respectively for batch processing.

[0066] Among them, batch rendering refers to: merging multiple objects to be rendered with certain commonalities (such as instance rendering units of the same type) into fewer rendering batches to reduce the processing burden of the CPU and GPU. In the present application, multiple instance rendering units of the same type contained in multiple scene modules have certain commonalities, for example, multiple instance rendering units of the same type have the same instance basic information. Therefore, multiple instance rendering units of the same type contained in multiple scene modules can be merged into at least one rendering batch. Correspondingly, only one drawing instruction needs to be called to complete the batch processing of multiple instance rendering units of the same type in the rendering batch, thereby greatly improving the rendering efficiency. It can be seen that since the present application splits the smallest object of instantiation rendering into instance rendering units that can be reused in different scene modules (the basic information of each instance rendering unit is the same), it is possible to perform batch rendering for the instance rendering units contained in multiple scene modules at the same time. As a result, the number of batch renderings can be determined according to the type of instance rendering unit, and has nothing to do with the number of target scene objects and the number of scene modules contained in a target scene object. It can be seen that in complex situations, assuming that the virtual scene includes a large number of target scene objects, and each target scene object further includes a large number of scene modules, however, since the multiple scene modules within each target scene object reuse the same type of instance rendering units, when the type of instance rendering unit remains unchanged, no matter how the scene scale expands or how complex the scene information is, the number of rendering times will not increase.

[0067] It can be seen that in this method, multiple scene modules reuse at least two instance rendering units, so that multiple different scene modules can be spliced ​​with the help of at least two instance rendering units, and in the rendering process, the same type of instance rendering units contained in multiple scene modules can be batched, so that one rendering process can simultaneously draw instance rendering units in multiple different scene modules. It can be seen that by splitting the constituent elements of the virtual scene into reusable, smaller-granularity instance rendering units, the batching range can be greatly increased and the rendering efficiency can be improved.

[0068] In addition, in order to achieve better technical effects, those skilled in the art may also make various changes and modifications to the above embodiments:

[0069] In an optional implementation, multiple instance rendering units of the same type correspond to the same instance basic information. The instance basic information is used to characterize the appearance and form of the instance rendering unit. Since instance rendering units of the same type have the same appearance and form, the corresponding instance basic information is exactly the same, and only one copy needs to be stored, which can greatly reduce the amount of data storage.

[0070] In addition, in order to improve the diversity of instance rendering units and facilitate flexible adaptation to various scene requirements, when batch rendering of multiple instance rendering units of the same type contained in multiple scene modules, it can be achieved in the following way: for any instance rendering unit, obtain the instance transformation data of the instance rendering unit; according to the instance transformation data, transform the instance basic information of the instance rendering unit, and render the transformed instance rendering unit. It can be seen that in this method, instance transformation data can be stored for each instance rendering unit, and the instance basic information can be transformed through the instance transformation data, so that the appearance and form of the instance rendering unit finally rendered can be changed. It can be seen that by configuring the instance transformation data, customized transformation processing can be implemented in the batch rendering process, which can not only expand the scope of batching, but also meet different rendering requirements.

[0071] In an optional implementation, the instance basic information may include: mesh information (e.g., vertex information) associated with the instance shape, and / or texture information (e.g., mapping information) associated with the instance material. The instance transformation data may include: first transformation data for transforming the mesh information, and / or second transformation data for transforming the texture information. Accordingly, when transforming the instance basic information of the instance rendering unit according to the instance transformation data, it can be implemented in the following manner: transforming the mesh information of the instance rendering unit through a vertex shader according to the first transformation data; and / or, transforming the texture information of the instance rendering unit through a pixel shader according to the second transformation data. Among them, the vertex shader may also be referred to as the first shader, which may be replaced by various shaders capable of processing the mesh shape. Through the vertex shader, the mesh shape of the instance rendering unit may be changed, so that the instance rendering unit may be changed in various shapes. The pixel shader may also be referred to as the second shader, which may be replaced by various shaders capable of processing the texture information. Through the pixel shader, the texture information of the instance rendering unit may be changed, so that the instance rendering unit may be changed in various texture mappings.

[0072] In an optional implementation, the instance transformation data of the instance rendering unit may be obtained in the following manner: the instance transformation data of the instance rendering unit is obtained from the instance attribute matrix of the instance rendering unit; wherein the instance attribute matrix may be an N*N matrix. In addition, the instance attribute matrix may include: a first matrix area and a second matrix area, and the instance transformation data is stored in the first matrix area, wherein the first matrix area at least includes: an area corresponding to the Nth row and the Nth column of the instance attribute matrix; wherein N is a natural number. For example, the second matrix area of ​​the instance attribute matrix is ​​used to store the instance orientation data of the instance rendering unit; wherein the instance orientation data may include: instance position data, instance rotation data, and / or instance scaling data. wherein the instance scaling data is used to characterize the scaling direction of the instance rendering unit; wherein the scaling direction includes: a first direction and / or a second direction perpendicular to the first direction. For example, when N is equal to 4, the first matrix area may be an area where the first three rows and the first three columns are located, and the second matrix area may be an area where the 4th row and the 4th column are located. It can be seen that in this manner, each instance rendering unit may correspond to an instance attribute matrix for storing personalized information of the instance rendering unit, thereby achieving personalized rendering of each instance rendering unit in the process of batch rendering.

[0073] In an optional implementation, the scene instantiation data includes: a plurality of module instantiation data corresponding to a plurality of scene modules; wherein the module instantiation data includes: an instance attribute data set generated according to the type and number of instance rendering units contained in the corresponding scene module. wherein the plurality of module instantiation data corresponds one-to-one to the plurality of scene modules, and each module instantiation data is used to store relevant data information in the corresponding scene module. generally, the module instantiation data corresponding to the scene module further includes at least two instance attribute data sets, wherein the number of instance attribute data sets included in the module instantiation data matches the number of types of instance rendering units contained in the corresponding scene module. for example, assuming that the number of types of instance rendering units contained in a scene module is 3, the number of instance attribute data sets included in the corresponding module instantiation data is 3, that is, each instance attribute data set corresponds to one instance rendering unit. in order to facilitate the distinction of the instance rendering units corresponding to each instance attribute data set, each instance attribute data set has a type identifier, which is used to characterize the type of the instance rendering unit corresponding to the instance attribute data set.

[0074] In an optional implementation, each instance attribute data set is used to store instance transformation data (e.g., instance attribute matrix) of multiple instance rendering units of the same type contained in a scene module and corresponding to the type identifier. Accordingly, before batch rendering of multiple instance rendering units of the same type contained in multiple scene modules, the following operations are further performed: determining instance attribute data sets with the same type identifier in multiple module instantiation data; and determining multiple instance rendering units of the same type contained in multiple scene modules based on the instance attribute data sets with the same type identifier. It can be seen that, since multiple instance rendering units of the same type are centrally stored in an instance attribute data set, multiple instance rendering units of the same type can be determined in batches by merging instance attribute data sets with the same type identifier in multiple scene modules.

[0075] In an optional implementation, the number of renderings of the batch rendering is the same as the number of types of instance rendering units included in the multiple scene modules. It can be seen that in the present application, a batch rendering process is performed once for each type of instance rendering unit. For example, when the number of types of instance rendering units included in the multiple scene modules is 5, the number of renderings of the batch rendering is also 5.

[0076] In an optional implementation, the virtual scene is a game scene in a game application. Accordingly, the scene instantiation data corresponding to the virtual scene can be obtained in the following manner: first, according to the camera position in the game scene, the current visible area in the virtual scene is determined; then, the scene instantiation data corresponding to the current visible area is dynamically obtained, and the scene instantiation data is loaded into the memory space. It can be seen that the range of the scene instantiation data loaded into the memory space changes dynamically according to the range of the current visible area. Therefore, it is only necessary to load the local data corresponding to the current visible area in the entire scene, without loading the full amount of data, thereby reducing the memory loading time and improving memory utilization.

[0077] In an optional implementation, after obtaining the scene instantiation data corresponding to the virtual scene, the following operations are further performed: according to the occlusion relationship, the invisible instance rendering units contained in the scene instantiation data are determined; the invisible instance rendering units are cropped to obtain the processed scene instantiation data, and the processed scene instantiation data is batch rendered. It can be seen that the cropping granularity in the present application is finer, and cropping can be performed in units of instance rendering units, thereby achieving refined cropping and avoiding the problem of time-consuming rendering due to coarse cropping granularity.

[0078] In an optional implementation, the scene module is determined in the following manner: First, according to the structural information of the virtual scene, the structure contained in the virtual scene and the reference plane contained in the structure are determined. Among them, the structure can be various types of polygonal structures, and accordingly, by analyzing the number of edges contained in the polygonal structure, the reference plane contained in the structure can be obtained. Among them, the reference plane can also be referred to as the virtual plane object mentioned above. Then, according to the size and / or shape of the reference plane, the reference line contained in the reference plane and the scene module corresponding to the reference line are determined. Among them, the reference plane can be composed of multiple reference lines, and the reference line can also be referred to as the virtual reference line object mentioned above. It can be seen that by performing geometric analysis on the structure in the virtual scene, multiple minimum units used to constitute the structure, that is, multiple different types of instance rendering units, can be split.

[0079] For ease of understanding, an example is used below to introduce the implementation of the above embodiment. This example can be applied to the field of games.

[0080] The following is an explanation of some of the concepts that may be involved in this example:

[0081] Stage: refers to the scene at runtime, which is a collection of all resource references and data in the scene used by the device to render a scene image. Stage can specifically be the virtual scene mentioned above.

[0082] Item: refers to the smallest data set unit for rendering images at runtime, corresponding to a real object.

[0083] IRI: Instance Render Item, refers to a scene object rendered using instantiation technology, which can be a collection of modules, corresponding to a specific object in the real scene corresponding to the mapping of the rendered graphics. For example, IRI can be a specific implementation form of the scene module mentioned above.

[0084] Block: refers to a collection of items at runtime that share the same context (such as spatial offset, rotation, and scaling).

[0085] IRI Volume: The data structure of IRI in the game engine. One object corresponds to one IRI Volume. Among them, IRI Volume can be the module instantiation data corresponding to the scene module mentioned above.

[0086] IRI Block: A controller in the game engine that manages a group of IRI Volumes. IRI Volumes in the same IRIBlock will share the same context (such as shared spatial rotation and spatial scaling). By setting up an IRIBlock, it is easy to group multiple IRI Volumes for unified control.

[0087] IRIData: A data file required for rendering IRI objects at runtime. It stores the module data referenced by the IRI object. Each module data mainly contains three pieces of information: 1. The model Mesh of the module; 2. The material of the module; 3. The point cloud data of the module. The point cloud data of the module can be a specific implementation form of the instance attribute data set mentioned above.

[0088] Point cloud data: There may be multiple identical modules in an IRI. The spatial matrix array of instances of these identical modules relative to the object coordinate system is called point cloud data. Among them, multiple identical modules in an IRI are multiple instance rendering units of the same type mentioned above. Point cloud data is a specific implementation form of the instance attribute data set mentioned above.

[0089] Regular expression: A continuous expression consisting of a series of characters and special symbols. The characters and symbols can be programmed to have specific meanings. In this solution, it can be used to record the splicing logic between multiple instance rendering units.

[0090] Basic Module: The smallest structure that is reused, which is a specific implementation of the instance rendering unit mentioned above.

[0091] IRI Module Unit: The smallest logical unit composed of 1 to N basic modules. The smallest unit operated in programmatic generation is the IRI Module Unit. The IRI Module Unit can correspond to one or more scene modules.

[0092] IRI Baseline: A straight / curved structure formed by arranging at least one splicing unit in a continuous order along the same axis according to a set rule. The length of the baseline can be precisely specified according to the length of the splicing unit, or it can be configured so that the length can be freely adjusted and there is only one layer in the height direction.

[0093] IRI Wall: A planar structure formed by vertically stacking (at least one) datum line of the same length sharing the same starting point, which can be configured in a form where the height can be freely extended.

[0094] IRI Structure: A complete three-dimensional structure composed of at least one reference plane. When the reference planes are arranged into a closed shape, a closed structure (such as a building) can be obtained. When the reference planes do not fill the closed shape, a type of protruding structure can be obtained.

[0095] Module group (IRI_Building): A complete scene structure consisting of at least one structure, such as a complete building with dozens of floors; a complete eight-lane road with detailed structures such as sidewalks, railings along the road, traffic lights, zebra crossings, and central green belts; a small square; a garden with Chinese courtyard buildings and artificial landscapes, etc. The module group contains all the used Module Units and the corresponding IRI Matrix Data collection.

[0096] IRI Matrix Data: A 4x4 float, based on a standard SRT matrix implementation for describing object positions, with the default (0, 0, 0, 1) in the last row replaced with custom data (i.e. instance transformation data). IRIMatrix Data is the matrix data of the instance attribute matrix mentioned above.

[0097] Rendering request: The drawing instruction sent by the CPU to the GPU mainly consists of two parts, namely PSO (Pipeline State Object) and Draw call. For modern games, the overhead generated by rendering requests accounts for 30% to 50% of the total CPU overhead.

[0098] Batch: A technology that combines the same rendering requests of multiple PSOs and shares one PSO.

[0099] GPU Instance: A technology that integrates multiple PSOs and Meshes with exactly the same rendering requests and shares a rendering request (PSO+Draw call).

[0100] At present, in order to realize larger and higher-precision game scenes, a large number of rendering assets are often laid into the scenes to achieve richer scene details. However, in virtual scenes dominated by man-made objects, scenes with the same theme (such as the same / different cities, villages, man-made functional buildings in the same country / culture / historical background, etc.) usually follow the same design language and use the same construction logic to construct scene elements (such as buildings, scene structures, etc.) with similar appearance but different volumes and detailed structures.

[0101] In the traditional scene production process, regardless of whether it belongs to the same type of building structure under the same design language, after being split into the appropriate granularity (for example, a building with the same length, width and height), each / group of scene structures needs to create a full set of 3D digital assets (such as 3D models, texture materials, etc.), and then be placed in the appropriate position in the space to form a complete game scene. At this time, each 3D digital asset needs to store complete asset information in memory. Therefore, a large number of digital assets will not only take up a lot of memory, but also generate a large number of rendering requests. Higher rendering requests will lead to increased power consumption, causing a series of problems such as device heating and frequency reduction, and fast power loss. In addition, if the memory exceeds the application upper limit specified by the device, it will directly cause a flash back. Therefore, how to reduce rendering requests and memory has become a very important issue in performance optimization.

[0102] In a related technology, memory optimization can be performed through LOD (Level of Detail) technology. In this technology, artists will generalize the same original asset and generate multiple levels of LOD assets with lower and lower precision. These assets have smaller memory and will be used on lower model configurations. Usually, the accuracy of the lowest level LOD is about 30% of the original accuracy. Although this method can optimize memory in most cases, when multiple levels of LOD exist at the same time, it may generate more memory. At the same time, this method requires a lot of manpower to produce and supervise LOD assets to reduce precision while ensuring consistency with the original assets.

[0103] In another related technology, rendering requests can be optimized through Batch and GPU Instance. Among them, the CPU overhead of each rendering request mainly consists of two parts, namely switching the rendering context (PSO) and initiating a draw request (Draw Call). Batch will package and draw rendering requests with the same context, so that multiple objects only need one PSO. GPU Instance will package and draw rendering requests that are exactly the same in PSO and Draw call, but the requirements for the objects to be rendered are very high: they must be exactly the same object model and exactly the same context. This shows that although GPU Instance has better performance, in most cases it is often difficult to use in actual production due to the more stringent restrictions.

[0104] This example aims to optimize the production and operation efficiency of large-scale digital scenarios from the following two aspects:

[0105] From the perspective of art production, as the scale of scenes becomes larger and larger (a single scene is more than 1 to 10 km long and wide), and the total amount of artificial structures that need to be used to fill the scene space increases, the number of digital scene assets that need to be produced also increases exponentially, and the production cycle is greatly lengthened to perform repetitive work. Among them, a large part of the work is used to repeatedly build parts with exactly the same logical structure, such as rebuilding models with tens of thousands to hundreds of thousands of triangles. However, the differences between these complex models are only in subtle aspects such as floor height, floor length and width, and window shape, which greatly wastes the production capacity of digital assets.

[0106] From a performance perspective, whether using LOD or Batch, there are limits to its effectiveness. To maintain the performance of assets at a good level, the accuracy of LOD should not be less than 30% of the original asset. Batch can only optimize PSO, and for scenes with a huge number of objects, it will still generate huge Draw Call overhead. However, for a modern metropolitan scene, there are a large number of buildings (one building has about hundreds of thousands of triangles and about dozens of texture resources) and a huge number of objects (the entire scene has about tens of thousands of small objects). Therefore, if you want to maintain good performance on mid- and low-end devices, it is far beyond the capabilities of LOD and Batch. Therefore, how to achieve high-precision metropolitan scenes on mid- and low-end devices while keeping the art budget and production costs within a reasonable range is the core optimization goal of this example.

[0107] In order to achieve the above optimization goals, when producing resources, the scene objects can be split into single, low-complexity, and widely reusable basic modules (i.e., instance rendering units) according to the art design structure and logic structure to reduce the total amount of assets. At this time, the artist only needs to make the basic modules, which can greatly reduce the production cost. Then, using the basic modules, programmatic tools are used to compile the generation logic based on the art design, and then the programmatic tools will copy / place simple basic modules according to the generation logic, and finally form a complex man-made structure. The advantage of doing this is that a large number of man-made structures of various shapes can be built with only a small number of module components.

[0108] For example, a building is actually composed of several reference planes (sides of the building), each of which is composed of several reference lines (each floor). Each reference line uses corresponding syntax (such as regular expressions) to describe how the reference line should be filled with basic modules. In the end, only a few basic modules are needed to build complex buildings and even huge scenes. Among them, these basic modules will eventually take up only a small amount of memory. At the same time, when running, the scene will collect the Transform matrix (4x4 matrix) of each module and store some data used to control custom effects in the last row of the matrix (this row is not used by normal Transform). The matrices of the same modules in the entire scene will be packaged and drawn by GPUInstance. Only one rendering request can complete the drawing of a large number of identical basic modules, minimizing the overhead of rendering requests.

[0109] This example specifically includes the pre-executed art production pipeline and the runtime executed rendering pipeline. Figure 2 FIG. 1 shows a schematic diagram of the art production process in this example. The art production process aims to produce scene instantiation data corresponding to the target scene object in the virtual scene, so that the scene instantiation data can be directly loaded in the subsequent rendering process. Figure 2 As shown, the art production process for any target scene object includes the following steps:

[0110] Step S201: Determine the scene module included in the target scene object.

[0111] Specifically, the structure contained in the target scene object and the reference surface contained in the structure can be determined according to the structural information of the target scene object. Then, according to the size and / or shape of the reference surface, the reference line contained in the reference surface and the scene module corresponding to the reference line are determined. It can be seen that the type and number of scene modules contained in a target scene object are determined according to the art structure of the target scene object.

[0112] Step S202: for any scene module, determining at least two different types of instance rendering units corresponding to the scene module.

[0113] Specifically, at least two different types of instance rendering units that are compatible with the scene module can be selected according to the types of pre-configured instance rendering units. In this example, multiple different types of instance rendering units are pre-configured according to the characteristics of the scene, and each instance rendering unit is a minimum rendering unit of a fixed form.

[0114] Specifically, the type and number of instance rendering units can be flexibly determined according to the art structure in the scene. For example, the types of instance rendering units may include: sphere, cube, cuboid, cone, etc. This application does not limit the type and number of instance rendering units.

[0115] Step S203: configuring unit arrangement data of at least two different types of instance rendering units according to the module form of the scene module.

[0116] The unit arrangement data is used to represent the number, order of occurrence, and / or arrangement information of any type of instance rendering units included in the scene module. The number, order of occurrence, and / or arrangement information of the instance rendering units can be represented by a regular expression.

[0117] For example, the same type of instance rendering units correspond to the same character in the regular expression; and the appearance order may include: single appearance order (appearing once alone), repeated appearance order (appearing multiple times repeatedly), combined appearance order (appearing in pairs in combination with at least one other instance rendering unit), etc. The arrangement information includes: arranging according to the original size, arranging after extending along the first direction, and / or arranging after extending along the second direction.

[0118] Accordingly, the instance rendering unit may include at least one of the following types: a non-scalable type for arrangement according to the original size (i.e., a fixed-length unit whose size cannot be adjusted), a first scalable type for arrangement after being extended along a first direction (e.g., a variable-length unit that can be adjusted along the length direction), a second scalable type for arrangement after being extended along a second direction (e.g., a variable-length unit that can be adjusted along the width direction), and a third scalable type for arrangement after being extended along the first direction and along the second direction (e.g., a variable-length unit that can be adjusted along both the length direction and the width direction).

[0119] Step S204: Generate corresponding instance attribute data sets for instance rendering units of the same type in the scene module according to the unit arrangement data.

[0120] In the same scene module, instance rendering units of the same type correspond to the same instance attribute data set, which can also be referred to as point cloud information of the corresponding instance rendering unit. The point cloud information is used to represent the number of instance rendering units of the same type in the scene module and the instance transformation data of each instance rendering unit from a spatial dimension.

[0121] It can be seen that the number of instance attribute data sets contained in the scene module is equal to the number of types of instance rendering units contained in the scene module. Each type of instance rendering unit corresponds to an instance attribute data set to centrally store instance transformation data of multiple instance rendering units of that type.

[0122] Step S205: Generate module instantiation data of the scene module according to at least two instance attribute data sets corresponding to at least two different types of instance rendering units in the scene module.

[0123] Each scene module corresponds to a module instantiation data, and the module instantiation data is used to centrally store relevant information of various instance rendering units included in the scene module.

[0124] Step S206: Generate scene instantiation data corresponding to the target scene object according to the multiple module instantiation data of the multiple scene modules.

[0125] It can be seen that the scene instantiation data corresponding to the target scene object specifically includes multiple module instantiation data of multiple scene modules.

[0126] Step S207: Export the scene instantiation data into a file in a preset format for subsequent loading.

[0127] In specific implementation, the above-mentioned art production process can be realized in the following ways:

[0128] On the one hand, it is necessary to configure the basic module assets (Basic Module), that is, to configure the type of instance rendering unit. Specifically, analyze the art design, comprehensively examine the building structure, and combine the building structure logic to split the smallest reusable structural unit as the basic module. Among them, the repetitive structure can be extracted from the target scene object to obtain the basic module, and the complex independent structure that needs to be customized can be identified, and this type of structure can be processed separately. Among them, for each instance rendering unit, the usage attributes of the basic module can be configured according to actual usage requirements. For example, according to the usage method of the basic module, its usage attributes can be configured as "4-way scalable", "2-way scalable", "reusable", etc., which specifically correspond to the non-scalable type and scalable type mentioned above.

[0129] On the other hand, it is necessary to split the scene modules in the target scene object. Specifically, it is necessary to analyze the logical architecture of the scene: perform structural analysis on the target artificial scene, refer to the logic of structure-reference plane to classify the scene structure into polygons, and identify the combination relationship between polygons. Each identified polygon is a structure; then, disassemble the structure, identify each plane that constitutes the structure, that is, the reference plane, and calculate the length and width of each plane; finally, map the art design onto the structure, identify what kind of appearance each reference plane should be reconstructed into, and what kind of architectural structural components each type of appearance is composed of.

[0130] The configuration logic of the basic module is described in detail below: According to the splitting logic, each basic module is configured as a prefab containing the mesh information and textures in the 3D model. Among them, the basic modules can be divided into the following three types according to the usage properties of the material: (1) Non-scalable type: The prefab of each basic module follows the initially set length, width and height ratio. It will only be copied and used repeatedly in the generation logic, and the preset ratio cannot be changed at will. (2) One-way scalable type: It can be scaled on demand in a certain direction. It can be scaled, lengthened or shortened in the horizontal direction according to the configuration, or stretched or shortened in the vertical direction to adapt to the target length / height required by the splicing unit. At the same time, the ratios in other directions remain unchanged, and the texture tiling accuracy of the scaled part also maintains the preset accuracy. (3) Two-way scalable type: It can be freely stretched, shortened, raised or lowered in both the horizontal and vertical directions. It usually corresponds to the construction of large-area similar structures such as walls and floors using tiled textures.

[0131] Through the above operations, the configuration of the properties and types of the basic modules can be completed. Next, it is necessary to configure the usage logic of the basic modules (such as the arrangement of multiple basic modules). By configuring the arrangement of multiple basic modules, a splicing unit (i.e. the above-mentioned scene module) can be obtained.

[0132] Among them, the basic modules can be combined to obtain the smallest unit involved in the generation logic, namely the splicing unit. Each splicing unit can contain several basic modules, and the basic modules can be displaced, rotated, and scaled within the splicing unit to achieve the required architectural structure. At the same time, the splicing unit can contain several simple internal logics to correspond to different usage environments: (1) Non-scalable units: splicing units whose length and height cannot be adjusted at will after the default ratio is configured. Such units will only be copied and used in the splicing logic, and their width cannot be changed at will. Therefore, non-scalable units can be obtained by splicing the non-scalable basic modules mentioned above. (2) Units with fixed length but adjustable height: splicing units whose length cannot be adjusted at will after the default ratio is configured, but whose occupied height can be adjusted. Such units will only be copied and used in the splicing logic in the length direction, but their height can be adjusted to match the changing requirements of the floor height according to the height occupied by the baseline. (3) Units with adjustable length but fixed height: After configuring the default ratio, the height cannot be adjusted at will, but the occupied length can be adjusted. Usually, there is only one such unit in each adjustable baseline, and the length requirement of the baseline is matched by adjusting the occupied length. Therefore, a unit with fixed length but adjustable height or a unit with adjustable length but fixed height can be obtained by splicing the basic modules of the one-way scalable type mentioned above. (4) Bidirectional scalable unit: After configuring the default ratio, the length and height of the splicing unit can be adjusted as needed. A scalable basic module can be configured in the length direction, and a scalable basic module can be configured in the height direction. In addition, a basic module that is fixed at the starting point of the scaling and remains stationary and a basic module that follows the end point of the scaling can be configured to meet the scaling requirements of the accessories at the edge of the scalable module.

[0133] In addition, based on the splicing unit, the baseline can be further configured: the baseline editor can be used to configure the arrangement logic and repetition logic of the splicing unit on the line, and the generation logic can be recorded through regular expressions. Specifically, the appearance logic of each splicing unit on the baseline can be configured through a visual editor. For example, each basic module or splicing unit on the baseline is recorded as a character, and the basic module or splicing unit can be configured to belong to a unique group ID. The module can configure the corresponding repetition mode for each group ID, and can choose to appear only once, repeat a fixed number of times, appear at least once, and repeat zero to any number of times. In this way, a baseline of fixed length can be prepared. In an optional manner, each group in the baseline can be configured to appear a fixed number of times and does not contain scalable splicing units. For example, assuming that the baseline is composed of 42 splicing units arranged in a specified order, and each splicing unit appears only once and does not repeat (this method does not require grouping), the grammatical logic of the baseline can be recorded as the following regular expression ABCACCCCDCBEEBDCCCACCCCDCCCABEFBDCCCCCCABDC; the regular expression represents different splicing units by different characters. Of course, a splicing unit can also be represented by multiple basic modules within it through the above regular expression.

[0134] In addition, a baseline with an adjustable length can also be configured. Among them, the baseline with an adjustable length can include a module combination that can appear repeatedly, and the length of the baseline can be adjusted by adjusting the number of repetitions of the module combination. For example, assuming that the baseline contains a module group that appears at least once or can appear repeatedly any number of times (the second and third modules together constitute a module group that can appear repeatedly), the need to adjust the length of the baseline can be met by multiple repetitions of the module group. For example, the grammatical logic of the baseline can be recorded as the following regular expression: A(BC)*D; wherein A, B, C, and D are different basic modules or splicing units, respectively, wherein module B and module C together constitute a module group that can appear repeatedly, and the value of "*" can be flexibly configured according to the number of repetitions.

[0135] Usually, there is at least one length-adjustable / bidirectionally scalable splicing unit in the adjustable length baseline. For example, the second splicing unit in the above sequence can be configured as a bidirectionally scalable unit, and the first and third splicing units can be configured as longitudinally scalable units. The length requirement of the baseline can be adapted by adjusting the scaling of the second unit, and the layer height of the baseline can be adjusted by adjusting the height of all units.

[0136] After configuring the baseline, you can further configure the reference plane based on the baseline. Stacking 1 to several baselines with the same starting point and the same length in the vertical direction can get a corresponding plane, which is a reference plane. Among them, the reference plane editing tool can be used to configure the reference plane. First, you can select the baseline as the base, the length of the baseline is the default length of the reference plane, and then select the baseline that can be adjusted to the same length to continue stacking upward to generate the basic arrangement of the reference plane. If the effect requirements are met at this time, the reference plane can be saved and participate in the subsequent structure generation. The editing tool supports replacing the splicing units in the reference plane on this basis. Each splicing unit that has been used can be directly replaced by other splicing units of the same length and width. Alternatively, several splicing units that can form a rectangle can be merged into the same placement space and replaced with other splicing units with the same length and width as this space. Moreover, the merged splicing units can also be split back to the original module splicing units from the baseline logic.

[0137] In addition, the baseline of the curved surface can also be generated using the baseline with adjustable length: in order to generate the baseline in the form of a curved surface, the tangent direction and tangent angle need to be configured. Accordingly, each splicing unit in the baseline will align with the tangent direction of the corresponding position on the curve, and scale and chamfer according to its own depth along the curved surface to form a continuous and complete circular baseline of the baseline, and the baselines of the same length can continue to be stacked in the height direction to form a curved high wall.

[0138] Finally, you can use the stitching tool to assemble IRI Buildings, which is the scene instantiation data.

[0139] In order to assemble the complete scene instantiation data, it is necessary to configure the structure first, that is, to form a complete structure based on the above-mentioned reference lines and reference planes. In an optional implementation, a complete man-made building structure can be composed of 1 to several reference planes, wherein a closed or non-closed structure of the structure can be constructed with a broken line through a structure editor, and then a corresponding reference plane is specified for each broken line to generate a complete structure, and then the tool can be used to automatically generate a top or bottom surface model that is not closed in the structure. In another optional implementation, the results of the aforementioned scene structure analysis can also be used to configure a corresponding reference plane for each plane in the analysis results, and the target height value of the reference plane can be input to adapt the reference plane to the plane to obtain a complete structure.

[0140] Among them, each complete structure (corresponding to a complete target scene object) can be marked as an IRI Building (i.e., a scene instantiation data) for subsequent data generation process.

[0141] In addition, in order to facilitate subsequent loading, you can use the tool to export IRI Building Data. Among them, each IRIBuilding will generate an IRI Matrix Data list as the smallest unit of data storage. The list uses a float4*4 to record the position, scale, rotation and custom data (i.e. instance transformation data) of each splicing unit in the module group coordinate system. The 4 floats in the last row are split into several digits to store the custom parameters of the basic module, such as the texture ID in the texture atlas, the RGB value of the color applied to the color-changing area, the deformation axis and deformation degree value, the basic magnification of UV tiling, and other parameters. Each IRI Building will save the MatrixData of each basic unit inside it for the client to load and rebuild the module at runtime.

[0142] It can be seen that the above method can realize the pre-construction of art resources. Among them, the most important optimization point of this example is: by dividing the basic modules into smaller granularity, different scene modules can reuse the same basic modules, so that for the same type of basic modules, only one instance basic information needs to be stored in IRI Building, without repeated storage, which greatly reduces memory overhead and loading time. In addition, since the basic information of the same type of basic modules is the same, only one drawing call needs to be triggered, which can greatly improve the efficiency of batching.

[0143] In an optional implementation, the instance attribute matrix (i.e., IRI MatrixData) of each instance rendering unit can be implemented by a 4*4 matrix. Among them, the first 3 rows and the first 3 columns are used to store the position, rotation, and scaling information of the instance rendering unit. The 4th row and the 4th column are used to store instance transformation data (also called custom data). Among them, the custom data can occupy 4*32 bits. Specifically, it may include: personalized color data, data used to characterize whether it is self-luminous, identification information corresponding to the texture map (through this information, any texture map can be flexibly selected from the texture atlas), density information of the texture map (through this information, the density of the texture map, sampling multiples, etc. can be adjusted), and information related to mesh deformation (through this information, the instance rendering module can perform deformation processing such as stretching and deformation). In short, this application does not limit the specific types and connotations of the instance transformation data stored in the matrix.

[0144] Based on the above scene instantiation data, scene rendering can be achieved. Figure 3 A possible structural diagram of a scene management device (also called an IRI management system) is shown. Figure 3As shown, the scene management device specifically includes: a resource cache module 31, an asynchronous loading queue management module 32, an IRI instance management module 33, and a space transformation management module 34.

[0145] During the game, due to the huge scale of the entire scene, it is difficult to load all the resources required for the scene into the device memory at one time. Therefore, the game client code needs to decide which IRI objects to create based on the current game context (including but not limited to the position of the game camera). Therefore, several code modules (i.e. Figure 3 The various modules in it maintain this dynamic scheduling process.

[0146] The resource cache module 31 is used to implement cache processing. Because different IRI objects in the scene may have submodules that reference the same model and material, if the corresponding resources have been loaded into the memory, the newly created IRI can directly use these resources without having to resend a read request to the storage device. Therefore, the resource cache module 31 is responsible for managing the reference relationship between the model's resources, materials, and point cloud data, and is responsible for responding to the read request of the IRI management system, returning existing resources, or initiating a new read request.

[0147] The asynchronous loading queue management module 32 is a queue management system, because the IRI object needs to load a lot of resources, and in order to avoid blocking the main loop logic of the game, and in order to optimize the reading efficiency and increase the I / O read and write throughput, these resources will adopt the asynchronous loading method. Therefore, the loading of IRI dependent resources is not completed immediately, so a queue system is needed to manage the resource reading state of IRI, and to be responsible for handling the external modification of IRI state during the reading process (including but not limited to canceling the loading of IRI during the reading process). At the same time, since initiating reading and executing reading and completing callbacks need to occupy the time of the main thread, therefore, in order to stabilize the frame rate, it is also necessary to limit the reading rate of IRI resources and the assembly rate after reading is completed, so a queue system is needed to schedule the loading task of IRI according to certain rules. Therefore, the asynchronous loading queue management module 32 is a module for solving the above problems, which is responsible for managing the creation and scheduling of the loading task of IRI.

[0148] The IRI instance management module 33 is a manager of the assembled IRI instance, responsible for providing query and modification operations of the currently created IRI instance to other logics within the game.

[0149] The space transformation management module 34 is an optional module responsible for responding to external space changes. When the external system needs to operate the IRI instance as a whole (including but not limited to space rotation), the space transformation management module 34 can be notified through an event, and then the space transformation management module 34 will process it to reassemble the IRI data and pass it to the game engine.

[0150] Figure 4 Describes the specific implementation process of scene rendering. Figure 4 As shown, the rendering process specifically includes the following steps:

[0151] Step S401: dynamically determine the current visible area in the virtual scene, and determine the virtual scene resources to be loaded according to the current visible area.

[0152] Specifically, during the running of the game, the scene object instances that need to be loaded can be dynamically determined based on the current game context.

[0153] Step S402: When the virtual scene resource to be loaded includes an IRI object, a loading request is sent.

[0154] Since there are a large number of virtual scene resources, in addition to the instance rendering objects constructed by the method in this application, virtual objects realized by other methods may also be included. Therefore, if it is determined that the virtual object to be loaded is the IRI object in this example, a loading request is initiated to the IRI management system. The IRI management system performs loading processing through the asynchronous loading queue management module 32.

[0155] Step S403: The asynchronous loading queue management module in the IRI management system performs on-demand loading processing.

[0156] Among them, the IRI management system schedules the loading request of IRI and initiates the actual resource loading operation on demand. Among them, the asynchronous loading queue management module is composed of two queues. When IRI needs to be loaded, a loading task is created and added to queue 1 (i.e., the first queue). Queue 1 will take out a number of loading tasks (the number is determined by the time consumed by the current frame loading task) in a first-in-first-out order at a specific time point in each frame, initiate a resource application to the resource cache module 31, and put the loading task into queue 2 (i.e., the second queue). Queue 2 will take out a number of loading tasks (the number is determined by the time consumed by the current frame loading task) whose resources have been loaded in a first-in-first-out order at a specific time point in each frame to assemble the data structure. After the assembly is completed, it will be passed back to the IRI management system to notify that the loading is complete.

[0157] Step S404: After the resources of the IRI are loaded, they are assembled and passed to the game engine for rendering.

[0158] After receiving the loading completion event of IRI, the IRI management system passes the assembled resource data structure to the game engine to create an IRI instance and render it.

[0159] In addition, it is also necessary to store the mapping relationship between the IRI instance and the corresponding data. The mapping relationship between the IRI instance and the corresponding data is used to respond to possible external queries and modification requests before the IRI instance is destroyed.

[0160] Step S405: In the event of spatial rotation or spatial displacement, the assembled data is updated and re-transmitted to the game engine for re-rendering.

[0161] This step is optional. When a global space rotation occurs in the game, or some IRIs are displaced by animation resources, the assembly data corresponding to the affected IRIs is updated and re-passed to the engine.

[0162] For example, overall modification operations may occur on IRI objects during the game (such as the world space is rotated, all IRIs must be rotated accordingly). These modification operations will be notified to the space transformation management module 34. The space transformation management module 34 is responsible for modifying the data structure of the IRI instance that has been created and notifying the game engine to update the corresponding data in order to correctly render the scene image.

[0163] Step S406: when the rendered IRI object moves out of the visible area, unload the corresponding data.

[0164] For example, due to the movement of the game camera, some IRI objects are invisible and need to be unloaded. During the game running process, it can dynamically decide which IRI instances need to be unloaded according to the current game context. At this time, the management system will notify the asynchronous loading queue management module 32 and the IRI instance management module 33 respectively to cancel the loading task of the corresponding unloaded IRI objects, or notify the engine to unload the created IRI instances. Subsequently, the resource cache module 31 will update the reference relationship and actively unload the models, materials and point cloud data that are not referenced by any IRI module.

[0165] The point cloud data of multiple instance rendering units (ie, instance attribute data sets) can be dynamically managed in memory. Specifically, the IRI data can be represented by a hierarchical structure. Figure 5 A schematic diagram of the hierarchical structure of IRI data is shown in FIG. Figure 5As shown, the top layer is IRI Block (i.e., scene instantiation data), which contains multiple IRI Volumes (corresponding to module instantiation data of scene modules). Each IRI Volume contains multiple IRI Types (i.e., type identifiers of instance attribute data sets). Each IRI Type corresponds to a type of instance attribute data set, which further contains instance attribute matrices (i.e., IRI Matrix Data) of multiple instance rendering units. Figure 5 The target scene object in contains three scene modules, corresponding to IRI Volume 1, IRI Volume 2, and IRI Volume 3. Among them, IRI Volume 1 uses 6 different types of instance rendering units, which are stored through 6 point cloud data (i.e., instance attribute data sets), namely IRI Type 1, IRI Type 2, IRI Type 3, IRI Type 4, IRI Type 5, and IRI Type 6 in the figure. In addition, IRI Volume 2 and IRI Volume 3 reuse 5 different types of instance rendering units, namely IRI Type 1, IRI Type 2, IRI Type 3, IRI Type 4, IRI Type 5 and IRI Type 1, IRI Type 2, IRI Type 3, IRI Type 4, IRI Type 6.

[0166] The unloading and hiding of IRI Block will trigger the unloading and hiding of multiple IRI Volumes. At the same time, IRI Block can also uniformly implement the position and posture transformation processing of multiple IRI Volumes inside it. IRI Volume is the streaming granularity, and IRI Block can be dynamically loaded or unloaded in units of IRI Volume.

[0167] In addition, when CPU clipping is performed, clipping can be performed in units of IRI Volume. Among them, IRI clipping can first be roughly clipped on the CPU side, and then clipped in units of IRI Volume, which includes two steps: occlusion clipping and frustum clipping. Among them, occlusion clipping can be implemented using various culling algorithms. Frustum clipping is based on whether the bounding box is outside a face. If the bounding box of the IRI Volume is not outside the six faces of the frustum, the bounding box must intersect with the frustum, and the frustum clipping passes. The IRI Volume that passes the clipping can be aggregated into multiple groups of rendering objects. Among them, all point cloud data can be cached through a special point cloud cache (i.e., point cloud GPU Buffer), which can be managed through a cache management algorithm. In addition, when implementing GPU precise clipping, point cloud GPU precise clipping can be clipped at the granularity of each point of the point cloud (i.e., an instance rendering unit).

[0168] in addition, Figure 6 The figure shows the rendering process of IRI in the shader. Figure 6 As shown, the rendering process in the shader includes the following steps:

[0169] Step S601: The vertex shader obtains instance orientation data and instance transformation data related to vertex transformation from the instance attribute matrix.

[0170] The instance orientation data and instance transformation data related to vertex transformation include at least one of the following: displacement, rotation, scaling, deformation, etc.

[0171] Step S602: performing vertex transformation processing according to the acquired instance orientation data and instance transformation data.

[0172] Step S603: The pixel shader obtains instance orientation data and instance transformation data related to texture transformation from the instance attribute matrix.

[0173] The instance orientation data and instance transformation data related to texture transformation include at least one of the following: color change, self-illumination, UV mapping control, etc.

[0174] Step S604: performing texture transformation processing according to the acquired instance orientation data and instance transformation data.

[0175] It can be seen that in the vertex shader, the instance transformation data, such as IRI Custom Data, is extracted from the last row of the instance attribute matrix. The information in IRI Custom Data is used for deformation control. Then the last row of the instance attribute matrix is ​​set to (0, 0, 0, 1), which turns it into a normal SRT matrix that can map each vertex to the correct world space coordinate. In the pixel shader, the separated IRI Custom Data is used to perform color change, self-luminous intensity control, UV mapping, etc.

[0176] In summary, through the method in this application, it is possible to achieve a high degree of reuse of instance rendering units. Since instance rendering units of the same type can be batched (even if they belong to multiple different scene modules in multiple different scene objects), the batching range can be greatly increased, the number of rendering requests can be reduced, and the system overhead can be reduced. In addition, since instance rendering units of the same type only need to have one copy of instance basic information and do not need to be stored repeatedly, memory overhead and loading time can be greatly reduced.

[0177] In short, this example can greatly optimize the total amount of assets. Compared with the traditional modeling production process, the total amount of assets that need to be digitally reconstructed has been reduced by 90-95%. This example can reuse the same basic modules, and the basic modules can make changes such as color change, material replacement, and limited range deformation according to the function of the shader. In addition, the same basic module can be adapted to many final results to meet the needs of the scene structure for the richness of visual expression in art design. Therefore, combined with the adjustment of the generation logic, different building combinations with the same design language but different volumes and details can be quickly reconstructed without adding new assets, greatly increasing the production flexibility of production staff.

[0178] For example, let's take a building as an example. Assuming the original number of faces of the building is 110,000, the repeated parts can be broken down into several parts, such as the side of the building, windows, stairs, etc. Each part is composed of repeated basic modules, and the total number of faces of all module assets is 2043. It can be seen that for only one building, memory consumption is reduced by 2%. At the same time, the artist only needs to make assets of repeated basic modules, and the amount of resources produced is also reduced by 2% compared to making a complete building.

[0179] Assume that the disassembled building is composed of 8 different types of basic modules, which contain 807 components in total. Then, at least 807 rendering requests need to be triggered in a conventional manner to render 807 components. However, after the improvement of the present application, only 8 rendering requests need to be triggered (corresponding to 8 different types of basic modules), thereby greatly reducing the number of Draw Calls and improving rendering performance.

[0180] In addition, for the entire scene, any building that uses the above basic modules will not generate additional resource memory even if its shape changes. Assuming that there are 100 buildings of different shapes in the scene, all composed of 100 repeated originals, the memory can be reduced to less than 0.1% of the original memory. It can be seen that the larger the data volume of the scene, the greater the reduction in rendering requests (that is, the higher the reuse degree of instance rendering units).

[0181] Figure 7 A schematic diagram of the structure of a building (corresponding to a target scene object in a virtual scene) is shown. Figure 8 Shown according to Figure 7 The three instance rendering units obtained by splitting the building in the figure are circle, triangle and rectangle. Figure 7 and Figure 8 This is only for illustration. In actual situations, the building structure may be more complex, and each type of instance rendering unit may be reused more frequently. It can be seen that through the method in this application, a complex scene can be divided into multiple different types of instance rendering units, thereby greatly reducing the number of rendering times and improving rendering efficiency.

[0182] Fig. 9 A schematic diagram of a virtual scene rendering device provided by another embodiment of the present disclosure is shown, and the device specifically includes:

[0183] An acquisition module 91 is adapted to acquire scene instantiation data corresponding to a virtual scene; wherein the scene instantiation data is used to characterize a plurality of scene modules included in the virtual scene, and the type and quantity of instance rendering units included in each scene module; the plurality of scene modules reuse at least two instance rendering units;

[0184] The rendering module 92 is adapted to batch render multiple instance rendering units of the same type contained in the multiple scene modules.

[0185] In an optional implementation, the multiple instance rendering units of the same type correspond to the same instance basic information;

[0186] The rendering module is specifically used for:

[0187] For any instance rendering unit, the instance transformation data of the instance rendering unit is obtained; according to the instance transformation data, the instance basic information of the instance rendering unit is transformed, and the instance rendering unit after the transformation is rendered.

[0188] In an optional implementation, the instance basic information includes: mesh information associated with the instance shape, and / or texture information associated with the instance material; the instance transformation data includes: first transformation data for transforming the mesh information, and / or second transformation data for transforming the texture information;

[0189] The rendering module is specifically used for:

[0190] According to the first transformation data, the mesh information of the instance rendering unit is transformed by a vertex shader; and / or,

[0191] According to the second transformation data, the texture information of the instance rendering unit is transformed by a pixel shader.

[0192] In an optional implementation, the rendering module is specifically used for:

[0193] Acquire instance transformation data of the instance rendering unit from an instance attribute matrix of the instance rendering unit;

[0194] The instance attribute matrix is ​​an N*N matrix, and includes: a first matrix area and a second matrix area, the instance transformation data is stored in the first matrix area, and the first matrix area at least includes: an area corresponding to the Nth row and the Nth column of the instance attribute matrix; wherein N is a natural number.

[0195] In an optional implementation, the second matrix area of ​​the instance attribute matrix is ​​used to store instance orientation data of the instance rendering unit;

[0196] Wherein, the instance orientation data includes: instance position data, instance rotation data, and / or instance scaling data;

[0197] The instance scaling data is used to represent a scaling direction of the instance rendering unit; wherein the scaling direction includes: a first direction and / or a second direction perpendicular to the first direction.

[0198] In an optional implementation, the scene instantiation data includes: a plurality of module instantiation data corresponding to the plurality of scene modules; wherein the module instantiation data includes: an instance attribute data set generated according to the type and number of instance rendering units included in the corresponding scene module;

[0199] Wherein, the number of instance attribute data sets contained in the module instantiation data corresponding to each scene module matches the number of types of instance rendering units contained in the scene module;

[0200] Furthermore, each instance attribute data set has a type identifier, and the type identifier is used to indicate the type of the instance rendering unit corresponding to the instance attribute data set.

[0201] In an optional implementation, each instance attribute data set is used to store instance transformation data of a plurality of instance rendering units of the same type and corresponding to the type identifier contained in the scene module;

[0202] The rendering module is further used to: determine instance attribute data sets having the same type identifier among the plurality of module instantiation data;

[0203] A plurality of instance rendering units of the same type included in the plurality of scene modules are determined according to the instance attribute data set having the same type identifier.

[0204] In an optional implementation, the number of rendering times of the batch rendering is the same as the number of types of instance rendering units included in the multiple scene modules.

[0205] In an optional implementation, the virtual scene is a game scene in a game application, and the acquisition module is specifically used to:

[0206] Determining a current visible area in the virtual scene according to a camera position in the game scene;

[0207] Dynamically obtain scene instantiation data corresponding to the current visible area, and load the scene instantiation data into the memory space.

[0208] In an optional implementation, the acquisition module is further used to:

[0209] Determining, according to the occlusion relationship, invisible instance rendering units included in the scene instantiation data;

[0210] The invisible instance rendering unit is cropped to obtain processed scene instantiation data, so as to perform batch rendering on the processed scene instantiation data.

[0211] In an optional implementation, the scenario module is determined in the following manner:

[0212] Determining a structure included in the virtual scene and a reference surface included in the structure according to the structural information of the virtual scene;

[0213] According to the size and / or shape of the reference plane, a reference line contained in the reference plane and a scene module corresponding to the reference line are determined.

[0214] The specific working principles of the above modules can be found in the description of the corresponding parts in the method embodiment, which will not be repeated here.

[0215] Fig.10 A schematic structural diagram of an electronic device according to another embodiment of the present disclosure is shown. The specific embodiment of the present disclosure does not limit the specific implementation of the electronic device.

[0216] like Fig.10 As shown, the electronic device may include: a processor (processor) 502 , a communication interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0217] in:

[0218] The processor 502 , the communication interface 504 , and the memory 506 communicate with each other via a communication bus 508 .

[0219] The communication interface 504 is used to communicate with other devices such as clients or other servers.

[0220] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above-mentioned video image detection method embodiment.

[0221] Specifically, the program 510 may include program codes, which include computer operation instructions.

[0222] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure. The one or more processors included in the electronic device may be processors of the same type, such as one or more CPUs. They may also be processors of different types, such as one or more CPUs and one or more ASICs.

[0223] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0224] The program 510 can be specifically used to enable the processor 502 to execute the corresponding operations in the above-mentioned video image detection method embodiment.

[0225] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device or other device. Various general purpose devices may also be used together with the teachings based thereon. According to the above description, it is apparent that the structure required for constructing such devices is. In addition, the present disclosure is not directed to any particular programming language either. It should be understood that various programming languages ​​may be utilized to implement the disclosure described herein, and the description of the specific languages ​​above is intended to disclose the best mode of implementation of the present disclosure.

[0226] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0227] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various disclosed aspects, in the above description of exemplary embodiments of the present disclosure, various features of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed disclosure requires more features than are expressly recited in each claim.

[0228] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0229] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present disclosure and form different embodiments. For example, any one of the claimed embodiments may be used in any combination.

[0230] The various component embodiments of the present disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the device according to the embodiments of the present disclosure. The present disclosure may also be implemented as a device or device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present disclosure may be stored on a computer-readable medium, or may have the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0231] It should be noted that the above embodiments illustrate the present disclosure rather than limit the present disclosure, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present disclosure may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A method for rendering a virtual scene, comprising: Acquire scene instantiation data corresponding to the virtual scene; wherein the scene instantiation data is used to characterize a plurality of scene modules included in the virtual scene, and the type and quantity of instance rendering units included in each scene module; the plurality of scene modules reuse at least two instance rendering units; Multiple instance rendering units of the same type contained in the multiple scene modules are batch rendered.

2. The method according to claim 1, wherein: The multiple instance rendering units of the same type correspond to the same instance basic information; The batch rendering of multiple instance rendering units of the same type contained in the multiple scene modules includes: For any instance rendering unit, obtaining instance transformation data of the instance rendering unit; According to the instance transformation data, the instance basic information of the instance rendering unit is transformed, and the instance rendering unit after the transformation is rendered.

3. The method according to claim 2, wherein: The instance basic information includes: mesh information associated with the instance shape, and / or texture information associated with the instance material; the instance transformation data includes: first transformation data for transforming the mesh information, and / or second transformation data for transforming the texture information; The transforming of the instance basic information of the instance rendering unit according to the instance transformation data comprises: According to the first transformation data, the mesh information of the instance rendering unit is transformed by a vertex shader; and / or, According to the second transformation data, the texture information of the instance rendering unit is transformed by a pixel shader.

4. The method according to claim 2 or 3, wherein: The acquiring the instance transformation data of the instance rendering unit comprises: Acquire instance transformation data of the instance rendering unit from an instance attribute matrix of the instance rendering unit; The instance attribute matrix is ​​an N*N matrix, and includes: a first matrix area and a second matrix area, the instance transformation data is stored in the first matrix area, and the first matrix area at least includes: an area corresponding to the Nth row and the Nth column of the instance attribute matrix; wherein N is a natural number.

5. The method according to claim 4, wherein: The second matrix area of ​​the instance attribute matrix is ​​used to store instance orientation data of the instance rendering unit; Wherein, the instance orientation data includes: instance position data, instance rotation data, and / or instance scaling data; The instance scaling data is used to represent a scaling direction of the instance rendering unit; wherein the scaling direction includes: a first direction and / or a second direction perpendicular to the first direction.

6. The method according to any one of claims 1 to 5, wherein: The scene instantiation data includes: a plurality of module instantiation data corresponding to the plurality of scene modules; wherein the module instantiation data includes: an instance attribute data set generated according to the type and quantity of instance rendering units included in the corresponding scene module; Wherein, the number of instance attribute data sets contained in the module instantiation data corresponding to each scene module matches the number of types of instance rendering units contained in the scene module; Furthermore, each instance attribute data set has a type identifier, and the type identifier is used to indicate the type of the instance rendering unit corresponding to the instance attribute data set.

7. The method according to claim 6, wherein: Each instance attribute data set is used to store instance transformation data of a plurality of instance rendering units of the same type corresponding to the type identifier contained in the scene module; Before batch rendering of the multiple instance rendering units of the same type contained in the multiple scene modules, the method further includes: Determine instance attribute data sets having the same category identifier among the plurality of module instantiation data; According to the instance attribute data set having the same type identifier, a plurality of instance rendering units of the same type included in the plurality of scene modules are determined.

8. The method according to any one of claims 1 to 7, wherein: The number of rendering times of the batch rendering is the same as the number of types of instance rendering units included in the multiple scene modules.

9. The method according to any one of claims 1 to 8, wherein: The virtual scene is a game scene in a game application, and obtaining scene instantiation data corresponding to the virtual scene includes: Determining a current visible area in the virtual scene according to a camera position in the game scene; Dynamically obtain scene instantiation data corresponding to the current visible area, and load the scene instantiation data into the memory space.

10. The method according to any one of claims 1 to 9, wherein: After acquiring the scene instantiation data corresponding to the virtual scene, the method further includes: Determining, according to the occlusion relationship, invisible instance rendering units included in the scene instantiation data; The invisible instance rendering unit is cropped to obtain processed scene instantiation data, so as to perform batch rendering on the processed scene instantiation data.

11. The method according to any one of claims 1 to 10, wherein: The scene module is determined in the following way: Determining a structure included in the virtual scene and a reference surface included in the structure according to the structural information of the virtual scene; According to the size and / or shape of the reference plane, a reference line contained in the reference plane and a scene module corresponding to the reference line are determined.

12. A virtual scene rendering device, comprising: An acquisition module, adapted to acquire scene instantiation data corresponding to a virtual scene; wherein the scene instantiation data is used to characterize a plurality of scene modules included in the virtual scene, and the type and quantity of instance rendering units included in each scene module; the plurality of scene modules reuse at least two instance rendering units; The rendering module is adapted to batch render multiple instance rendering units of the same type contained in the multiple scene modules.

13. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the virtual scene rendering method according to any one of claims 1-11.

14. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and wherein the executable instruction enables a processor to execute the virtual scene rendering method according to any one of claims 1 to 11.

Citation Information

Cited By

  • Dynamic creation and management method and device for multiple rendering scenes, equipment and medium

    CN119919558A