Detail level asset generation method and system, electronic equipment and medium

Abstract

The invention relates to a detail level asset generation method and system, electronic equipment and a medium. The method comprises the following steps: acquiring context information of a virtual object in a rendering environment; determining a perception urgency degree of the virtual object based on the context information; determining whether the LOD level of the virtual object needs to be raised to a target LOD level or not based on the perception urgency degree of the virtual object and the current computing power budget; and if the LOD level of the virtual object needs to be raised to the target LOD level, generating the target LOD level asset for the virtual object according to the current LOD level asset of the virtual object, and replacing the current LOD level asset of the virtual object with the target LOD level asset. By adopting the technical scheme, the balance between the rendering performance and the visual quality can be effectively improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically to a method, system, electronic device, and medium for generating assets at a level of detail. Background Technology

[0002] In modern real-time rendering systems, especially in applications such as large-scale open-world games, virtual reality, and the metaverse, Level of Detail (LOD) technology is widely used to maintain high frame rates and visual quality with limited computing resources. Traditional LOD systems rely on artists pre-creating models and textures at multiple levels of precision. While this approach alleviates rendering pressure to some extent, it also introduces fundamental drawbacks such as huge storage costs, rigid content, and cumbersome production processes.

[0003] Currently, generative AI offers new possibilities for dynamic content generation. However, when applied directly to real-time rendering environments, in scenarios where multiple virtual objects compete for limited computing power, it can lead to disordered computational competition, resulting in a poor balance between rendering performance and visual quality. Summary of the Invention

[0004] This application provides a method, system, electronic device, and medium for generating level-of-detail assets, which can effectively improve the balance between rendering performance and visual quality.

[0005] In a first aspect, embodiments of this application provide a method for generating level-of-detail assets, including: Obtain context information of virtual objects in the rendering environment; Based on the context information, the perceived urgency of the virtual object is determined; Based on the perceived urgency of the virtual object and the current computing power budget, determine whether it is necessary to upgrade the LOD level of the virtual object to the target LOD level; If it is necessary to upgrade the LOD level of the virtual object to the target LOD level, then based on the current LOD level assets of the virtual object, the target LOD level assets are generated for the virtual object, and the current LOD assets of the virtual object are replaced with the target LOD level assets.

[0006] Secondly, embodiments of this application provide a level-of-detail asset generation system, including: The data acquisition module is used to acquire context information of virtual objects in the rendering environment; The determination module is used to determine the perceived urgency of the virtual object based on the context information; and then, based on the perceived urgency of the virtual object and the current computing power budget, determine whether it is necessary to raise the LOD level of the virtual object to the target LOD level. The generation execution module is used to generate the target LOD level asset for the virtual object based on the current LOD level asset of the virtual object when the LOD level of the virtual object needs to be raised to the target LOD level, and replace the current LOD asset of the virtual object with the target LOD level asset.

[0007] Thirdly, embodiments of this application also provide an electronic device, including a memory storing multiple instructions; a processor loads instructions from the memory to execute the steps of any of the level-of-detail asset generation methods provided in embodiments of this application.

[0008] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the steps of any of the level-of-detail asset generation methods provided in embodiments of this application.

[0009] Fifthly, embodiments of this application also provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps in any of the level-of-detail asset generation methods provided in embodiments of this application.

[0010] The solution adopted in this application embodiment is based on the context information of virtual objects in the rendering environment; the perceived urgency of the virtual object is determined; based on the perceived urgency of the virtual object and the current computing power budget, it is determined whether the LOD level of the virtual object needs to be upgraded to the target LOD level; if the LOD level of the virtual object needs to be upgraded to the target LOD level, then according to the current LOD level asset of the virtual object, the target LOD level asset is generated for the virtual object, and the current LOD asset of the virtual object is replaced with the target LOD level asset; therefore, by using the perceived urgency of the virtual object and the current computing power budget, when it is determined that the LOD level needs to be upgraded, the current LOD asset of the virtual object is replaced with the target LOD level asset, so that the virtual object is dynamically switched to an appropriate LOD level based on the perceived urgency of the virtual object and the current computing power budget, thereby improving the balance between visual quality and rendering efficiency, and making the user experience better. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart of an embodiment of the method for generating assets at the level of detail provided in this application. Figure 2 This is a schematic diagram of a material projection back to baking process provided in the embodiments of this application; Figure 3 This is a schematic diagram of a LOD level upgrade / downgrade process provided in the embodiments of this application; Figure 4 This is a schematic diagram of the module of the Level of Detail (LOD) asset generation system provided in the embodiments of this application; Figure 5 This is a schematic diagram of the system architecture of the Level of Detail (LOD) asset generation system provided in the embodiments of this application; Figure 6 This is a schematic diagram of the client-to-cloud structure of the Level of Detail (LOD) asset generation system provided in this application embodiment; Figure 7 This is a schematic diagram of the synchronization mechanism of the Level of Detail (LOD) asset generation system provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. At the same time, in the description of the embodiments of this application, the terms "first," "set," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. Thus, features defined with "first" or "set" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0014] Specifically, this embodiment will be described from the perspective of a level-of-details asset generation device, which can be integrated into an electronic device. That is, the level-of-details asset generation method of this application embodiment can be executed by an electronic device. Optionally, the electronic device may include a terminal device. The terminal device may be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, game console, or personal computer (PC), etc.

[0015] The method for generating Level of Detail (LOD) assets provided in this application can be applied to systems such as LOD asset generation systems. This LOD asset generation system may include a terminal device and a server. The terminal device may be a device including receiving and transmitting hardware, i.e., a device with receiving and transmitting hardware capable of performing bidirectional communication over a bidirectional communication link. The terminal device and the server can communicate bidirectionally via a network.

[0016] Optionally, the server can be a standalone server, or a server network or server cluster, including but not limited to computers, network hosts, single network servers, multiple network server sets, or cloud servers composed of multiple servers. Cloud servers consist of a large number of computers or network servers based on cloud computing.

[0017] The following detailed description is provided in conjunction with the accompanying drawings. In this embodiment, the execution subject is a terminal device as an example. It should be noted that the order of description in the following embodiments is not intended to limit the preferred order of the embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the accompanying drawings.

[0018] To facilitate understanding of the technical solution of this application, the relevant concepts involved in this application will be introduced first.

[0019] Please refer to Figure 1 The specific process of this detailed asset generation method can be summarized in steps 101 to 104, where: Step 101: Obtain the context information of virtual objects in the rendering environment; Specifically, it can collect contextual information of virtual objects in the rendering environment in real time. The contextual information includes at least one of the following parameters: the screen projection area of ​​the virtual object, the distance between the virtual object and the camera, the gaze point information, the relative motion vector between the virtual object and the camera, the probability of occlusion, and the importance label.

[0020] In some embodiments, importance tags can be tags assigned by game logic. Importance tags can be, for example, quest items, key NPCs, and rare items. This specification does not impose specific limitations.

[0021] In some embodiments, there are typically multiple virtual objects in the rendering environment, and the operations in steps 101-104 are performed for each virtual object.

[0022] Step 102: Determine the perceived urgency of the virtual object based on contextual information; In some embodiments, contextual information can be input into a linearly weighted model to obtain the perceived urgency of virtual objects; alternatively, contextual information can be input into a nonlinear model trained by machine learning, which also yields the perceived urgency of virtual objects. When training the nonlinear model, historical contextual information can be used as the model input, while historical perceived urgency labels of virtual objects can be used as the model output. Thus, by training the model using its input and output, a trained nonlinear model can be obtained. The following example uses a linearly weighted model.

[0023] Specifically, if the context information includes at least two parameters among the following: the screen projection area of ​​the virtual object, the distance between the virtual object and the camera, the gaze point information, the relative motion vector between the virtual object and the camera, the probability of occlusion, and the importance label, then the perceptual urgency score (PUS) of the virtual object can be determined based on the weighted sum of each parameter in the context information.

[0024] For example, if the contextual information includes the screen projection area of ​​the virtual object, the distance between the virtual object and the camera, the gaze point information, the relative motion vector between the virtual object and the camera, the probability of occlusion, and all parameters in the importance label, then the linear weighted model can be specifically represented by the following formula (1), as follows: PUS = A×ProjectedArea + B / Distance + C×Gaze + D×Motion + E×(1 OcclusionProb) + F×Importance (1) In the above formula (1), A, B, C, D and F are all weight coefficients, which can be calibrated according to the actual situation. ProjectedArea represents the screen projection area of ​​the virtual object. At this time, the larger the virtual object is in the picture, the easier it is for the player to notice the details, so the urgency of its LOD upgrade is higher. Distance represents the distance between the virtual object and the camera. At this time, the closer the object is, the more high-quality rendering is needed. Since it is related to ProjectedArea, but Distance can provide additional information such as distant but huge buildings. Gaze represents the gaze point information. At this time, if the player is looking at an object, even if it is not large in the picture, the LOD should be upgraded first. Motion represents the relative motion vector between the virtual object and the camera. At this time, objects that are fast approaching or about to enter the player's field of vision need to generate a higher LOD in advance to avoid the image quality collapse when suddenly getting close. OcclusionProb represents the probability that the virtual object is occluded. At this time, the higher the occlusion probability, the more the LOD upgrade can be postponed. Virtual objects with low occlusion probability should be upgraded faster. (1 OcclusionProb) indicates the visibility of virtual objects; Importance indicates the logical importance label. In this case, even if these virtual objects are not currently in the direction of gaze or the screen is small, the PUS may still be improved because the game logic requires high-quality display.

[0025] In particular, if the virtual object is larger in the screen by ProjectedArea, the player is more likely to notice the details, and therefore the urgency of upgrading its LOD is higher.

[0026] In some embodiments, PUS can also be obtained based on the screen projection area, the distance between the virtual object and the camera, gaze point information, the relative motion vector between the virtual object and the camera, the probability of occlusion, and a weighted sum of two, three, four, or four parameters from the importance label. For example, PUS can be A×ProjectedArea + B / Distance, A×ProjectedArea + B / Distance + D×Motion, or A×ProjectedArea + C×Gaze + D×Motion + F×Importance, etc.

[0027] In this way, the linear weighted model can uniformly weight the screen projection area of ​​the virtual object, the distance between the virtual object and the camera and the probability of the virtual object being occluded, the player behavior factors corresponding to the gaze point information and the relative motion vector between the virtual object and the camera, and the game logic factors corresponding to the importance label, to obtain a score, namely PUS. The higher the PUS value, the more urgently the virtual object needs computing power for LOD upgrade.

[0028] Step 103: Based on the perceived urgency of the virtual object and the current computing power budget, determine whether it is necessary to upgrade the LOD level of the virtual object to the target LOD level; In some embodiments, the number of LOD levels can be increased or decreased according to application requirements. For example, it can be simplified to 3 levels (placeholder → 2.5D shell → 3D model) or expanded to 4 or 5 levels, etc. This specification does not impose specific limitations.

[0029] In some embodiments, if a virtual object appears in the rendering environment for the first time, an initial LOD asset of the virtual object is generated based on the basic information of the virtual object, and the initial LOD asset is used as the current LOD asset of the virtual object.

[0030] Specifically, when a virtual object first appears in the rendering environment, its basic information is obtained. This basic information may include the virtual object's description or category. The description may be a text prompt such as a warrior running in a forest. The category of the virtual object may be such as flowers, rocks, mountains, clouds, rivers, icebergs, houses, castles, bridges, lighthouses, road signs, walls, spaceship landing pads, magic portals, humans, elves, orcs, and robots, etc. This manual does not impose specific limitations.

[0031] Specifically, when a virtual object first appears in the rendering environment, a minimum placeholder visual must be displayed. At this time, the LOD level of the virtual object can be determined to be LODO. Then, based on the basic information of the virtual object, the initial LOD asset of the virtual object, namely LOD0 asset, is generated.

[0032] In some embodiments, a low-resolution image can be generated using a text-to-image model based on the basic information of the virtual object, serving as a LOD0 asset. Alternatively, a basic geometry can be generated directly, and a color can be assigned to the basic geometry, with the resulting colored geometry serving as the LOD0 asset. This achieves the shortest time to first pixel (TTFP), avoiding holes or white models in the rendering environment and improving the overall quality of the rendering environment.

[0033] In some embodiments, since multiple LOD levels are set, in addition to the LOD level corresponding to the placeholder, a corresponding preset urgency threshold is set for each LOD level. For example, if there are 5 LOD levels, LOD0 represents placeholder visibility, LOD1 represents high-resolution texture, LOD2 represents 2.5D depth shell, LOD3 represents low-polygon coarse model and LOD4 represents high-precision model and high-resolution physically based rendering (PBR).

[0034] In some embodiments, after obtaining the perceived urgency and the current computing power budget, the perceived urgency is compared with a first preset urgency, and it is detected whether the first set computing power required for the virtual object's LOD level to rise to the target LOD level is less than the current computing power budget. If it is determined that the perceived urgency is greater than the first preset urgency and the current computing power budget is greater than the first set computing power, it is determined that the virtual object's LOD level needs to be raised to the target LOD level. If the perceived urgency of the virtual object is greater than the first preset urgency and the current computing power budget is not greater than the first set computing power, it is determined that the virtual object's current LOD level remains unchanged.

[0035] In some embodiments, the target LOD level is the adjacent level to the current LOD level of the virtual object. Typically, the target LOD level is the level above the current LOD level; for example, if the current LOD level is LOD0, then the target LOD level is LOD1. Of course, the target LOD level can also be any LOD level above the current LOD level; for example, if the current LOD level is LOD0, then the target LOD level is LOD2 or LOD3, etc. This specification does not impose specific limitations.

[0036] In some embodiments, it can also be determined whether the LOD level of a virtual object needs to be downgraded to a set LOD level based on context information or the current computing power budget; if it is necessary to downgrade the LOD level of a virtual object to a set LOD level, the current LOD asset of the virtual object is replaced with the set LOD level asset based on the pre-stored set LOD level asset of the virtual object.

[0037] Specifically, in order to ensure that the current computing budget is sufficient to render all virtual objects in the rendering environment, thereby improving the balance between rendering performance and visual quality, it is also necessary to determine whether the LOD level of the virtual objects should be reduced based on context information or the current computing budget.

[0038] In some embodiments, the LOD level is set to the level adjacent to the current LOD level of the virtual object. Typically, the LOD level is the next level below the current LOD level. For example, if the current LOD level is LOD1, then the LOD level is set to LOD0. Of course, the LOD level can also be any LOD level below the current LOD level. For example, if the current LOD level is LOD3, then the LOD level is set to LOD2, LOD1, or LOD0, etc. This specification does not impose any specific limitations.

[0039] In some embodiments, the context information includes the distance between the virtual object and the camera. First, it is detected whether the distance between the virtual object and the camera is greater than a set camera distance corresponding to the current LOD level, and whether the current computing power budget is less than a second set computing power required for the virtual object to be at the current LOD level. If the distance between the virtual object and the camera is detected to be greater than the set camera distance, or the current computing power budget is less than the second set computing power, then it is determined that the LOD level of the virtual object needs to be lowered to the set LOD level. If the distance between the virtual object and the camera is detected to be no greater than the set camera distance, and the current computing power budget is no less than the second set computing power, then it is determined that the LOD level of the virtual object remains unchanged.

[0040] In some embodiments, the context information includes the screen projection area of ​​the virtual object. In this case, it is first detected whether the screen projection area of ​​the virtual object is less than a set projection area required for the virtual object to be at the current LOD level, and whether the current computing power budget is less than a second set computing power required for the virtual object to be at the current LOD level. If the screen projection area of ​​the virtual object is detected to be less than the set projection area, or the current computing power budget is less than the second set computing power, it is determined that the LOD level of the virtual object needs to be lowered to the set LOD level. If the screen projection area of ​​the virtual object is detected to be not less than the set projection area, and the current computing power budget is not less than the second set computing power, it is determined that the LOD level of the virtual object remains unchanged.

[0041] In this way, the LOD level of virtual objects can be dynamically adjusted based on context information and the current computing power budget to improve the balance between rendering performance and visual quality.

[0042] In one embodiment, the current computing budget can be determined from available computing resources both locally and in the cloud, making the current computing budget a dynamic value.

[0043] In some embodiments, when multiple virtual objects are being rendered, rendering tasks can be processed based on the level of each PUS. Alternatively, when multiple virtual objects are being rendered, global optimization can be performed to determine which virtual objects need LOD upgrades and execute those upgrades; and to determine which virtual objects maintain their LOD or downgrade their LOD, thereby dynamically adjusting the LOD level of virtual objects and further improving the balance between rendering performance and visual quality.

[0044] Specifically, during global optimization, the optimization objective is to maximize the total weighted perceptual quality improvement in the rendering environment within the next time window. This can be modeled as a variant of a multidimensional, multi-choice knapsack problem, as shown in the following formula (2): (2) The specific constraints are shown in the following formula (3): (3) In formulas (2) and (3), This represents the perceived urgency of virtual object i. This indicates the current LOD level of virtual object i. This represents the target LOD level of virtual object i, and ΔQuality indicates whether virtual object i is from... Upgraded to The expected quality gain, Indicates virtual object i from Upgraded to The required computing power, Budget represents the current computing power budget. In actual implementation, it can be approximated by a greedy algorithm or heuristic method, for example, by using a ratio sorting of (PUS·ΔQuality) / Cost. In this way, the LOD level of each virtual object can be dynamically adjusted by the above formulas (2) and (3) to ensure that the total weighted perceptual quality improvement in the rendering environment is maximized under the current computing power budget, and further improve the balance between rendering performance and visual quality.

[0045] In some embodiments, player movement trajectory and gaze direction data can be utilized to predict virtual objects that the player might focus on in the next few seconds using a simple motion model or a more complex Recurrent Neural Network (RNN) or Long Short-Term Memory (LSTM) model. For these predicted virtual objects, the LOD asset generation task can be initiated in advance to generate corresponding LOD assets. At this time, the LOD assets can be one or more of LOD0, LOD1, and LOD2, even if the PUS of these predicted virtual objects has not yet reached the threshold, thereby reducing perceptual latency and improving the user experience.

[0046] Step 104: If it is necessary to upgrade the LOD level of a virtual object to the target LOD level, generate the target LOD level asset for the virtual object based on the current LOD level asset of the virtual object, and replace the current LOD asset of the virtual object with the target LOD level asset.

[0047] In some embodiments, when it is necessary to upgrade the LOD level, a task to generate assets of the target LOD level is triggered. The current LOD level assets are input into the asset generation model corresponding to the target LOD level to obtain the target LOD level assets. Then, the current LOD assets of the virtual object are replaced with the target LOD level assets, and then the target LOD level assets are rendered, so that the rendered virtual object can be visually presented with higher quality.

[0048] For example, taking a LOD level of 5 as an example, it includes: LOD0 represents a placeholder visual with a very low-cost initial representation, such as a bulletin board or basic geometry; LOD1 represents a high-resolution textured plane / shell with a 2D or 2.5D representation with higher resolution textures; LOD2 represents a 2.5D depth patch with a parallax shell (impostor) containing depth and normal information, which can respond to limited viewpoint changes; LOD3 represents a low-polygon coarse model with a simplified 3D mesh with correct topology and basic physical properties (collision capability); LOD4 represents a high-precision model and high-resolution PBR, with a high-precision model in its final form and fully physically rendered (PBR) materials.

[0049] The generation and enhancement of a virtual object's LOD follows a progressive process: first, it enters LOD0 generation. The trigger condition for LOD0 is the virtual object entering the player's view frustum for the first time. The input for LOD0 is the basic description or category of the virtual object. The generation process of LOD0 is as follows: execute an extremely fast T2I model to generate a low-resolution (e.g., 64x64) image as a billboard texture; or directly generate a basic geometry (e.g., a cube or capsule) and assign it a solid color. The output of LOD0 is a renderable LOD0 asset. LOD0 can achieve the shortest first pixel time, avoiding "holes" or "white models" in the scene.

[0050] After LOD0 generation, LOD1 generation can begin. The trigger condition for LOD1 is that PUS reaches the LOD1 threshold and the computing power required for LOD1 is less than the current computing power budget. The input for LOD1 is the low-resolution image of LOD0 and the basic description or category of the virtual object. The generation process of LOD1 is as follows: using a super-resolution model or an inpainting model, the image of LOD0 is upscaled to a higher resolution (such as 512x512). The output of LOD1 is the LOD1 asset with a high-resolution Albedo map.

[0051] After LOD1 generation, LOD2 generation can begin. LOD2 is triggered when PUS reaches the LOD2 threshold and the required computing power for LOD2 is less than the current computing power budget. The input for LOD2 is the high-resolution image of LOD1. The LOD2 generation process involves calling monocular depth estimation and normal estimation models to generate a depth map and normal map corresponding to the LOD1 image. Then, the Impostor builder is used to construct a 2.5D depth patch with parallax effects from the depth map and normal map corresponding to the LOD1 image. The output of LOD2 is a LOD2 asset containing Albedo, Depth, and Normal maps. This LOD2 asset can provide a sense of depth and delay the appearance of weaknesses at mid-range distances when the player has slight displacement or rotation. The depth map and normal map generated by LOD2 will serve as strong geometric priors for LOD3 reconstruction. After LOD2 generation, LOD3 generation can begin. LOD3 is triggered when PUS reaches the LOD3 threshold and the required computing power is less than the current computing power budget. LOD3 is typically triggered when an object approaches or when physical interaction is required. The input to LOD3 is the image from LOD1, the depth map or normal map from LOD2, and a basic description or category of the virtual object. The LOD3 generation process involves calling a single-view or multi-view 3D reconstruction model, such as neural surface reconstruction (SDF / NeRF). At this point, the depth map from LOD2 is used to initialize or constrain the reconstruction process, ensuring that the generated mesh is consistent with the previous 2.5D representation in macroscopic shape, and performing mesh simplification and generating simplified colliders after generation. The output of LOD3 is a low-polygon, topologically correct, and collision-enabled LOD3 mesh model. LOD3 assets can meet the basic shape requirements for close-up observation and physical interaction. Furthermore, the LOD3 mesh topology provides the basic framework for the geometric refinement of LOD4.

[0052] After LOD3 generation, LOD4 generation can begin. LOD4 is triggered when PUS reaches the LOD4 threshold and the required computing power is less than the current computing power budget. It is typically triggered when the player observes or focuses at very close range. The input for LOD4 is the LOD3 mesh, the LOD1 image, and a basic description or category of the virtual object. The LOD4 generation process involves: first, geometric refinement, specifically adding geometric details to the LOD3 mesh, such as using subdivision surfaces or generating displacement maps; then, PBR material completion, which involves calling a material generation model to decompose and complete the LOD1 Albedo into a full PBR channel, including high-frequency details such as normals, roughness, and metallicity. The output of LOD4 is a high-precision mesh and a complete high-resolution PBR texture set, providing the highest quality visual presentation.

[0053] Thus, the above approach decomposes the time-consuming generation process into multiple stages, directly mapping the intermediate products of each stage to a renderable Level of Detail (LOD). This allows for a trade-off between time and quality, dynamically and progressively improving the visual details of objects in the rendering environment without blocking the rendering thread. This approach eliminates the need for pre-storing multiple LOD assets; only generation instructions are required, significantly reducing package size and network transmission volume, thereby lowering storage and bandwidth requirements. Furthermore, it can respond in real-time to scene changes in the rendering environment, dynamically creating LODs for procedural or user-generated content, greatly enhancing the flexibility and diversity of the world and improving content dynamism. Additionally, asynchronous scheduling and progressive updates can be used for one or more LOD levels to ensure the generation process does not affect the stability of the rendering frame rate, avoiding stuttering and blank screens, thus guaranteeing rendering performance.

[0054] In some embodiments, to avoid abrupt jumps or misalignments during LOD switching, geometric alignment, material continuity, and temporal smoothing can be employed.

[0055] In some embodiments, when using geometric alignment, during the transition from LOD2 to LOD3, when generating the LOD3 coarse model, the depth map generated by LOD2 can be converted into a point cloud. This point cloud serves as a strong constraint or initial shape for 3D reconstruction algorithms such as SDF, ensuring that the macroscopic shape of LOD3 closely matches the depth map of LOD2. Furthermore, during the transition from LOD3 to LOD4, since LOD4 is a refinement of LOD3 with a consistent topology, a smooth transition can be achieved through vertex displacement or skeleton skinning. If the topology is inconsistent, fine alignment is performed using a fast Iterated ClosestPoints (ICP) algorithm before the transition.

[0056] In some embodiments, when material continuity is employed, the first texture corresponding to the current LOD asset can be projected onto the mesh surface of the target LOD asset to generate projected color information; the projected color information is baked into the UV space corresponding to the target LOD asset to generate a second texture corresponding to the target LOD asset; and the current LOD asset of the virtual object is replaced with the target LOD asset based on the second texture.

[0057] Specifically, material continuity employs a projection-rebaking mechanism to ensure smooth transfer of texture information between Levels of Dimensions (LODs) with different UV layouts. For example, taking LOD levels LOD0-LOD4 as an example, after LOD4 is generated, the system projects the high-resolution texture of LOD1 or LOD2 onto the mesh surface of LOD4 from the perspective of the rendering camera or multiple virtual cameras. Then, the color information projected onto the surface is baked into the new UV layout of LOD4, generating a new Albedo map. Through this mechanism, the initial texture of LOD4 is visually highly consistent with LOD1 or LOD2, laying the foundation for a smooth transition. Of course, the same operation can be performed on LOD3 to ensure that the initial texture of LOD3 is visually highly consistent with LOD1 or LOD2.

[0058] Furthermore, to address the seam and stretching issues caused by projection, multi-view projection fusion can be employed, and the boundaries of the chart can be expanded by pixels in UV space to fill the gaps, thus resolving the seam and stretching issues in the generation of LOD4 assets.

[0059] For example, see Figure 2First, camera 20 executes step S21, sending rendering parameters. Camera 20 sends the view matrix and projection matrix to baker 23, where the view matrix and projection matrix are used to project the texture coordinates of LOD1 / 2 onto the geometry surface of LOD3 / 4. Then, LOD1 / 2 texture 21 executes step S22, inputting the source texture. Here, the source texture is a high-resolution Albedo map generated in the LOD1 or LOD2 stage. LOD1 / 2 texture 21 inputs the source texture to baker 23. Next, LOD3 / 4 mesh 22 executes step S23, target geometry and UVs, inputting the LOD3 mesh model and its UV layout information into baker 23. Then, baker 23 sequentially executes step S24... Steps S25 and S26 are as follows: Step S24: Project L1Tex onto the L3Mesh surface. At this time, baker 23 uses the rendering parameters obtained in step S21 to project the source texture obtained in step S22 onto the surface of the LOD3 mesh model in step S23. Step S25: Bake the surface color into the UV space of L3Mesh. Baker 23 bakes or samples the temporary color information obtained from the previous projection into the UV layout of the LOD3 mesh model, thereby generating a brand new texture map that perfectly matches the UVs of the LOD3 mesh model. Step S26: Output the new texture that matches L3Mesh. Baker 23 outputs the final generated texture map, thus forming a new PBR texture 24. In this way, the above technical solution can project the texture of LOD1 / 2 onto the LOD3 / 4 geometry through camera projection, and then bake it onto the UV map of LOD3 / 4. This effectively avoids material misalignment or style discontinuity when the UV layout is inconsistent in different LOD stages.

[0060] In some embodiments, when temporal smoothing is used, the current LOD asset and the target LOD asset can be rendered simultaneously within a preset switching time window; color mixing is performed on the rendering output of the current LOD asset and the target LOD asset.

[0061] Specifically, when using temporal smoothing, alpha blending can be performed first. During a short time window of LOD switching (e.g., 100-200ms), both the old and new LODs are rendered simultaneously, and their outputs are alpha-blended. The opacity of the old LOD gradually changes from 1 to 0, while the new LOD changes from 0 to 1. Then, temporal accumulation is performed. Here, historical frame accumulation mechanisms from post-processing, such as Temporary Anti-Alising (TAA), are used to distribute the impact of the LOD switching across multiple frames, further smoothing the transition. In this way, temporal smoothing can further reduce the occurrence of abrupt changes and misalignments.

[0062] For example, see Figure 3 When virtual object 30 first enters the field of view, it enters LOD0 and generates LOD0 assets. After entering LOD0, it checks whether the virtual object's PUS is greater than the preset urgency level corresponding to LOD1, denoted as T1, and whether the first set computing power required for the virtual object to rise to LOD1 is less than the current computing power budget. If PUS > T1 and the first set computing power is less than the current computing power budget, then... Figure 2 The display shows PUS>T1&&budget OK, where && indicates that if PUS>T1, then it enters LOD1; when entering LOD2 from LOD1, it is also necessary to confirm PUS>T2&&budget OK, where PUS>T2 means that the virtual object's PUS is greater than the preset urgency level corresponding to LOD2, denoted as T2, and budget OK means that the set computing power required for the virtual object to rise from LOD1 to LOD2 is less than the current computing power budget; and when entering LOD3 from LOD2, it is also necessary to confirm PUS>T3&&budget OK, where PUS>T1, T2, T3 ... T3 indicates that the virtual object's PUS is greater than the preset urgency level corresponding to LOD3, denoted as T3. Budget OK indicates that the set computing power required for the virtual object to rise from LOD2 to LOD3 is less than the current computing power budget. Similarly, when moving from LOD3 to LOD4, it is also necessary to determine that PUS>T4&&Budget OK, where PUS>T4 means that the virtual object's PUS is greater than the preset urgency level corresponding to LOD4, denoted as T4, and Budget OK means that the set computing power required for the virtual object to rise from LOD3 to LOD4 is less than the current computing power budget.

[0063] Specifically, when downgrading from LOD4 to LOD3, the system checks whether the distance between the virtual object and the camera is greater than the set camera distance corresponding to LOD4, and whether the current computing power budget is less than the second set computing power required for the virtual object to be at LOD4. If the distance is greater than the set camera distance corresponding to LOD4, or the current computing power budget is less than the second set computing power, then it is determined that the LOD level of the virtual object needs to be downgraded from LOD4 to LOD3. Figure 2 The display shows a downgrade (distance / budget); and, when downgrading from LOD3 to LOD2, in Figure 2 The display indicates a downgrade. At this point, it can be determined that the distance between the virtual object and the camera is greater than the set camera distance corresponding to LOD3, or that the current computing power budget is less than the set computing power required for the virtual object to be at LOD3; and, when downgrading from LOD2 to LOD1, in Figure 2 The display indicates a downgrade. At this point, it can be determined that the distance between the virtual object and the camera is greater than the set camera distance corresponding to LOD2, or that the current computing power budget is less than the set computing power required for the virtual object to be at LOD2. When downgrading from LOD1 to LOD0, in... Figure 2If the display shows a downgrade, it can be determined that the distance between the virtual object and the camera is greater than the set camera distance corresponding to LOD1, or that the current computing power budget is less than the set computing power required for the virtual object to be at LOD1.

[0064] Furthermore, when the LOD level of the virtual object is any level from LOD0 to LOD4, if the virtual object 30 leaves the field of view, it will enter the cache reclamation 31. The virtual object 30 will first enter the cache in the ICache instruction cache, and after a period of time, it will be reclaimed from the ICache, that is, the LOD asset of the virtual object will be deleted from the ICache.

[0065] Based on the above technical solution, when it is determined that the LOD level needs to be upgraded, the current LOD asset of the virtual object can be replaced with the target LOD level asset by using the perceived urgency of the virtual object and the current computing power budget. This allows the virtual object to be dynamically switched to the appropriate LOD level based on its perceived urgency and current computing power budget, thereby improving the balance between visual quality and rendering efficiency and resulting in a better user experience.

[0066] This application also provides a level-of-detail asset generation system, such as... Figure 4 As shown, it includes: Data acquisition module 401 is used to acquire context information of virtual objects in the rendering environment; The determination module 402 is used to determine the perceived urgency of the virtual object based on context information; and then, based on the perceived urgency of the virtual object and the current computing power budget, to determine whether it is necessary to raise the LOD level of the virtual object to the target LOD level. The generation execution module 403 is used to generate a target LOD level asset for the virtual object based on the current LOD level asset of the virtual object when it is necessary to raise the LOD level of the virtual object to the target LOD level, and replace the current LOD asset of the virtual object with the target LOD level asset.

[0067] The generation and execution module includes multiple AI sub-modules, each responsible for generating one or more LOD-level assets.

[0068] In some embodiments, the generation execution module can be designed as a highly modular open framework, allowing developers to replace or add generation submodules as needed without modifying the upper-layer scheduling logic. All generation submodules must implement a standard interface, which may include three key methods: capability reporting, asynchronous execution of generation tasks, and result self-checking. Capability reporting is called when the generation submodule starts, returning its capability description, which contains one or more (quality, cost) tuples forming a latency-quality Pareto front curve. The Quality of Service (QoS) parameter in asynchronous execution of generation tasks can be passed in by the system's determination module, specifying the performance target of this call (e.g., "please complete within 80ms"). At this point, an appropriate internal configuration can be selected based on the QoS parameter. Result self-checking is used to self-check the generation results, ensuring they conform to basic specifications (e.g., normal range, geometric closure, etc.).

[0069] In some embodiments, the generation system may further include a determination module for maximizing the overall visual quality improvement of the scene while meeting real-time budget constraints. The determination module first obtains the capability description from the capability report, and then selects the most suitable "working point" (i.e. a quality-cost configuration) from the Pareto curve based on the current frame budget and the perceptual urgency of the virtual object, and passes it as a QoS parameter to the asynchronously executed generation task. In this way, the dynamic matching of upper-layer requirements and lower-layer capabilities is realized through this mechanism.

[0070] In practical applications, this generation system can typically be divided into five collaborative layers, such as... Figure 5 As shown, the system may include a scene awareness layer 51, a decision scheduling layer 52, a generation execution layer 53, an asset management and caching layer 54, and a rendering and transition layer 55. The scene awareness layer 51 is used to collect contextual information in the rendering environment in real time. This contextual information can serve as the basis for the decision scheduling layer's decisions. Contextual information includes the screen projection area of ​​virtual objects, the distance between virtual objects and the camera, foveation point information, the relative motion vector between virtual objects and the camera, the probability of occlusion, and importance labels, etc.

[0071] Furthermore, the decision scheduling layer 52 receives context information collected in real time by the scene awareness layer 51, calculates the corresponding PUS for each virtual object, and performs global optimization based on the PUS of all virtual objects and the current computing power budget of the generation system, dynamically adjusting the LOD level of each virtual object. This allows for the upgrading or downgrading of the LOD level of virtual objects, and the conversion of each upgrade request and downgrade request into a specific LOD generation task. The decision scheduling layer 52 is also used to predict player behavior and perform LOD level pre-fetching and pre-generation.

[0072] Furthermore, the generation execution layer 53 can contain a series of pluggable, independent generative AI sub-modules, each responsible for generating one or more LOD levels. Examples include text-to-image modules, super-resolution modules, monocular depth estimation modules, and 3D reconstruction modules. All modules execute asynchronously, without blocking the rendering thread. The asset management and caching layer 54 stores and manages the LOD assets generated by the generation execution layer 53 at each level. It typically uses (Stage, Ver, Seed) versioning for assets and employs LRU / TTL strategies for cache reclamation. This layer ensures the reusability and consistency of the generated results. Finally, the rendering and transition layer 55 seamlessly integrates the LOD assets cached in the asset management and caching layer 54 into the rendering pipeline. When a new LOD asset for a virtual object is generated, the rendering and transition layer 55 performs a seamless replacement operation, including geometric alignment, material continuity processing, and temporal smoothing transitions, ultimately submitting the appropriate LOD to the renderer.

[0073] In some embodiments, the decision scheduling layer 52 includes a scheduler, which is used to maximize the overall visual quality improvement of the scene based on PUS while satisfying real-time budget constraints. The scheduler does not simply process tasks according to PUS, but its optimization goal is to maximize the total weighted perceptual quality improvement of the entire scene in the next time window. For details, please refer to the discussion on global optimization above. For the sake of brevity, it will not be repeated here.

[0074] In some embodiments, the generation execution layer 53 is designed as a highly modular open framework, allowing developers to replace or add generation submodules as needed without modifying the upper-layer scheduling logic. The generation execution layer 53 typically uses a unified interface, and all generation modules must implement the standard interface IGenStage.<T_in, T_out> `capabilities() ->CapabilityProfile`: Called when the module starts, returns its basic description. `CapabilityProfile` contains one or more (quality, cost) tuples, forming a Pareto front curve of latency-quality. For example, `[(quality=0.6, latency=40ms), (quality=0.8, latency=120ms)]`. `infer_async(inputs: T_in, qos:dict) -> T_out`: Executes the generation task asynchronously. The `qos` parameter is passed in by the scheduler, specifying the performance target for this call (e.g., "complete within 80ms"). The module selects an appropriate internal configuration based on the `qos`. `validate(outputs: T_out) -> bool`: Performs a self-check on the generated results to ensure they conform to basic specifications (e.g., normal range, geometric closure, etc.).

[0075] In some embodiments, the generation system is applied to a client-to-cloud collaborative architecture to balance the low latency advantage of local computing with the powerful computing power of cloud computing, and to address network limitations through a streaming mechanism.

[0076] For example, see Figure 6 After receiving the LOD generation task, client 60 first executes step S610 to obtain local resource constraints, such as battery level, device temperature, CPU or GUP usage, and memory limits. Next, it executes step S620 to make a local decision, specifically deciding whether to execute the LOD generation task locally or upload it to the cloud based on resource availability and task urgency. This decision includes processing low-LOD and medium-LOD tasks locally, processing LOD2 generation tasks locally if the network is good, and processing computationally intensive LOD2, LOD3, and LOD4 generation tasks in the cloud. Following the allocation decision, it executes step S630 to process latency-sensitive tasks, used to generate the LOD level with the highest real-time requirements. Latency-sensitive tasks include LOD0 generation tasks corresponding to low-LOD tasks, LOD1 generation tasks corresponding to medium-LOD tasks, and LOD2 generation tasks. Client 60 also includes a rendering system 601, used for real-time rendering of the highest available LOD asset, LOD switching, cache management, and user interaction processing.

[0077] If the LOD generation task is determined to be a computationally intensive task, it enters the edge computing node 61. First, step S640 is executed to perform local caching, store hot assets, and support prefetching and region specialization. If the edge computing node 61 is determined to have sufficient computing power for the corresponding LOD generation task, step S650 is executed for intermediate layer processing to handle LOD2 / 3 generation tasks, providing region-based computing and reducing latency. If the edge computing node 61 is determined to have insufficient computing power for the corresponding LOD generation task, it enters the network transport layer 62.

[0078] In this process, the network transport layer 62 continuously executes steps S660 and network adaptation, real-time detection of bandwidth, latency, and connection type, and adaptively adjusts transmission strategies, such as prioritizing the transmission of low-frequency MIPMAPs. The scheduler can monitor network status in real time, i.e., monitor network adaptation in real time. If it is determined that the edge computing node 61 does not meet the computing power required for the corresponding LOD generation task, the network transport layer 62 executes step S670 and uplink, sending the LOD generation request to the cloud 63. The LOD generation request includes ObjectID, LOD level, QoS constraints, seed parameters, etc. The network transport layer 62 can also execute step S680 and downlink to receive the streaming generation results from the cloud 63. It can support streaming transmission of geometric chunks, texture fragments, and MIPMAP levels, and supports breakpoint resumption, pausing when the network is interrupted and continuing after recovery.

[0079] Furthermore, after receiving the uplink request sent in step S680, cloud 63 executes step S690, a computationally intensive task, including LOD3 generation task and LOD4 generation task. During the execution of step S610, cloud 63 will process LOD3 generation task and LOD4 generation task according to computing power resources 631, data processing 632 and caching system 633. The LOD3 generation task and LOD4 generation task will transmit their respective generated LOD products to rendering system 601 for rendering through step S690.

[0080] Thus, on the client side (60), tasks most sensitive to latency and with relatively low computational complexity are prioritized. Typically, this involves generating LOD0, LOD1, and LOD2, as well as small-scale, low-complexity LOD3 generation. For example, in a VR application requiring a 60Hz refresh rate, the client must render LOD0 placeholders within a 16.6ms frame budget, while the high-precision LOD4 model can be progressively filled in by the cloud within 1-3 seconds. On the cloud side (63), computationally intensive and time-consuming tasks are handled, primarily high-quality LOD3 reconstruction, LOD4 geometric refinement, and high-resolution PBR material generation. The cloud side (63) utilizes a large-scale GPU cluster, capable of completing computations that the client side cannot handle within a reasonable timeframe. At this point, the determination module can dynamically decide the execution location of LOD3 / 4 tasks based on real-time network conditions (bandwidth, latency), cloud server load, and the QoS requirements of the task itself.

[0081] In some embodiments, to avoid stuttering when transmitting large assets, the generation system can employ a chunked and progressive streaming mechanism. This can involve geometric chunking for data transmission; for example, a high-resolution mesh (LOD4) can be divided into multiple chunks based on a spatial octree, prioritizing the transmission of the chunk closest to the viewpoint. Alternatively, texture-based chunking can be used, such as transmitting PBR textures by MIPMAP layer and tiles, following a low-frequency-to-high-frequency principle. Low-resolution MIPMAP layers are transmitted first to quickly obtain the basic appearance, followed by high-resolution details downloaded in the background. Furthermore, if the network connection is interrupted and transmission pauses, the rendering layer continues to use the highest quality LOD that has been fully received. Once the network is restored, transmission can resume from the point of interruption, achieving seamless failure and breakpoint resumption.

[0082] In some embodiments, the generation system may also employ a hierarchical caching architecture, including a video memory hot cache for storing LOD assets for the current frame or the next frame to be rendered; and a system memory / disk cold cache for storing LOD artifacts that have been recently accessed but have been removed from the view frustum for quick recovery.

[0083] In some embodiments, to address the problem of redundant generation of numerous similar objects in the rendering environment, a template-based and instantiated mechanism is introduced. A complete Level of Detail (LOD) template is generated for the first instance of a category, such as a pine tree. Subsequent instances directly reuse this template, and visual diversity is increased through lightweight procedural perturbations (such as color shifts and stain map overlays). This template-based and instantiated mechanism contrasts sharply with existing LOD schemes, which require pre-creating and storing multiple complete models and textures for each variant. The template-based and instantiated mechanism employed in this invention, through template reuse and real-time perturbations, significantly saves generation computational power and storage space.

[0084] In some embodiments, the generation system may further include a task exception handling module, which can cancel or downgrade the LOD task during the LOD generation task ultrasound; and discard the currently generated LOD asset and roll back to the previous LOD asset when the LOD generation task consistency check fails.

[0085] In some embodiments, the generation system further includes a system-level resource degradation processing module, used to prioritize the reclamation (degradation) of high-order LOD assets of objects with the smallest screen projection area and the furthest distance when VRAM / video memory is tight; and to automatically suspend unnecessary cloud tasks and prioritize the transmission of low-frequency data when bandwidth is limited. Thus, the generation system detects... Even if a background LOD generation task fails, the lower LOD assets of the virtual object that have already been generated will still be rendered; and when network or computing power conditions improve, the LOD generation task of the downgraded object will be automatically reactivated, and the system can automatically revert to a higher quality LOD level, resulting in a better user experience.

[0086] In some embodiments, traditional pre-built asset synchronization typically involves a complete asset library on a server. For example, the models and textures for all LOD levels (from LOD0 to LOD4) of the object "Ancient Statue" are pre-made by artists and packaged in the game client or downloaded in advance as an update package. When the statue appears, the server synchronizes only one instruction: "Generate the object with asset ID #12345 at coordinates (X,Y,Z)". After receiving the instruction, all clients find the model and texture with ID #12345 from their locally downloaded asset packages and render it. For example, if players A, B, and C's clients receive the server instruction simultaneously, all three clients will find files such as "Statue_Low-poly", "Statue_High-poly", and "Statue_Texture" locally and complete the rendering, thus achieving a visually consistent effect. Since all versions of the statue must be pre-downloaded, they occupy MB-level space. If the game has tens of thousands of such objects, the package size will be enormous, and server-side download synchronization will result in high bandwidth and storage consumption, as well as content stagnation. Existing technologies can also employ cloud gaming synchronization. In this case, all rendering calculations are completed on a cloud server. The cloud server renders the game screen containing the "ancient idols," then encodes the rendering results into a video stream and sends it to various clients. For example, clients (players A, B, and C) are only responsible for receiving and decoding the video stream and displaying it on the screen. However, this solution requires extremely high bandwidth to continuously transmit Mbps-level video data, and there is also interactive latency because every action of the player (such as moving or clicking) needs to be uploaded to the cloud, processed, and then returned as a video stream. This is latency-sensitive, and the video quality is lost due to compression during transmission.

[0087] The embodiments of this application employ a scheme of instruction stream + deterministic generation, such as... Figure 7 As shown, during the instruction stream synchronization process, server 70 does not distribute any assets or videos. Instead, it broadcasts a lightweight generation instruction, which includes ObjectID, TargetLOD, Seed, Ver, and Input Spec. This generation instruction is then sent to clients 71, 72, and 73 via the instruction stream. Upon receiving the instruction, all clients enter the local reproduction process 74, that is, they run the generation process independently locally using the same version-locked generation model and the same seed. Specifically, after receiving the instruction stream, client 71 performs deterministic generation 711, obtaining generation result 712; client 72 performs deterministic generation 721, obtaining generation result 722; and client 73 performs deterministic generation 731, obtaining generation result 732. Since deterministic generation 711, 721, and 731 use the same input, the same model, and the same seed, generation results 712, 722, and 732 are completely identical assets. For example, instruction synchronization: The server broadcasts the instruction: (ObjectID: "Ancient_Statue_001",TargetLOD: 2, Seed: 0x5F3A2C, Ver: "SD-v1.5", Input Spec: "a moss-covered ancient stone statue"). Clients 71, 72, and 73 simultaneously receive this instruction and then each invokes the local version "SD-v1.5" text-to-image model, inputting the exact same prompt and random seed 0x5F3A2C. This deterministic generation ensures that clients 71, 72, and 73 generate identical LOD1 images. Then, clients 71, 72, and 73 use the version "ZoeDepth-v1.0" depth estimation model and the same seed to estimate the depth and normals of the previously generated image, resulting in completely identical LOD2 (2.5D depth shell) assets.

[0088] In some embodiments, this generation system is applied across multiple clients. Each client, upon receiving an instruction, uses the same deterministic generation module and seed to independently generate visually identical assets locally. To ensure this strict consistency across clients, this application employs five sets of conditions—instruction stream, seed, module version number, input specification, and tolerance check—to ensure visual consistency in a multi-user environment. The instruction stream can be a lightweight instruction broadcast by the server, containing (ObjectID, TargetLOD, Seed, Ver); the seed is a deterministic seed used to initialize all random processes; the module version number ensures all clients use the exact same generation model version, operator implementation, and weights; and the input specification standardizes the processing flow for inputs such as prompt text and initial images, avoiding differences in results due to encoding or preprocessing variations. Additionally, Tolerance Check is used on the client side to verify key features of the product (such as hash values) after generation. If the value exceeds the preset tolerance, it is determined to be inconsistent, triggering a rollback or resynchronization, thereby ensuring the robustness of the system.

[0089] Thus, through instruction stream synchronization, the data volume is typically at the byte level. This contrasts sharply with the video streams (megabits per second, Mbps) relied upon by existing cloud rendering or cloud gaming, or the model streams (megabytes, MB) of traditional multiplayer game synchronization, effectively reducing network overhead by a significant margin. Clients do not need to pre-download or cache all possible generated results, leading to more optimized storage and support for consistent playback of game recordings, resulting in a better user experience.

[0090] In some embodiments, the generation system may further include an asset management and caching module for storing and managing LOD assets at all levels generated by the generation execution module 403, and using (Stage, Ver, Seed) to version the LOD assets, and using LRU / TTL and other strategies for cache reclamation, thereby ensuring the reusability and consistency of the generation results.

[0091] In some embodiments, the generation system may further include a rendering and transition module for seamlessly integrating LOD assets cached in the asset management and caching module into the rendering pipeline. Once a new LOD for an object is generated, the layer performs a seamless replacement operation, including geometric alignment, material continuity processing, and temporal smooth transition, ultimately submitting the appropriate LOD to the renderer.

[0092] In some embodiments, the generation system can be integrated into a game engine such as Unreal Engine as a plugin or engine module, or it can be applied to mobile or edge computing devices, as well as to massively multiplayer online games (MMOs) or metaverse platforms.

[0093] Accordingly, this application also provides an electronic device, which can be a terminal, such as a smartphone, tablet computer, laptop computer, touch screen, game console, personal computer (PC), personal digital assistant (PDA), or other terminal device. Alternatively, the electronic device can be a server.

[0094] like Figure 8 As shown, Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1100 includes a processor 1101 with one or more processing cores, a memory 1102 with one or more computer-readable storage media, and a computer program stored on the memory 1102 and executable on the processor. The processor 1101 and the memory 1102 are electrically connected. Those skilled in the art will understand that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0095] The processor 1101 is the control center of the electronic device 1100. It connects various parts of the electronic device 1100 via various interfaces and lines. By running or loading software programs and / or units stored in the memory 1102, and by calling data stored in the memory 1102, it executes various functions of the electronic device 1100 and processes data, thereby providing overall monitoring of the electronic device 1100. The processor 1101 can be a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc., and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0096] In this embodiment, the processor 1101 in the electronic device 1100 loads the instructions corresponding to the processes of one or more applications into the memory 1102 according to the following steps, and the processor 1101 runs the applications stored in the memory 1102 to realize various functions, such as: obtaining the context information of virtual objects in the rendering environment; determining the perceived urgency of the virtual object based on the context information; determining whether the LOD level of the virtual object needs to be raised to the target LOD level based on the perceived urgency of the virtual object and the current computing power budget; if the LOD level of the virtual object needs to be raised to the target LOD level, generating the target LOD level asset for the virtual object according to the current LOD level asset of the virtual object, and replacing the current LOD asset of the virtual object with the target LOD level asset.

[0097] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0098] Optional, such as Figure 8 As shown, the electronic device 1100 also includes: a touch display screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch display screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107. Those skilled in the art will understand that... Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0099] The touch display screen 1103 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 1103 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1101. It can also receive and execute commands from the processor 1101. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 1101 to determine the type of touch event. Subsequently, the processor 1101 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the touch display screen 1103 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 1103 can be implemented as two independent components to achieve input and output functions. That is, the touch display screen 1103 can also be used as part of the input unit 1106 to achieve input functions.

[0100] The radio frequency circuit 1104 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other electronic devices, and to transmit and receive signals with network devices or other electronic devices.

[0101] Audio circuit 1105 can be used to provide an audio interface between a user and an electronic device via a speaker and a microphone. Audio circuit 1105 can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuit 1105, converted back into audio data, and then processed by processor 1101 before being transmitted via radio frequency circuit 1104 to, for example, another electronic device, or output to memory 1102 for further processing. Audio circuit 1105 may also include an earphone jack to provide communication between peripheral headphones and electronic devices.

[0102] The input unit 1106 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0103] Power supply 1107 is used to supply power to various components of electronic device 1100. Optionally, power supply 1107 can be logically connected to processor 1101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 1107 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0104] although Figure 8 As not shown in the diagram, the electronic device 1100 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0105] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0106] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0107] To this end, embodiments of this application provide a computer-readable storage medium storing multiple computer programs that can be loaded by a processor to execute any of the Level of Detail (LOD) asset generation methods provided in this application. The methods include: acquiring context information of virtual objects in a rendering environment; determining the perceived urgency of the virtual object based on the context information; determining whether the LOD level of the virtual object needs to be raised to a target LOD level based on the perceived urgency and the current computing power budget; if the LOD level of the virtual object needs to be raised to the target LOD level, generating the target LOD level asset for the virtual object based on its current LOD level asset, and replacing the current LOD asset of the virtual object with the target LOD level asset.

[0108] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0109] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0110] Since the computer program stored in the computer-readable storage medium can execute any of the Level of Detail (LOD) asset generation methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the LOD asset generation methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0111] According to one aspect of this application, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations of the above embodiments.

[0112] In the above embodiments of the LOD asset generation method, system, computer-readable storage medium, electronic device, and computer program product, the descriptions of each embodiment have different focuses. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes and beneficial effects of the above-described LOD asset generation apparatus, computer-readable storage medium, computer program product, electronic device, and their corresponding units can be referred to the description of the LOD asset generation method in the above embodiments, and will not be repeated here.

[0113] The above provides a detailed description of a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating Level of Detail (LOD) assets according to embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for generating assets at a level of detail, characterized in that, include: Obtain context information of virtual objects in the rendering environment; Based on the context information, the perceived urgency of the virtual object is determined; Based on the perceived urgency of the virtual object and the current computing power budget, determine whether it is necessary to upgrade the LOD level of the virtual object to the target LOD level; If it is necessary to upgrade the LOD level of the virtual object to the target LOD level, then based on the current LOD level assets of the virtual object, the target LOD level assets are generated for the virtual object, and the current LOD assets of the virtual object are replaced with the target LOD level assets.

2. The method as described in claim 1, characterized in that, The process of determining whether to elevate the LOD level of the virtual object to the target LOD level based on the perceived urgency of the virtual object and the current computing power budget includes: If the perceived urgency of the virtual object is greater than the first preset urgency corresponding to the target LOD level, and the current computing power budget is greater than the first set computing power required for the virtual object's LOD level to rise to the target LOD level, then it is determined that the virtual object's LOD level needs to be raised to the target LOD level.

3. The method as described in claim 2, characterized in that, The method further includes: If the perceived urgency of the virtual object is greater than the first preset urgency, and the current computing power budget is not greater than the first set computing power, then the current LOD level of the virtual object is determined to remain unchanged.

4. The method as described in claim 1, characterized in that, The method further includes: Based on the context information or the current computing power budget, determine whether the LOD level of the virtual object needs to be reduced to the set LOD level; If it is necessary to lower the LOD level of the virtual object to the set LOD level, the current LOD asset of the virtual object is replaced with the set LOD level asset according to the pre-stored set LOD level asset of the virtual object.

5. The method as described in claim 4, characterized in that, The context information includes the distance between the virtual object and the camera. Determining whether to downgrade the virtual object's LOD level to a set LOD level based on the context information or the current computing power budget includes: If the distance between the virtual object and the camera is greater than the set camera distance corresponding to the current LOD level, or if the current computing power budget is less than the second set computing power required for the virtual object to be at the current LOD level, then it is determined that the LOD level of the virtual object needs to be reduced to the set LOD level.

6. The method as described in claim 1, characterized in that, The method further includes: If the virtual object appears in the rendering environment for the first time, an initial LOD asset of the virtual object is generated based on the basic information of the virtual object, and the initial LOD asset is used as the current LOD asset of the virtual object.

7. The method as described in claim 1, characterized in that, The step of replacing the current LOD asset of the virtual object with the target LOD level asset includes: Project the first texture corresponding to the current LOD asset onto the mesh surface of the target LOD level asset to generate the projected color information; The projected color information is baked into the UV space corresponding to the target LOD level asset to generate the second texture corresponding to the target LOD level asset; Based on the second texture, the current LOD asset of the virtual object is replaced with the target LOD level asset.

8. The method as described in claim 1, characterized in that, The method further includes: Within a preset switching time window, both the current LOD asset and the target LOD level asset are rendered simultaneously. Color mixing is performed on the rendering output of the current LOD asset and the target LOD level asset.

9. The method according to any one of claims 1-8, characterized in that, The target LOD level is the adjacent level of the current LOD level of the virtual object.

10. The method according to any one of claims 1-8, characterized in that, The context information includes at least one of the following: the screen projection area of ​​the virtual object, the distance between the virtual object and the camera, gaze point information, the relative motion vector between the virtual object and the camera, the probability of the virtual object being occluded, and an importance label.

11. The method as described in claim 10, characterized in that, The context information includes at least two parameters among the following: the screen projection area of ​​the virtual object, the distance between the virtual object and the camera, gaze point information, the relative motion vector between the virtual object and the camera, the probability of the virtual object being occluded, and an importance label. Determining the perceptual urgency of the virtual object based on the context information includes: The perceived urgency of the virtual object is determined by the weighted sum of each parameter in the context information.

12. A system for generating assets at a level of detail, characterized in that, include: The data acquisition module is used to acquire context information of virtual objects in the rendering environment; The determination module is used to determine the perceived urgency of the virtual object based on the context information; and then, based on the perceived urgency of the virtual object and the current computing power budget, determine whether it is necessary to raise the LOD level of the virtual object to the target LOD level. The generation execution module is used to generate the target LOD level asset for the virtual object based on the current LOD level asset of the virtual object when the LOD level of the virtual object needs to be raised to the target LOD level, and replace the current LOD asset of the virtual object with the target LOD level asset.

13. The system as described in claim 12, characterized in that, The generation and execution module includes multiple AI sub-modules, each of which is responsible for generating one or more LOD-level assets.

14. An electronic device, characterized in that, It includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps of the detail level asset generation method as described in any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the detail level asset generation method as described in any one of claims 1 to 11.

Images