A rendering method and electronic device
By parsing scene data into sub-scenes and rendering them in parallel, the efficiency and scalability issues of existing rendering engines in large-scale scenes are solved, achieving efficient and accurate rendering results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LENOVO NEW VISION BEIJING TECH CO LTD
- Filing Date
- 2026-05-08
- Publication Date
- 2026-06-05
AI Technical Summary
Existing rendering engines have significant bottlenecks when handling ultra-large-scale industrial scenes, such as sharp drops in frame rate, low hardware utilization, poor scalability, inability to dynamically adapt to scene complexity, and difficulty in cross-platform porting.
The scene input data is parsed into multiple sub-scene data, and rendering tasks are executed in parallel through multiple rendering pipelines. Sub-rendered images are integrated based on the correlation between sub-scene data, and a suitable rendering pipeline is dynamically selected to optimize the load score.
It improves rendering efficiency and frame rate, reduces video memory usage, supports smooth operation of scenes with hundreds of millions of polygons, ensures the accuracy and timeliness of rendered images, and has wide scalability.
Smart Images
Figure CN122156423A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image rendering technology, and in particular to a rendering method and electronic device. Background Technology
[0002] Current mainstream rendering engines (such as Unreal Engine / Unity) adopt a single rendering pipeline architecture, which has significant bottlenecks when handling ultra-large-scale industrial scenes (such as hundreds of millions of polygons): 1. The frame rate drops sharply when the number of triangles exceeds a certain number (such as 20 million), requiring the reliance on distributed rendering or reduced precision, sacrificing more than 30% efficiency; 2. It cannot dynamically adapt to scene complexity, resulting in low hardware utilization; 3. It is difficult to port across platforms (such as domestic OS, MIPS architecture), and it depends on specific hardware (such as NVIDIA RTX), resulting in poor scalability. Summary of the Invention
[0003] The purpose of this application is to provide a rendering method and an electronic device.
[0004] In a first aspect, embodiments of this application provide a rendering method, including: Load scene input data; The scene input data is parsed to obtain multiple sub-scene data and the relationships between the sub-scene data; For each sub-scene data, the rendering pipeline corresponding to the sub-scene data is selected from multiple rendering pipelines, so that the rendering pipeline performs the rendering task based on the sub-scene data to obtain the sub-rendered image, and different rendering pipelines execute their corresponding rendering tasks in parallel. Based on the relationships between the sub-scene data, all sub-rendered images are integrated to obtain the target rendered image of the scene input data.
[0005] In one possible implementation, parsing the scene input data to obtain multiple sub-scene data and the relationships between the sub-scene data includes: Based on the scene input data, scene attribute information is determined; wherein, the scene attribute information can at least characterize the assembly-level metadata of the scene corresponding to the scene input data; Based on the scene attribute information and the assembly attribute information of each assembly, the scene input data is parsed into multiple sub-scene data, and the relationship between the sub-scene data is determined.
[0006] In one possible implementation, The assembly attribute information includes the dimensions, geometry, and material of the corresponding assembly.
[0007] In one possible implementation, selecting the rendering pipeline corresponding to the sub-scene data from multiple rendering pipelines includes: Determine the load fraction for each rendering pipeline; where the GPU of an electronic device comprises multiple rendering pipelines; Based on the quantity of the sub-scene data, N rendering pipelines are selected from multiple rendering pipelines; Based on the assembly attribute information corresponding to the sub-scene data, select the rendering pipeline corresponding to the sub-scene data from N rendering pipelines.
[0008] In one possible implementation, determining the load fraction for each rendering pipeline includes: For each rendering pipeline, determine the corresponding video memory usage, the volume of the geometry to be processed, and the execution time of the previous rendering task. The load score of the rendering pipeline is obtained by weighting the amount of video memory used, the size of the geometry to be processed, and the execution time.
[0009] In one possible implementation, If assemblies with the same material exist, determine that the sub-scene data of assemblies with the same material belong to the same type of rendering pipeline, and the materials corresponding to the same type of rendering pipeline are the same.
[0010] In one possible implementation, Different rendering pipelines correspond to different materials, and multiple shaders are pre-configured based on the materials corresponding to the rendering pipelines.
[0011] In one possible implementation, Based on the load score of each rendering pipeline, determine whether there is a rendering pipeline that meets preset conditions; wherein, the preset conditions include the load score being greater than a threshold. If it exists, the target rendering task is determined based on the sub-scene data corresponding to each rendering task in the rendering pipeline that meets the preset conditions and the relationship between the sub-scene data. The target rendering task corresponding to the rendering pipeline that meets the preset conditions is migrated to other rendering pipelines so that the other rendering pipelines can execute the target rendering task.
[0012] In one possible implementation, the step of integrating all sub-rendered images based on the correlation between the sub-scene data to obtain the target rendered image of the scene input data includes: Based on the relative positions and relative poses between the sub-scene data, the target rendered image is constructed using all the sub-rendered images.
[0013] Secondly, embodiments of this application also provide an electronic device, including a CPU and a GPU; The CPU loads scene input data; parses the scene input data to obtain multiple sub-scene data and the relationships between the sub-scene data; and selects the rendering pipeline corresponding to each sub-scene data from multiple rendering pipelines for each sub-scene data. The GPU schedules the rendering pipeline to execute rendering tasks based on sub-scene data to obtain sub-rendered images, and different rendering pipelines execute their corresponding rendering tasks in parallel; based on the correlation between the sub-scene data, all sub-rendered images are integrated to obtain the target rendered image of the scene input data. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0015] Figure 1 A flowchart of a rendering method provided in this application is shown; Figure 2 This invention illustrates a method flow for selecting the rendering pipeline corresponding to the sub-scene data from multiple rendering pipelines in a rendering method provided in this application. Figure 3 A schematic diagram of the structure of an electronic device provided in this application is shown; Figure 4 A schematic diagram of the structure of another electronic device provided in this application is shown. Detailed Implementation
[0016] Various embodiments and features of this application are described herein with reference to the accompanying drawings.
[0017] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.
[0018] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.
[0019] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.
[0020] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application, which have the features described in the claims and are therefore all within the scope of protection defined herein.
[0021] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.
[0022] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.
[0023] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.
[0024] To facilitate understanding of this application, a rendering method provided in this application will be described in detail below. It should be noted that the execution subject of the rendering method in this application is an electronic device, which includes a central processing unit (CPU) and a graphics processing unit (GPU).
[0025] As an example, Figure 1 A flowchart of the rendering method provided in an embodiment of this application is shown, wherein the specific steps include S101-S104.
[0026] S101, Load scene input data.
[0027] Optionally, the scene input data can be obtained based on the deconstruction of a scene video. For example, to simulate and render an actual vehicle, video information of the vehicle is collected and deconstructed to obtain the corresponding scene input data. This scene input data can then be used to achieve simulation rendering for that vehicle. Similarly, the scene input data for buildings, street scenes, etc., can be obtained in the same way for simulation rendering. Furthermore, for constructed virtual scenes such as scenes in film and animation, or game scenes, the scene input data for each frame of the scene image can be manually set by the user.
[0028] The scene input data may include spatial parameters of the scene (such as vertex coordinates), structural parameters (such as the size and number of triangle meshes), assembly level metadata, assembly name, assembly attribute information, number of assemblies, assembly coordinates, assembly material, and environmental parameters.
[0029] S102, parse the scene input data to obtain multiple sub-scene data and the relationships between the sub-scene data.
[0030] After the scene input data is fully loaded, it is parsed. Specifically, to ensure the accuracy and reasonableness of the parsed scene input data, the parsing process is performed only after the scene input data has been fully loaded.
[0031] After parsing the scene input data, multiple sub-scene data and the relationships between the sub-scene data are obtained.
[0032] Optionally, the scene input data is parsed using the CPU of the electronic device. For example, when parsing the scene input data, scene attribute information is determined based on the scene input data. The scene attribute information at least represents the assembly-level metadata of the scene corresponding to the scene input data. When the scene input data represents a vehicle, the corresponding scene is the vehicle, and the corresponding assembly-level metadata includes the names of all assemblies included in the vehicle and the relationships between each assembly. Assemblies can be outer shells (roof, doors, trunk lid), interior trim (seats, center console, steering wheel), engine, tires, screws, etc.
[0033] After determining the scene attribute information, based on the scene attribute information and the assembly attribute information of each assembly, the scene input data is parsed into multiple sub-scene data. The assembly attribute information includes the size, geometry, and material of the corresponding assembly. Sub-scene data can include data from one or more assemblies. For example, if the individual geometry of two or more adjacent assemblies indicates low complexity, and the combined geometry of two or more adjacent assemblies indicates low complexity (e.g., a small number of triangular meshes), then it can be determined that the data of these two or more adjacent assemblies belong to the same sub-scene data.
[0034] Correspondingly, sub-scene data can include the dimensions (boundary spheres), geometry (number and pose of triangle meshes), and materials (at least one) of all assemblies. Optionally, an independent rendering container can be constructed for each sub-scene data, whose data structure includes vertex buffer handles (e.g., for indicating vertex coordinates), index buffer handles (e.g., for indicating triangle mesh parameters), material parameter sets (Uniform Buffer Objects, including surface texture values such as color, metallicity, roughness, etc.), texture binding lists (for indicating the texture information required for rendering), and world transformation matrices (for implementing the position, rotation, and scaling of objects in the 3D world).
[0035] Based on the assembly-level metadata, the relationships between assemblies can be clearly understood, such as hierarchical relationships, positional relationships, constraint relationships, connection relationships, dependency relationships, and coupling strength. After determining multiple sub-scene data, the relationships between sub-scene data are determined based on the relationships between each assembly, which also include hierarchical relationships, positional relationships, constraint relationships, connection relationships, dependency relationships, and coupling strength.
[0036] S103: For each sub-scene data, select the rendering pipeline corresponding to the sub-scene data from multiple rendering pipelines, so that the rendering pipeline can perform rendering tasks based on the sub-scene data to obtain a sub-rendered image, and different rendering pipelines can execute their corresponding rendering tasks in parallel.
[0037] Optionally, a rendering task can be constructed based on the sub-scene data. Based on this, for each sub-scene data, the rendering pipeline corresponding to that sub-scene data is selected from multiple rendering pipelines.
[0038] The GPU has multiple rendering pipelines, each capable of executing rendering tasks to achieve simulation rendering. Each rendering pipeline can have different rendering capabilities, rendering effects, and target objects.
[0039] For each sub-scene data, a corresponding rendering pipeline is selected. Then, the sub-scene data (or its corresponding rendering container) is transferred to its corresponding rendering pipeline so that the corresponding rendering pipeline can perform rendering tasks based on the sub-scene data to obtain a sub-rendered image. The sub-rendered image rendered by its corresponding rendering pipeline has better effect.
[0040] It should be noted that each rendering pipeline has its own independent task list generator and task queue. The task queue can include one or more rendering tasks, which can be determined according to actual needs. This embodiment does not limit this. Furthermore, each rendering pipeline executes its corresponding rendering tasks independently, and the rendering pipelines do not affect each other. As long as the execution dependency relationship is established between the task queues of different rendering pipelines through the timeline semaphore, the true decoupling and overlapping execution of computation tasks (such as view frustum clipping and LOD selection) and rendering tasks (such as geometry drawing and post-processing) can be achieved.
[0041] In other words, the above configuration enables different rendering pipelines to execute their corresponding rendering tasks in parallel. Correspondingly, it also allows for the simultaneous transmission of sub-scene data to multiple rendering pipelines, effectively improving the overall rendering efficiency.
[0042] S104, based on the correlation between sub-scene data, integrates all sub-rendered images to obtain the target rendered image of the scene input data.
[0043] After all rendering pipelines have completed their rendering tasks, all sub-rendered images are integrated based on the relationships between sub-scene data to obtain the target rendered image of the scene input data. Optionally, when integrating all sub-rendered images based on the relationships between sub-scene data, the target rendered image is constructed using all sub-rendered images based on the relative positions and relative poses between the sub-scene data, ensuring the accuracy of the simulation rendering and avoiding problems such as misalignment and inconsistency.
[0044] In this embodiment, by parsing the scene input data into multiple sub-scene data and using multiple rendering pipelines to execute rendering tasks in parallel based on the sub-scene data, compared with the single rendering pipeline architecture of traditional technologies, this application effectively improves rendering efficiency and the timeliness of displaying the target rendered image. After obtaining all sub-rendered images, based on the correlation between the sub-scene data, all sub-rendered images are integrated to obtain the target rendered image to complete the rendering interaction and ensure the accuracy of the target rendered image. In addition, this application parses the assembly-level metadata of the scene corresponding to the scene input data, which can be dynamically applied to different scenes (i.e., simulation rendering objects) and has a wide range of applications.
[0045] As an example, Figure 2 A flowchart of a method for selecting the rendering pipeline corresponding to the sub-scene data from multiple rendering pipelines is shown, wherein the specific steps include S201-S203.
[0046] S201, determine the load fraction for each rendering pipeline; wherein, the GPU of the electronic device comprises multiple rendering pipelines.
[0047] S202 selects N rendering pipelines from multiple rendering pipelines based on the number of sub-scene data.
[0048] S203, based on the assembly attribute information corresponding to the sub-scene data, select the rendering pipeline corresponding to the sub-scene data from N rendering pipelines.
[0049] For example, when selecting the rendering pipeline corresponding to each sub-scene data from multiple rendering pipelines, the GPU first determines the load score of each rendering pipeline among the multiple rendering pipelines. The load score of a rendering pipeline can characterize the load level of that rendering pipeline; for example, the higher the load score, the heavier the load, and correspondingly, the lower the load score, the lighter the load.
[0050] When determining the load score for each rendering pipeline, the memory usage, the volume of the geometry to be processed (e.g., the number of triangle meshes), and the execution time of the previous rendering task are determined for each pipeline. Specifically, the volume of the geometry to be processed is obtained from the sub-scene data corresponding to the rendering pipeline.
[0051] Next, a weighted calculation is performed based on memory usage, the size of the geometry to be processed, and the execution time to obtain the load score of the rendering pipeline. For example, the load score for each rendering pipeline can be determined using the following formula: "Load Score = α * (Memory Usage / Total Memory) + β * (Number of Triangle Meshes / Maximum Capable Number) + γ * (Execution Time of Previous Frame Rendering Task / Preset Time Threshold)". Here, α, β, and γ are weighting coefficients, and their sum is 1.
[0052] It should be noted that each rendering pipeline executes the rendering task of multiple frames of images. For example, for a vehicle, when the user wants to observe the side of the vehicle, the corresponding image of the side is rendered. When the user wants to switch to the interior, the perspective of entering from the outside to the inside is simulated to determine multiple consecutive images to be rendered, which are then rendered and displayed in sequence, so that the user can visually experience the process of entering from the outside to the inside.
[0053] After determining the load score of each rendering pipeline, N rendering pipelines are selected from multiple rendering pipelines based on the number of sub-scene data. For example, N rendering pipelines can be selected from low to high load scores. Specifically, if the number of sub-scene data is greater than or equal to the number of rendering pipelines, N rendering pipelines can be selected from multiple rendering pipelines to execute the rendering task, with each rendering pipeline corresponding to one or more sub-scene data. In this case, N is less than or equal to the number of rendering pipelines. If the number of sub-scene data is less than the number of rendering pipelines, for each sub-scene data, one rendering pipeline corresponding to each sub-scene data can be selected from multiple rendering pipelines. In this case, N is the number of sub-scene data. Alternatively, when the number of sub-scene data is less than the number of rendering pipelines, N can also be limited to be less than the number of sub-scene data. In this case, each rendering pipeline corresponds to one or more sub-scene data. This embodiment does not impose such limitations; rendering pipelines can be selected according to actual needs.
[0054] After selecting N rendering pipelines, based on the assembly attribute information corresponding to the sub-scene data, the rendering pipeline corresponding to the sub-scene data is selected from the N rendering pipelines. Each rendering pipeline undergoes initialization before executing a rendering task. That is, based on the material characteristics of each assembly in the scene (such as whether it is transparent or requires subdivision), a rendering pipeline is generated through Just-In-Time (JIT) compilation and / or pre-compilation branch selection, so that different materials correspond to different rendering pipelines. Based on this, the rendering pipeline with the same material as the assembly in the sub-scene data is matched to execute the rendering task corresponding to the sub-scene data. Optionally, at least one rendering pipeline can be configured for the same material. For example, in application scenarios with a large amount of scene input data, multiple rendering pipelines can be configured for the same material; in application scenarios with a small amount of scene input data, one rendering pipeline can be configured for the same material, and so on.
[0055] For example, each rendering pipeline is pre-configured with multiple shaders, and these shaders are determined based on the material corresponding to that rendering pipeline. A corresponding rendering program is set up for each shader to achieve optimal rendering results when simulating and rendering the assembly to which the corresponding material belongs through that rendering pipeline.
[0056] In practice, if the assemblies in the sub-scene data contain multiple materials, the size and geometry of each material's assembly are determined to identify the target material. This allows the rendering pipeline corresponding to the target material to execute the rendering task for that sub-scene data. For example, the material of the assembly with the largest size and / or the most triangle meshes can be identified as the target material.
[0057] As another example, if assemblies with the same material exist, it is determined that the sub-scene data belonging to assemblies of the same material corresponds to the same type of rendering pipeline, and the materials corresponding to the same type of rendering pipeline are the same. Of course, if the sub-scene data belonging to assemblies of the same material is large, and the material is configured with multiple rendering pipelines, the rendering tasks corresponding to the sub-scene data can be distributed to two or more rendering pipelines to ensure that the rendering tasks can be executed normally, while ensuring high rendering efficiency.
[0058] In addition, to improve the simulation rendering effect, an independent ray tracing channel can be constructed. The ray tracing channel performs intersection detection and reflection and refraction calculations between the light rays and each assembly based on the material corresponding to the rendering channel and its ray tracing requirements, so as to make the rendering effect of the target image better.
[0059] Optionally, the ray tracing channel can adopt a two-layer acceleration structure: the top layer is a bounding box hierarchy (BVH) based on sub-scene data, and the bottom layer is an adaptive subdivision mesh (ASM) based on Z-Engine. It can work asynchronously and in parallel with the rendering pipeline, while supporting efficient intersection of complex industrial surfaces, ultimately achieving a balance between realistic lighting effects and high performance.
[0060] During the execution of rendering tasks in each rendering pipeline, the load score of each rendering pipeline can be calculated in real time, or it can be calculated in response to trigger conditions, such as the rendering time exceeding a preset duration. Furthermore, based on the load score of each rendering pipeline, it can be determined whether there are rendering pipelines that meet preset conditions; these preset conditions include a load score greater than a threshold. Meeting these preset conditions indicates that the rendering pipeline is heavily loaded, which may affect rendering efficiency and the execution of rendering tasks.
[0061] If a rendering pipeline that meets the preset conditions exists, the target rendering task is determined based on the sub-scene data corresponding to each rendering task of the rendering pipeline that meets the preset conditions and the relationship between the sub-scene data. The target rendering task corresponding to the rendering pipeline that meets the preset conditions is then migrated to other rendering pipelines so that the other rendering pipelines can execute the target rendering task.
[0062] For example, when migrating a target rendering task to another rendering pipeline, first determine if there exists a rendering pipeline with the same material as the rendering pipeline that meets the preset conditions. If so, identify the rendering pipeline with the same material as the rendering pipeline that meets the preset conditions as another rendering pipeline, and migrate the target rendering task to that other rendering pipeline. This allows the rendering pipeline with the same material as the rendering pipeline that meets the preset conditions to execute the target rendering task, ensuring high rendering efficiency and that the rendering task can be executed normally, while also guaranteeing good rendering results. If there are multiple rendering pipelines with the same material as the rendering pipeline that meets the preset conditions, the rendering pipeline with the lowest load score among these multiple rendering pipelines can be identified as another pipeline.
[0063] If there is no rendering pipeline with the same material as the rendering pipeline that meets the preset conditions, the rendering pipeline with the lowest load score is determined as another rendering pipeline, and the target rendering task is migrated to that other rendering pipeline so that the rendering pipeline with the lowest load score can execute the target rendering task. At this time, it can be ensured that the rendering efficiency is high and the rendering task can be executed normally.
[0064] It should be noted that the other rendering pipeline can be any one of the N rendering pipelines, or any rendering pipeline other than the N rendering pipelines. This application embodiment does not limit this.
[0065] Specifically, for each rendering task, the number of triangular meshes included in the sub-scene data corresponding to the rendering task and the coupling strength of the assemblies included in the sub-scene data are determined, and the rendering task with the largest number of triangular meshes and the lowest coupling strength of the assemblies is determined as the target rendering task.
[0066] The embodiments of this application can schedule rendering tasks reasonably and evenly based on the real-time load scores of each rendering pipeline during the execution of rendering tasks. That is, the rendering pipeline with the lowest load score is used to execute the target rendering task, which ensures rendering efficiency while avoiding resource waste.
[0067] The second aspect of this application also provides an electronic device corresponding to the rendering method. Since the principle of solving the problem by the electronic device in this application is similar to the rendering method described above, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be described again.
[0068] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application is shown, with reference to... Figure 3 It is known that electronic devices include CPUs and GPUs; The CPU loads scene input data; parses the scene input data to obtain multiple sub-scene data and the relationships between the sub-scene data; and selects the rendering pipeline corresponding to each sub-scene data from multiple rendering pipelines for each sub-scene data. The GPU schedules the rendering pipeline to execute rendering tasks based on sub-scene data to obtain sub-rendered images, and different rendering pipelines execute their corresponding rendering tasks in parallel; based on the correlation between the sub-scene data, all sub-rendered images are integrated to obtain the target rendered image of the scene input data.
[0069] In another example, when the CPU parses the scene input data to obtain multiple sub-scene data and the relationships between the sub-scene data, it includes: Based on the scene input data, scene attribute information is determined; wherein, the scene attribute information can at least characterize the assembly-level metadata of the scene corresponding to the scene input data; Based on the scene attribute information and the assembly attribute information of each assembly, the scene input data is parsed into multiple sub-scene data, and the relationship between the sub-scene data is determined.
[0070] As another example, the assembly attribute information includes the dimensions, geometry, and material of the corresponding assembly.
[0071] As another example, when the CPU selects the rendering pipeline corresponding to the sub-scene data from multiple rendering pipelines, it includes: Determine the load fraction for each rendering pipeline; where the GPU of an electronic device comprises multiple rendering pipelines; Based on the quantity of the sub-scene data, N rendering pipelines are selected from multiple rendering pipelines; Based on the assembly attribute information corresponding to the sub-scene data, select the rendering pipeline corresponding to the sub-scene data from N rendering pipelines.
[0072] As another example, when the CPU determines the load fraction for each rendering pipeline, it includes: For each rendering pipeline, determine the corresponding video memory usage, the volume of the geometry to be processed, and the execution time of the previous rendering task. The load score of the rendering pipeline is obtained by weighting the amount of video memory used, the size of the geometry to be processed, and the execution time.
[0073] As another example, if there are assemblies with the same material, determine that the sub-scene data of the assemblies with the same material corresponds to the same type of rendering pipeline, and the materials corresponding to the same type of rendering pipeline are the same.
[0074] As another example, different rendering pipelines correspond to different materials, and multiple shaders are pre-configured based on the materials corresponding to the rendering pipelines.
[0075] As another example, the CPU: Based on the load score of each rendering pipeline, determine whether there is a rendering pipeline that meets preset conditions; wherein, the preset conditions include the load score being greater than a threshold. If it exists, the target rendering task is determined based on the sub-scene data corresponding to each rendering task in the rendering pipeline that meets the preset conditions and the relationship between the sub-scene data. The target rendering task corresponding to the rendering pipeline that meets the preset conditions is migrated to other rendering pipelines so that the other rendering pipelines can execute the target rendering task.
[0076] In another example, when the GPU integrates all sub-rendered images based on the correlation between the sub-scene data to obtain the target rendered image of the scene input data, it includes: Based on the relative positions and relative poses between the sub-scene data, the target rendered image is constructed using all the sub-rendered images.
[0077] It should be noted that for specific instruction sets of domestically produced GPUs (such as specific SIMD widths or special function units), kernel shader assembly optimization templates can be pre-written to execute the steps in the rendering method described above. For common graphics mathematical operations (such as matrix multiplication and vector normalization), the system detects the GPU model during initialization and dynamically loads the corresponding manually tuned microkernel code to replace the standard GLSL / HLSL compilation results.
[0078] A shared physical memory pool under a unified virtual address space is established between the CPU and GPU. This memory pool is used for physics simulation data such as particle position and velocity, which reside directly within the pool. The CPU performs logical updates, and the GPU reads the data for rendering, avoiding costly memory copies. Furthermore, CPU and GPU synchronization is achieved through hardware-supported atomic operations and memory barrier instructions, ensuring data consistency.
[0079] A separate dynamic scheduling engine can be set up for this electronic device to schedule multiple rendering pipelines. The load of each rendering pipeline (such as the number of rigid bodies and the complexity of collision pairs) is also included in the overall load balancing calculation to ensure that both rendering and physics simulation can make full use of heterogeneous computing resources, thereby guaranteeing the utilization rate of each rendering pipeline and reasonable load balancing. It is worth noting that scheduling and calculation can also be performed directly using the CPU.
[0080] The electronic device of this application embodiment, without upgrading the hardware, performs image rendering according to the above rendering method, and the frame rate is increased by at least 35%, and the video memory usage is reduced by 40%, which can support the smooth operation of scenes with hundreds of millions of polygons; furthermore, the assembly rendering latency is reduced to within 5ms (the traditional latency is greater than or equal to 20ms).
[0081] A third aspect of this application also provides a computer device, the structural schematic diagram of which can be shown as follows: Figure 4 As shown, it includes at least a memory 401 and a processor 402. The memory 401 stores a computer program, and the processor 402 implements the method provided in any embodiment of this application when executing the computer program in the memory 401. Exemplarily, the steps of the electronic device computer program are as follows: S11-S14: S11, Load scene input data; S12, parse the scene input data to obtain multiple sub-scene data and the correlation between the sub-scene data; S13, for each sub-scene data, select the rendering pipeline corresponding to the sub-scene data from multiple rendering pipelines, so that the rendering pipeline performs rendering tasks based on the sub-scene data to obtain a sub-rendered image, and different rendering pipelines execute their corresponding rendering tasks in parallel. S14. Based on the correlation between the sub-scene data, integrate all sub-rendered images to obtain the target rendered image of the scene input data.
[0082] A fourth aspect of this application also provides a storage medium, which is a computer-readable medium, carrying one or more computer programs that, when executed by a processor, implement the method provided in any embodiment of this application, including the following steps S21-S24: S21, Load scene input data; S22, parse the scene input data to obtain multiple sub-scene data and the correlation between the sub-scene data; S23, for each sub-scene data, select the rendering pipeline corresponding to the sub-scene data from multiple rendering pipelines, so that the rendering pipeline performs rendering tasks based on the sub-scene data to obtain a sub-rendered image, and different rendering pipelines execute their corresponding rendering tasks in parallel. S24. Based on the correlation between the sub-scene data, integrate all sub-rendered images to obtain the target rendered image of the scene input data.
[0083] In this embodiment, by parsing the scene input data into multiple sub-scene data and using multiple rendering pipelines to execute rendering tasks in parallel based on the sub-scene data, compared with the single rendering pipeline architecture of traditional technologies, this application effectively improves rendering efficiency and the timeliness of displaying the target rendered image. After obtaining all sub-rendered images, based on the correlation between the sub-scene data, all sub-rendered images are integrated to obtain the target rendered image to complete the rendering interaction and ensure the accuracy of the target rendered image. In addition, this application parses the assembly-level metadata of the scene corresponding to the scene input data, which can be dynamically applied to different scenes (i.e., simulation rendering objects) and has a wide range of applications.
[0084] It should be understood that in the embodiments of this application, the processor may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0085] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk. Optionally, in this embodiment, the processor executes the method steps described in the above embodiments according to the program code stored in the storage medium. Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, which will not be repeated here. Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed on a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be executed in a different order than those described here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any specific hardware and software combination.
[0086] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on this application that have equivalent elements, modifications, omissions, combinations (e.g., schemes involving intersections of various embodiments), adaptations, or alterations. Elements in the claims will be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or during the implementation of this application, which will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered illustrative only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.
[0087] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify the application. This should not be construed as an intention that a disclosed feature not claimed is necessary for any claim. Rather, the subject matter of this application may be less than all the features of a particular disclosed embodiment. Thus, the following claims are incorporated herein by reference as examples or embodiments, wherein each claim is an independent, separate embodiment, and these embodiments are contemplated as being possible in various combinations or arrangements. The scope of this application should be determined by reference to the appended claims and the full scope of their equivalents.
[0088] The foregoing has described in detail several embodiments of this application, but this application is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications based on the concept of this application, and all such variations and modifications should fall within the scope of protection claimed in this application.
Claims
1. A rendering method, comprising: Load scene input data; The scene input data is parsed to obtain multiple sub-scene data and the relationships between the sub-scene data; For each sub-scene data, the rendering pipeline corresponding to the sub-scene data is selected from multiple rendering pipelines, so that the rendering pipeline performs the rendering task based on the sub-scene data to obtain the sub-rendered image, and different rendering pipelines execute their corresponding rendering tasks in parallel. Based on the relationships between the sub-scene data, all sub-rendered images are integrated to obtain the target rendered image of the scene input data.
2. The rendering method according to claim 1, wherein parsing the scene input data to obtain multiple sub-scene data and the correlation between the sub-scene data includes: Based on the scene input data, scene attribute information is determined; wherein, the scene attribute information can at least characterize the assembly-level metadata of the scene corresponding to the scene input data; Based on the scene attribute information and the assembly attribute information of each assembly, the scene input data is parsed into multiple sub-scene data, and the relationship between the sub-scene data is determined.
3. The rendering method according to claim 2, The assembly attribute information includes the dimensions, geometry, and material of the corresponding assembly.
4. The rendering method according to claim 2, selecting the rendering pipeline corresponding to the sub-scene data from multiple rendering pipelines, includes: Determine the load fraction for each rendering pipeline; where the GPU of an electronic device comprises multiple rendering pipelines; Based on the quantity of the sub-scene data, N rendering pipelines are selected from multiple rendering pipelines; Based on the assembly attribute information corresponding to the sub-scene data, select the rendering pipeline corresponding to the sub-scene data from N rendering pipelines.
5. The rendering method according to claim 4, wherein determining the load fraction of each rendering pipeline includes: For each rendering pipeline, determine the corresponding video memory usage, the volume of the geometry to be processed, and the execution time of the previous rendering task. The load score of the rendering pipeline is obtained by weighting the amount of video memory used, the size of the geometry to be processed, and the execution time.
6. The rendering method according to claim 4, If assemblies with the same material exist, determine that the sub-scene data of assemblies with the same material belong to the same type of rendering pipeline, and the materials corresponding to the same type of rendering pipeline are the same.
7. The rendering method according to claim 4, Different rendering pipelines correspond to different materials, and multiple shaders are pre-configured based on the materials corresponding to the rendering pipelines.
8. The rendering method according to claim 4 or 5, Based on the load score of each rendering pipeline, determine whether there exists a rendering pipeline that meets preset conditions; where, The preset condition includes the load score being greater than a threshold; If it exists, the target rendering task is determined based on the sub-scene data corresponding to each rendering task in the rendering pipeline that meets the preset conditions and the relationship between the sub-scene data. The target rendering task corresponding to the rendering pipeline that meets the preset conditions is migrated to other rendering pipelines so that the other rendering pipelines can execute the target rendering task.
9. The rendering method according to claim 1, wherein integrating all sub-rendered images based on the correlation between the sub-scene data to obtain the target rendered image of the scene input data comprises: Based on the relative positions and relative poses between the sub-scene data, the target rendered image is constructed using all the sub-rendered images.
10. An electronic device comprising a CPU and a GPU; The CPU loads scene input data; parses the scene input data to obtain multiple sub-scene data and the relationships between the sub-scene data; and selects the rendering pipeline corresponding to each sub-scene data from multiple rendering pipelines for each sub-scene data. The GPU schedules the rendering pipeline to execute rendering tasks based on sub-scene data to obtain sub-rendered images, and different rendering pipelines execute their corresponding rendering tasks in parallel; based on the correlation between the sub-scene data, all sub-rendered images are integrated to obtain the target rendered image of the scene input data.