A method and related equipment for deploying resources from a 3D Gaussian target perspective

By evaluating the imaging contribution of Gaussian primitives under multiple training perspectives, a set of target perspective master calling primitives is generated, and a lookup table-based orientation color response evaluation mechanism is constructed. This solves the problem of unreasonable deployment of computing resources in existing technologies and improves the efficiency of 3D Gaussian rendering.

CN123118884APending Publication Date: 2026-11-17SOUTH CHINA UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611234682.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-14
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing 3D Gaussian rendering methods fail to effectively utilize imaging contribution information from the training perspective, resulting in computational resources being occupied by low-contribution primitives and reducing rendering efficiency.

Method used

By determining the contribution weights of high-level primitives in the pixel transparency synthesis link under multiple training perspectives, accumulating imaging contributions, generating a set of target perspective main calling primitives, and constructing a lookup table-based directional color response evaluation mechanism, the computational resources are prioritized to high-contribution primitives.

Benefits of technology

It improves the efficiency of computational resource utilization in the process of generating new perspectives from the target viewpoint, reduces the burden of ineffective computation, and enhances rendering efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118884A_ABST
    Figure CN123118884A_ABST
Patent Text Reader

Abstract

This application provides a method and related equipment for deploying 3D Gaussian target viewpoint resources, belonging to the field of image data processing technology. The method includes: acquiring a 3D Gaussian model, which includes multiple Gaussian primitives; determining the contribution weight of each Gaussian primitive in the pixel transparency synthesis link under multiple training viewpoints, the contribution weight being jointly determined by the opacity contribution of the Gaussian primitive at the current pixel and the forward transmission relationship formed by the Gaussian primitives in front of it; accumulating the contribution weights of the Gaussian primitives to multiple pixels in each training viewpoint to obtain the in-view imaging contribution of each Gaussian primitive under each training viewpoint; and determining the computational resource deployment priority of the Gaussian primitives in the target viewpoint rendering process based on the in-view imaging contributions of the Gaussian primitives under multiple training viewpoints. This application improves the efficiency of computational resource utilization in the target viewpoint new viewpoint generation process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of image data processing technology, and in particular to a method and related equipment for deploying three-dimensional Gaussian target viewpoint resources. Background Technology

[0002] 3D Gaussian Splatting (3DGS) uses anisotropic 3D Gaussian as a renderable primitive. It generates target viewpoint images through projection, screen space coverage determination, depth organization, color evaluation, and transparency mixing. It has high image quality and real-time rendering capabilities in generating new viewpoint images, and is therefore widely used for interactive 3D content display.

[0003] In practical rendering, 3D Gaussian models typically contain a large number of Gaussian primitives, each of which must participate in computational processes such as projection, sorting, rasterization, color evaluation, memory access, and transparency blending. However, the contributions of different Gaussian primitives to the final image vary significantly: some primitives are located in key areas of the object's surface and have a significant impact on pixel color; while many primitives, due to being behind occlusion, located at the edge of the scene, or contributing very little, have extremely limited impact on image quality. Existing rendering methods allocate computational resources indiscriminately to all Gaussian primitives when generating the target viewpoint, resulting in a large amount of computational resources being consumed on low-contribution or even invalid primitives, severely limiting rendering efficiency.

[0004] Various optimization schemes have been proposed in existing technologies. For example, some schemes improve rasterization efficiency from the perspective of rendering kernel execution by optimizing tile partitioning, load balancing, or memory access; some schemes reduce the total number of Gaussian primitives by focusing on model compression or primitive pruning during training; and some schemes predict primitive importance and selectively render primitives based on their own attributes (such as opacity and scale). However, all of the above schemes share a common technical flaw: they do not utilize the actual imaging contribution information of each Gaussian primitive in the pixel transparency synthesis link from the training perspective to evaluate the rendering value of the primitives.

[0005] For a 3D Gaussian model trained from multi-view images, each training viewpoint inherently contains the imaging contribution information of each Gaussian primitive in different occlusion relationships, depth orders, and transparency hybrid paths—this contribution depends simultaneously on the primitive's own opacity, its depth order within the pixel, and the transmission relationship with the primitives in front of it. However, existing schemes have not yet established a mechanism to utilize this information to distinguish between high-contribution primitives and low-contribution primitives, resulting in a lack of reasonable primitive-level basis for the deployment of computational resources during target viewpoint rendering. Summary of the Invention

[0006] The main objective of this application is to propose a three-dimensional Gaussian target view resource deployment method, device, electronic device, storage medium, and program product, which aims to solve the problem that existing three-dimensional Gaussian new view generation methods lack primitive-level computing resource deployment basis during target view rendering and that some computing resources fail to prioritize serving high-contribution imaging primitives.

[0007] To achieve the above objectives, one aspect of this application proposes a three-dimensional Gaussian target view resource deployment method, the method comprising: Obtain a three-dimensional Gaussian model, wherein the three-dimensional Gaussian model includes multiple Gaussian elements; Under multiple training perspectives, the contribution weight of each Gaussian element in the pixel transparency synthesis link is determined. The contribution weight is jointly determined by the opacity contribution of the Gaussian element at the current pixel and the forward transmission relationship formed by other Gaussian elements located in front of the Gaussian element. The contribution weights of the Gaussian elements to multiple pixels are accumulated in each of the training viewpoints to obtain the in-view imaging contribution of the Gaussian elements in each of the training viewpoints. The in-view imaging contribution is used to characterize the effective contribution of the Gaussian elements to the imaging results in the corresponding training viewpoint. Based on the Gaussian primitive's contribution to in-view imaging under multiple training viewpoints, the computational resource deployment priority of the Gaussian primitive in the target viewpoint rendering process is determined.

[0008] In some embodiments, the contribution weight is determined in the following manner:

[0009] in, Indicates the first From a training perspective, the individual high-level elements pixels The pixel transparency composition link contribution weight at that location, Indicates the first Each Gaussian element in a pixel The contribution of opacity at the location, Indicates the first Forward transmission relationship before each Gaussian element; The forward transmission relationship Determined in the following ways:

[0010] in, Indicates the Gaussian unit in the current pixel's transparency composition order. Located in Gaussian Before.

[0011] In some embodiments, the intra-view imaging contribution is determined by the following method:

[0012] in, Indicates the first From a training perspective, the individual high-level elements The contribution of in-view imaging, Indicates training perspective A set of pixels.

[0013] In some embodiments, after obtaining the in-view imaging contribution of the Gaussian element at each of the training viewpoints, the method further includes: The Gaussian element is sorted according to its imaging contribution within the multiple training perspectives, and the peak perspectives with the highest contribution are selected for peak perspective aggregation to obtain the core contribution evaluation result of the Gaussian element. The priority of computational resource deployment for the Gaussian element in the target viewpoint rendering process is determined based on the core contribution evaluation results.

[0014] In some embodiments, the peak view aggregation is determined by the following method:

[0015]

[0016] in, Indicates the first The highest contributor among the individual Gaussian elements A set of training viewpoint indices Indicates the first Evaluation results of the core contributions of each Gaosky element Indicates the first From a training perspective, the individual high-level elements The contribution of imaging within the lower field of view.

[0017] In some embodiments, determining the computational resource deployment priority of the Gaussian primitive in the target view rendering process based on the core contribution evaluation results includes: Based on the core contribution evaluation results of each Gaussian primitive, a target perspective main call primitive set is generated; In this context, the Gaussian primitives in the target view main call primitive set are given a higher priority in the deployment of computing resources than non-main call primitives during the generation of a new view of the target view, so that computing resources for projection, sorting, rasterization, color evaluation, video memory access, and transparency mixing are given priority to serve the Gaussian primitives in the target view main call primitive set.

[0018] In some embodiments, the target perspective main call primitive set is generated in the following manner: All Gaussian elements are sorted in descending order according to the core contribution evaluation results to obtain a sorting index. ; Based on the preset contribution energy maintenance threshold According to the sorting index Following the indicated order, the core contribution evaluation results of each Gaussian primitive are sequentially accumulated to determine the minimum number of main call primitives that satisfy the following conditions. :

[0019] The sorted first Each Gaussian primitive is determined as the set of primary invocation primitives for the target viewpoint:

[0020] in, This represents the total number of Gaussian elements. This indicates the threshold for maintaining energy contribution. This represents the set of primitives that are invoked from the target's perspective. Indicates the sorted order of the first... Evaluation results of the core contributions of each Gaosky element Indicates the sorted order of the first... A high-ranking official.

[0021] In some embodiments, Gaussian primitives that have not entered the target view main call primitive set are set to a non-main call state, an on-demand call state, a low-frequency call state, or a quality enhancement mode call state.

[0022] In some embodiments, a lookup table-based orientation color response evaluation mechanism is constructed for the Gaussian primitives in the target viewpoint main call primitive set, the lookup table-based orientation color response evaluation mechanism including: The color of each Gaussian element is represented as a combination of the base color, the blending coefficient vector, and the shared directional color response matrix:

[0023]

[0024] in, Indicates the first A Gaussian element in the direction of observation The color below, Indicates the first The basic colors of the Gaussian element. Indicates the first A Gaussian mixture coefficient vector, Indicates the direction of observation The corresponding shared directional color response matrix, Indicates a direction color response generator. Indicates the direction of observation Directional encoding.

[0025] In some embodiments, the method further includes: Before rendering the target viewpoint, the observation direction space is discretized into multiple direction units, and the direction color response is pre-calculated for each direction unit to generate a direction color response lookup table. During target view rendering, the directional color response lookup table is indexed according to the observation direction between the target camera and the Gaussian unit, the corresponding directional color response is read or interpolated to obtain the corresponding directional color response, and the target view color is calculated by combining the mixing coefficient vector of the Gaussian unit.

[0026] In some embodiments, the orientation color response lookup table is generated in the following manner:

[0027] in, The first element in the direction color response lookup table represents the first element. Each item in the table, Indicates the first Each discrete directional unit represents the observation direction. Indicates the total number of discrete directional elements; Color values ​​for rendering from the target viewpoint are determined in the following way:

[0028]

[0029] in, Indicates the direction of observation The corresponding lookup table index, This represents the directional distance metric function. This indicates a table lookup or interpolation function.

[0030] In some embodiments, the computing resources include at least one of projection resources, sorting resources, rasterization resources, color evaluation resources, video memory access resources, and transparency mixing resources.

[0031] To achieve the above objectives, another aspect of this application proposes a three-dimensional Gaussian target view resource deployment device, the device comprising: The model acquisition module is used to acquire a three-dimensional Gaussian model, which includes multiple Gaussian elements; The contribution observation module is used to determine the contribution weight of each Gaussian element in the pixel transparency synthesis link under multiple training perspectives. The contribution weight is jointly determined by the opacity contribution of the Gaussian element at the current pixel and the forward transmission relationship formed by other Gaussian elements located in front of the Gaussian element. The contribution evaluation module is used to accumulate the contribution weights of the Gaussian element to multiple pixels in each of the training viewpoints to obtain the in-view imaging contribution of the Gaussian element in each of the training viewpoints. The resource deployment module is used to determine the computational resource deployment priority of the Gaussian unit in the target view rendering process based on the in-view imaging contribution of the Gaussian unit under multiple training viewpoints.

[0032] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0033] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0034] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0035] The embodiments of this application include at least the following beneficial effects: This application provides a three-dimensional Gaussian target viewpoint resource deployment method, apparatus, electronic device, storage medium, and program product. This solution introduces contribution weights in the pixel transparency synthesis link and accumulates these contribution weights across multiple training viewpoints to obtain the in-viewpoint imaging contribution of each Gaussian primitive. These contribution weights simultaneously consider the opacity contribution of the Gaussian primitive itself and the forward transmission relationship formed by the Gaussian primitives in front of it, thus truly reflecting the effective contribution of each Gaussian primitive to the final pixel color during the actual imaging process at the training viewpoint. This avoids relying solely on the primitive's own properties (such as opacity). This addresses the issue of the disconnect between evaluation metrics (such as degree and scale) and actual imaging contributions. Based on this, the priority of computational resource deployment in the target viewpoint rendering process is determined according to the in-view imaging contribution of each Gaussian primitive across multiple training viewpoints. This ensures that computational resources such as projection, sorting, rasterization, color evaluation, memory access, and transparency blending are preferentially deployed to primitives with higher imaging contributions. This avoids the problem of low-contribution primitives consuming large amounts of computational resources while high-contribution primitives suffer from insufficient resources due to unfounded resource allocation. Consequently, it improves the utilization efficiency of computational resources during the generation of new viewpoints in the target viewpoint and reduces the ineffective computational burden in the main rendering path. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the overall process of a three-dimensional Gaussian target view resource deployment method provided in the embodiments of this application.

[0037] Figure 2 This is a schematic diagram of the target perspective main call primitive set generation process in the embodiments of this application.

[0038] Figure 3 This is a schematic diagram of the directional color response lookup table and the lookup-based color evaluation process in the embodiments of this application.

[0039] Figure 4 This is a flowchart of a three-dimensional Gaussian target view resource deployment method provided in an embodiment of this application.

[0040] Figure 5 This is a schematic diagram of the structure of a three-dimensional Gaussian target view resource deployment device provided in an embodiment of this application.

[0041] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0042] In the diagram, 101 represents the initial 3D Gaussian model acquisition step, 102 the pixel transparency synthesis link contribution observation step, 103 the core contribution evaluation and target view main call primitive set generation step, 104 the direction color response evaluation mechanism construction step, 105 the computational resource deployment model output step for the new viewpoint generation for the target viewpoint, 106 the direction color response lookup table generation step, 107 the lookup table color evaluation step, and 108 the target viewpoint computational resource deployment rendering step; 201 represents the training viewpoint input, 202 the opacity contribution statistics, 203 the forward transmission relationship statistics, 204 the peak viewpoint aggregation, 205 the contribution energy preservation or sorting selection, and 206 the target viewpoint main call primitive set; 301 represents the target observation direction input, 302 the direction index generation, 303 the direction color response lookup table, 304 the per-Gaussian mixing coefficient, 305 the lookup table color evaluation, 306 the transparency blending rendering, and 307 the new viewpoint image output. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0045] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0046] 1) 3D Gaussian Splatting (3DGS) is a scene representation and differentiable rasterization rendering technique based on explicit 3D Gaussian ellipsoid sets. It can achieve real-time (>100FPS) rendering while maintaining photorealistic quality and is widely used in new perspective compositing, digital twins and XR fields.

[0047] With the development of 3D web page display, mobile 3D interaction, and virtual reality and augmented reality applications, 3D scenes not only need to be reconstructed offline, but also need to generate new perspective images with low latency as the user continuously moves their viewpoint. 3D Gaussian sputtering uses anisotropic 3D Gaussians as renderable primitives, generating target perspective images through projection, screen space coverage determination, depth organization, color evaluation, and transparency mixing. The 3D Gaussian sputtering method disclosed in the first prior art solution has high image quality and real-time rendering capabilities in generating new perspective images, and is therefore increasingly being used for interactive 3D content display.

[0048] In practical low-latency rendering systems, to generate new perspectives for the target image on web pages, mobile devices, head-mounted displays, and other interactive terminals, 3D Gaussian models typically require a large number of Gaussian primitives to be used in computational processes such as projection, sorting, rasterization, color evaluation, memory access, and transparency blending. While different Gaussian primitives consume runtime computational resources, their effective contributions to the target image imaging process are not consistent. Without a prior assessment of the relationship between the imaging contribution of Gaussian primitives and runtime resource consumption, computational resources for projection, sorting, rasterization, color evaluation, and transparency blending can easily be allocated to primitives with low or unstable imaging contributions, thus reducing the overall computational efficiency of generating new perspectives.

[0049] Existing solutions for reducing the computational burden of 3D Gaussian model rendering have optimized the computational resource consumption during the target viewpoint rendering process from different perspectives. For example, some solutions improve the execution efficiency of projection, rasterization, and transparency blending by optimizing the rendering kernel, parallel execution, and graphics processor scheduling through tile partitioning, load balancing, memory access optimization, or sorting process adjustments. Other solutions adjust the rendering execution process or the representation layers involved in rendering based on hardware resource limitations, focusing on device adaptation, layer detail, or local sampling strategies. A second existing technology discloses a 3D Gaussian rendering method combining Gaussian-level parallelism and fine-grained tiles to alleviate workload imbalance during rendering. A third existing technology discloses a 3D Gaussian rasterization optimization method for large-scale and high-resolution scenes, improving computational efficiency through rendering process analysis, redundancy elimination, scheduling control, and memory access optimization. A fourth existing technology discloses a 3D Gaussian rendering process for mobile devices, enabling real-time inference under mobile resource constraints. The above solutions can reduce some computing resource consumption in specific operational phases, but their focus is usually on rendering kernel execution methods, parallel scheduling, hierarchical details, or device adaptation strategies.

[0050] For 3D Gaussian models obtained from multi-view images, the training views already contain imaging contribution information of each Gaussian primitive in different viewing directions, occlusion relationships, and pixel transparency synthesis links. Existing solutions have not fully utilized this type of training view imaging information to determine the effectiveness of different Gaussian primitives after occupying projection, sorting, rasterization, color evaluation, memory access, and transparency mixing resources, nor have they formed a target view master call primitive generation mechanism that prioritizes the deployment of computing resources to high-contribution core Gaussian primitives.

[0051] Furthermore, the evaluation of view-dependent colors in the 3D Gaussian model continuously consumes computational resources and slows down color calculation efficiency. Traditional methods often use spherical harmonics to store multi-order view-dependent color coefficients for each Gaussian unit; other methods utilize multi-layer perceptrons or other neural networks to predict orientation-dependent colors. The former causes the color evaluation stage to frequently read a large number of high-dimensional color parameters and occupy video memory bandwidth, while the latter, if orientation-dependent inference is repeatedly performed on each Gaussian unit in every frame, will further increase computational resource consumption. Therefore, the process of generating new viewpoints from the target perspective not only needs to determine which Gaussian units should be prioritized for computational resources, but also needs to provide a color evaluation path more suitable for low-latency operation for the main calling Gaussian unit.

[0052] Therefore, existing technologies still require a computational resource deployment scheme for the generation of new perspectives in the target viewpoint of a 3D Gaussian model. This scheme should introduce contribution weights from the pixel transparency synthesis link during the model building stage, evaluate the imaging contribution of each Gaussian primitive under multiple training perspectives, and select high-contribution core Gaussian primitives through contribution thresholds, contribution energy preservation conditions, sorting selection mechanisms, or combinations thereof. Computational resources for projection, sorting, rasterization, color evaluation, memory access, and transparency blending in the target viewpoint rendering should be preferentially deployed to these high-contribution core Gaussian primitives. Simultaneously, the point-by-point viewpoint-related color evaluation of the main calling primitive should be converted into pre-calculated directional response lookup tables and a small amount of blending operations, thereby improving the efficiency of computational resource utilization in the generation of new perspectives in the target viewpoint.

[0053] In view of this, this application provides a method, apparatus, electronic device, storage medium, and program product for deploying 3D Gaussian target viewpoint resources. This solution does not aim to simply reduce the number of 3D Gaussian model parameters or compress model file size. Instead, it addresses the computational resource deployment problem in the generation of new target viewpoints. It introduces contribution weights in the pixel transparency synthesis link, evaluates the imaging contribution of each Gaussian primitive under multiple training views, and determines high-contribution core Gaussian primitives through contribution thresholds, contribution energy preservation conditions, sorting selection mechanisms, or combinations thereof, generating a target viewpoint main calling primitive set. During target viewpoint rendering, the system prioritizes the deployment of computational resources such as projection, sorting, rasterization, color evaluation, video memory access, and transparency blending onto the target viewpoint main calling primitive set, and outputs the new viewpoint image using a lookup-table directional color response evaluation mechanism.

[0054] The 3D Gaussian target view resource deployment method provided in this application relates to the fields of image data generation, 3D scene representation, neural rendering, graphics computing resource deployment, and interactive graphics display technology. The 3D Gaussian target view resource deployment method provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the 3D Gaussian target view resource deployment method, but is not limited to the above forms.

[0055] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0056] like Figure 4 As shown, this embodiment provides a method for deploying resources from a three-dimensional Gaussian target perspective, specifically including the following steps: Step S1: Obtain a three-dimensional Gaussian model, which includes multiple Gaussian primitives; Step S2: Under multiple training perspectives, determine the contribution weight of each Gaussian element in the pixel transparency synthesis link. The contribution weight is jointly determined by the opacity contribution of the Gaussian element at the current pixel and the forward transmission relationship formed by other Gaussian elements located in front of the Gaussian element. Step S3: Accumulate the contribution weights of the Gaussian elements to multiple pixels in each of the training viewpoints to obtain the in-view imaging contribution of the Gaussian elements in each of the training viewpoints; the in-view imaging contribution is used to characterize the effective contribution of the Gaussian elements to the imaging results in the corresponding training viewpoints. Step S4: Determine the computational resource deployment priority of the Gaussian unit in the target view rendering process based on the in-view imaging contribution of the Gaussian unit under multiple training viewpoints.

[0057] The following section provides a detailed description and explanation of the solutions in the embodiments of this application, using specific application examples. This embodiment provides a method for deploying resources from a three-dimensional Gaussian target perspective, specifically including the following steps: Step 1: Obtain the initial 3D Gaussian model.

[0058] Obtain a 3D Gaussian model constructed from multi-view images, camera intrinsics, and camera extrinsics, or obtain a pre-trained 3D Gaussian model. The 3D Gaussian model includes multiple renderable Gaussian primitives, each of which includes at least spatial position, scale, rotation, opacity, and color parameters.

[0059] Step 2: Introduce contribution weights in the pixel transparency composition chain.

[0060] The 3D Gaussian model is rendered from multiple training perspectives. For each Gaussian primitive, its contribution weight in the pixel opacity composition chain is determined. The contribution weight is not determined solely by the opacity of the Gaussian primitive itself, but is jointly determined by the opacity contribution of the Gaussian primitive at the pixel, its blending order in the current pixel, and the forward transmission relationship formed by the Gaussian primitives located in front of it.

[0061] For ease of description, regarding the training perspective Pixels The pixel color synthesis and single-primary contribution weights are represented as:

[0062]

[0063] in, Indicates training perspective Medium pixel Synthetic colors, Indicates participating pixels The rendered Gaussian metaset, Indicates the first A high-ranking dollar, Indicates the first A Gaussian element in the direction of observation The color below, Indicates the first Each Gaussian element in a pixel The contribution of opacity at the location, This indicates the forward transmission relationship preceding the Gaussian element. Indicates the Gaussian unit in the current pixel's transparency composition order. Located in Gaussian Before, Indicates the first From a training perspective, the individual high-level elements pixels The pixel transparency composition link contribution weight.

[0064] Step 3: Evaluate the primitive imaging contribution from the training perspective.

[0065] For each Gaussian primitive, its contribution weights to multiple pixels are accumulated within each training viewpoint to obtain the image contribution of that Gaussian primitive within the corresponding training viewpoint. Since the same Gaussian primitive may have different contributions under different viewpoints, for example, boundary structures, fine structures, occluded regions, or highlight regions may only have a significant effect in a few viewpoints, the method in this embodiment does not directly use the simple average result of all training viewpoints as the evaluation basis.

[0066]

[0067] in, Indicates the first From a training perspective, the individual high-level elements The in-view imaging contribution is obtained based on the pixel transparency synthesis link contribution weight. Indicates training perspective A set of pixels.

[0068] Step 4: Determine high-contribution core high-ski elements based on peak perspective aggregation.

[0069] For each Gaussian primitive, it is sorted according to its imaging contribution across multiple training perspectives. Peak perspective aggregation is then performed on several training perspectives with high contributions to obtain the core contribution evaluation result for that Gaussian primitive. This peak perspective aggregation is used to retain Gaussian primitives that play a crucial imaging role in a few perspectives, preventing such primitives from being misclassified as low-value primitives due to their low average contribution.

[0070]

[0071] in, Indicates the first Individual high-level units contribute significantly to multiple training perspectives. A set of training viewpoint indices Indicates the first The core contribution evaluation results of each Gaussian element. In other implementations, The Gaussian primitive can also be modified by combining the number of pixels covered, the number of color evaluations, the amount of video memory reads, or the number of sorting participations in the corresponding training viewpoint to obtain the unit resource imaging contribution evaluation result.

[0072] Step 5: Generate the target perspective main call primitive set.

[0073] Based on the core contribution evaluation results, the set of primary call primitives in the target perspective generation process is determined through contribution thresholds, contribution energy preservation conditions, sorting selection mechanisms, resource budget constraints, or combinations thereof. The set of primary call primitives for the target perspective refers to the set of high-contribution core primitives that preferentially participate in projection, sorting, rasterization, color evaluation, memory access, and transparency mixing during the target perspective generation process.

[0074] Based on the Gaussian primitive's contribution to in-view imaging under multiple training viewpoints, the computational resource deployment priority of the Gaussian primitive in the target viewpoint rendering process is determined.

[0075] In one implementation, Gaussian elements are sorted in descending order based on the core contribution evaluation results to obtain a sorting index. The sorting index This is used to indicate the order of the Gaussian elements according to the core contribution evaluation results from high to low; and to maintain a threshold based on contribution energy. According to the sorting index The core contribution evaluation results of each Gaussian primitive are accumulated sequentially from front to back in the indicated order to determine the main calling primitive set:

[0076]

[0077]

[0078] in, This represents the total number of Gaussian elements. This indicates the threshold for maintaining energy contribution. This represents the set of primitives that are invoked from the target's perspective. Indicates the sorted order of the first... Evaluation results of the core contributions of each Gaosky element Indicates the sorted order of the first... Each high-performance unit. In another implementation, the resource budget can also be set based on the target frame rate, target video memory budget, target power consumption budget, or terminal device capabilities. And the resource budget constraint and the energy contribution maintenance condition are used together to determine .

[0079] For those who have not entered The system can configure high-contribution core Gaussian elements to be called in non-primary, on-demand, low-frequency, or quality-enhanced modes based on application needs, rather than treating them as objects that continuously and preferentially consume computing resources in the target view's main rendering path. Therefore, the main computing resources in the target view's rendering process are preferentially deployed to high-contribution core Gaussian elements.

[0080] Step 6: Construct a lookup table-based mechanism for evaluating directional color responses.

[0081] A lookup-based directional color response evaluation mechanism is constructed for Gaussian primitives in the target viewpoint's main call primitive set. Specifically, the observation direction space is discretized into multiple directional units, and a directional color response table is pre-calculated for each directional unit before deployment or target viewpoint rendering. When the target viewpoint is generated, the system determines the observation direction based on the camera position and Gaussian primitive positions, obtains the corresponding directional color response by reading or interpolating the direction index, and combines it with the color mixing parameters of the Gaussian primitives to obtain the target viewpoint color.

[0082] For the target perspective, the first in the set of primary call primitives A Gaussian element, in the direction of observation The color below can be represented as:

[0083] in, Indicates the first The basic colors of the Gaussian element. Indicates the first A Gaussian mixture coefficient vector, Indicates the direction of observation The corresponding shared directional color response matrix, Indicates a direction color response generator. Indicates the direction of observation Directional encoding.

[0084] Discretize the view orientation space before deployment or target view rendering. Each direction unit generates a direction color response lookup table.

[0085] During target view rendering, the color values ​​are obtained using the direction index function and lookup table interpolation function:

[0086] in, Indicates the first Each discrete directional unit represents the observation direction. The first element in the direction color response lookup table Each item in the table, Indicates directional distance measurement. This indicates a lookup table or interpolation function. Therefore, during target viewpoint generation, the viewpoint-related color calculation of the main caller Gaussian unit can be transformed from point-by-point direction function evaluation or high-dimensional color parameter reading into direction indexing, table lookup, interpolation, and a small amount of mixed operations.

[0087] Step 7: Perform target viewpoint computational resource deployment rendering and output the new viewpoint image.

[0088] The camera parameters of the target viewpoint are obtained, and computational resources are preferentially deployed to the main calling primitive set of the target viewpoint. Projection, sorting, rasterization, lookup table-based color evaluation, and transparency blending are performed on the main calling primitive set to output a new viewpoint image of the target viewpoint. Thus, the main computational resources in the new viewpoint generation process are concentrated on core Gaussian primitives that contribute significantly to imaging, thereby improving the efficiency of computational resource utilization.

[0089] Example 1: Primitive Imaging Contribution Evaluation Based on Pixel Transparency Synthesis Link Contribution Weights In this embodiment, the input is a 3D Gaussian model constructed from multi-view images. This model can be obtained through a 3D Gaussian sputtering training process or through other 3D reconstruction or neural rendering processes. Each Gaussian primitive includes at least 3D position, scale, rotation, opacity, and color parameters.

[0090] like Figure 1 As shown, the initial 3D Gaussian model acquisition step (labeled 101) is executed first. The model is then rendered from multiple training views, and the pixel transparency synthesis link contribution observation step (labeled 102) is executed. For each pixel, the system calculates the transparency mixing weights according to the Gaussian unit depth order.

[0091] Specifically, regarding the training perspective Pixels The Gaussian elements involved in rendering this pixel are sorted from near to far according to depth. Each Gaussian element... In pixels Opacity contribution The forward transmission relationship formed by the Gaussian elements in front of it is: Then the Gaussian pixel is in the pixel The pixel transparency composition link contribution weight at that location is This weight simultaneously reflects the Gaussian pixel's own opacity, foreground occlusion, pixel coverage, and color composition contribution. Therefore, it can serve as an evaluation criterion for whether the Gaussian pixel should preferentially occupy projection, sorting, rasterization, color evaluation, and transparency mixing resources.

[0092] In each training perspective Inside, for the first The contribution weights of each Gaussian element across all pixels are summed to obtain the in-view imaging contribution of that Gaussian element under the training viewpoint. .

[0093] In one specific implementation, the number of training viewpoints is: The image resolution for each training viewpoint is Pixels. The system for all. The above contribution weight statistics and in-view imaging contribution calculation are performed on each training viewpoint to obtain the contribution distribution of each Gaussian unit under 100 training views.

[0094] Example 2: Peak View Aggregation and Target View Main Call Primitive Set Generation In this embodiment, the following steps are performed based on Embodiment 1. Figure 1 The core contribution evaluation and target perspective main call primitive set generation steps corresponding to the number 103.

[0095] For a Gaussian unit, instead of directly averaging the contributions of all training perspectives, we select the perspectives with the highest contributions for peak perspective aggregation. Specifically, for the _th A high-ranking Yuan, which he placed in Intra-view imaging contribution from each training perspective Sort in descending order and select the top... The training perspective that contributed the most, The core contribution evaluation result of this Gaussian element is obtained by summing the contribution values ​​from each perspective. .

[0096] In one specific implementation, In other words, for each Gaussian primitive, the five training perspectives that contribute the most are selected for peak perspective aggregation. This avoids the erroneous underestimation of primitives such as thin rods, object edges, specular highlights, and occlusion boundaries, which only have high imaging value from a few perspectives, when deploying computational resources.

[0097] See Figure 2 Then, based on the energy contribution preservation condition, the set of target perspective main call primitives is determined. Specifically, for all Each Gao Siyuan was evaluated based on his core contributions. Perform descending sorting to obtain the sorted index. The sorting index This is used to indicate the order in which the Gaussian elements are arranged from highest to lowest according to the core contribution evaluation results. A contribution energy maintenance threshold is set. According to the sorting index The core contribution evaluation results of each Gaussian primitive are accumulated sequentially from front to back in the indicated order to determine the minimum number of main call primitives that meet the conditions. :

[0098]

[0099] In one specific implementation, the original three-dimensional Gaussian model includes After the above screening, each Gaussian element... That is, about 40% of the Gaussian primitives enter the target perspective main call primitive set.

[0100] In this set of Gaussian primitives, projection, sorting, rasterization, color evaluation, and transparency composition are given priority in subsequent target viewpoint rendering. Gaussian primitives that are not included in the target viewpoint's main call primitive set can be set to non-main call, on-demand call, low-frequency call, or quality enhancement mode call states according to application needs. Therefore, during target viewpoint runtime, the system can deploy the main computing resources to high-contribution core Gaussian primitives, reducing the sorting queue length, pixel composition count, and memory access pressure in the main rendering path.

[0101] In another implementation, the resource budget can be set based on the target frame rate, target memory budget, or target power consumption budget. Resource budget constraints and energy contribution maintenance conditions are used together to determine .

[0102] Example 3: Construction of a lookup-based directional color response evaluation mechanism In this embodiment, a lookup table-based direction color response evaluation mechanism is constructed based on the target viewpoint main call primitive set obtained in Embodiment 2. For example... Figure 1 As shown in reference numeral 104, this mechanism no longer requires each Gaussian primitive to independently store a large number of direction-dependent color functions, nor does it require each main calling primitive to repeatedly execute complex direction functions when rendering the target viewpoint. Instead, it represents the color of each Gaussian primitive as a combination of a base color, a small number of blending coefficients, and a shared direction color response.

[0103] Specifically, for the first A Gaussian element, in the direction of observation The color is determined by the base color, blending coefficient, and directional color response matrix:

[0104] in, Indicates the first The basic color of each high-order element (such as RGB three channels). Indicates the first A Gaussian mixture coefficient vector, Indicates the direction of observation The corresponding directional color response matrix.

[0105] In one specific implementation, the mixing coefficient dimension A value of 5 can be used. Directional color response matrix. By direction color response generator Directional coding of the observation direction Mapping yields: In one implementation, It could be a lightweight multilayer perceptron (MLP) containing 2-3 fully connected layers; in another implementation... It can also be a combination of low-order spherical harmonic functions, a learnable response function, or a combination thereof.

[0106] This representation allows different Gaussian elements to share directional color response. Each Gaussian unit stores only a small number of mixing coefficients. (In this embodiment, it is 5-dimensional) and basic color. (3D) Compared to the traditional spherical harmonic function method (which typically stores 16-45 spherical harmonic coefficients per Gaussian), the storage requirements for color parameters are significantly reduced. Since the target viewpoint color calculation does not require reading a large number of independent high-order directional color parameters, nor does it require repeatedly executing complex directional functions for each Gaussian unit, it helps to reduce runtime memory access, cache pressure, and the resource consumption for color evaluation calculations.

[0107] In other implementations, the mixing coefficient dimension It can be any integer between 2 and 16, or it can be set adaptively according to the target image quality, target frame rate and target device capabilities.

[0108] Example 4: Generation of Directional Color Response Lookup Table and Lookup-based Color Evaluation In this embodiment, a directional color response lookup table is generated before deployment or target view rendering. For example... Figure 3 As shown, label 301 is the target observation direction input, label 302 is the direction index generation, and label 303 is the direction color response lookup table.

[0109] Specifically, the unit spherical orientation space is discretized into multiple orientation units. For each orientation unit, its representative orientation is input into an orientation color response generator to obtain the corresponding orientation color response, and this orientation color response is written into a lookup table.

[0110] In one specific implementation, the sphere is discretized using a latitude and longitude sampling method. There are 12 directional units, with the longitude direction divided into 32 parts and the latitude direction divided into 16 parts. For the 12th unit... Each directional unit represents a direction. The corresponding directional color response is .

[0111] In another implementation, the lookup table can be sampled using a cube map (6 faces, each face) Sampling (384 directional elements in total), hierarchical spherical sampling, or adaptive directional sampling. Lookup table size. The value can be set according to the target image quality requirements and the target device capabilities, with a typical range of 256 to 2048.

[0112] During target view rendering, the system calculates the viewing direction based on the camera center and Gaussian unit position. Use the directional index function to locate the lookup table cell:

[0113] in The directional distance metric is represented, such as cosine distance or spherical distance. The directional color response is then obtained by reading or interpolation. In one implementation, a nearest neighbor lookup table is used for direct reading. In another implementation, bilinear or trilinear interpolation is used to interpolate the responses of adjacent directional units to obtain a smoother directional color response.

[0114] After obtaining the directional color response, the target's viewpoint color is calculated by combining the mixing coefficients of the Gaussian element:

[0115] This process transforms runtime orientation color generation from neural network inference or high-dimensional function evaluation into table lookup and a small number of multiplication and addition operations (5 multiplications and 5 additions in this embodiment), which is more suitable for parallel execution by graphics processors and can reduce the computational resource consumption of the color evaluation stage.

[0116] Example 5: Target-view computational resource deployment rendering and new perspective image output In this embodiment, the inputs are the camera intrinsic and extrinsic parameters of the target viewpoint. First, the projection range of each Gaussian primitive in the target viewpoint main calling primitive set on the image plane is determined according to the target camera parameters, and the Gaussian primitives participating in the rendering are organized according to depth order or approximately depth order.

[0117] Specifically, such as Figure 1 As shown in reference number 108, the system performs the target viewpoint computational resource deployment rendering steps: The first step is to analyze the target perspective's main call primitive set. Each Gaussian primitive in the image is projected from three-dimensional space onto a two-dimensional image plane to obtain an elliptical coverage area of ​​each Gaussian primitive on the image plane.

[0118] The second step is to sort all the Gaussian elements involved in the rendering according to their depth, either from far to near or from near to far.

[0119] The third step is to collect all the high-order pixels covering that pixel for each pixel and perform transparency blending in sorted order.

[0120] The fourth step is to calculate the target's visual color by reading or interpolating the directional color response from the directional color response lookup table based on the target's viewing direction, and then combining this with the Gaussian mixing coefficient to calculate the target's visual color.

[0121] The fifth step is to synthesize pixel colors according to the transparency blending method and output a new perspective image of the target viewpoint.

[0122] The output should include at least: first, a 3D Gaussian representation that can be directly called by a low-resource-consumption rendering system, which includes the target viewpoint's main calling primitive set, base colors, blending coefficients, and lookup table parameters; and second, a new viewpoint image generated from this representation under the target viewpoint.

[0123] Depending on the application requirements, it can also output engineering indicators such as the number of main call primitives, lookup table size, rendering frame rate, peak video memory usage, end-to-end frame time, number of color evaluations, and number of non-main call primitives, as reference information for system deployment.

[0124] Example 6: Application of Low-Latency Renderable Representations in Resource-Constrained Scenarios In this embodiment, the present application can also be used to construct a low-resource-consumption, renderable 3D representation in resource-constrained scenarios. For a 3D Gaussian model trained from multi-view images, pixel transparency synthesis link contribution observation, training view imaging contribution evaluation, peak view aggregation, target view main call primitive set generation, and orientation color response lookup table generation are performed during or after training. This directly yields a 3D Gaussian representation suitable for low-resource-consumption deployment and low-latency rendering.

[0125] It should be noted that "low-resource deployment" in this embodiment refers to prioritizing the deployment of major computing resources such as projection, sorting, rasterization, color evaluation, memory access, and transparency blending to high-contribution Gaussian primitives during the generation of the new target perspective. It does not limit the output to a mesh model, point cloud model, or computer-aided geometric model. If the application system requires further output of a mesh or other geometric formats, subsequent conversions can be performed based on the 3D Gaussian representation output in this application.

[0126] In a specific application scenario, this method can be deployed on mobile devices for real-time 3D scene browsing. By reducing the number of Gaussian primitives from the original 200,000 to 80,000 in the main call set (a 60% reduction), and combining lookup-based color evaluation (converting Gaussian harmonic function evaluation into lookup tables and 5D vector multiplication), the end-to-end rendering frame rate is increased from the original 15 FPS to over 45 FPS, and the video memory usage is reduced from 2.5 GB to 1.2 GB, meeting the real-time rendering requirements of mobile devices.

[0127] In another application scenario, this method can be deployed on a web page to display 3D content via WebGL or WebGPU. Because the main calling primitive set significantly reduces the number of Gaussian primitives that need to be transmitted and rendered, both model loading time and first-frame rendering time are significantly reduced, resulting in an improved user experience.

[0128] In summary, compared with the prior art, this application has at least the following advantages and beneficial effects: 1) By introducing contribution weights in the pixel transparency synthesis link, this application can determine whether Gaussian elements should preferentially occupy projection, sorting, rasterization, color evaluation, video memory access and transparency mixing resources in the target viewpoint new viewpoint generation process based on their contribution to the real imaging process of the training viewpoint.

[0129] 2) By employing in-view contribution accumulation and peak view aggregation, this application is able to retain boundary structures, fine structures, occlusion structures or specular structures that have significant contributions only in a few training views, thus avoiding the underestimation of core primitives due to simple average evaluation.

[0130] 3) By determining the set of primary call primitives for the target perspective through contribution threshold, contribution energy maintenance conditions, sorting selection mechanism or resource budget constraints, this application can prioritize the deployment of the main computing resources in the target perspective generation process to high contribution core primitives, thereby reducing the runtime burden caused by unfounded resource allocation and inefficient calls in the main rendering path.

[0131] 4) Due to the adoption of a lookup table-based directional color response evaluation mechanism, this application can transform the calculation of view-related colors during target view generation from Gaussian-wise complex directional function evaluation or high-dimensional color parameter reading into directional indexing, table lookup, interpolation, and a small amount of mixed operations, thereby reducing the computational resource consumption and memory bandwidth pressure in the color evaluation stage.

[0132] 5) By combining the observation of the synthetic link contribution of the training perspective transparency, the generation of the target perspective main call primitive set, and the lookup table-based evaluation of the directional color response, this application can simultaneously improve the deployment of primitive-level computing resources and the color evaluation path, and reduce end-to-end frame time, memory access pressure, and the operating burden on resource-constrained devices.

[0133] 6) Because this application outputs a new perspective image of the target viewpoint and a three-dimensional Gaussian representation that can be directly called by a low-resource-consumption rendering system, it can facilitate the interactive display of three-dimensional content, online transmission, terminal deployment, and low-latency operation in web pages, mobile terminals, virtual reality and augmented reality devices.

[0134] See Figure 5 This application also provides a three-dimensional Gaussian target view resource deployment device that can implement the above-described method. The device includes: The model acquisition module is used to acquire a three-dimensional Gaussian model, which includes multiple Gaussian elements; The contribution observation module is used to determine the contribution weight of each Gaussian element in the pixel transparency synthesis link under multiple training perspectives. The contribution weight is jointly determined by the opacity contribution of the Gaussian element at the current pixel and the forward transmission relationship formed by other Gaussian elements located in front of the Gaussian element. The contribution evaluation module is used to accumulate the contribution weights of the Gaussian element to multiple pixels in each of the training viewpoints to obtain the in-view imaging contribution of the Gaussian element in each of the training viewpoints. The resource deployment module is used to determine the computational resource deployment priority of the Gaussian unit in the target view rendering process based on the in-view imaging contribution of the Gaussian unit under multiple training viewpoints.

[0135] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0137] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0138] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 602 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called and executed by the processor 601 using the methods described in the embodiments of this application. The input / output interface 603 is used to implement information input and output; The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604); The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.

[0139] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0140] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0141] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0142] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0143] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0144] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0145] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0146] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0148] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0149] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0151] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0153] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0154] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for deploying resources from a three-dimensional Gaussian target perspective, characterized in that, The method includes the following steps: Obtain a three-dimensional Gaussian model, wherein the three-dimensional Gaussian model includes multiple Gaussian elements; Under multiple training perspectives, the contribution weight of each Gaussian element in the pixel transparency synthesis link is determined. The contribution weight is jointly determined by the opacity contribution of the Gaussian element at the current pixel and the forward transmission relationship formed by other Gaussian elements located in front of the Gaussian element. The contribution weights of the Gaussian elements to multiple pixels are accumulated in each of the training viewpoints to obtain the in-view imaging contribution of the Gaussian elements in each of the training viewpoints. Based on the Gaussian primitive's contribution to in-view imaging under multiple training viewpoints, the computational resource deployment priority of the Gaussian primitive in the target viewpoint rendering process is determined.

2. The method according to claim 1, characterized in that, The contribution weight is determined in the following way: in, Indicates the first From a training perspective, the individual high-level elements pixels The pixel transparency composition link contribution weight at that location, Indicates the first Each Gaussian element in a pixel The contribution of opacity at the location, Indicates the first Forward transmission relationship before each Gaussian element; The forward transmission relationship Determined in the following ways: in, Indicates the Gaussian unit in the current pixel's transparency composition order. Located in Gaussian Before.

3. The method according to claim 1, characterized in that, After obtaining the in-view imaging contribution of the Gaussian element at each of the training viewpoints, the method further includes: The Gaussian element is sorted according to its imaging contribution within the multiple training perspectives, and the peak perspectives with the highest contribution are selected for peak perspective aggregation to obtain the core contribution evaluation result of the Gaussian element. The priority of computational resource deployment for the Gaussian element in the target viewpoint rendering process is determined based on the core contribution evaluation results.

4. The method according to claim 3, characterized in that, The peak view aggregation is determined in the following way: in, Indicates the first The highest contributor among the individual Gaussian elements A set of training viewpoint indices Indicates the first Evaluation results of the core contributions of each Gaosky element Indicates the first From a training perspective, the individual high-level elements The contribution of imaging within the lower field of view.

5. The method according to claim 3, characterized in that, The step of determining the computational resource deployment priority of the Gaussian element in the target view rendering process based on the core contribution evaluation results includes: Based on the core contribution evaluation results of each Gaussian primitive, a target perspective main call primitive set is generated; In this context, the Gaussian primitives in the target view main call primitive set are given a higher priority in the deployment of computing resources than non-main call primitives during the generation of a new view of the target view, so that computing resources for projection, sorting, rasterization, color evaluation, video memory access, and transparency mixing are given priority to serve the Gaussian primitives in the target view main call primitive set.

6. The method according to claim 5, characterized in that, The target perspective main call primitive set is generated in the following way: All Gaussian elements are sorted in descending order according to the core contribution evaluation results to obtain a sorting index. ; Based on the preset contribution energy maintenance threshold According to the sorting index Following the indicated order, the core contribution evaluation results of each Gaussian primitive are sequentially accumulated to determine the minimum number of main call primitives that satisfy the following conditions. : The sorted first Each Gaussian primitive is determined as the set of primary invocation primitives for the target viewpoint: in, This represents the total number of Gaussian elements. This indicates the threshold for maintaining energy contribution. This represents the set of primitives that are invoked from the target's perspective. Indicates the sorted order of the first... Evaluation results of the core contributions of each Gaosky element Indicates the sorted order of the first... A high-ranking official.

7. The method according to claim 5, characterized in that, A lookup table-based orientation color response evaluation mechanism is constructed for the Gaussian primitives in the target viewpoint main call primitive set. This lookup table-based orientation color response evaluation mechanism includes: The color of each Gaussian element is represented as a combination of the base color, the blending coefficient vector, and the shared directional color response matrix: in, Indicates the first A Gaussian element in the direction of observation The color below, Indicates the first The basic colors of the Gaussian element. Indicates the first A Gaussian mixture coefficient vector, Indicates the direction of observation The corresponding shared directional color response matrix, Indicates a direction color response generator. Indicates the direction of observation Directional encoding.

8. The method according to claim 1, characterized in that, The method further includes: Before rendering the target viewpoint, the observation direction space is discretized into multiple direction units, and the direction color response is pre-calculated for each direction unit to generate a direction color response lookup table. During target view rendering, the directional color response lookup table is indexed according to the observation direction between the target camera and the Gaussian unit, the corresponding directional color response is read or interpolated to obtain the corresponding directional color response, and the target view color is calculated by combining the mixing coefficient vector of the Gaussian unit.

9. The method according to claim 8, characterized in that, The directional color response lookup table is generated in the following way: in, The first element in the direction color response lookup table represents the first element. Each item in the table, Indicates the first Each discrete directional unit represents the observation direction. Indicates the total number of discrete directional elements; Color values ​​for rendering from the target viewpoint are determined in the following way: in, Indicates the direction of observation The corresponding lookup table index, This represents the directional distance metric function. This indicates a table lookup or interpolation function.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 9.