Large-scene federated 3d gaussian sputtering scene reconstruction method and system for resource-constrained edge network

Abstract

The application discloses a large-scene federated 3D Gaussian sputtering field scene reconstruction method and system for resource-constrained edge networks, and the method comprises the following steps: constructing a resource-aware federated 3D-GS system composed of a single edge cloud server and a group of ubiquitous edge perception devices; on the edge of the ubiquitous edge perception device, based on the collected local multi-view real scene image dataset and the adaptive lightweight mechanism, the optimal Gaussian point subset is adaptively selected for local training under the given GPU memory budget and the maximum time delay constraint, the updated local spatial features, position gradients and binary masks are obtained, and the updated local spatial features, position gradients and binary masks are returned to the single edge cloud server through the wireless network; on the single edge cloud server side, the local spatial features, position gradients and binary masks uploaded by the device are used to map the structure-heterogeneous local model back to the global index space, the structure consistency is recovered, and then weighted aggregation is implemented to generate a three-dimensional real scene map.

Description

Technical Field

[0001] This invention belongs to the technical field of high-precision 3D collaborative reconstruction of large-scale outdoor real physical scenes, specifically involving a method and system for large-scale federated 3D Gaussian sputtering scene reconstruction for resource-constrained edge networks. Background Technology

[0002] In practical engineering applications such as urban digital inspection, rapid mapping of post-disaster ruins, and high-precision map construction for autonomous driving, 3D reconstruction of large-scale real-world physical scenes plays a crucial role. In recent years, 3D Gaussian Splatting (3D-GS), as a novel 3D content representation technology, utilizes a massive number of learnable 3D Gaussian primitives to parametrically represent physical space, demonstrating significant advantages in visual realism and rendering speed. However, when the reconstructed object expands from a single local object to a large-scale real-world outdoor environment, the number of Gaussian points representing physical features in the 3D-GS model explodes, causing a dramatic increase in model size. This results in extremely high memory consumption and computational overhead on the Graphics Processing Unit (GPU) during the training phase, severely hindering the engineering implementation of this technology on computing-constrained devices.

[0003] With the development of IoT and Mobile Edge Computing (MEC) technologies, mobile edge sensing devices such as autonomous mapping vehicles, inspection drone swarms, and AR / VR headsets have been widely deployed in real-world physical environments for distributed acquisition of multi-view high-definition images. In current engineering practices, the training of large-scale 3D-GS models often employs a centralized cloud architecture: each edge sensing device must upload a massive amount of raw high-definition images and the intrinsic and extrinsic parameter data of its physical cameras to a cloud data center for unified training via a wireless network. However, in real-world wireless edge network environments, this centralized architecture faces severe physical challenges: it not only rapidly depletes extremely valuable wireless uplink bandwidth, causing significant communication latency, but also exposes sensitive geographic visual data collected by the devices to the risk of direct privacy breaches.

[0004] To overcome the aforementioned communication bottlenecks and data security issues, edge-collaborative distributed and federated 3D-GS training has become an important technical approach for solving large-scale scene reconstruction. Typical approaches include: spatially dividing the large-scale physical scene into multiple sub-regions and assigning them to different edge devices for independent training; or introducing a federated learning mechanism, allowing edge devices to train on local airborne / vehicle platforms, uploading model parameter updates only via wireless links, thereby merging local models at cloud nodes. This approach, to some extent, avoids the wireless transmission of original images, alleviates communication pressure, and enhances privacy protection.

[0005] Despite the progress made by the aforementioned distributed / federated 3D-GS training methods, practical deployment in wireless edge environments still faces the following prominent challenges: (1) The stringent resource constraints of wireless edge physical devices are not fully considered, which can easily lead to computing power crashes and memory overflows: Real wireless edge sensing devices (such as UAV onboard computing modules) are usually limited by size, power consumption and heat dissipation. Their physical GPU memory, local computing power and wireless uplink bandwidth are extremely limited, and the network channel often fluctuates. Existing solutions generally lack a model size adaptive adjustment mechanism that can sense the heterogeneous constraints of the underlying hardware (such as dynamic memory limits and end-to-end physical communication / computation latency). This makes it very easy for edge devices to experience memory overflow (OOM) crashes when they are forced to carry and train large-scale 3D-GS models of large scenes, resulting in the system being unable to run stably in real mapping environments.

[0006] (2) Misalignment of physical space structures from multiple perspectives leads to severe visual artifacts in the global large-scale scene stitching: In edge-coordinated 3D reconstruction, different devices independently update Gaussian points based on their own physical motion trajectories and limited observation perspectives (e.g., each performs different addition, deletion, or splitting operations). This results in huge differences in the 3D spatial geometry and point cloud density of the generated local models. If these heterogeneous models are directly merged using conventional parameters on the edge cloud server, the original physical space consistency will be destroyed, resulting in severe boundary blurring, ghosting, and spatial floating objects in the reconstructed global large-scale scene.

[0007] (3) Over-reliance on global refinement by central nodes deviates from the real application scenario of "decentralization" in edge networks: In order to eliminate the above-mentioned artifacts, some existing solutions rely heavily on central servers to collect physical camera parameters of each edge device for global model distillation or retraining. This approach not only leads to a large amount of idle edge computing power and unnecessary wireless transmission overhead, but also violates the actual physical requirements of "naturally dispersed data and real-time node processing" in wireless edge networks, making it difficult to meet the application scenarios with extremely high timeliness requirements, such as post-disaster relief. Summary of the Invention

[0008] To address the challenges of collaboratively training large-scale 3D-GS models using ubiquitous edge sensing devices (such as surveying drones, mobile inspection terminals, and smart cameras) on wireless edge networks in large-scale outdoor physical scene 3D reconstruction tasks, which are constrained by the physical GPU memory capacity, computing power, and wireless communication bandwidth of airborne / vehicle-mounted devices, memory overflow and extremely high end-to-end training latency are common. Furthermore, the limited viewing angles of different devices lead to spatial geometric misalignment and severe visual artifacts when heterogeneous models are aggregated in the cloud. This invention provides a method and system for large-scene federated 3D Gaussian sputtering scene reconstruction for resource-constrained edge networks. Under the stringent physical resource limitations of wireless edge networks, it achieves an optimal balance between large-scene physical space rendering quality and system training overhead. This not only effectively avoids memory overflow crashes of edge devices, significantly accelerates the convergence speed of the global spatial model, and greatly reduces end-to-end communication and computing latency, but also effectively eliminates ghosting and floating object artifacts at model stitching points using a mask recovery mechanism, maintaining extremely high rendering fidelity and physical geometric consistency for large scenes.

[0009] To achieve the above objectives, the present invention provides the following solution: A method for large-scale federated 3D Gaussian sputtering scene reconstruction for resource-constrained edge networks, the method comprising: Construct a resource-aware federated 3D-GS system consisting of a single edge cloud server and a set of ubiquitous edge sensing devices; On the edge of the ubiquitous edge sensing device, based on the collected local multi-view real-scene image dataset and the adaptive lightweight mechanism, under the given GPU memory budget and maximum latency constraints, the optimal Gaussian point subset is adaptively selected for local training to obtain updated local spatial features, position gradients and binary masks. Ubiquitous edge sensing devices transmit updated local spatial features and location gradients back to a single edge cloud server via wireless network; On a single edge cloud server, local spatial features, location gradients, and binary masks uploaded by devices are used to map heterogeneous local models back to the global index space. After structural consistency is restored, weighted aggregation is performed to generate a 3D real-world map.

[0010] Preferably, based on the acquired local multi-view real-scene image dataset and an adaptive lightweight mechanism, under given GPU memory budget and maximum latency constraints, the method adaptively selects the optimal Gaussian point subset for local training to obtain updated local spatial features, location gradients, and binary masks includes: In the In each communication round, the device Utilize the local camera parameter set to process the distributed global model. ,in This represents the total number of global Gaussian points. Including the For each Gaussian point, perform a lightweight operation on its various properties: construct a reduced version of the initial local model by selectively retaining some Gaussian points. And simultaneously generate a binary mask vector. ; equipment Use lightweight model High-definition real-scene datasets collected locally implement One cycle of reconstruction training, making Indicates the number of communication rounds ,cycle The local model at that time, the device calculates the gradient of the sampled data: ; And update the local model parameters: ; in For learning rate, Meanwhile, the gradient of the Gaussian points accumulated and retained by the device with respect to their 3D spatial location is: ; in, It is the gradient of the complete model parameters. A subset of represents the partial derivative of a Gaussian point with respect to its three-dimensional spatial coordinates; This indicates that the device is in The sum of gradients at all spatial locations within a local training cycle.

[0011] Preferably, the method for generating a 3D reality map by mapping structurally heterogeneous local models back to the global index space using local spatial features, positional gradients, and binary masks uploaded by the device, performing structural consistency restoration, and then implementing weighted aggregation includes: A single edge cloud server utilizes the received mask The local models and position gradients of each device are reconstructed back into a unified structure aligned with the global model. After the structure is restored and aligned, a single edge cloud server performs a weighted average of the local model and gradients to obtain the aggregated global position gradient. A single edge cloud server performs density control based on the aggregated global location gradient, generates the next round of global model and broadcasts it to the device, and iterates until the 3D reconstruction target of the entire physical scene converges.

[0012] Preferably, a single edge cloud server utilizes the received mask. Methods for reconstructing the local models and position gradients of each device back into a unified structure aligned with the global model include: ; in, For lightweight models, The aggregated global position gradient, For the restored model, The recovered position gradient, This represents the structure consistency recovery function based on binary masks.

[0013] Preferably, after structural restoration and alignment, the method for obtaining the aggregated global location gradient by performing a weighted average of the local model and gradients on a single edge cloud server includes: ; ; in, K For the number of devices, For the aggregated global model, This represents the aggregated global position gradient.

[0014] This invention also provides a large-scene federated 3D Gaussian sputtering scene reconstruction system for resource-constrained edge networks. The system is used to implement the aforementioned method and includes: a system architecture construction module, an edge-side adaptive training module, a feature gradient backpropagation module, and a cloud-based aggregation mapping module. The system architecture building module is used to build a resource-aware federated 3D-GS system consisting of a single edge cloud server and a set of ubiquitous edge sensing devices; The edge-side adaptive training module is used on the edge side of the ubiquitous edge sensing device to adaptively select the optimal subset of Gaussian points for local training under the given GPU memory budget and maximum latency constraints, based on the collected local multi-view real-scene image dataset and adaptive lightweight mechanism, to obtain updated local spatial features, position gradients and binary masks. The feature gradient backhaul module is used by the ubiquitous edge sensing device to backhaul the updated local spatial features and location gradients to a single edge cloud server via a wireless network. The cloud-based aggregation mapping module is used on a single edge cloud server to map heterogeneous local models back to the global index space using local spatial features, position gradients, and binary masks uploaded by devices. After restoring structural consistency, it performs weighted aggregation to generate a 3D real-world map.

[0015] Preferably, based on the acquired local multi-view real-scene image dataset and an adaptive lightweight mechanism, under given GPU memory budget and maximum latency constraints, the process of adaptively selecting the optimal Gaussian point subset for local training to obtain updated local spatial features, location gradients, and binary masks includes: In the In each communication round, the device Utilize the local camera parameter set to process the distributed global model. ,in This represents the total number of global Gaussian points. Including the For each Gaussian point, perform a lightweight operation on its various properties: construct a reduced version of the initial local model by selectively retaining some Gaussian points. And simultaneously generate a binary mask vector. ; equipment Use lightweight model High-definition real-scene datasets collected locally implement One cycle of reconstruction training, making Indicates the number of communication rounds ,cycle The local model at that time, the device calculates the gradient of the sampled data: ; And update the local model parameters: ; in For learning rate, Meanwhile, the gradient of the Gaussian points accumulated and retained by the device with respect to their 3D spatial location is: ; in, It is the gradient of the complete model parameters. A subset of represents the partial derivative of a Gaussian point with respect to its three-dimensional spatial coordinates; This indicates that the device is in The sum of gradients at all spatial locations within a local training cycle.

[0016] Preferably, the cloud-based aggregated mapping module includes: a structure alignment unit, a weighted average unit, and a density control unit; The structure alignment unit is used by a single edge cloud server to utilize the received mask. The local models and position gradients of each device are reconstructed back into a unified structure aligned with the global model. The weighted averaging unit is used to perform a weighted average of the local model and gradient on a single edge cloud server after structural restoration and alignment, to obtain the aggregated global position gradient. The density control unit is used by a single edge cloud server to perform density control based on the aggregated global position gradient, generate the next round of global model and broadcast it to the device, and iterate in a loop until the 3D reconstruction target of the entire physical scene converges.

[0017] Preferably, a single edge cloud server utilizes the received mask. The process of reconstructing the local models and position gradients of each device back into a unified structure aligned with the global model includes: ; in, For lightweight models, The aggregated global position gradient, For the restored model, The recovered position gradient, This represents the structure consistency recovery function based on binary masks.

[0018] Preferably, after structural restoration and alignment, the process by which a single edge cloud server performs a weighted average of the local model and gradients to obtain the aggregated global location gradient includes: ; ; in, K For the number of devices, For the aggregated global model, This represents the aggregated global position gradient.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) A model-adaptive lightweighting mechanism that is aware of the underlying hardware latency and video memory is designed. This mechanism establishes a resource consumption model of physical devices, quantitatively evaluates the importance of each 3D Gaussian primitive to the rendering of the local physical view, and innovatively proposes the "importance-to-latency ratio" criterion. Based on this criterion, edge devices can adaptively select and prune redundant Gaussian points with low contribution to the spatial structure, under the premise of strictly meeting the upper limit of their own physical GPU video memory and the end-to-end wireless communication / computing latency constraints.

[0020] (2) A cloud-based structural consistency restoration mechanism based on binary mask was designed. To address the spatial structural heterogeneity problem caused by local model lightweighting, the edge server uses the uploaded mask to losslessly align the local model parameters and position gradients of each edge device to the global physical spatial structure without accessing the underlying privacy visual data, thereby achieving high-fidelity model weighted aggregation and global density control.

[0021] By employing the aforementioned edge-side adaptive lightweight mechanism based on the "importance-latency ratio" and the cloud-based structural consistency recovery mechanism based on "binary masks," this invention achieves an optimal balance between large-scene physical space rendering quality and system training overhead under stringent wireless edge physical resource constraints. It not only effectively avoids memory overflow crashes on edge devices, significantly accelerates the convergence speed of the global spatial model, and substantially reduces end-to-end communication and computation latency, but also effectively eliminates ghosting and floating artifacts at model stitching points using the mask recovery mechanism, maintaining extremely high rendering fidelity and physical geometric consistency for large scenes. Attached Figure Description

[0022] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the resource-constrained edge network federated 3D-GS training framework for outdoor large-scene reconstruction tasks according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the convergence performance of different schemes in an urban building scenario according to embodiments of the present invention; Figure 3 This is a schematic diagram illustrating the convergence performance of different schemes in an outdoor sand and gravel scene according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the PSNR performance of different reconstruction schemes in an urban building scenario according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the PSNR performance of different reconstruction schemes in an outdoor sand and gravel scene according to an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Example 1 This invention is specifically applied to high-precision 3D collaborative reconstruction tasks in large-scale outdoor physical scenes (such as digitizing urban building complexes and rapidly mapping post-disaster ruins). In this scenario, multiple ubiquitous edge sensing devices (such as mapping drones equipped with cameras and inspection vehicles) move through the physical space to collect real-world images. This invention uses a federated 3D-GS framework for edge collaboration to perform 3D reconstruction of this scene: First, when faced with the physical constraints of limited onboard GPU memory and wireless backhaul bandwidth, each device does not directly process the entire large scene. Instead, it uses an adaptive lightweight mechanism to evaluate the visibility and rendering importance of physical Gaussian points to its current acquisition perspective (such as the ruins area directly below the drone). It actively prunes physical structures outside the field of view or that are occluded, retaining only the core Gaussian points that meet its current onboard computing power and latency limits for local training. Subsequently, the devices only transmit the updated local spatial features and positional gradients (rather than the high-resolution original images) back via the wireless network. Edge cloud servers (such as ground control stations) utilize uploaded binary masks to initiate a consistency recovery mechanism. This mechanism precisely and losslessly "pieces together" and aligns these incomplete and heterogeneous local physical structures from different perspectives into a globally unified physical coordinate system, ultimately fusing them to generate a complete, large-scale, high-precision 3D real-world map. Specifically, this invention provides a method for large-scene federated 3D Gaussian sputtering scene reconstruction for resource-constrained edge networks. The method includes: Construct a resource-aware federated 3D-GS system consisting of a single edge cloud server and a set of ubiquitous edge sensing devices; On the edge of the ubiquitous edge sensing device, based on the collected local multi-view real-scene image dataset and the adaptive lightweight mechanism, under the given GPU memory budget and maximum latency constraints, the optimal Gaussian point subset is adaptively selected for local training to obtain updated local spatial features, position gradients and binary masks. Ubiquitous edge sensing devices transmit updated local spatial features and location gradients back to a single edge cloud server via wireless network; On a single edge cloud server, local spatial features, location gradients, and binary masks uploaded by devices are used to map heterogeneous local models back to the global index space. After structural consistency is restored, weighted aggregation is performed to generate a 3D real-world map.

[0027] In this embodiment, based on the acquired local multi-view real-scene image dataset and an adaptive lightweight mechanism, under given GPU memory budget and maximum latency constraints, the method of adaptively selecting the optimal Gaussian point subset for local training to obtain updated local spatial features, positional gradients, and binary masks includes: In the In each communication round, the device Utilize the local camera parameter set to process the distributed global model. ,in This represents the total number of global Gaussian points. Including the For each Gaussian point, perform a lightweight operation on its various properties: construct a reduced version of the initial local model by selectively retaining some Gaussian points. And simultaneously generate a binary mask vector. ; equipment Use lightweight model High-definition real-scene datasets collected locally implement One cycle of reconstruction training, making Indicates the number of communication rounds ,cycle The local model at that time, the device calculates the gradient of the sampled data: ; And update the local model parameters: ; in For learning rate, Meanwhile, the gradient of the Gaussian points accumulated and retained by the device with respect to their 3D spatial location is: .

[0028] In this embodiment, the method for generating a 3D reality map by mapping structurally heterogeneous local models back to the global index space using local spatial features, positional gradients, and binary masks uploaded by the device, performing structural consistency restoration, and then implementing weighted aggregation includes: A single edge cloud server utilizes the received mask The local models and position gradients of each device are reconstructed back into a unified structure aligned with the global model. After the structure is restored and aligned, a single edge cloud server performs a weighted average of the local model and gradients to obtain the aggregated global position gradient. A single edge cloud server performs density control based on the aggregated global location gradient, generates the next round of global model and broadcasts it to the device, and iterates until the 3D reconstruction target of the entire physical scene converges.

[0029] In this embodiment, a single edge cloud server utilizes the received mask. Methods for reconstructing the local models and position gradients of each device back into a unified structure aligned with the global model include: ; in, For lightweight models, This represents the aggregated global position gradient.

[0030] In this embodiment, after structural restoration and alignment, the method for obtaining the aggregated global location gradient by performing a weighted average of the local model and gradient on a single edge cloud server includes: ; ; in, K The number of devices.

[0031] Example 2 The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0032] This invention aims to solve the problem of high-precision 3D reconstruction of large-scale outdoor physical scenes (such as urban building complexes, post-disaster ruins, etc.). In this specific application scenario, this invention constructs a system consisting of a single edge cloud server (e.g., a ground control station or MEC edge node) and a set of... A resource-aware federated 3D-GS system is composed of ubiquitous edge sensing devices (such as mapping drones equipped with high-definition cameras and autonomous inspection vehicles). The set of ubiquitous edge sensing devices is denoted as […]. As attached Figure 1 As shown.

[0033] System model definition: In this resource-aware federated 3D-GS system, each edge sensing device Responsible for surveying and exploring the physical area it is located in, and maintaining a local 3D Gaussian physical scene model on the local airborne / vehicle-mounted computing platform, denoted as... The edge cloud server is responsible for coordinating the various devices, aggregating and maintaining a global physical scene model covering the entire large-scale reconstruction area. .

[0034] In actual 3D reconstruction tasks, equipment Independently acquire local multi-view real-scene image datasets along their physical motion trajectories The physical meanings of each parameter are as follows: Indicates resolution as And has three color channels Zhang high-definition real-scene RGB image, among which Indicates the length of the image. This represents the width of the image; the total number of pixels an image contains is... This image records the true photometric information of the physical environment (such as the texture of building surfaces, the geometric outline of ruins, etc.); This indicates the spatial pose (extrinsic parameters, i.e., three-dimensional coordinates and rotation orientation in the real-world coordinate system) and intrinsic parameters (such as focal length and optical center) of the physical camera mounted on the device when acquiring the real-world image; The total number of locally acquired real-scene images is denoted as Because the physical cruise trajectories of different mapping drones / inspection vehicles may overlap, the physical views collected by different devices often have some overlapping areas. That is, for some different devices... ,exist .

[0035] At each edge sensing device, the physical objective of local training is to adjust the spatial parameters (position, scaling, rotation, opacity, spherical harmonic color) of the 3D Gaussian primitives through gradient descent to minimize the error between real aerial / inspection images and virtual views rendered using the current onboard 3D-GS model. The local rendering loss function is defined as: in, The single-sample physical rendering error is calculated by taking the pose parameters of the physical camera... Input to the current physical space model The predicted image is obtained through rasterization rendering and compared with high-resolution images captured by real physical sensors. Perform pixel-level difference comparison calculations.

[0036] Accordingly, in order to obtain the globally optimal 3D representation of the entire large-scale outdoor scene, the system constructs a global physical reconstruction total loss function based on the real dataset collected by all distributed edge devices, which is expressed as: The overall engineering optimization objective of the air-ground collaborative reconstruction system of this invention is to collaboratively minimize the aforementioned global loss function under limited physical computing and communication resources, thereby obtaining a three-dimensional Gaussian parameterized representation model that is optimal for real large-scale physical scenes. : .

[0037] Federated Learning (FL) Pipeline: The 3D-GS training architecture of this invention adopts a protocol based on Federated Averaging Algorithm (FedAvg). The collaborative reconstruction task of the entire large-scale physical scene is performed between edge cloud servers and distributed ubiquitous edge sensing devices (such as autonomous vehicles, mobile inspection robots, wearable / portable smart terminals, etc.). Round-based wireless communication interaction. In each round of communication, the device utilizes its local computing power to perform... There are 10 local training epochs. The index sets of the communication epochs and the local epochs are denoted as follows: and The complete physical process for large-scale scene reconstruction executed by the system is as follows: Step 1: Global Model Broadcasting. In each round of communication... Initially, the edge server broadcasts a global 3D-GS model to all devices. ,in This represents the total number of global Gaussian points. Including the The attribute parameters of each Gaussian point.

[0038] Step 2: On-Device Model Lightweighting. Considering the limited GPU memory, computation latency, and communication constraints of edge devices, the device... Lightweighting is performed on the global model using the local camera parameter set. This operation constructs a scaled-down version of the initial local model by selectively preserving some Gaussian points. And simultaneously generate a binary mask vector. The expression for the mask vector is of the form: Among them, there are a total of These values ​​are obtained from the model's lightweight module. (In the mask) Indicates that the first [number] is reserved. A Gaussian point, This indicates that the point should be clipped. The lightweight operation is defined as: Step 3: Local Model Training. Equipment Use lightweight model High-definition real-scene datasets collected locally implement One cycle of reconstruction training. Let Indicates the number of communication rounds ,cycle The local model at that time. The device calculates the gradient of the sampled data: And update the local model parameters: in The learning rate is used. Simultaneously, the device accumulates and retains the gradients of the Gaussian points with respect to their 3D spatial locations, which are then used for subsequent density control on the server side. These accumulated location gradients do not contain the original visual data, ensuring privacy. The accumulated location gradients are expressed as: in, It is the gradient of the parameters of the complete model mentioned above. A subset of represents the coordinates of a Gaussian point with respect to its three-dimensional spatial location (i.e., its coordinates in the physical world). Partial derivatives of the x-axis; This indicates that the device is in The sum of all spatial location gradients over a local training cycle. These spatial gradients reflect the intensity of local fine-tuning of the physical geometry and are used for subsequent spatial density control in the cloud. More importantly, these spatial location gradients and Gaussian attribute parameters do not contain any original high-resolution real-world pixel data, thus isolating the privacy risks of sensitive real-world views at the source of the reconstruction task.

[0039] Step 4: Local Update Uploading. After local training is complete, the device... Only three pieces of data are uploaded to the edge server: the lightweight model with updated parameters. Accumulated position gradient and binary mask This step significantly reduces the burden on wireless bandwidth compared to directly transmitting massive amounts of acquired images.

[0040] Step 5: On-Server Model Recovery. In federated large-scale scene reconstruction, the core challenge lies in the fact that different devices explored different physical sub-regions during the data acquisition phase (e.g., device A acquired the front section of a street, device B acquired the rear section), resulting in each device cropping and optimizing different subsets of Gaussian points during local training. This local observation-based training leads to significant heterogeneity in the uploaded local model's 3D spatial structure. Directly performing conventional parameter aggregation would result in severe geographic coordinate distortion and visual ghosting. The edge server utilizes the received mask... This involves reconstructing the lightweight local models (lightweight models) and position gradients at each device back into a unified structure aligned with the global model. For each device... The restored model is The recovered position gradient is The process is represented as follows: in, This represents a structure consistency recovery function based on binary masks. Specifically, this recovery function... Based on the initial global model of this round Lightweight local model uploaded by the device Local location gradient and binary mask As joint input variables, the lightweight local 3D Gaussian attribute parameters and gradients are losslessly aligned and backfilled into the global index space through masking rules, ultimately outputting a restored model with a consistent topological structure with the global model. and the recovered position gradient Regarding the structural consistency recovery function For details on the specific execution formulas and logic, please refer to the Structural Consistency Model and Gradient Recovery Mechanism.

[0041] Step 6: Global Model Aggregation and Update. After structural restoration and alignment, the server performs a weighted average of the local models and gradients to obtain the aggregated global model. The global position gradient is The process is represented as follows: Finally, the server uses the aggregated global location gradient. Perform density control (e.g., split and densify the physical Gaussian points of high-frequency textures such as building edges, and prune and remove transparent redundant points caused by occlusion) to generate the next round of global model. It then broadcasts to the device, iterating in a loop until the 3D reconstruction target of the entire physical scene converges.

[0042] Lightweight mechanism for edge models based on Importance-to-Latency Ratio: To achieve efficient training under the resource constraints of edge devices, this invention designs an adaptive Gaussian point selection and pruning strategy. This strategy does not rely on simple random pruning, but is based on a novel "importance-latency ratio" criterion, aiming to maximize the rendering quality gain per unit of latency cost. The specific implementation steps are as follows: Step 1: Importance assessment of Gaussian points. Equipment Based on local dataset For the global model Each Gaussian point in the dataset is assigned an importance score. The importance score reflects the point's contribution to the local view rendering. For the _th _ _ Each view (corresponding camera parameters) and image pixel set The set of coordinates of the Gaussian points associated with this view can be represented as: In the model, the first If a Gaussian point is related to this view, that is The contribution of the Gaussian point in this view The calculation, which takes into account the relationship between opacity and depth occlusion, can be expressed as: in, Point For pixels The contribution of opacity This represents the depth of the point. The formula shows that if a point is occluded by an object in front of it, its importance decreases. If the... The Gaussian points are irrelevant to the view, that is... Contribution Then it is zero. Combining all local views, the first... Global importance score of Gaussian points Defined as: The device generates an importance score vector based on this. .

[0043] Step 2: Resource Consumption Modeling. To quantify the resource-saving effect of pruning, this invention establishes an analytical model for memory usage and end-to-end latency.

[0044] GPU memory model: Device Peak memory usage during training With the selected number of Gaussian points A linear relationship exists: This expression characterizes the composition of peak GPU memory usage during 3D-GS training, which mainly consists of Gaussian model parameters, gradient parameters, temporary variables, training data, and fixed hardware overhead. and These represent the number of various parameters and the number of bytes required, respectively. This represents the number of parameters at a Gaussian point in the model. This indicates the number of bytes occupied by each parameter of this type; This represents the number of gradient parameters corresponding to a Gaussian point parameter. This indicates the number of bytes occupied by a gradient parameter of this type; Indicates the number of parameters for the temporary variable; Indicates the number of camera parameters; The number of image pixels parameter This refers to fixed memory overhead (such as image loading cache and CUDA fixed memory overhead) that is independent of the number of Gaussian points in the model. This model ensures that the device meets memory constraints when making lightweight choices. , Indicates device The maximum amount of video memory it can handle.

[0045] End-to-end delay model: Total delay Includes computational latency And communication latency (downlink broadcast and uplink upload) : The computational latency is modeled based on the GPU's Roofline performance model, covering both Gaussian splatting and rendering stages, and taking into account the number of parallel threads. Device-specific delay coefficient : in, For equipment The number of Gaussian points in the model. Number of image pixels. Sputtering stage delay: .in, This is a rounding up operation. It is the maximum number of parallel threads on the GPU. These are device-specific latency coefficients derived from FLOPs and memory bandwidth. Rendering stage latency: . It is a scaling factor for the number of Gaussian points, used to correct for the additional computational overhead caused by the increase in point cloud density. This represents the latency factor during the rendering phase. Number of training epochs in local mode. Communication latency. Communication latency includes downlink broadcast latency Upload latency : in This represents the total number of Gaussian points in the global model. and These are the downlink and uplink data transmission rates, respectively. This represents the dimension of the location parameters (typically 3). In summary, the specific expression for the total end-to-end delay is: Step 3: Optimal subset selection based on "importance-latency ratio". This invention proposes the "importance-latency ratio" metric. , defined as the ratio of the cumulative importance of the selected Gaussian point set to the total training latency: in, The importance scores are sorted in descending order. This is to determine the optimal number of points to retain. The equipment solves the following constrained optimization problem: in, This represents the maximum total latency that a single device can tolerate in a single communication round. This represents the maximum peak memory consumption that the device can handle during training.

[0046] Implementation Results: By solving the above problem (e.g., using binary search or block search algorithms), the device can adaptively truncate the importance ranking list, retaining only the top-ranked items. A Gaussian point is used for training, and a corresponding binary mask is generated after lightweighting. This ensures that core features contributing to rendering quality are preserved to the greatest extent possible without exceeding the device's video memory and time budget.

[0047] In summary, the edge-adaptive lightweight mechanism based on the "importance-to-latency ratio" (distinct from traditional full-scene unified training or random pruning) addresses the common problem of existing technologies that typically segment large scenes through spatial partitioning or directly train complete models on the device (such as large 3D scene models corresponding to entire disaster areas or city blocks). These methods ignore the heterogeneous resource limitations (GPU memory, computing power, communication bandwidth) of edge devices, leading to extremely high training latency and even Out of Memory (OOM). The key difference of this invention lies in the introduction of a novel metric: the "Importance-to-Latency Ratio." It not only considers the contribution of Gaussian points to visual rendering (importance) but also establishes an accurate latency / GPU memory cost model by combining the underlying GPU execution characteristics and network transmission conditions. Based on this ratio, the device adaptively selects the subset of Gaussian points with the highest "cost-effectiveness" for training, while satisfying its own hard constraints.

[0048] Cloud-based Structure-Consistent Recovery Model and Gradient Recovery Mechanism: Because each ubiquitous edge device explores different sub-regions of physical space during the data acquisition phase, they each prune and retain distinct subsets of Gaussian elements during edge-side lightweighting. This local-view-based training results in highly heterogeneous local models uploaded by each device in terms of 3D geometry and feature dimensions. Directly applying traditional federated averaging to the parameters of these structurally mismatched models would completely destroy the original global physical coordinate system, leading to severe structural misalignment, ghosting, and visual artifacts such as floating objects in the aggregated large scene. The specific implementation steps are as follows: Step 1: Mask-based index mapping. Server receiving device. Uploaded lightweight model (Contains only) (points), location gradient and binary mask Since the local model uploaded by the device is a sparse subset of the global model, in order to ensure that subsequent parameters can be losslessly restored to a unified 3D space, the server needs to use a binary mask to build a global index. To Local Model Index Accurate computational mapping. Utilizing masks. The server builds a global index. Upload model index locally The mapping relationship. Specifically, the binary mask vector is represented as... Its dimension is the same as the total number of Gaussian points in the global model in this round. Strict alignment. The first in the mask. Elements satisfy: If , representing the first in the global model Each Gaussian point is retained on the device and updated locally; if If the value is zero, it means that the point has been clipped. The server performs a prefix sum calculation on the uploaded binary mask, and constructs the index mapping formula as follows: Using the above formula, the server accumulates the mask sequence from the starting position to the current global index. All values ​​are By using the flag bit, the preserved Gaussian points in the local one-dimensional storage structure can be calculated. absolute and relative offset positions .

[0049] Step 2: Model parameter recovery. For the first parameter in the global model... At a Gaussian point, the server reconfigures the device according to the following rules. Complete model parameters : in, Indicates device Uploaded lightweight local model The first in A set of Gaussian point parameters.

[0050] Technical principle: When When this occurs, it indicates that the physical structure represented by the Gaussian point (such as a wall) is within the equipment's range. Since the parameters are within the effective field of view and have been trained and optimized locally, the latest parameters uploaded by the user are used. This indicates that the Gaussian point is located at the equipment. The observation blind spot (such as beyond the line of sight or obstructed). At this time, the physical structure has not disappeared in the real world, but the latest observation data from that perspective is lacking. Therefore, the system directly extracts the parameters of that point from the previous round of global model for backfilling. This perfectly avoids the "catastrophic forgetting" of unseen physical regions by the local model, ensuring that unselected physical features can still maintain their spatial position during subsequent aggregation.

[0051] Step 3: Gradient Field Recovery. Similarly, for the location gradient, the server constructs the recovered gradient field. : in, Indicates device Uploaded cumulative position gradient (i.e. In the set, corresponding to the lightweight local model, the first... The three-dimensional position gradient vector of a Gaussian point.

[0052] Technical principle: For physical Gaussian points within the equipment's observation blind zone ( The structure adjustment gradient is strictly set to a zero vector. This physically ensures that the device will not blindly fine-tune the geometric coordinates of unobserved physical regions using unverified data. This operation effectively eliminates mathematical noise during heterogeneous model splicing, providing an accurate basis for subsequent global spatial structure density control (gradient-based splitting and cloning) of structural evolution.

[0053] Implementation Results: Through the aforementioned recovery mechanism, the server aligns all heterogeneous local models into a unified global structure. The subsequent weighted averaging effectively eliminates visual artifacts (such as ghosting and floating objects) caused by structural mismatches, significantly improving the rendering quality of the aggregated global model in unseen views. Experimental testing of the proposed invention yielded the following simulation results: Figure 2 , Figure 3 , Figure 4 , Figure 5 As shown in Tables 1 and 2.

[0054] Table 1 compares the performance of the proposed invention and the Fed3D-GS algorithm in 3D reconstruction for urban buildings and outdoor sand and gravel scenarios. Table 2 compares the proposed invention with the Fed3D-GS algorithm and the initial 3D-GS algorithm in terms of training latency and GPU memory consumption. In summary, the cloud-based structural consistency restoration mechanism based on "binary masks" (distinct from direct averaging aggregation or aggregation relying on viewpoint information) addresses the following issues with existing methods: Firstly, existing methods often face inconsistencies in local model structures during aggregation (different devices have added or removed different points). Direct averaging leads to severe ghosting and blurring. Secondly, relying on servers for retraining / distillation using shared data increases server load and often requires uploading camera parameters, raising privacy concerns. The key difference in this invention lies in its binary mask-based restoration strategy. Devices only need to upload a lightweight sparse model and mask. New parameters are received for updated Gaussian points within the device's field of view, and global historical parameters are backfilled for Gaussian points clipped in the device's blind spots to prevent "catastrophic forgetting." The gradient in the blind spots is set to zero to prevent the introduction of spatial geometric noise. Therefore, the server can "restore" and align heterogeneous local models to the global structure without accessing any original images or camera parameters, achieving efficient gradient aggregation.

[0055] Example 3 This invention also provides a large-scene federated 3D Gaussian sputtering scene reconstruction system for resource-constrained edge networks. The system is used to implement the method described in Embodiment 1. The system includes: a system architecture construction module, an edge-side adaptive training module, a feature gradient backpropagation module, and a cloud-based aggregated mapping module. The system architecture building module is used to build a resource-aware federated 3D-GS system consisting of a single edge cloud server and a set of ubiquitous edge sensing devices; The edge-side adaptive training module is used on the edge of the ubiquitous edge sensing device to adaptively select the optimal subset of Gaussian points for local training under the given GPU memory budget and maximum latency constraints, based on the collected local multi-view real-scene image dataset and adaptive lightweight mechanism, to obtain updated local spatial features, position gradients and binary masks. The feature gradient backhaul module is used by ubiquitous edge sensing devices to transmit updated local spatial features and location gradients back to a single edge cloud server via a wireless network. The cloud-based aggregation mapping module is used on a single edge cloud server to map heterogeneous local models back to the global index space using local spatial features, position gradients, and binary masks uploaded by devices. After restoring structural consistency, it performs weighted aggregation to generate a 3D real-world map.

[0056] In this embodiment, based on the acquired local multi-view real-scene image dataset and an adaptive lightweight mechanism, under given GPU memory budget and maximum latency constraints, the process of adaptively selecting the optimal Gaussian point subset for local training to obtain updated local spatial features, positional gradients, and binary masks includes: In the In each communication round, the device Utilize the local camera parameter set to process the distributed global model. ,in This represents the total number of global Gaussian points. Including the For each Gaussian point, perform a lightweight operation on its various properties: construct a reduced version of the initial local model by selectively retaining some Gaussian points. And simultaneously generate a binary mask vector. ; equipment Use lightweight model High-definition real-scene datasets collected locally implement One cycle of reconstruction training, making Indicates the number of communication rounds ,cycle The local model at that time, the device calculates the gradient of the sampled data: ; And update the local model parameters: ; in For learning rate, Meanwhile, the gradient of the Gaussian points accumulated and retained by the device with respect to their 3D spatial location is: ; in, It is the gradient of the complete model parameters. A subset of represents the partial derivative of a Gaussian point with respect to its three-dimensional spatial coordinates; This indicates that the device is in The sum of gradients at all spatial locations within a local training cycle.

[0057] In this embodiment, the cloud-based aggregated mapping module includes: a structure alignment unit, a weighted average unit, and a density control unit; Structure alignment unit, used by a single edge cloud server to utilize the received mask The local models and position gradients of each device are reconstructed back into a unified structure aligned with the global model. The weighted average unit is used after structural restoration and alignment. A single edge cloud server performs a weighted average of the local model and gradient to obtain the aggregated global position gradient. The density control unit is used by a single edge cloud server to perform density control based on the aggregated global location gradient, generate the next round of global model and broadcast it to the device, and iterate until the 3D reconstruction target of the entire physical scene converges.

[0058] In this embodiment, a single edge cloud server utilizes the received mask. The process of reconstructing the local models and position gradients of each device back into a unified structure aligned with the global model includes: ; in, For lightweight models, The aggregated global position gradient, For the restored model, The recovered position gradient, This represents the structure consistency recovery function based on binary masks.

[0059] In this embodiment, after structural restoration and alignment, the process of a single edge cloud server performing a weighted average of the local model and gradients to obtain the aggregated global location gradient includes: ; ; in, K For the number of devices, For the aggregated global model, This represents the aggregated global position gradient.

[0060] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for large-scale federated 3D Gaussian sputtering scene reconstruction for resource-constrained edge networks, characterized in that: The method includes: Construct a resource-aware federated 3D-GS system consisting of a single edge cloud server and a set of ubiquitous edge sensing devices; On the edge of the ubiquitous edge sensing device, based on the collected local multi-view real-scene image dataset and the adaptive lightweight mechanism, under the given GPU memory budget and maximum latency constraints, the optimal Gaussian point subset is adaptively selected for local training to obtain updated local spatial features, position gradients and binary masks. Ubiquitous edge sensing devices transmit updated local spatial features and location gradients back to a single edge cloud server via wireless network; On a single edge cloud server, local spatial features, location gradients, and binary masks uploaded by devices are used to map heterogeneous local models back to the global index space. After structural consistency is restored, weighted aggregation is performed to generate a 3D real-world map. The dataset of locally acquired multi-view real-world images and the adaptive lightweight mechanism include: Step 1: Gaussian point importance assessment: equipment Based on local dataset For the global model Each Gaussian point in the matrix is ​​assigned an importance score; for the first Gaussian point... A view, and the set of coordinates of the Gaussian points associated with that view is represented as: ,in, This represents the total number of Gaussian points in the global model. For the training rounds of communication; the first round in the model. If a Gaussian point is related to this view, that is The contribution of the Gaussian point in this view The calculation, which takes into account the relationship between opacity and depth occlusion, is expressed as follows: ； in, Point For pixels The contribution of opacity Indicates the depth of a point; if a point is occluded by an object in front of it, its importance decreases; if the first... The Gaussian points are irrelevant to the view, that is... Contribution Then it is zero; combining all local views, the first... Global importance score of Gaussian points Defined as: ； The device generates an importance score vector based on the global importance score. ; Step 2: Resource Consumption Modeling: An analytical model for video memory usage and end-to-end latency was established. GPU memory model: Device Peak memory usage during training With the selected number of Gaussian points The relationship is linear; it satisfies the memory constraints. , Indicates equipment The maximum video memory overhead that can be accommodated; End-to-end delay model: Total delay Includes computational latency and communication latency The computational latency is modeled based on the GPU's Roofline performance model, covering both Gaussian sputtering and rendering stages, and taking into account the number of parallel threads. Device-specific delay coefficient Communication delay Including downlink broadcast delay Upload latency ; Step 3: Optimal subset selection based on "importance-latency ratio": "Importance-latency ratio" metric , defined as the ratio of the cumulative importance of the selected Gaussian point set to the total training latency: ； in, The importance scores are sorted in descending order; this is to determine the optimal number of points to retain. The equipment solves a constrained optimization problem: ； in, This represents the maximum total latency that a single device can tolerate in a single communication round. This represents the maximum peak memory consumption that the device can handle during training.

2. The method according to claim 1, characterized in that, Based on a locally acquired multi-view real-world image dataset and an adaptive lightweight mechanism, under given GPU memory budget and maximum latency constraints, the method adaptively selects the optimal subset of Gaussian points for local training to obtain updated local spatial features, location gradients, and binary masks includes: In the In each communication round, the device Utilize the local camera parameter set to process the distributed global model. ,in This represents the total number of global Gaussian points. Including the For each Gaussian point, perform a lightweight operation on its various properties: construct a reduced version of the initial local model by selectively retaining some Gaussian points. And simultaneously generate a binary mask vector. ; equipment Use lightweight model High-definition real-scene datasets collected locally implement One cycle of reconstruction training, making Indicates the number of communication rounds ,cycle The local model at that time, the device calculates the gradient of the sampled data: ； And update the local model parameters: ； in For learning rate, Meanwhile, the gradient of the Gaussian points accumulated and retained by the device with respect to their 3D spatial location is: ； in, It is the gradient of the complete model parameters. A subset of represents the partial derivative of a Gaussian point with respect to its three-dimensional spatial coordinates; This indicates that the device is in The sum of gradients at all spatial locations within each local training cycle. For the first A Gaussian point, This indicates the selection of the first [device] from the local training dataset of device k. RGB image corresponding to Gaussian points Indicates the relationship with the first The camera parameters corresponding to the training image for each Gaussian point.

3. The method according to claim 2, characterized in that, Methods for generating 3D reality maps by mapping heterogeneous local models back to the global index space using local spatial features, positional gradients, and binary masks uploaded by devices, performing structural consistency restoration, and then implementing weighted aggregation include: A single edge cloud server utilizes the received mask The local models and position gradients of each device are reconstructed back into a unified structure aligned with the global model. After the structure is restored and aligned, a single edge cloud server performs a weighted average of the local model and gradients to obtain the aggregated global position gradient. A single edge cloud server performs density control based on the aggregated global location gradient, generates the next round of global model and broadcasts it to the device, and iterates until the 3D reconstruction target of the entire physical scene converges.

4. The method according to claim 3, characterized in that, A single edge cloud server utilizes the received mask Methods for reconstructing the local models and position gradients of each device back into a unified structure aligned with the global model include: ； in, For lightweight models, The aggregated global position gradient, For the restored model, The recovered position gradient, This represents the structure consistency recovery function based on binary masks.

5. The method according to claim 4, characterized in that, After structural restoration and alignment, a single edge cloud server performs a weighted average of the local model and gradients to obtain the aggregated global location gradient. Methods include: ； ； in, K For the number of devices, For the aggregated global model, This represents the aggregated global position gradient.

6. A large-scale federated 3D Gaussian sputtering scene reconstruction system for resource-constrained edge networks, the system being used to implement the method described in any one of claims 1-5, characterized in that, The system includes: a system architecture construction module, an edge-side adaptive training module, a feature gradient backpropagation module, and a cloud-based aggregation and mapping module; The system architecture building module is used to build a resource-aware federated 3D-GS system consisting of a single edge cloud server and a set of ubiquitous edge sensing devices; The edge-side adaptive training module is used on the edge side of the ubiquitous edge sensing device to adaptively select the optimal subset of Gaussian points for local training under the given GPU memory budget and maximum latency constraints, based on the collected local multi-view real-scene image dataset and adaptive lightweight mechanism, to obtain updated local spatial features, position gradients and binary masks. The feature gradient backhaul module is used by the ubiquitous edge sensing device to backhaul the updated local spatial features and location gradients to a single edge cloud server via a wireless network. The cloud-based aggregation mapping module is used on a single edge cloud server to map heterogeneous local models back to the global index space using local spatial features, position gradients, and binary masks uploaded by devices. After restoring structural consistency, it performs weighted aggregation to generate a 3D real-world map.

7. The system according to claim 6, characterized in that, The cloud-based aggregated mapping module includes: a structure alignment unit, a weighted average unit, and a density control unit; The structure alignment unit is used at a single edge cloud server to utilize the received mask. The local models and position gradients of each device are reconstructed back into a unified structure aligned with the global model. The weighted averaging unit is used to perform a weighted average of the local model and gradient on a single edge cloud server after the structure is restored and aligned, so as to obtain the aggregated global position gradient. The density control unit is used to perform density control on a single edge cloud server based on the aggregated global position gradient, generate the next round of global model and broadcast it to the device, and iterate until the 3D reconstruction target of the entire physical scene converges.

Images