Three-dimensional modeling water surface hole filling method and system based on federated learning

Abstract

The application relates to the field of information technology, in particular to a three-dimensional modeling water surface cavity filling method and system based on federated learning. The method comprises the following steps: receiving a sample three-dimensional model of a water surface cavity to be filled submitted by a user, and obtaining a cause description of the formation of the water surface cavity corresponding to the sample three-dimensional model; reading a plurality of pre-trained sub-models and sub-model descriptions; calculating the semantic correlation degree of the cause description and each sub-model description, screening a plurality of sub-models as candidate models, and assigning corresponding weights to each candidate model; constructing a weighted federated learning framework, taking the candidate models as weighted teacher models, and generating a lightweight student model through knowledge distillation training; filling the water surface cavity area in the sample three-dimensional model by using the student model, and evaluating the filling result; and adjusting the weight when the filling result does not meet the expectation.

Description

Technical Field

[0001] This application relates to the field of information technology, specifically to a method and system for filling voids in water surfaces based on federated learning in 3D modeling. Background Technology

[0002] With the widespread application of 3D modeling and digital twin technologies, high-precision reconstruction of water surface areas has become a crucial requirement in fields such as water conservancy, urban planning, and environmental monitoring. However, due to limitations imposed by sensor characteristics, environmental interference, and the physical properties of the water surface (such as specular reflection, signal penetration, and occlusion effects), the collected water surface point cloud or image data often contains voids or missing regions, severely impacting model integrity and subsequent analysis accuracy. Traditional void-filling methods often rely on single interpolation strategies or fixed prior assumptions, making them ill-suited to the complex and varied causes of voids and exhibiting limited generalization ability across different scenarios. Furthermore, while existing deep learning methods can improve filling performance, they typically require large amounts of labeled data for training, leading to issues such as data privacy leaks and poor cross-domain adaptability. Therefore, further research is needed on techniques for repairing water surface voids. Summary of the Invention

[0003] This specification describes a method and system for filling voids in 3D water surface based on federated learning through several embodiments.

[0004] Firstly, this specification provides an embodiment of a 3D modeling method for filling water surface voids based on federated learning, comprising the following steps:

[0005] Receive sample 3D models of water surface cavities submitted by users and obtain descriptions of the causes of the water surface cavities corresponding to the sample 3D models;

[0006] Read multiple pre-trained sub-models and their descriptions;

[0007] Calculate the semantic correlation between the cause description and the description of each sub-model, select several sub-models as candidate models based on the semantic correlation, and assign corresponding weights to each candidate model based on the semantic correlation.

[0008] A weighted federated learning framework is constructed, in which the candidate model is used as a weighted teacher model, and a lightweight student model is generated through knowledge distillation.

[0009] The student model was used to fill in the voids in the water surface of the example 3D model, and the filling results were evaluated.

[0010] When the filling result does not meet expectations, the weight is adjusted; when the filling result meets expectations, the calling interface of the student model is provided to the user.

[0011] Secondly, embodiments of this specification provide a 3D modeling water surface cavity filling system based on federated learning, comprising:

[0012] The receiving module receives a sample 3D model of a water surface cavity to be filled submitted by the user, and obtains a description of the cause of the water surface cavity corresponding to the sample 3D model.

[0013] The pre-loading module reads multiple pre-trained sub-models and their descriptions.

[0014] The calculation module calculates the semantic correlation between the cause description and the description of each sub-model, selects several sub-models as candidate models based on the semantic correlation, and assigns corresponding weights to each candidate model based on the semantic correlation.

[0015] The training module constructs a weighted federated learning framework, using the candidate model as a weighted teacher model, and generates a lightweight student model through knowledge distillation.

[0016] The filling module uses the student model to fill the water surface voids in the sample 3D model and evaluates the filling results.

[0017] The interface module adjusts the weights when the filling result does not meet expectations, and provides the calling interface of the student model to the user when the filling result meets expectations.

[0018] Thirdly, embodiments of this specification provide an electronic device, including a processor and a memory;

[0019] The processor is connected to the memory;

[0020] The memory is used to store executable program code;

[0021] The processor runs a program corresponding to the executable program code stored in the memory to perform the method described in any of the above aspects.

[0022] Fourthly, embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above aspects.

[0023] Fifthly, embodiments of this specification provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the above aspects.

[0024] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0025] In several embodiments of this specification, the provided method and system for filling voids in 3D modeling water surfaces based on federated learning effectively improves the filling accuracy and geometric consistency of void regions by integrating multi-scale contextual information and semantic prior knowledge. Compared to traditional interpolation methods or single network models, this solution can adaptively identify the causes of voids, such as occlusion, signal attenuation, or dynamic disturbances, and generate repair results that conform to the actual water body morphology by combining neighborhood structural features and global semantic constraints. Simultaneously, by introducing federated learning or lightweight transfer mechanisms, cross-scene model generalization is achieved while ensuring data privacy, reducing dependence on large-scale labeled data.

[0026] Other features and advantages of various embodiments of this specification will be further revealed in the following detailed description and accompanying drawings. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of filling the voids in the water surface using a 3D model provided in this manual.

[0029] Figure 2 This is a schematic diagram of the process for filling voids in a 3D-modeled water surface provided in this manual.

[0030] Figure 3 This is a schematic diagram of the training sub-model method provided in this specification.

[0031] Figure 4 This is a schematic diagram of the method for calculating semantic relevance provided in this specification.

[0032] Figure 5 This document provides a flowchart illustrating the method for assigning weights to candidate models.

[0033] Figure 6 This is a schematic diagram of the training and generation method for student models provided in this manual.

[0034] Figure 7 This is a schematic diagram of the 3D modeling water surface cavity filling system provided in this manual.

[0035] Figure 8 This is a schematic diagram of the electronic device provided in this manual. Detailed Implementation

[0036] The technical solutions of the embodiments of this specification will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of this specification and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of this specification.

[0037] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0038] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to facilitate the description of the embodiments and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this specification.

[0039] All data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0040] Before introducing the technical solutions described in this manual, the application scenarios and related technologies of the technical solutions will be introduced.

[0041] In 3D modeling, water surface voids refer to discontinuous areas on the water surface caused by missing data, typically manifesting as sparse point clouds, geometric breaks, or texture interruptions. The causes of water surface voids are diverse, primarily including sensor obstruction (e.g., bridges, ships, etc. blocking the laser or photogrammetric line of sight), specular reflection preventing effective signal return to the receiver, electromagnetic or acoustic waves penetrating the water surface causing underwater information to be misinterpreted as invalid, dynamic water level changes resulting in inconsistent elevation values ​​for the same area at different acquisition times, and insufficient density of the original point cloud preventing the reconstruction of water surface details. Common methods for repairing these voids include geometric imputation based on interpolation (e.g., radial basis function interpolation, Kriging interpolation), constraint reconstruction using prior knowledge (e.g., assuming the water surface is planar or smooth), combining external hydrological data (e.g., historical water level lines, DEM data) to assist in inferring missing areas, and generative imputation methods based on deep learning that have emerged in recent years. Interpolation methods are prone to producing unnatural transitions under conditions of large-scale voids or complex boundaries; prior assumptions fail when waves, current velocities, or topographical undulations exist in real water bodies; methods relying on external data are limited by data availability and spatiotemporal matching accuracy; and general-purpose deep learning models often lack semantic understanding of the causes of voids, making it difficult to generalize to scenarios with missing data caused by different mechanisms, and the training process usually requires a large amount of labeled data, making it difficult to deploy in privacy-sensitive or distributed data environments. Therefore, there is an urgent need for an intelligent repair mechanism that can integrate multi-source heterogeneous filling strategies, adaptively select and integrate models based on the causes of voids, and simultaneously take into account data privacy and computational efficiency.

[0042] This manual provides a method and system for filling voids in 3D water surfaces based on federated learning. Please refer to the appendix. Figure 1 This method extracts features and analyzes the context of 3D models containing water surface cavities to identify their geometric shape, boundary continuity, neighborhood point cloud density, and possible causes, such as sensor occlusion, specular reflection, or water level changes. Then, based on pre-defined causal classification rules or a trained machine learning model, the cavities are categorized into several typical types. For each type, the most suitable repair sub-module is dynamically invoked; for example, surface interpolation is used for small-scale smooth defects, while historical hydrological data or digital elevation models are fused for semantic inference of large-area cavities caused by external environmental interference. Through consistency verification and smooth transition processing, the outputs of each sub-module are seamlessly integrated to generate a geometrically continuous and semantically reasonable complete water surface. This helps overcome the insufficient generalization ability of existing single repair methods in complex scenarios.

[0043] Specifically, this specification provides a method for filling voids in 3D water surfaces based on federated learning. Please refer to the appendix. Figure 2 The steps include:

[0044] Step S1: Receive the sample 3D model 10 of the water surface cavity to be filled submitted by the user, and obtain the description 11 of the cause of the water surface cavity corresponding to the sample 3D model 10.

[0045] Users upload 3D model files containing voids in the water surface, such as point clouds or mesh models generated by UAV oblique photography or LiDAR scanning. They also fill in or select a description of the void's cause in a form provided by the platform, such as "data loss due to bridge obstruction," "signal loss due to water surface reflection," or "inconsistent elevation caused by seasonal water level changes." For example, in a river modeling project, a user uploads a river segment model missing water surface data due to bridge pier obstruction and labels the cause field as "sensor line of sight blocked by bridge structure." Based on this, the platform classifies the sample as an "obstruction-type void," providing semantic basis for matching personalized repair strategies in subsequent federated learning.

[0046] Step S2: Read the pre-trained sub-models 21 and their descriptions.

[0047] After receiving the sample 3D model 10 and information on the causes of voids submitted by the user, the platform loads a set of repair sub-models that have been collaboratively trained through federated learning from local or distributed nodes. Each sub-model 21 is accompanied by a structured description explaining its applicable scenarios, input requirements, and technical characteristics. In a cross-regional water conservancy modeling collaboration project, one participant uploaded a lake model with missing data due to mirror reflection of the water surface caused by strong sunlight, and labeled the cause as "loss of optical sensor signal." The platform then reads multiple sub-models 21, including "ReflectionAwareFiller," and their descriptions. The description of "ReflectionAwareFiller" clearly states that the model is designed specifically for highly reflective water surfaces, utilizing neighborhood geometric symmetry and multi-view consistency for reconstruction, and is suitable for void repair in clear-sky aerial photography scenarios. At the same time, the platform may also load other sub-models 21, such as "ObstructionInpainter" and "TemporalHydroMapper," corresponding to occlusion-type and time-series water level change-type voids, respectively.

[0048] Please see the appendix Figure 3 Methods for pre-training multiple sub-models 21 and generating sub-model descriptions include:

[0049] Step S21: Pre-train two sets of sub-models 21, wherein the first set of sub-models are dedicated filling sub-models corresponding to multiple void formation mechanisms, and the sub-model description of the first set of sub-models is generated according to the corresponding void formation mechanism.

[0050] Within the federated learning framework, multiple stakeholders collaborate to train specialized repair models for typical causes of water surface voids, such as sensor occlusion, specular reflection, water level fluctuations, and electromagnetic wave penetration. For example, in a 3D modeling project of a coastal city, due to frequent data loss caused by strong light reflection, the platform, in collaboration with multiple regional nodes, used local desensitized data to jointly train a specialized sub-model called "SpecularHoleFiller"21. Based on its training objectives and applicable conditions, the platform automatically generates a description: "Applicable to local water surface voids caused by specular reflection under optical remote sensing conditions, relying on multi-view geometric consistency and neighborhood curvature smoothing constraints."

[0051] Step S22: The second group of sub-models is a general filling sub-model based on multiple filling logics. The sub-model description of the second group of sub-models is generated according to the corresponding filling logic.

[0052] The second set of sub-models is not limited to specific causes, but is constructed based on general repair strategies, such as logic based on radial basis function interpolation, Poisson reconstruction, depth image completion, or physical hydrodynamic simulation. For example, the platform integrates a general sub-model 21 called "PoissonWaterSurface", whose filling logic is based on global gradient field reconstruction. It is suitable for arbitrarily shaped voids with closed boundaries but missing interiors. It is described as: "Using the Poisson equation to solve the water surface elevation field, it is suitable for medium-sized voids with complete geometric boundaries and no external semantic priors."

[0053] By organizing specialized and general models in parallel and generating structured descriptions based on causal mechanisms and filling logic respectively, the platform retains both high-precision repair capabilities for typical scenarios and generalized adaptability to handle unknown or mixed-cause voids, thus achieving more robust and flexible water surface void filling in complex and ever-changing real-world modeling tasks.

[0054] For example, the first set of sub-models includes an occlusion filling sub-model, a specular reflection filling sub-model, a signal penetration filling sub-model, a water level change filling sub-model, and a point cloud sparse filling sub-model, while the second set of sub-models includes a priori filling sub-model, an interpolation filling sub-model, and a water level data filling sub-model.

[0055] In this embodiment, all sub-models are recommended to be pre-trained within a federated learning framework. This ensures full utilization of data from various participants (such as surveying agencies in different cities). Each participant trains using local, anonymized data, keeping the original data within their local machine. A central server coordinates and aggregates gradients or parameter updates from each participant's model to collectively optimize a global model (or a set of models), thereby achieving stronger generalization capabilities.

[0056] For the occlusion filling sub-model, this sub-model uses complete geometric information outside the hole boundary, combined with the prior that the water surface should be smooth and continuous, to infer the shape of the occluded area inward.

[0057] The recommended hierarchy includes a boundary feature extractor, a context encoder, a generator decoder, and a physical constraint layer. The boundary feature extractor encodes the 3D point cloud coordinates, normals, curvature, and other geometric features of the hole boundary. The context encoder uses graph convolutional networks (GCNs) or PointNet++ to process the neighborhood point cloud, capturing a larger range of contextual structure. The generator decoder typically uses a U-Net or Transformer-based decoder to decode the encoded contextual information into a point cloud or mesh for filling the region. The physical constraint layer is optional and enforces constraints that meet water surface smoothness requirements (such as minimum curvature) or consistency with upstream and downstream water elevations.

[0058] The training process generally includes data preparation, federated initialization, local training, federated aggregation, and iteration. Data preparation requires collecting a large amount of real or synthetic point cloud data showing water surface gaps due to occlusion by objects such as bridges and ships, and generating corresponding complete water surfaces as the Ground Truth. During federated initialization, the model is initialized on each participating node. In local training, each node uses its local occlusion dataset and minimizes the difference between the predicted and real point clouds using L1 / L2 loss or Chamfer Distance loss functions. During federated aggregation, the central server aggregates the model weight updates from each node (e.g., using the FedAvg algorithm) to form a new global model. During iteration, the aforementioned two steps are repeated until the model converges.

[0059] Key parameters include: loss function weights (geometric loss vs. smoothness regularization term), and learning rate (typically 1e). -4 To 1e -3 ), Batch Size (which depends on GPU memory), and Number of Communication Rounds.

[0060] For the specular reflection infill sub-model, this sub-model utilizes multi-view geometric consistency (if available) and the symmetry and smoothness of the neighborhood water surface to reconstruct the data lost due to strong light reflection.

[0061] The recommended hierarchy includes a multi-view feature fusion module, a symmetry detection and utilization module, and a curvature smoothing constraint. In the multi-view feature fusion module, if the input contains multi-view images or point clouds, this module aligns and fuses information from different perspectives. The symmetry detection and utilization module analyzes the geometric patterns of the surrounding water surface, assuming that the reflection area has some symmetry (e.g., symmetry about the normal). The curvature smoothing constraint includes adding a penalty term to the output curvature in the loss function to ensure a smooth and natural filling result.

[0062] The training process includes data preparation, data augmentation, and federated training. Data preparation involves collecting water surface data with specular reflection holes under clear weather and low-angle lighting conditions. Data augmentation simulates reflection effects at different intensities and angles to increase data diversity. The federated training process is similar, but the loss function must emphasize geometric consistency and curvature smoothness.

[0063] Key parameters should generally include the weights of the curvature smoothing term, the weights of the multi-view consistency loss, and the number of input views.

[0064] For the signal penetration filling sub-model, it is necessary to understand the relationship between the physical properties of the signal and the optical / acoustic properties of the water body. This is because electromagnetic waves (such as LiDAR) or sound waves undergo refraction, scattering, and energy attenuation when penetrating the water surface, resulting in weak or distorted returned signals, thus creating holes in the point cloud data.

[0065] The recommended layers include a multimodal input encoder, a physical model embedding layer, and a generative decoder.

[0066] The multimodal input encoder comprises a point cloud encoder, a signal feature extractor, and an environmental context encoder. The point cloud encoder processes the 3D point cloud geometry (coordinates, normal vectors, intensity values) surrounding the hole. Architectures such as PointNet++ or KPConv can be used. For the signal feature extractor, if raw sensor data is available, this module analyzes signal features such as echo intensity, pulse width, and multiple echoes to infer penetration depth and water turbidity. The environmental context encoder encodes auxiliary information such as the angle of incidence, weather conditions (affecting atmospheric refraction), and known water type (freshwater / saline water, affecting refractive index).

[0067] The physical model embedding layer is the core of this sub-model, integrating optical / acoustic refraction principles such as Snell's law as hard or soft constraints into the network. For example, it could be a differentiable ray tracing module used to simulate the path of a signal entering water from air and predict the theoretically correct return point.

[0068] Generative decoders generate point clouds to fill in regions based on the encoding context and physical model guidance. Autoregressive or diffusion-based generative strategies are commonly used.

[0069] The training process generally includes data preparation, federated initialization, local training, federated aggregation, and iteration.

[0070] During data preparation, a large amount of data was collected using airborne / spaceborne LiDAR or sonar under different water conditions (clear, turbid). The actual underwater topography of the penetration area was obtained as the Ground Truth through precise field measurements or high-precision reference data. Training data with controllable penetration effects was generated using a physically based rendering engine (such as a ray-tracing-based underwater scene simulator).

[0071] During federated initialization, the initial model is deployed locally by each participating party (such as institutions possessing mapping data for different waters). During local training, each participating party trains its model using its local dataset. The loss function is obtained by combining multiple loss terms, specifically including geometric loss, physical consistency loss, and signal fidelity loss.

[0072] Geometric loss uses Chamfer Distance or Earth Mover's Distance (EMD) to measure the geometric difference between the generated point cloud and the real point cloud. Physical consistency loss is used to penalize generation results that violate Snell's law. Signal fidelity loss is used to ensure that the intensity / timestamp of the generated points is consistent with the physical model prediction when there is multiple echo information.

[0073] During federated aggregation, each participant encrypts and uploads its local model's weight updates (gradients) to the central server. The server uses algorithms such as FedAvg to aggregate the updates, generate a new global model, and distribute it to all participants. This iterative optimization continues until the model converges.

[0074] Key parameters include physical loss weights, learning rate, batch size, number of federated communication rounds, and water refraction prior.

[0075] The physics loss weights are hyperparameters that balance geometric accuracy and physical plausibility. The learning rate is typically around 1e. -4 up to 5e -4 The batch size depends on GPU memory, typically 8-32. The number of federated communication rounds is typically 100-500, depending on data distribution and model complexity. Water refraction is used as a priori as a learnable or fixed parameter in the model.

[0076] The water level change filling sub-model is essentially a spatiotemporal prediction model. It is used to repair gaps in the data collected at different times due to the dynamic changes in the water levels of rivers and lakes with factors such as seasons, tides, and rainfall.

[0077] The recommended hierarchy includes a spatiotemporal feature encoder, an external driving factor fusion module, and a conditional generation decoder. The spatiotemporal feature encoder consists of a spatial encoder and a time-series encoder. The spatial encoder processes the static terrain (riverbank, riverbed) around the cavity in the current scan. The time-series encoder uses LSTM, GRU, or Temporal Transformer to process historical water level time-series data (from hydrological stations, historical DEMs, or remote sensing imagery). The external driving factor fusion module fuses external factors that may affect the water level, such as recent rainfall, upstream reservoir discharge, tide tables, and seasonal cycles. These can serve as additional input features for the time-series encoder. The conditional generation decoder generates a water surface geometry that matches the current water level based on the encoded static terrain, the predicted current water level, and the spatial location of the cavity. The output is typically an elevation grid or point cloud filling the region.

[0078] The training process generally includes data preparation, task definition, federated learning setup, local training, and federated aggregation and iteration.

[0079] In the data preparation, a dataset containing the following elements is constructed: multi-period 3D scan data (showing the water surface boundary at different times), accurate water level records for the corresponding time periods (from hydrological stations), and related meteorological and hydrological driving data (rainfall, flow).

[0080] In the task definition, the problem is formalized as follows: given the historical water level and topography from time tn to time t-1, and a partial scan at time t (including voids), predict the complete water surface at time t.

[0081] In a federated learning setup, nodes in different river basins or cities possess their own hydrological and mapping data. The model learns general water level-topography relationships while protecting data privacy.

[0082] During local training, the loss function is primarily the L1 / L2 loss between the predicted and actual water level elevations. Smoothness constraints can be added to ensure the generated water surface is spatially continuous. Federated aggregation and iteration are as described previously and will not be repeated here.

[0083] Key parameters include the time window length, the number of LSTM / Transformer layers and units, the learning rate, and the feature dimensions of external factors. The time window length refers to how many days / months of historical data the model needs for prediction (e.g., the past 30 days).

[0084] The point cloud sparse filling sub-model aims to upsample and reconstruct details from the point cloud. It is used to repair water surface voids caused by extremely low density in certain areas (especially distant or edge areas) of the original point cloud due to sensor performance limitations, flight altitude, or scanning angle, which prevent the formation of a continuous surface.

[0085] The recommended hierarchy includes a local geometric feature extractor, a global context encoder, a hierarchical upsampling decoder, and a detail enhancement module.

[0086] The local geometric feature extractor uses DGCNN (Dynamic Graph CNN) or similar methods to construct a k-nearest neighbor graph around each point in the sparse point cloud, extracting local geometric structures (such as curvature and normal vector variations). The global context encoder uses Point Transformer or Set Abstraction layers (such as the SA layer in PointNet++) to capture the global shape and topological information of the entire point cloud. The hierarchical upsampling decoder employs a progressive strategy. In the first stage, interpolation is performed between sparse points to generate a medium-density point cloud. In the second stage, the medium-density point cloud is further refined to recover high-frequency details (such as small ripples). Common techniques include feature duplication and interpolation, adaptive weighting, and residual learning. The detail enhancement module is a GAN (Generative Adversarial Network) discriminator used to improve the realism of details and the plausibility of local geometry in the generated point cloud.

[0087] The training process generally includes data preparation, supervised learning, federated learning setup, loss function setup, and federated aggregation and iteration.

[0088] Data preparation requires acquiring high-density, high-quality complete water surface point clouds as real data. This is achieved by randomly downsampling the high-density point cloud or simulating sensor viewpoint occlusion, thus generating corresponding sparse input.

[0089] In supervised learning, the model can learn the mapping from sparse inputs to high-density outputs.

[0090] In a federated learning setup, each mapping unit possesses its own (sparse-dense) point cloud pairs. Federated learning allows them to jointly train a more robust upsampling model without sharing the original high-precision data.

[0091] The loss function includes a main loss term, a regularization term, and an adversarial loss. The main loss term uses ChamferDistance (CD) or Earth Mover's Distance (EMD). The regularization term consists of normal vector consistency loss and density uniformity loss. If a GAN is used, the adversarial loss can improve visual quality.

[0092] Key parameters include the upsampling factor, the k-value of the k-nearest neighbors, the weight balance of the CD / EMD loss, and the learning rate. For example, upsampling factors of 4x, 8x, and 16x are used. The k-value of the k-nearest neighbors is used to construct the local graph. The weight balance of the CD / EMD loss is also considered.

[0093] Signal penetration models require fusing data from different sensors (such as lidar and sonar) or learning the relationship between penetration depth and signal attenuation. Water level change models must incorporate time-series information. At their core is a time-series model (such as LSTM, GRU, or Temporal Transformer) that uses historical water level data or scan data from different periods of the same area to predict the currently missing water level. Sparse point cloud models focus more on super-resolution or density completion, drawing on techniques from image super-resolution or point cloud upsampling (such as PU-Net) to recover dense, detailed water surfaces from sparse point clouds. General-purpose filling sub-models do not concern themselves with the causes of holes but rather provide a set of general mathematical or physical tools to fill holes of arbitrary shapes.

[0094] For the interpolation-filling sub-model, the values ​​of the interior points are estimated using mathematical interpolation functions (such as radial basis functions RBF and Kriging) based on known points on the cavity boundary.

[0095] Implementation: The interpolation imputation sub-model is not a machine learning model, but a deterministic algorithm module. Required parameters include the interpolation kernel function type (e.g., Gaussian, Multiquadric), smoothness parameter, and regularization coefficient. Traditional "training" is not required, but the optimal interpolation parameters can be learned within the federated framework. Each node evaluates the effectiveness of different parameter combinations on local data and uploads the optimal parameters or gradients for aggregation.

[0096] The prior filler sub-model assumes the water surface is a smooth, physically conforming curved surface. The entire elevation field is reconstructed from the known boundary gradient field using the Poisson equation.

[0097] The core of this approach is solving partial differential equations. This can be encapsulated as a differentiable layer for integration into deep learning frameworks.

[0098] The necessary modules include a boundary gradient estimator and a Poisson solver. The boundary gradient estimator is used to estimate the normal vector and gradient from the point cloud of the hole boundary. The Poisson solver is a numerical solver (such as the conjugate gradient method) used to solve the discretized Poisson equation. As a differentiable layer, it can be jointly trained with other modules in an end-to-end network to learn how to better estimate boundary gradients.

[0099] For the water level data infilling sub-model, authoritative external hydrogeographic data (such as digital elevation model DEM, historical water level lines, and river flow data) are used as strong priors to guide the infilling process.

[0100] Essential modules include a multi-source data alignment module and a data fusion network. The multi-source data alignment module is used to accurately register user-submitted local 3D models with large-scale DEM or GIS data. The data fusion network is a lightweight network (such as an MLP or a small CNN) that fuses local geometric features with global hydrological prior features to output the final infill result.

[0101] The training process includes data preparation and federated training. Data preparation involves constructing a dataset containing local point cloud holes and corresponding high-precision DEM / GIS data for the area. During federated training, each node uses its local point cloud and accessible public / private hydrological data for training. Since hydrological data is typically public or centralized, this federated aspect may be relatively weak, primarily focusing on privacy protection of the point cloud data. Key parameters include the fusion weights of local features and global priors, and the accuracy threshold for data registration.

[0102] Step S3: Calculate the semantic correlation between the cause description 11 and each sub-model description, select several sub-models 21 as candidate models 22 based on the semantic correlation, and assign corresponding weights to each candidate model 22 based on the semantic correlation.

[0103] Please see the appendix Figure 4 The method for calculating the semantic relevance between the cause description 11 and the descriptions of each sub-model includes:

[0104] Step S311: The cause description 11 and each sub-model description are processed into text vectors to obtain corresponding semantic embedding vectors. The natural language processing module is used to clean, segment, and standardize the user-input cause description 11 (e.g., "point cloud loss on water surface due to strong light reflection") and the structured description of each sub-model 21 (e.g., "applicable to scenarios where optical sensors lose data due to specular reflection under sunny conditions"), and then converts them into high-dimensional semantic embedding vectors through a pre-loaded context-aware encoder.

[0105] Step S312: Using the accessed word embedding model, map the semantic embedding vectors corresponding to the cause description 11 and the sub-model description to a unified semantic vector space. Using the accessed word embedding model (such as Sentence-BERT or SimCSE) ensures that texts from different sources are comparable in the same semantic space, even if the expressions differ. For example, if a user uses the colloquial expression "the water surface reflection was too strong and couldn't be scanned," while the sub-model description uses the technical term "signal attenuation caused by specular reflection," they can still be effectively aligned in the vector space.

[0106] Step S313: Within a unified semantic vector space, calculate the similarity between the semantic embedding vector of the cause description 11 and the semantic embedding vectors of each sub-model, using this similarity as the semantic relevance. Cosine similarity is used as the metric. Taking a lake modeling case as an example, when a user submits the description "severe water surface reflection, blank data during aerial photography," its embedding vector has a similarity of 0.89 with the description vector of the dedicated sub-model 21 "SpecularHoleFiller," only 0.22 with "ObstructionInpainter," and 0.53 with the general interpolation model "RBFWaterFill." This similarity value is directly used as the semantic relevance for subsequent candidate model 22 selection and weight allocation. Through this process, a semantic bridge is established between unstructured user input and structured model scheduling, improving the intelligence level and scene adaptability of the repair strategy selection.

[0107] Please see the appendix Figure 5 The method for assigning corresponding weights to each candidate model 22 based on the semantic relevance includes:

[0108] Step S321: Normalize the semantic relevance of each selected candidate model 22 to obtain normalized relevance values. After obtaining the original semantic similarity (e.g., 0.89, 0.65, 0.53) between the user reason description 11 and each sub-model description, use Softmax or linear normalization to convert them into a probability distribution with a sum of 1. For example, normalize the above three values ​​to 0.48, 0.35, and 0.17, which are used as the initial credibility ratios of each candidate model 22.

[0109] Step S322: Assign the normalized correlation value as the initial weight to the corresponding candidate model 22. In a certain river restoration task, if “SpecularHoleFiller”, “PoissonWaterSurface”, and “TemporalHydroMapper” are selected as candidates, they will participate in subsequent ensemble inference with initial weights of 0.48, 0.35, and 0.17, respectively.

[0110] Step S323: During the knowledge distillation training process, if the contribution of a candidate model 22 to the loss function during distillation is consistently lower than a threshold, its weight is reduced; if the consistency between its output and the high-quality imputation result is higher than a preset threshold, its weight is increased. When performing knowledge distillation under the federated learning framework, the performance of each candidate model 22 is continuously monitored. For example, in multiple rounds of training, "TemporalHydroMapper" lacks local hydrological prior data, and its prediction results deviate significantly from the actual water surface elevation, resulting in its gradient contribution to the total distillation loss being consistently lower than the set threshold. Therefore, the platform gradually reduced its weight from 0.17 to 0.08. Conversely, "SpecularHoleFiller" is dynamically increased to 0.62 because its output is highly consistent with the reference high-quality imputation result in terms of geometric continuity and reflection symmetry.

[0111] Step S324: Continue to participate in the teacher model integration in the weighted federated learning framework with the adjusted weights.

[0112] Weights, initialized semantically and dynamically calibrated by distillation feedback, are used to construct a weighted ensemble teacher model, guiding the learning direction of student model 23 during the federated aggregation phase. Through this mechanism, the platform not only utilizes prior semantics for intelligent initial screening but also continuously optimizes weight allocation based on actual performance during training. This ensures that the repair strategy not only fits the causal context described by the user but also withstands data-driven performance verification, thereby achieving high-precision, adaptive hole filling under distributed and privacy-preserving conditions.

[0113] Step S4: Construct a weighted federated learning framework, using the candidate model 22 as a weighted teacher model, and generate a lightweight student model 23 through knowledge distillation training.

[0114] Please see the appendix Figure 6 The method for constructing a weighted federated learning framework, using the candidate model 22 as a weighted teacher model, and generating a lightweight student model 23 through knowledge distillation training includes:

[0115] Step S41: Load each candidate model 22 as a teacher model in the distributed environment. Deploy the selected candidate sub-models 21 on the local nodes of multiple participants (such as surveying agencies or water conservancy departments in different cities). For example, one node runs "SpecularHoleFiller" and another node runs "PoissonWaterSurface". These models keep the original parameters unchanged and only provide output in inference mode. The original point cloud or sensitive geographic data is always kept locally and is not transmitted across domains.

[0116] Step S42: Construct student model 23. Its input consists of the local 3D geometric features and contextual information of the area to be filled, and its output is the filled water surface point cloud or elevation value. Student model 23 typically employs a lightweight graph neural network or a variant of U-Net, using the coordinates of the hole boundary points, normal vectors, neighborhood density, and a rough water surface trend as input. It ensures sufficient expressive power while maintaining computational efficiency, making it suitable for deployment on mobile or edge devices. Taking a cross-basin modeling task as an example, student model 23 needs to repair water surface holes caused by bridge obstruction in real time on a low-computing-power UAV ground station. Therefore, the structural design emphasizes parameter simplification and inference speed.

[0117] Step S43: Define the distillation loss function, which includes the weighted soft target loss between the output of student model 23 and the outputs of each weighted teacher model, and the supervision loss between the output of student model 23 and the real or pseudo label. The platform uses a composite loss function, where the soft target loss term is weighted and fused with the probability distribution or elevation prediction of each teacher model output according to the dynamically adjusted weights (e.g., 0.62, 0.30, 0.08) in step S3. If high-quality reference data (e.g., historical measured water levels) exists in a local area, an L2 supervision loss is introduced to anchor physical authenticity.

[0118] Step S44: By iteratively optimizing the distillation loss function, during the training process, each teacher model does not share the original training data, but only participates in distillation by outputting intermediate features. In each round of federated training, the teacher model of each node generates prediction results for local void samples, such as the filled elevation field or point cloud confidence, encrypts them, and uploads them to the coordination server. The platform aggregates the weighted soft targets and distributes them to student models 23 for global updates. The entire process follows the principle of privacy protection.

[0119] Step S5: Use the student model 23 to fill the water surface cavity area in the sample 3D model 10 and evaluate the filling result.

[0120] The trained lightweight student model 23 is deployed to the inference environment. For the water surface voids identified in the user-submitted sample 3D model 10, its local geometric context is extracted, including features such as void boundary point clouds, neighborhood curvature, normal consistency, and elevation trends. These features are then input into the student model 23 to generate a filled water surface point cloud or continuous elevation field. Taking a 3D modeling project of an urban river as an example, the original model has a missing water surface area of ​​approximately 15 square meters due to obstruction under a bridge. The student model 23, based on distillation results that integrate knowledge from "ObstructionInpainter" and "PoissonWaterSurface," reconstructs a smooth water surface geometry that naturally connects with the elevations of the upstream and downstream water bodies. After filling, the platform automatically performs multi-dimensional evaluation: on the one hand, it calculates geometric indicators such as the normal deviation, curvature continuity, and point density consistency between the filled surface and its neighborhood; on the other hand, it combines semantic rationality judgment, such as checking whether the filled area conforms to the water flow direction or historical water level constraints. If available reference data exists, such as concurrent remote sensing imagery or measured water levels, quantitative error measures such as RMSE or Hausdorff distance will also be introduced. The evaluation results are not only used to provide feedback to users on the quality of the infill, such as "high confidence in the infill, good geometric continuity".

[0121] Step S6: When the filling result does not meet expectations, adjust the weight; when the filling result meets expectations, provide the calling interface of the student model 23 to the user.

[0122] After the infill assessment is completed, the results are judged to meet the standards based on preset quality thresholds or user feedback. If the infill results have obvious geometric distortions, abrupt elevation changes, or discontinuities with surrounding water bodies, as in a lake restoration case where student model 23 over-relyed on general interpolation logic and ignored local seasonal water level decline characteristics, resulting in the filled water surface being higher than the actual shoreline, the weight allocation will be backtracked, reducing the weight of teacher models with low previous contributions, such as "PoissonWaterSurface," and increasing the influence of sub-model 21, which is more suitable for the scenario, such as "TemporalHydroMapper." This triggers a new round of knowledge distillation and fine-tuning of student model 23 until the infill quality meets the requirements. Conversely, if the evaluation indicators show that the infill results are geometrically smooth, semantically reasonable, and highly consistent with the context, the infill results will be deemed satisfactory. For example, in a river project, student model 23 accurately reproduced the water surface morphology of the area obstructed by the bridge, with a normal deviation of less than 2 degrees and an elevation error within ±5 centimeters. The platform then encapsulated student model 23 as a standardized API service, generating dedicated calling interfaces, such as RESTful endpoints or draggable components in a low-code platform, for users to reuse directly in subsequent similar tasks. This ensured the reliability of each repair and also enabled the accumulation and sharing of high-quality model assets, strengthening the continuous learning and value transformation capabilities under the federated learning framework.

[0123] On the other hand, this specification provides a 3D modeling water surface cavity filling system based on federated learning; please refer to the appendix. Figure 7 ,include:

[0124] The receiving module 100 receives a sample 3D model 10 of the water surface cavity to be filled submitted by the user, and obtains a description 11 of the cause of the water surface cavity corresponding to the sample 3D model 10.

[0125] The preload module 200 reads multiple pre-trained sub-models 21 and their descriptions.

[0126] The calculation module 300 calculates the semantic correlation between the cause description 11 and the description of each sub-model, selects several sub-models 21 as candidate models 22 based on the semantic correlation, and assigns corresponding weights to each candidate model 22 based on the semantic correlation.

[0127] Training module 400 constructs a weighted federated learning framework, using the candidate model 22 as a weighted teacher model, and generates a lightweight student model 23 through knowledge distillation training;

[0128] The filling module 500 uses the student model 23 to fill the water surface cavity area in the sample 3D model 10 and evaluates the filling result;

[0129] The interface module 600 adjusts the weights when the filling result does not meet expectations, and provides the calling interface of the student model 23 to the user when the filling result meets expectations.

[0130] Please see Figure 8 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this specification.

[0131] like Figure 8As shown, the electronic device 1100 may include: at least one processor 1101, at least one network interface 1104, a user interface 1103, a memory 1105, and at least one communication bus 1102. The communication bus 1102 can be used to connect and communicate with the various components mentioned above. The user interface 1103 may include buttons, and optionally may include standard wired or wireless interfaces. The network interface 1104 may include, but is not limited to, a Bluetooth module, an NFC module, or a Wi-Fi module. The processor 1101 may include one or more processing cores. The processor 1101 connects to various parts within the electronic device 1100 using various interfaces and lines, and performs various functions of the routing device and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1105, and by calling data stored in the memory 1105. Optionally, the processor 1101 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 1101 may integrate one or more combinations of CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content that the display screen needs to show; and the modem is used for wireless communication.

[0132] It is understandable that the aforementioned modem may not be integrated into the processor 1101, but may be implemented using a separate chip.

[0133] The memory 1105 may include RAM or ROM. Optionally, the memory 1105 may include a non-transitory computer-readable medium. The memory 1105 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1105 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1105 may also be at least one storage device located remotely from the aforementioned processor 1101. As a computer storage medium, the memory 1105 may include an operating system, a network communication module, a user interface module, and application programs. The processor 1101 may be used to call the application programs stored in the memory 1105 and execute the methods in the above-described embodiments.

[0134] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform multiple steps as described in the above embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0135] This specification also provides a computer program product, including a computer program that, when executed by a processor, implements the multiple steps described in the above embodiments.

[0136] Where there is no conflict, the technical features in this embodiment and implementation scheme can be combined arbitrarily.

[0137] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes multiple computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating multiple available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0138] When implemented through hardware or firmware, the aforementioned method flow is programmed into the hardware circuit to obtain the corresponding hardware circuit structure and achieve the corresponding function. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit, whose logic function is determined by the user programming the device. Designers can program a digital system onto a PLD themselves, eliminating the need for chip manufacturers to design and fabricate dedicated integrated circuit chips. Furthermore, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, similar to the software compiler used in program development. The original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There is not just one HDL, but many. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of the aforementioned hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logic method flow can be easily obtained.

[0139] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. A method for filling voids in 3D water surface modeling based on federated learning, characterized in that, Including the following steps: Receive sample 3D models of water surface cavities submitted by users and obtain descriptions of the causes of the water surface cavities corresponding to the sample 3D models; Read multiple pre-trained sub-models and their descriptions; Calculate the semantic correlation between the cause description and the description of each sub-model, select several sub-models as candidate models based on the semantic correlation, and assign corresponding weights to each candidate model based on the semantic correlation. A weighted federated learning framework is constructed, in which the candidate model is used as a weighted teacher model, and a lightweight student model is generated through knowledge distillation. The student model was used to fill in the voids in the water surface of the example 3D model, and the filling results were evaluated. When the filling result does not meet expectations, the weight is adjusted; when the filling result meets expectations, the calling interface of the student model is provided to the user.

2. The method for filling water surface voids based on federated learning in 3D modeling according to claim 1, characterized in that, Methods for pre-training multiple sub-models and generating sub-model descriptions include: Two sets of sub-models are pre-trained. The first set of sub-models are dedicated filling sub-models corresponding to multiple void formation mechanisms. Sub-model descriptions of the first set of sub-models are generated according to the corresponding void formation mechanisms. The second set of sub-models is a general filling sub-model based on multiple filling logics. The sub-model description of the second set of sub-models is generated according to the corresponding filling logic.

3. The method for filling water surface voids based on federated learning in 3D modeling according to claim 2, characterized in that, The first set of sub-models includes the occlusion filling sub-model, specular reflection filling sub-model, signal penetration filling sub-model, water level change filling sub-model, and point cloud sparse filling sub-model. The second set of sub-models includes the prior filling sub-model, interpolation filling sub-model, and water level data filling sub-model.

4. The method for filling water surface voids based on federated learning in 3D modeling according to claim 1, characterized in that, The method for calculating the semantic relevance between the cause description and the descriptions of each sub-model includes: The causal description and each sub-model description are respectively processed into text vectors to obtain the corresponding semantic embedding vectors; Using the accessed word embedding model, the semantic embedding vectors corresponding to the cause description and the sub-model description are mapped to a unified semantic vector space; Within a unified semantic vector space, the similarity between the semantic embedding vector describing the cause and the semantic embedding vectors describing each sub-model is calculated, and this similarity is used as the semantic relevance.

5. The method for filling water surface voids based on federated learning in 3D modeling according to claim 1, characterized in that, The method for assigning corresponding weights to each candidate model based on the semantic relevance includes: The semantic relevance of each selected candidate model is normalized to obtain the normalized relevance value. The normalized correlation value is used as the initial weight and assigned to the corresponding candidate model; During the knowledge distillation training process, if a candidate model's contribution to the loss function is consistently lower than a threshold during distillation, its weight is reduced; if the consistency between its output and the high-quality imputation result is higher than a preset threshold, its weight is increased. They will continue to participate in teacher model integration within the weighted federated learning framework with adjusted weights.

6. The method for filling water surface voids based on federated learning in 3D modeling according to claim 1, characterized in that, The method for constructing a weighted federated learning framework, using the candidate model as a weighted teacher model, and training a lightweight student model through knowledge distillation includes: Load each candidate model as a teacher model in a distributed environment; Construct a student model, whose input is the local three-dimensional geometric features and contextual information of the region to be filled, and whose output is the point cloud or elevation value of the filled water surface; Define a distillation loss function, which includes the weighted soft objective loss between the student model output and the outputs of each weighted teacher model, and the supervision loss between the student model output and the real or pseudo label; By iteratively optimizing the distillation loss function, during the training process, each teacher model does not share the original training data, but only participates in distillation by outputting intermediate features.

7. A 3D modeling water surface cavity filling system based on federated learning, characterized in that, include: The receiving module receives a sample 3D model of a water surface cavity to be filled submitted by the user, and obtains a description of the cause of the water surface cavity corresponding to the sample 3D model. The pre-loading module reads multiple pre-trained sub-models and their descriptions. The calculation module calculates the semantic correlation between the cause description and the description of each sub-model, selects several sub-models as candidate models based on the semantic correlation, and assigns corresponding weights to each candidate model based on the semantic correlation. The training module constructs a weighted federated learning framework, using the candidate model as a weighted teacher model, and generates a lightweight student model through knowledge distillation. The filling module uses the student model to fill the water surface voids in the sample 3D model and evaluates the filling results. The interface module adjusts the weights when the filling result does not meet expectations, and provides the calling interface of the student model to the user when the filling result meets expectations.

8. An electronic device, characterized in that, Including the processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images