A Real-Scene 3D Evolutionary Update Method Based on Dynamic Neural Radiation Field
By constructing a four-dimensional implicit field and applying boundary continuity constraints through a real-scene 3D evolutionary update method based on dynamic neural radiation fields, the problems of error accumulation and high cost in real-scene 3D updates are solved, achieving seamless continuous updates and high-precision change detection.
Patent Information
- Application Number
- CN202611431042.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-15
- Publication Date
- 2026-11-17
AI Technical Summary
Existing real-scene 3D update technologies suffer from problems such as error accumulation, difficulty in eliminating splicing seams, insufficient autonomous evolution capabilities, high computing power costs, and insufficient update robustness, making it difficult to support the routine monthly update needs of city-level scenes.
A real-world 3D evolutionary update method based on dynamic neural radiation field is adopted. By constructing a four-dimensional implicit field, decoupling the temporal state of components, applying boundary continuity constraints, and combining a generative replay mechanism, targeted parameter optimization is achieved, forming an evolutionary closed loop.
It achieves seamless continuous evolution and updates, improves change detection accuracy, reduces update computing costs, has autonomous evolution capabilities, and solves the problems of splicing seams and error accumulation.
Smart Images

Figure CN123118944A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of 3D reality modeling and digital twin model updating technology, and in particular to a real-scene 3D evolutionary updating method based on dynamic neural radiation fields. Background Technology
[0002] Real-world 3D models are the core spatial foundation for building digital twin cities and a real-world 3D China, and the model's current availability directly determines its application value. Traditional real-world 3D updates employ a sequential patching approach: "two-phase data registration - change area detection - local reconstruction - stitching and embedding," which has significant drawbacks in engineering practice. (1) Errors in each link of the update link accumulate downstream, which can easily lead to problems such as geometric discontinuity, texture misalignment, and semantic disconnection. The splicing seams are difficult to eliminate, and the model is always in a cycle of "static construction-local repair" and does not have the ability to evolve autonomously. (2) Although implicit modeling techniques such as neural radiation fields can eliminate splicing seams from the root of expression, existing dynamic neural radiation fields are mostly anchored to a single offline modeling paradigm. When the scene changes such as the addition of new land features or the demolition and reconstruction of buildings, a full retraining is required, which is costly and cannot support the normal monthly update needs of city-level scenes. (3) Existing incremental neural radiation field research only focuses on memory optimization at the parameter level, without incorporating the semantic attributes and geoscientific evolution laws of urban scenes. It is difficult to distinguish between real land feature changes and pseudo-interferences such as vegetation seasonal changes, light fluctuations, and temporary shading, resulting in insufficient update robustness.
[0003] To address these issues, a real-world 3D evolutionary update method based on dynamic neural radiation fields is proposed. Summary of the Invention
[0004] The purpose of this invention is to solve the problems in the prior art by proposing a real-scene three-dimensional evolutionary update method based on dynamic neural radiation field.
[0005] A real-scene 3D evolutionary update method based on dynamic neural radiation fields includes the following steps: S1. Input image, extract and stitch into fusion features, and construct queue meta-prior knowledge base; S2. Based on the queue meta-prior knowledge base, the system is integrated with the state space model (SSM) for spatiotemporal joint modeling, and outputs a four-dimensional implicit field. S3. Construct a three-layer PINN constraint loss and perform regularization optimization on the four-dimensional implicit field; S4. Decouple the temporal state of the components, retain the trend term, and construct the normal evolution baseline of the scenario by combining the state space model SSM. S5. Perform multi-dimensional residual calculations on the extrapolation results of the evolutionary baseline and the actual observations to determine and generate change masks; S6. Based on the change mask, divide the three-level update influence domain, apply boundary continuity constraints, and combine the generative replay mechanism to complete the targeted parameter optimization and update the scene state library to form an evolutionary closed loop.
[0006] Preferably, in step S1, semantic tokens, visual embeddings, and deep geometric features are extracted from the basic model group respectively, and then spliced together to form a fused feature; The basic model set includes the SAM2 semantic segmentation model, the DINOv2 visual encoding network, and the Depth Anything depth estimation network; The queue-based prior knowledge base organizes the fusion features and semantic tags of object instances from various regions into a reusable knowledge queue.
[0007] Preferably, in step S2, the fusion of the queue element prior knowledge base adopts token-based hierarchical modulation fusion. The token-based hierarchical modulation fusion maps the queue element prior to feature scaling coefficients and offset parameters through a multilayer perceptron, and performs element-level modulation on the basic fusion features. The state space model SSM receives the modulation features, spatial coordinates and timestamps, and iteratively updates the hidden scene state through the state transition equation.
[0008] Preferably, in step S3, a three-layer PINN physical constraint loss is constructed, which includes geometric consistency, semantic stability, and functional attributes.
[0009] Preferably, in step S4, the temporal state is decoupled by multivariate empirical mode decomposition (MEMD), and divided into noise component, periodic fluctuation component, and long-term trend component according to the period threshold. High-frequency noise and mid-frequency periodic disturbance are removed, low-frequency long-term trend term is retained, and a normal evolution baseline of the scenario is constructed by combining it with the state space model (SSM).
[0010] Preferably, in step S5, the multi-dimensional residual is a weighted sum of the differences in the geometric, color, and semantic dimensions; the kernel density estimation is used to calculate the residual distribution of the entire scene, an adaptive change threshold is set, and the area exceeding the threshold is marked as a real change, thereby identifying and generating a three-dimensional binary change mask.
[0011] Preferably, in step S6, the update influence domain is divided into three levels based on the change mask: core change region, transition buffer, and static region. Full parameter update, lightweight fine-tuning, and parameter freezing are performed respectively, and zero-order value continuity and first-order gradient continuity constraints are applied to the transition buffer. The generative replay mechanism renders pseudo-labels for static areas using historical network snapshots, collaboratively optimizes and suppresses catastrophic forgetting, and synchronizes the scene state to the global state library after the update is completed, thereby completing targeted parameter optimization and providing historical state initialization for the next round of scene evolution.
[0012] Preferably, the zeroth-order value continuity constraint is:
[0013] in, This represents the total number of paired sampling points in the buffer. Points inside the buffer zone Points outside the buffer zone The SSM outputs the implicit state feature vector corresponding to the spatial location. The square of the vector's 2-norm. The implicit feature values are constrained to be continuous.
[0014] Preferably, the first-order gradient continuity constraint is:
[0015] in, The gradient of the implicit state. Constrain the continuous rate of change of features to eliminate texture jumps.
[0016] Compared with existing technologies, the advantages of this invention are: 1. This invention constructs a four-dimensional implicit representation system integrating semantics, geometry, appearance, and attributes. By transferring cross-scene ground object knowledge through queue priors and combining it with PINN geoscientific rule constraints, it solves the problems of building facade distortion and semantic loss under sparse aerial photography perspectives, and realizes the leap from visual replication to semantic parsing of scenes.
[0017] 2. This invention proposes a temporally decoupled evolutionary baseline change detection paradigm, which removes pseudo-interferences such as periodic vegetation and light fluctuations, and completes change identification and 3D localization in implicit space. No explicit point cloud registration is required throughout the process, avoiding the accumulation of registration errors and significantly improving the change detection accuracy in complex urban scenes.
[0018] 3. This invention designs a three-level influence domain directional incremental update mechanism, which optimizes network parameters only for the changed regions. Combined with high-order boundary smoothing constraints and generative replay, it significantly reduces the update computing cost while avoiding catastrophic forgetting, and achieves seamless continuous evolution update. Attached Figure Description
[0019] Figure 1 This is a schematic diagram of the principle of the present invention.
[0020] Figure 2 This is a flowchart illustrating the overall workflow of the present invention.
[0021] Figure 3 This is a flowchart of the pre-training process of the queue prior cMeta-INR in this invention.
[0022] Figure 4This is a diagram of the token modulation structure in this invention.
[0023] Figure 5 This is a schematic diagram of the PINN geoscience three-layer constraint in this invention.
[0024] Figure 6 This is a flowchart of the multi-temporal evolution baseline and change detection process in this invention.
[0025] Figure 7 This is a flowchart of the partition-oriented incremental update process of the present invention. Detailed Implementation
[0026] To facilitate understanding of this application and to make the above-mentioned objectives, features and advantages of this application more apparent, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0027] Reference Figure 1-7 As shown, a real-scene 3D evolutionary update method based on dynamic neural radiation fields includes the following steps: S1. Multimodal feature extraction and queue meta-prior construction: Input multi-temporal UAV oblique images, extract multimodal complementary features through basic model groups, and construct a cross-scene ground feature queue meta-prior knowledge base; The basic model group includes the SAM2 semantic segmentation model, the DINOv2 visual encoding network, and the Depth Anything depth estimation network, which extract semantic tokens, visual embeddings, and deep geometric features respectively, and then concatenate them to form fused features. The queue meta-prior knowledge base organizes the fused features and semantic labels of object instances from various places into a reusable knowledge queue, which is used to initialize the downstream implicit representation network.
[0028] S2, Construction of a four-dimensional scene collaborative representation: Based on queue meta-prior, token hierarchical modulation and fusion are performed, and spatiotemporal joint modeling is carried out in combination with the state space model SSM to output a four-dimensional implicit representation integrating geometric, appearance, semantic and functional attributes. Token-based hierarchical modulation fusion uses a multilayer perceptron to map queue priors to feature scaling factors and offset parameters, performing element-wise modulation on the basic fusion features. The State Space Model (SSM) receives the modulation features, spatial coordinates, and timestamps, and iteratively updates the hidden scene state through state transition equations.
[0029] in, for t The scene state is hidden at all times. This indicates the state of the scene at the previous moment. Represents the state transition matrix. Represents the input mapping matrix. Represents the state output matrix. This represents the input residual mapping matrix. The feature dimension concatenation operator merges multiple vectors end-to-end into a single long vector. This represents the scene features after fusion. This represents the spatiotemporal fusion features of the final output; the final output head synchronously outputs four types of results: volume density, RGB color, semantic features, and functional attributes.
[0030] S3, Geoscience Regularization Constraints: Construct a three-layer PINN physical constraint loss of geometric consistency, semantic stability, and functional attributes to optimize the regularization of the four-dimensional implicit field; The specific loss of the three-layer PINN constraint is as follows: (Geometric consistency constraint) constrains the geometric smoothness of building facades, ground surfaces, and other structures through volume density spatial gradient constraints; (Semantic stability constraint) suppresses unreasonable class jumps in temporal evolution through cross-entropy loss; (Functional attribute constraints) Differentiated change weights are set for different land features to constrain the temporal continuity of functional characteristics; The total constraint loss is a weighted sum of the three losses, and the residuals are calculated automatically using differentiation, requiring no additional annotation.
[0031] S4. Temporal decoupling and evolutionary baseline construction: The temporal state is decoupled by multivariate empirical mode decomposition (MEMD), the long-term trend term is extracted, and the normal evolutionary baseline of the scenario is constructed by combining it with SSM. MEMD decomposes the temporal state sequence of spatial points into multi-order intrinsic mode functions and residual trend terms; it further divides the data into noise components, periodic fluctuation components, and long-term trend components according to a period threshold, removes high-frequency noise and mid-frequency periodic disturbances, and retains only the low-frequency long-term trend term as input to the SSM, iteratively constructing a normal evolution baseline for the scene:
[0032] in, The initial state for meta-prior initialization. for Real-time extracted ground feature trend characteristics Forward mapping of state space, for The sequence of normal evolutionary states of the scene obtained through continuous learning. These are timestamps for two historical periods of video footage.
[0033] S5. Multi-dimensional residual change detection and localization: Multi-dimensional residual calculation is performed by extrapolating the evolutionary baseline results and the actual observations, and a three-dimensional change mask is generated by adaptive threshold discrimination; The multidimensional residuals are the weighted sum of the differences in the geometric, color, and semantic dimensions; kernel density estimation is used to calculate the residual distribution across the entire scene, and the 95th percentile is taken as the adaptive change threshold.
[0034] in, To adaptively change the threshold, For the infimum operator, choose the smallest one that satisfies the condition. Assign to , For residual kernel density distribution, This is an integral dummy variable, and the integration condition represents the upper bound of the 95% normal fluctuation range of the residual. Regions exceeding the threshold are marked as true changes, generating a three-dimensional binary change mask.
[0035] S6. Targeted Incremental Update and State Iteration: Based on the change mask, a three-level update influence domain is divided, boundary continuity constraints are applied, and a generative replay mechanism is combined to complete targeted parameter optimization and update the scene state library to form an evolutionary closed loop.
[0036] In step S6, the three-level update influence domain includes the core change region, the transition buffer, and the static region. Full parameter update, lightweight fine-tuning, and parameter freezing are performed respectively. Zero-order value continuity and first-order gradient continuity constraints are applied to the transition buffer, corresponding to the finite element C0 / C1 continuity conditions. Zero-order value continuity:
[0037] in, This represents the total number of paired sampling points in the buffer. Points inside the buffer zone Points outside the buffer zone The SSM outputs the implicit state feature vector corresponding to the spatial location. The square of the vector's 2-norm. The implicit feature values are constrained to be continuous.
[0038] First-order gradient continuity:
[0039] in, The gradient of the implicit state. Constrain the continuous rate of change of features to eliminate texture jumps.
[0040] The generative replay mechanism renders pseudo-labels for static areas using historical network snapshots, collaboratively optimizes and suppresses catastrophic forgetting, and synchronizes the scene state to the global state library after the update to provide historical state initialization for the next round of scene evolution.
[0041] in, It serves as a global temporal scene state memory library. The latest target phase to be executed for incremental update for The scene state vector after the update is complete.
[0042] Example S1. Multimodal Feature Extraction and Queue Meta-Prior Construction Given a fixed input multi-temporal tilted image sequence, for the input time t tilted image of time Complementary features are extracted in parallel using three basic models: semantic token features are extracted using the SAM2 model. Visual embedding features are extracted using the DINOv2 network. Deep geometric features are extracted using the Depth Anything network. The three types of features are concatenated to obtain the basic fusion feature, such as... Figure 3 As shown:
[0043] in, This represents the tilted image acquired at time t. and These represent the image space dimensions, Indicates the fusion features of the base model. This indicates a feature splicing operation.
[0044] For the first in the scenario For each ground object, its basic fusion features and semantic labels are combined into a meta-prior vector. ,like Figure 4 As shown:
[0045] in, Indicates the first The basic model features corresponding to each object This represents the semantic category information corresponding to the object. All land feature instances together form the queue meta-prior knowledge base, which is used for the initialization of the downstream implicit representation network.
[0046] S2, Construction of Four-Dimensional Scene Collaborative Representation The prior information of the queue elements is mapped using a multilayer perceptron to obtain feature scaling coefficients and offset parameters. Element-level modulation is then employed to fuse the prior and observed features. The fused features, 3D spatial coordinates, and timestamps are input into the state-space model (SSM) for spatiotemporal joint state iteration.
[0047] in, for t The scene state is hidden at all times. This indicates the state of the scene at the previous moment. Represents the state transition matrix. Represents the input mapping matrix. Represents the state output matrix. This represents the input residual mapping matrix. The feature dimension concatenation operator merges multiple vectors end-to-end into a single long vector. This represents the scene features after fusion. This represents the spatiotemporal fusion features of the final output; the final output head synchronously outputs four types of results: volume density, RGB color, semantic features, and functional attributes.
[0048] S3, Geoscience Regular Constraints A three-layer PINN geoscientific constraint loss is constructed, and the residuals are obtained entirely through automatic differentiation by the network, such as... Figure 5 As shown.
[0049] Geometric consistency constraints are used to constrain the spatial gradient of volume density and suppress geometric distortions of building facades.
[0050] in, Represents geometric constraint loss. Indicates spatial location Bulk density, This represents the spatial gradient operator.
[0051] Semantic stability constraints are implemented using cross-entropy to suppress unreasonable temporal semantic category jumps.
[0052] in, For semantic evolution regularization loss, This is the semantic category index number of the land feature. Represents the true semantic category label, This indicates the semantic probability predicted by the model.
[0053] Functional attribute constraints assign differentiated weights to different land features, constraining the temporal continuity of functional characteristics:
[0054] in, This represents the functional attribute constraint loss. Indicates the first Weighting of changes in individual land features Indicates the functional characteristics at the current moment. It indicates the functional characteristics of historical moments.
[0055] The total PINN constraint loss is expressed as a weighted sum:
[0056] in, These represent the weights of different constraint terms.
[0057] S4. Temporal decoupling and evolutionary baseline construction For spatial points Interpolation generates equally spaced time-series state sequences. Through MEMD decomposition, the original time series state is broken down into a series of narrowband oscillatory components and a monotonic trend term. The decomposition results satisfy:
[0058] in, No. k Interpolated, equally spaced continuous timestamps Represents the total length of the interpolation time series. For the first i The first-order intrinsic mode function corresponds to the wave components of different frequencies. This represents the residual trend term.
[0059] The decomposed modes are divided into noise components, periodic fluctuation components, and long-term trend components according to the modal average period. High-frequency noise components are directly removed, mid-frequency periodic fluctuations are judged as normal scene disturbances, and only low-frequency long-term trend terms are retained.
[0060] Using the scenario state obtained from the prior initialization of the queue element as the initial baseline, input , The trend features of the two periods are iteratively updated through a state-space model to construct a baseline for the normal evolution of the scenario, such as... Figure 6 As shown:
[0061] in, The initial state for meta-prior initialization. for t Real-time extracted ground feature trend characteristics Forward mapping of state space, for t The sequence of normal evolutionary states of the scene obtained through continuous learning. These are timestamps for two historical periods of video footage.
[0062] S5. Multidimensional Residual Change Detection and Localization The evolutionary baseline is extrapolated to the detection time to obtain the network's predicted state. The weighted residuals of this predicted state are then calculated point-by-point between the predicted state and the actual observed state, considering geometric, color, and semantic dimensions. Kernel density estimation is used to calculate the residual distribution across the entire scene, and the 95th percentile is taken as the adaptive change threshold. :
[0063] in, To adaptively change the threshold, For the infimum operator, choose the smallest one that satisfies the condition. Assign to , For residual kernel density distribution, This is an integral dummy variable, and the integration condition represents the upper bound of the 95% normal fluctuation range of the residual. Regions exceeding the threshold are marked as true changes, generating a three-dimensional binary change mask:
[0064] in, represent x There are real changes in the terrain. This indicates no substantial change.
[0065] S6. Targeted Incremental Updates and State Iteration The scene is discretized using grid points, and a three-level update influence domain is defined by combining a change mask and a spatial distance field: a core change region, a transition buffer, and a static region. Full parameter updates, lightweight fine-tuning, and parameter freezing are performed on these regions respectively. Zero-order value continuity and first-order gradient continuity constraints are applied to the transition buffer, corresponding to the finite element C0 / C1 continuity conditions. Zero-order value continuity:
[0066] in, This represents the total number of paired sampling points in the buffer. Points inside the buffer zone Points outside the buffer zone The SSM outputs the implicit state feature vector corresponding to the spatial location. The square of the vector's 2-norm. The implicit feature values are constrained to be continuous.
[0067] First-order gradient continuity:
[0068] in, The gradient of the implicit state. Constrain the continuous rate of change of features to eliminate texture jumps.
[0069] Generative replay mechanisms render pseudo-labels for static areas using historical network snapshots, collaboratively optimizing and suppressing catastrophic forgetting. After updates, scene states are synchronized to the global state library to provide historical state initialization for the next round of scene evolution, such as... Figure 7 As shown:
[0070] in, It serves as a global temporal scene state memory library. The latest target phase to be executed for incremental update for The scene state vector after the update is complete.
[0071] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative in all respects and are not the only ones. All modifications within the scope of this invention or its equivalents are included in this invention.
Claims
1. A real-scene 3D evolutionary update method based on dynamic neural radiation fields, characterized in that, Includes the following steps: S1. Input image, extract and stitch into fusion features, and construct queue meta-prior knowledge base; S2. Based on the queue meta-prior knowledge base, the system is integrated with the state space model (SSM) for spatiotemporal joint modeling, and outputs a four-dimensional implicit field. S3. Construct a three-layer PINN constraint loss and perform regularization optimization on the four-dimensional implicit field; S4. Decouple the temporal state of the components, retain the trend term, and construct the normal evolution baseline of the scenario by combining the state space model SSM. S5. Perform multi-dimensional residual calculations on the extrapolation results of the evolutionary baseline and the actual observations to determine and generate change masks; S6. Based on the change mask, divide the three-level update influence domain, apply boundary continuity constraints, and combine the generative replay mechanism to complete the target parameter optimization and update the scene state library to form an evolutionary closed loop.
2. The real-scene three-dimensional evolutionary update method based on dynamic neural radiation field according to claim 1, characterized in that: In step S1, semantic tokens, visual embeddings, and deep geometric features are extracted from the basic model group and then spliced together to form fused features. The basic model set includes the SAM2 semantic segmentation model, the DINOv2 visual encoding network, and the Depth Anything depth estimation network; The queue-based prior knowledge base organizes the fusion features and semantic tags of object instances from various regions into a reusable knowledge queue.
3. The real-scene three-dimensional evolutionary update method based on dynamic neural radiation field according to claim 1, characterized in that: In step S2, the queue prior knowledge base is fused using Token hierarchical modulation fusion. Token hierarchical modulation fusion maps the queue prior knowledge to feature scaling coefficients and offset parameters through a multilayer perceptron, and modulates the basic fused features at the element level. The state space model SSM receives the modulation features, spatial coordinates and timestamps, and iteratively updates the hidden scene state through the state transition equation.
4. The real-scene three-dimensional evolutionary update method based on dynamic neural radiation field according to claim 1, characterized in that: In step S3, a three-layer PINN physical constraint loss is constructed, which includes geometric consistency, semantic stability, and functional attributes.
5. The real-scene three-dimensional evolutionary update method based on dynamic neural radiation field according to claim 1, characterized in that: In step S4, the temporal state is decoupled by multivariate empirical mode decomposition (MEMD), and divided into noise component, periodic fluctuation component, and long-term trend component according to the period threshold. High-frequency noise and mid-frequency periodic disturbance are removed, low-frequency long-term trend term is retained, and a normal evolution baseline of the scenario is constructed by combining it with the state space model (SSM).
6. The real-scene three-dimensional evolutionary update method based on dynamic neural radiation field according to claim 1, characterized in that: In step S5, the multi-dimensional residual is the weighted sum of the differences in the geometric, color, and semantic dimensions; the kernel density estimation is used to calculate the residual distribution of the entire scene, an adaptive change threshold is set, and the area exceeding the threshold is marked as a real change, thereby generating a three-dimensional binary change mask.
7. The real-scene three-dimensional evolutionary update method based on dynamic neural radiation field according to claim 1, characterized in that: In step S6, the update influence domain is divided into three levels based on the change mask: core change region, transition buffer, and static region. Full parameter update, lightweight fine-tuning, and parameter freezing are performed respectively, and zero-order value continuity and first-order gradient continuity constraints are applied to the transition buffer. The generative replay mechanism renders pseudo-labels for static areas using historical network snapshots, collaboratively optimizes and suppresses catastrophic forgetting, and synchronizes the scene state to the global state library after the update is completed, thereby completing targeted parameter optimization and providing historical state initialization for the next round of scene evolution.
8. The real-scene three-dimensional evolutionary update method based on dynamic neural radiation field according to claim 7, characterized in that: The zeroth-order value continuity constraint is: in, This represents the total number of paired sampling points in the buffer. Points inside the buffer zone Points outside the buffer zone The SSM outputs the implicit state feature vector corresponding to the spatial location. The square of the vector's 2-norm. The implicit feature values are constrained to be continuous.
9. The real-scene three-dimensional evolutionary update method based on dynamic neural radiation field according to claim 8, characterized in that: The first-order gradient continuity constraint is: in, The gradient of the implicit state. Constrain the continuous rate of change of features to eliminate texture jumps.
