Multi-level updating method and device for digital twin scene based on explicit-implicit fusion model

Abstract

The application belongs to the technical field of digital twin three-dimensional scene updating, and discloses a digital twin scene multi-level updating method and equipment based on an explicit-implicit fusion model, which comprises the following steps: based on the pre-acquisition data and real-time acquisition data of the scene, the explicit scene model is subjected to structured processing, and the explicit scene model is updated in real time according to the motion state of the physical equipment in the scene; at the same time, the low dynamic scene is subjected to implicit scene model construction using a neural radiation field, the scene motion is subjected to segmentation and time consistency constraint, and the implicit scene model is updated using an IBR-based neural network and a motion-adjusted feature aggregation mode. The application divides the scene updating into main parts such as explicit scene model updating, implicit scene model updating and explicit-implicit scene model mutual conversion, and then sets the updating frequency in a targeted manner, so that the whole can have high fidelity and high reliability while also having good running efficiency.

Description

Technical Field

[0001] This invention belongs to the technical field of digital twin 3D scene update, and more specifically, relates to a method and device for multi-level update of digital twin scenes based on an explicit-implicit fusion model. Background Technology

[0002] After multiple iterations, the five-dimensional digital twin model is currently one of the most mature theoretical frameworks for digital twins, specifically including physical entities, virtual scenes, services, twin data, and the connections and interactions between them. Among these, the virtual scene is the most direct means of reflecting physical entities, and constructing the virtual scene is the first and one of the most crucial steps in building a digital twin system. The performance of scene updates directly affects the dynamic stability of the digital twin system and its judgment and decision-making regarding physical entities. As digital twin technology expands into fields such as intelligent manufacturing, intelligent construction, intelligent maintenance, and product lifecycle management, these fields are demanding high fidelity and real-time performance in digital twin scene construction and updating technologies. Accurate and efficient technology for updating virtual twin scenes in response to dynamic changes in physical scenes has become one of the key factors restricting the development of digital twin systems.

[0003] The hierarchy, rules, and complexity of the digital twin scenario update mechanism directly affect the performance and reliability of the digital twin system. Currently, research on digital twin scenario update schemes is limited, and these schemes are highly dependent on the scenario construction methods. Existing research lacks effective scenario update strategies to ensure dynamic stability of the scenario. Summary of the Invention

[0004] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a multi-level update method and device for digital twin scenes based on an explicit-implicit fusion model. This method divides scene updates into main parts such as explicit scene model updates, implicit scene model updates, and mutual conversion between explicit and implicit scene models, thereby setting the update frequency in a targeted manner, so that the overall system can achieve both high fidelity and high reliability while also maintaining good operating efficiency.

[0005] To achieve the above objectives, according to one aspect of the present invention, a multi-level update method for digital twin scenes based on an explicit-implicit fusion model is provided, the method comprising the following steps:

[0006] The explicit scene model is structured based on pre-collected data and real-time collected data, and the explicit scene model is updated in real time according to the motion state of the physical devices in the scene.

[0007] Meanwhile, a neural radiation field is used to construct an implicit scene model for low-dynamic scenes, while scene motion is segmented and time consistency is constrained. The implicit scene model is updated using an IBR-based neural network and a motion-adjusted feature aggregation method.

[0008] Furthermore, while updating the explicit scene model, the motion state of the explicit scene model is also recorded to form an explicit scene model action database.

[0009] Furthermore, the explicit scene model action database is traversed to determine the low-dynamic explicit scene model, and the low-dynamic explicit scene model is converted into an implicit scene model.

[0010] Furthermore, a period T is defined based on the equipment performance. Within each period, the explicit model action database is traversed and searched. A frequency threshold υ and a motion amplitude threshold ω are set to filter low-dynamic explicit objects. The distinction between low-dynamic and high-dynamic objects is as follows: First, it is determined whether the object has its own kinematic pairs; objects that generate kinematic pairs are high-dynamic objects. Second, the motion frequency of the object is determined; objects with a motion frequency greater than the frequency threshold υ are high-dynamic objects. Third, the single motion amplitude of the object is determined; objects with a single motion amplitude greater than the motion amplitude threshold ω are high-dynamic objects. Objects that do not meet any of the above three conditions are low-dynamic objects, and the rest are high-dynamic objects. The motion amplitude is the weighted sum of the object's rotation and translation amplitudes.

[0011] Furthermore, the implicit scene models corresponding to low-dynamic objects that originally belonged to the implicit scene are converted into explicit scene models because their motion properties have changed, making them difficult for the corresponding implicit scene models to track. The objects that are difficult to track are those whose errors cannot be reduced to below the threshold κ within a given time when performing joint optimization of IBRNet and MNet.

[0012] Furthermore, the explicit scene model is updated more frequently than the implicit scene model.

[0013] Furthermore, a motion trajectory field described by a learned basis function is used to represent scene motion. For a 3D point x sampled along the new viewpoint ray r at time i, an MLP is used as the trajectory encoder T. en :

[0014]

[0015] in Let γ be the basis function, and γ be the position encoder.

[0016] Introduce a globally adaptive motion basis And initialize it using the DCT basis;

[0017] This movement base Spanning each time step i of the input video, then with T en Joint optimization is performed, and the motion trajectory is defined as:

[0018]

[0019] Therefore, the motion compensation at position x at time j is expressed as:

[0020] Δ x,i (j)=θ x,i (j)-θ x,i (i).

[0021] Furthermore, when converting the implicit scene model into an explicit scene model: First, MNet is called to find the approximate location of the object corresponding to the implicit scene model. Then, the object detection algorithm is called to identify the object category. Furthermore, the stereo vision algorithm is used to calculate the three-dimensional pose information of the object. Then, the explicit model of the object is called from the pre-built model database and loaded into the scene. For the implicit scene model of the object, it is isolated by a mask and then locally modified and deleted.

[0022] The present invention also provides a multi-level update system for digital twin scenes based on an explicit-implicit fusion model. The system includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it performs the multi-level update method for digital twin scenes based on an explicit-implicit fusion model as described above.

[0023] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the multi-level update method for digital twin scenarios based on an explicit-implicit fusion model as described above.

[0024] In summary, compared with the prior art, the multi-level update method and device for digital twin scenes based on explicit-implicit fusion model provided by the present invention have the following beneficial effects:

[0025] 1. This invention can decompose the static and dynamic objects of the implicit scene using IBRNet and Mnet, and then perform motion adjustment compensation for the dynamic objects in the scene in the time domain. Based on the IBR idea, the implicit scene view is obtained by volume rendering by aggregating features of 3D sampling points from a new perspective. Finally, by combining the above methods, the scene update that is difficult to perform by traditional implicit scene models is realized.

[0026] 2. This invention constructs a scheme for mutual conversion between explicit and implicit scene models. Based on the local editing algorithm of the implicit scene model, it realizes a flexible scene update strategy and enhances the adaptive performance of the system. At the same time, it provides a mechanism for handling abnormal activity states of dynamic objects in the scene, thereby enhancing the robustness of the system.

[0027] 3. This invention adopts a hierarchical update strategy, which performs high-frequency updates for explicit high-dynamic models and low-frequency updates for implicit low-dynamic scenes, and constructs a channel for dynamic transformation between levels. This ensures, under the condition of limited hardware performance, a high-fidelity real-time mapping of the digital twin scene to the physical scene, thereby improving the performance ceiling of the entire digital twin system. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a multi-level update method for digital twin scenarios based on an explicit-implicit fusion model provided by the present invention.

[0029] Figure 2 It separates the dynamic and static models in the implicit scene model using IRBNet and MNet;

[0030] Figure 3 This is a schematic diagram illustrating how trajectory prediction adjusts and corrects the interference of dynamic objects on the feature aggregation process. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0032] This invention provides a multi-level update method for digital twin scenes based on an explicit-implicit fusion model. The update method mainly includes the following steps:

[0033] Step 1: Based on the pre-collected data and real-time collected data of the scene, the explicit scene model is structured and updated in real time according to the motion state of the physical devices in the scene, and the motion state of the explicit scene model is recorded to form an explicit scene model action database.

[0034] Specifically, it includes the following sub-steps:

[0035] First, an explicit scene model structured database is constructed based on prior information and device data. For highly dynamic devices and objects with kinematic pairs in the scene, as well as potential objects with regular motion, explicit scene models are pre-constructed. The structural information and motion mechanism of virtual devices are defined according to the actual devices.

[0036] Secondly, the state and structural shape of the explicit scene model are updated in real time based on sensor information.

[0037] Furthermore, an explicit scene model action database is set up to record the historical action states of the explicit scene models in the scene, for subsequent analysis and handling of abnormal situations.

[0038] The classification of low-dynamic and high-dynamic objects is as follows: First, determine whether the object possesses its own kinematic pairs; objects exhibiting kinematic pairs are considered high-dynamic objects. Second, determine the object's motion frequency; objects with a motion frequency greater than a frequency threshold υ are considered high-dynamic objects. Finally, determine the object's single-motion amplitude; objects with a single-motion amplitude greater than a motion amplitude threshold ω are considered high-dynamic objects. Objects that do not satisfy any of these three criteria are low-dynamic objects; all others are high-dynamic objects. The motion amplitude is a weighted sum of the object's rotational and translational amplitudes.

[0039] Step 2: Traverse the explicit scene model action database to determine the low-dynamic explicit scene model, and convert the low-dynamic explicit scene model into an implicit scene model.

[0040] Specifically, a period T is defined based on device performance. Within each period, the explicit model action database is traversed and searched. A frequency threshold υ and a motion amplitude threshold ω are set to filter low-dynamic explicit objects. Objects with kinematic pairs, objects with motion frequencies greater than the frequency threshold υ, and objects with single motion amplitudes greater than ω are excluded. The remaining objects are considered low-dynamic objects. If any explicit objects still exist that have not been excluded, they can be identified as low-dynamic explicit objects. For low-dynamic explicit models, an explicit model to implicit scene model conversion algorithm is called to transform them into implicit scene models, thereby reducing the high-frequency update pressure of the explicit model and improving visual realism.

[0041] The method for converting the explicit model to the implicit scene model involves calling the image of the target object from the camera, retraining the local data of the implicit scene, deleting the corresponding explicit model in the virtual scene, and deleting the corresponding record in the explicit model action database.

[0042] Step 3: Construct an implicit scene model for the low-dynamic scene using neural radiation fields. Simultaneously, segment the scene motion and impose temporal consistency constraints. Convert the implicit scene models of objects that were originally part of the implicit scene but whose motion properties have changed, making them difficult for the corresponding implicit scene models to track, into explicit scene models. Then, update the implicit scene models using IBR-based neural networks and motion-adjusted feature aggregation methods, and also update the converted explicit scene models.

[0043] Among these, the objects that are difficult to track are those whose error cannot be reduced below the threshold κ within a given time when performing joint optimization with IBRNet and MNet. The update frequency of the explicit scene model is greater than that of the implicit scene model.

[0044] For low-dynamic scenes, Neural Radiance Fields (NeRF) are applied to construct a high-fidelity implicit scene model. Simultaneously, motion segmentation and temporal consistency constraints are applied to the scene. Then, an IBR-based neural network and motion-adjusted feature aggregation method are used to update the implicit scene model. Specifically:

[0045] (1) The first step is the initialization of the implicit scene. This step aims to minimize the proportion of highly dynamic objects in the scene, making the time-invariant scene dominant, thereby improving the robustness of the system in subsequent operations. Image information of the scene is acquired using a camera, ensuring that the camera's viewpoint covers all areas of the scene as much as possible. Then, an initial implicit scene model of the scene is generated based on the PRNeRF (Pre-Rendered Neural Radiance Fields) network.

[0046] (2) Accurate decomposition of static and dynamic objects in a scene is achieved by training the segmentation mask of the motion segmentation module separately using a Bayesian learning framework. The Bayesian learning framework combines deep learning and probabilistic graphical models, offering advantages in tasks involving the fusion of perception and reasoning at this temporal scale. The specific steps are as follows:

[0047] First, static scene content is modeled using IBRNet. Feature aggregation is performed along epipolar lines of sampled views of the nearby region, and volume rendering is used along each ray to render pixel color I. IBR .

[0048] For dynamic scene content, a lightweight 2D convolutional encoder-decoder MNet (MotionNetwork) is trained. Unlike DeepLab, since it does not need to acquire semantic information, MNet does not require a complex network to speed up rendering. The input frame F is predicted using MNet. i RGB values density and confidence level

[0049]

[0050] The outputs of the IBRNet model and the MNet model are combined pixel by pixel along the observation ray r to create a complete reconstructed image:

[0051]

[0052] In the formula, I i (r) represents the fused color information of the sampling points on the observed ray r at time i.

[0053] By jointly training and optimizing the IBRNet and MNet models using multiple time-series images, the convergence results of the two models are geometrically consistent, and the transition between static backgrounds and dynamic objects is more natural.

[0054] (3) Motion adjustment feature aggregation in the temporal domain. The new perspective synthesis algorithm based on the IBR concept needs to aggregate features from neighboring source images. For purely static scenes, the path rendered along the new perspective volume can be projected as the epipolar line of neighboring images, so feature aggregation can be performed along the epipolar lines of neighboring images. However, if there are dynamic objects in the scene, it will violate the epipolar geometry constraint, and if the moving objects are not considered, it will lead to inconsistent feature aggregation. Therefore, it is necessary to adjust and compensate for image motion in the temporal domain.

[0055] Scene motion is represented using a motion trajectory field described by a learned basis function. For a 3D point x sampled along the new viewpoint ray r at time i, an MLP is used as the trajectory encoder T. en :

[0056]

[0057] in Let γ be the basis function, and γ be the position encoder.

[0058] Introduce a globally adaptive motion basis It is then initialized using a DCT basis. This motion basis... Spanning each time step i of the input video, then with T en Perform joint optimization. Define the motion trajectory as:

[0059]

[0060] Therefore, the motion compensation at position x at time j can be expressed as:

[0061] Δ x,i (j)=θ x,i (j)-θ x,i (i).

[0062] After performing motion adjustment compensation on the 3D points on the IBR rendering path, the volume rendering is then performed to obtain the RGB values ​​of the target pixels.

[0063] (4) Since the implicit scene model update is based on the assumption of low dynamism, the implicit scene update frequency is low in order to save system performance overhead and prioritize the real-time update of the highly dynamic explicit model.

[0064] Implicit scene model update algorithms are based on the assumption of low dynamism. However, in real-world scenarios, there may be low-dynamic objects that originally belonged to the implicit scene but have changed their motion properties and generated significant dynamic changes. In such cases, it will be difficult for the implicit scene model to track the object with low-frequency updates. Therefore, an implicit scene model to explicit model algorithm is designed to convert the object into an explicit model and allow it to be updated and managed by the explicit model.

[0065] For the implicit scene model to explicit model conversion method, firstly, MNet is called to find the approximate location of the object corresponding to the implicit scene model. Then, an object detection algorithm is called to identify the object category. Further, a stereo vision algorithm is used to calculate the 3D pose information of the object. Finally, the explicit model of the object is retrieved from a pre-built model database and loaded into the scene. For the implicit scene model of the object, it is isolated by a mask and then locally modified and deleted.

[0066] The present invention will be further described in detail below with reference to specific embodiments.

[0067] Please see Figure 1 This embodiment takes the reconstruction of a digital twin scene in a tower crane construction scenario as an example to detail a multi-level update method for digital twin scenes based on an explicit-implicit fusion model provided by the present invention. This method includes a frequently updated explicit model update layer, a frequently updated implicit scene model update layer, and a bridge for mutual conversion between explicit and implicit scene models.

[0068] The method mainly includes the following steps:

[0069] S1, Explicit model update.

[0070] The explicit model is structured based on pre-collected data and real-time sensor data. The explicit scene model is updated in real time according to the motion state of the physical devices, and the motion state of the model is recorded to form an explicit scene model action database.

[0071] The explicit scene model update steps are as follows:

[0072] S11 constructs an explicit model structured database based on prior information and device data. For highly dynamic devices and objects with kinematic pairs in the scene, as well as potentially movable regular objects, explicit virtual models are pre-built, and the structural information and motion mechanism of the virtual devices are defined according to the actual devices.

[0073] S12 updates the state and structural shape of the explicit scene model in real time based on feedback data from sensors, especially for the accurate mapping of the mechanical structure of equipment such as tower cranes.

[0074] S13, design a database to record motion information, which will be used to record the historical motion information of each explicit model in the scene, for subsequent anomaly analysis and to provide judgment data for model transformation.

[0075] S2, Implicit Scene Model Update.

[0076] To more realistically represent complex curved environments and low-dynamic objects in a scene, using Neural Radiation Field (NeRF) to construct a high-fidelity implicit scene model is a good approach. However, the editability of implicit scene models is relatively low, and there is currently no reliable solution for updating implicit scene models in digital twin scenes. This embodiment simultaneously performs motion segmentation and temporal consistency constraints on the scene, and then uses an IBR-based neural network and motion-adjusted feature aggregation method to update the implicit scene model.

[0077] The steps for updating the implicit scene model are as follows:

[0078] S21 begins with implicit scene initialization. Before initial data acquisition, the proportion of highly dynamic objects in the scene should be minimized to make the time-invariant scene dominant, thus improving the robustness of the system's subsequent operations. Image information of the scene is acquired using a camera, ensuring that the camera's viewpoint covers the entire scene area as much as possible. Then, an initial implicit scene model is generated based on the PRNeRF network. PRNeRF has the capability to synthesize new viewpoints.

[0079] RGB,σ=PRNeRF(x,v),

[0080] Where x is the position, v is the viewing direction, and RGB and σ are the color and density, respectively.

[0081] S22, Scene Dynamic and Static Object Decomposition, employs a Bayesian framework to address this type of task that integrates perception and reasoning at a temporal scale. Specifically, it requires training a segmentation mask for a motion segmentation module to achieve accurate scene dynamic and static object decomposition.

[0082] Please see Figure 2 IBRNet is used to model static objects in a continuous time sequence. An image sequence radius r is set, and IBRNet features are aggregated for images within the range of i+r frames before and after the target frame i and ir frames in the sequence.

[0083] For the feature aggregation, a continuous neural radiation field is dynamically generated from multiple source views. At the core of this method is a small MLP network that aggregates feature information from the source views at its 3D sampling points along a new viewpoint ray. The resulting feature information is then decoded into scalar color and density information using a Transformer decoder, and finally, volume rendering is used along each ray to represent the pixel information.IBR To synthesize a new view.

[0084] For dynamic scene content on a continuous temporal sequence, a lightweight 2D convolutional encoder-decoder MNet is trained to represent it. MNet does not need to acquire semantic information from the image, so it does not require a complex network structure, thus achieving fast training and rendering. This embodiment predicts the input frame F using MNet. i RGB values density and confidence level

[0085]

[0086] The outputs of the IBRNet and MNet models need to be fused. Specifically, a complete scene decomposition map (I) needs to be synthesized pixel by pixel along the observation ray. i (r):

[0087]

[0088] By jointly optimizing the IBRNet and MNet models through temporal continuity constraints and epipolar geometry constraints, the convergence results of the two models are geometrically consistent, thereby making the transition between static backgrounds and dynamic objects more natural.

[0089] S23 performs motion adjustment feature aggregation in the time domain. (See also...) Figure 3 The new perspective synthesis algorithm based on the IBR concept needs to aggregate features from neighboring source images, and motion adjustment compensation is required for dynamic objects in this process.

[0090] The idea of ​​IBR is to synthesize a new two-dimensional image from multiple perspectives of two-dimensional images. Inspired by this, for the 3D query point of the neural radiation field, the direct color and density information is not stored in advance. Instead, the feature information of the neighboring images is aggregated during the query and then decoded by the decoder to obtain the required information.

[0091] For purely static scenes, the path rendered along the new viewpoint volume can be projected as the epipolar line of the neighboring images, so feature aggregation can be performed efficiently by following the epipolar line of the neighboring images.

[0092] For scenes with dynamic objects that do not meet epipolar geometry constraints, it is impossible to obtain consistent feature aggregation results directly through epipolar lines. Therefore, it is necessary to adjust and compensate for moving objects in the image in the time domain.

[0093] Scene motion is represented using a motion trajectory field described by a learned basis function. For a 3D point x sampled along the new viewpoint ray r at time i, an MLP is used as the trajectory encoder T.en :

[0094]

[0095] in Let be the basis function, and γ be the position encoder.

[0096] Introduce a globally adaptive motion basis Using DCT basis to motion basis Initialize. This motion base Spanning each time step i of the input video, then with T en Joint optimization is performed, and the motion trajectory is defined as:

[0097]

[0098] Therefore, the motion compensation at position x at time j can be expressed as:

[0099] Δ x,i (j)=θ x,i (j)-θ x,i (i)

[0100] Please see Figure 3 After motion adjustment compensation is performed on the 3D query points (points on the solid line) on the IBR rendering path (points on the dashed line), color density information is obtained through the decoder, and finally volume rendering is performed to obtain the RGB value of the target pixel.

[0101] S24, the implicit scene update model is based on the assumption of low dynamism, so the implicit scene update frequency can be reasonably reduced to save system performance overhead and prioritize the real-time update of the highly dynamic explicit model.

[0102] Steps S1 and S2 can be performed simultaneously without any specific order.

[0103] Furthermore, this embodiment may also include step S3: converting the explicit model into an implicit scene model.

[0104] For low-dynamic explicit scene models, an explicit-to-implicit scene model algorithm is used to convert them into implicit scene models, thereby reducing the pressure of frequent updates to the explicit model and improving visual realism. Specifically, this includes the following sub-steps:

[0105] S31, for low-dynamic explicit models, the explicit model action database is queried and filtered. The retrieval period T is defined based on equipment performance, and a frequency threshold υ and a motion amplitude threshold ω are set. Within each period, the explicit model action database is traversed and searched, excluding objects with kinematic pairs, objects with a motion frequency greater than the frequency threshold υ, and objects with a single motion amplitude greater than ω. If any explicit objects still exist that have not been excluded, they can be identified as low-dynamic explicit objects.

[0106] S32. For the method of converting explicit scene model to implicit scene model, it is necessary to change the IBR source image of the region where the model is located, then retrain the local data of the implicit scene, delete the corresponding structured explicit model in the digital twin virtual scene, and delete the corresponding record in the explicit model action database.

[0107] Furthermore, this embodiment may also include step S4: converting the implicit scene model into an explicit model.

[0108] Implicit scene model update algorithms are based on the assumption of low dynamism. However, in real-world scenarios, there may be low-dynamic objects that originally belonged to the implicit scene but have changed their motion properties, resulting in significant dynamic changes. In such cases, the implicit scene model will struggle to track the object with infrequent updates. Therefore, an algorithm to convert the implicit scene model to an explicit model is designed to transform the object into an explicit model, allowing it to be updated and managed by the explicit model. Specifically, this includes the following steps:

[0109] S41. For objects that are difficult to track by the implicit scene model update mechanism, specifically when performing joint optimization of IBRNet and MNet, the error cannot be reduced to below the threshold κ within a given time.

[0110] S42, for the method of converting implicit scene models to explicit models: First, the approximate region where the moving object is located is determined using MNet. Then, a pre-trained semantic segmentation algorithm is used to separate the pixel range of the moving object in the given region and obtain its category information. Further, the 3D pose information of the object is calculated using a stereo vision algorithm, and then the explicit model of the object is loaded into the scene from a pre-built model database. To eliminate the object's original implicit scene model, the scene needs to be retrained locally using a mask isolation method.

[0111] The present invention also provides a multi-level update system for digital twin scenes based on an explicit-implicit fusion model. The system includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it performs the multi-level update method for digital twin scenes based on an explicit-implicit fusion model as described above.

[0112] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the multi-level update method for digital twin scenarios based on an explicit-implicit fusion model as described above.

[0113] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-level update method for digital twin scenes based on an explicit-implicit fusion model, characterized in that, The method includes the following steps: The explicit scene model is structured based on pre-collected data and real-time collected data, and the explicit scene model is updated in real time according to the motion state of the physical devices in the scene. Meanwhile, a neural radiation field is used to construct an implicit scene model for low-dynamic scenes, while scene motion is segmented and time consistency is constrained. The implicit scene model is updated using an IBR-based neural network and a motion-adjusted feature aggregation method. While updating the explicit scene model, the motion state of the explicit scene model is also recorded to form an explicit scene model action database; Traverse the explicit scene model action database to determine low-dynamic explicit scene models, and convert the low-dynamic explicit scene models into implicit scene models. Define a cycle based on equipment performance. Within each cycle, the explicit model action database is traversed and retrieved, and a frequency threshold is set. and motion amplitude threshold Filter out low-dynamic explicit objects; the distinction between low-dynamic and high-dynamic objects is as follows: First, determine whether the object has its own kinematic pairs; objects that produce kinematic pairs are high-dynamic objects; Second, determine the object's motion frequency; objects with a motion frequency greater than a frequency threshold are considered high-dynamic objects. The object is a highly dynamic object; third, the amplitude of a single motion of the object is judged, and the amplitude of a single motion is greater than the motion amplitude threshold. The object is a high-dynamic object; an object that does not satisfy any of the above three points is a low-dynamic object, and the rest are high-dynamic objects; the motion amplitude is a weighted sum of the object's rotation and translation amplitudes. The implicit scene model is converted into an explicit scene model for objects that were originally low-dynamic objects in the implicit scene but whose motion properties have changed, making them difficult for the corresponding implicit scene model to track. These difficult-to-track objects are those whose error cannot be reduced to a threshold within a given time during joint optimization using IBRNet and MNet. the following; The explicit scene model is updated more frequently than the implicit scene model.

2. The multi-level update method for digital twin scenarios based on an explicit-implicit fusion model as described in claim 1, characterized in that: Scene motion is represented using a trajectory field described by a learned basis function, for time... Along the new perspective ray 3D points obtained from sampling An MLP is used as the trajectory encoder. : in As basis functions, For position encoder, Introduce a globally adaptive motion basis It is initialized using the DCT basis; This movement base Spanning every time step of the input video , and then with Joint optimization is performed, and the motion trajectory is defined as: So in Time and location The motion compensation at point is expressed as:

3. The multi-level update method for digital twin scenarios based on an explicit-implicit fusion model as described in claim 2, characterized in that: When converting an implicit scene model to an explicit scene model: First, MNet is called to find the approximate location of the object corresponding to the implicit scene model. Then, an object detection algorithm is called to identify the object category. Furthermore, a stereo vision algorithm is used to calculate the three-dimensional pose information of the object. Then, the explicit model of the object is called from the pre-built model database and loaded into the scene. For the implicit scene model of the object, it is isolated by a mask and then locally modified and deleted.

4. A multi-level update system for digital twin scenarios based on an explicit-implicit fusion model, characterized in that: The system includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it performs the multi-level update method for digital twin scenes based on an explicit-implicit fusion model as described in any one of claims 1-3.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the multi-level update method for digital twin scenarios based on an explicit-implicit fusion model as described in any one of claims 1-3.

Images