Method and system for rapid generation of digital twin model based on AI large model

Abstract

The application discloses an AI large model-based digital twin model rapid generation method and system, which comprises a data layer, a visual model layer and a visual application layer, the data layer is connected with the visual model layer, the visual model layer is connected with the visual application layer, the visual model layer comprises an AI model construction module, a multi-modal integration module, a model training module, a dynamic updating module, a visual twin modeling module, a real-time rendering simulation module, an intelligent decision support module, a model integration module and a model framework module, and is loaded with an AI large model engine.The AI large model-based digital twin model rapid generation method and system can shorten the modeling period from several weeks to several hours, significantly improve the model precision, generalization ability and self-adaptive updating ability, are suitable for multiple scenes such as industrial manufacturing, smart city and energy conservation, and reduce the technical threshold of digital twin modeling.

Description

Technical Field

[0001] This invention relates to the fields of digital twin visualization, 3D virtual simulation, and AI automatic modeling and rendering technology. Specifically, it relates to a method and system for rapidly generating visualized digital twin models based on large AI models, which is applicable to scenarios such as industrial manufacturing, smart cities, and energy conservation that require the construction of high-precision, dynamically visualized, and interactive digital twin models. Background Technology

[0002] Digital twin technology, by constructing dynamic and visual mappings of physical entities in virtual space, enables real-time monitoring, 3D visualization simulation, interactive prediction, and optimized control of real-world system states, and has become one of the core technologies driving industrial digital transformation. Currently, the construction of visual digital twin models mainly relies on manual 3D modeling, manual rendering, mechanism binding, or static modeling methods based on small-scale data, which suffer from problems such as long modeling cycles, limited visualization accuracy, poor dynamic rendering effects, and weak real-time interactive capabilities.

[0003] In existing technologies, the visualization modeling process typically involves multiple stages, including data acquisition, geometric modeling, material rendering, behavior binding, and scene integration. This requires the participation of a large number of professional modelers and has a low degree of automation. Especially when dealing with complex equipment, urban scenes, and other nonlinear systems, traditional methods struggle to effectively integrate multi-source heterogeneous data and automatically generate real-time renderable 3D visualization twins. This results in models that cannot accurately and intuitively reflect the dynamic visualization characteristics of the real system. Furthermore, when physical entities undergo structural changes or operational status shifts, existing visualization models struggle to rapidly iterate and update, synchronize the visuals in real time, and lack adaptive rendering capabilities.

[0004] In recent years, large AI models (such as Transformer, VisionTransformer, and multimodal large models) have demonstrated powerful representation learning and generation capabilities in 3D modeling, image rendering, and dynamic simulation, possessing the potential for automatic generation of 3D meshes, real-time rendering, and dynamic visualization. However, a systematic solution has yet to be developed for effectively integrating large AI models into the visualization digital twin modeling process to achieve rapid end-to-end generation of directly renderable visualization models from data.

[0005] Therefore, there is an urgent need for a method to rapidly generate digital twin models that can combine the powerful visualization and modeling capabilities of AI large models, in order to solve the technical bottlenecks of low visualization and modeling efficiency, poor rendering effects, and weak real-time interaction in existing technologies. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for rapid generation of digital twin models based on AI large models. It aims to overcome the shortcomings of existing digital twin visualization modeling processes, such as reliance on manual labor, low efficiency, low visualization accuracy, and poor dynamic interactivity. By introducing AI large models, it achieves automated, intelligent, and high-precision visualization modeling driven by multi-source data, significantly improving the generation speed, rendering effect, and real-time interactive value of visualized digital twin models.

[0007] Technical Solution: To achieve the above objectives, the technical solution adopted by this invention is as follows: a rapid generation system for digital twin models based on AI large-scale models, comprising a data layer, a visualization model layer, and a visualization application layer. The data layer is connected to the visualization model layer, and the visualization model layer is connected to the visualization application layer. The visualization model layer includes an AI model construction module, a multimodal integration module, a model training module, a dynamic update module, a visualization twin modeling module, a real-time rendering and simulation module, an intelligent decision support module, a model integration module, and a model framework module, and is equipped with an AI large-scale model engine, a visualization adapter, and a training optimization unit. The visualization application layer includes a 3D visualization display module, a rendering effect verification module, a decision support module, a model output feedback module, a visualization simulation and prediction module, and a central processing module, providing model generation, visualization, and deployment management functions.

[0008] As a preferred technical solution of this application, the output ends of the multimodal integration module, dynamic update module, visual twin modeling module, real-time rendering simulation module and intelligent decision support module are connected to the input end of the model integration module, and the model framework module is divided into a model integration module and a model training module.

[0009] As a preferred technical solution of this application, the central processing module is interconnected with the 3D visualization display module, the rendering effect verification module, the decision support module, the model output feedback module, and the visualization simulation prediction module.

[0010] As a preferred technical solution of this application, the visualization model layer supports the fusion architecture of graph neural networks and AI large models, and can use a federated learning framework to achieve cross-subject collaborative training. The data layer, visualization model layer and visualization application layer are suitable for the construction of digital twin models in the fields of industrial manufacturing, smart cities and energy and environmental protection.

[0011] The method for rapidly generating digital twin models based on large AI models includes the following steps: S1: Multi-source data acquisition and visualization preprocessing: Collect multi-modal data from the target physical entity and its operating environment, including structural CAD data, real-time sensor data, historical operation data, image and video data, and scene environment data. Clean, align, normalize, and label the multi-modal data, and convert it into a unified serialization or embedded representation format suitable for AI visualization modeling.

[0012] S2: Large Model Selection and Visualization Adaptation: Based on the type and data characteristics of the target system, a pre-trained general-purpose AI large model is selected as the basic architecture. The AI ​​large model includes, but is not limited to, Transformer-based sequence models, VisionTransformer, and multimodal large models. By introducing visualization domain adaptation modules, 3D rendering attention mechanisms, and scene knowledge injection interfaces, the basic model is fine-tuned and enhanced to enable it to have the ability to visualize digital twin modeling and automatic rendering.

[0013] S3: Automatic generation of visual digital twin models: Input the preprocessed multimodal data into the adapted AI large model, and the model automatically extracts 3D geometric features, physical rendering attributes, dynamic behavior logic, and real-time interaction relationships. The AI ​​large model outputs a complete visual digital twin model file that can be directly loaded and rendered, including 3D mesh structure, material texture, dynamic rendering rules, and state visualization equations.

[0014] S4: Visualization Model Optimization and Rendering Verification: Render simulation verification of the generated visualization model is performed using real running screens and state data. The consistency error between the model visualization effect and the actual physical entity screen is calculated. Based on the error feedback, online learning or incremental training strategies are used to jointly optimize the AI ​​large model and the visualization digital twin model to improve the model visualization accuracy, dynamic rendering smoothness and real-time adaptability.

[0015] S5: Visualization Model Deployment and Real-time Update: Deploy the optimized visualization digital twin model to the digital twin platform to achieve real-time synchronization of physical entities and virtual models, 3D visualization simulation prediction, and interactive decision support. When the physical system undergoes structural changes or new data accumulates to a set threshold, the model visualization automatic update mechanism is triggered, calling the AI ​​large model to regenerate or locally correct the rendered screen and 3D structure.

[0016] As a preferred technical solution of this application, the AI ​​large model has a built-in "human-machine-environment" visual collaborative modeling module, which simultaneously models the visual behavior of operators, the screen of equipment operation status, and the rendering effect of the external environment, so as to realize a global system-level visual digital twin.

[0017] As a preferred technical solution of this application, the method of steps S1-S5 supports small sample visualization modeling. When the initial data is insufficient, the method uses knowledge distillation technology to transfer prior rendering and modeling knowledge from existing visualization digital twin models of similar scenarios to assist in the generation of new visualization models.

[0018] Beneficial effects: Compared with the prior art, the present invention provides a method and system for rapid generation of digital twin models based on large AI models, which has the following beneficial effects: The method and system for rapid generation of digital twin models based on large AI models can shorten the visualization modeling cycle from several weeks to several hours, significantly improve the model visualization accuracy, rendering smoothness, real-time interaction capability and adaptive screen update capability, and is applicable to multiple scenarios such as industrial manufacturing, smart cities, energy and environmental protection, and lowers the technical threshold for visualization digital twin modeling. Significantly improved efficiency in visualization modeling: Through AI-powered large-scale models, end-to-end automatic generation of visual digital twin models from raw data is achieved, reducing manual modeling and rendering interventions and shortening the modeling cycle from several weeks to several hours.

[0019] Higher visualization and accuracy: The AI ​​large model can capture complex 3D structures, dynamic rendering relationships and multimodal image coupling features, and the generated visualized digital twin model is closer to the real physical entity and the rendering effect is more realistic.

[0020] Strong dynamic visualization and adaptive capabilities: Supports cross-scene visualization knowledge transfer and online incremental rendering learning. The model can update the screen and optimize the rendering effect in real time as the physical system evolves.

[0021] Supports multi-domain general visualization modeling: Through modular design, it can be widely used in the 3D visualization twin construction of various industry scenarios such as intelligent manufacturing, smart cities, and energy and power.

[0022] Lowering the technical threshold for visual modeling: Even non-professional modelers can quickly build high-quality, interactive, and real-time rendered visual digital twin models through this system, promoting the popularization of the technology. The entire visual digital twin model rapid generation system has a simple structure, is easy to operate, and has better results than traditional methods. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall structure of the method and system for rapid generation of digital twin models based on large AI models according to the present invention.

[0024] Figure 2 This is a flowchart illustrating the process of generating a visual digital twin model in the method and system for rapid generation of digital twin models based on large AI models, as described in this invention.

[0025] Figure 3This is a schematic diagram of the internal structure of the AI ​​large model in the method and system for rapid generation of digital twin models based on AI large models according to the present invention.

[0026] Figure 4 This is a comparison chart of the visual digital twin modeling effect of a wind turbine in Implementation Example 1 of the method and system for rapid generation of digital twin models based on AI large models of the present invention.

[0027] Figure 5 This is a schematic diagram of the visual model update mechanism in the method and system for rapid generation of digital twin models based on large AI models, as described in this invention.

[0028] Figure 6 This is a schematic diagram of the visualization model layer in the method and system for rapid generation of digital twin models based on large AI models, as described in this invention.

[0029] Figure 7 This is a schematic diagram of the visualization application layer in the method and system for rapid generation of digital twin models based on large AI models, as described in this invention.

[0030] Figure 8 This is a schematic diagram of a wind turbine model, which is an embodiment of the rapid generation method and system for digital twin models based on AI large models of the present invention.

[0031] Figure 9 This is a schematic diagram of a traffic intersection model, which is an embodiment 2 of the rapid generation method and system for digital twin models based on AI large models of the present invention. Detailed Implementation

[0032] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. However, those skilled in the art will understand that the embodiments described below are some embodiments of the present invention, but not all embodiments, and are only used to illustrate the present invention, and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Where specific conditions are not specified in the embodiments, conventional conditions or conditions recommended by the manufacturer shall be followed. Where the manufacturers of reagents or instruments are not specified, they are all conventional products that can be purchased commercially.

[0033] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0034] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0035] like Figure 1-9 As shown, the rapid generation system for digital twin models based on AI large-scale models includes a data layer, a visualization model layer, and a visualization application layer. The data layer connects to the visualization model layer, and the visualization model layer connects to the visualization application layer. The visualization model layer includes an AI model building module, a multimodal integration module, a model training module, a dynamic update module, a visualization twin modeling module, a real-time rendering and simulation module, an intelligent decision support module, a model integration module, and a model framework module. It is equipped with an AI large-scale model engine, a visualization adapter, and a training optimization unit. The visualization application layer includes a 3D visualization display module, a rendering effect verification module, a decision support module, a model output feedback module, a visualization simulation and prediction module, and a central processing module, providing model generation, visualization, and deployment management functions.

[0036] The outputs of the multimodal integration module, dynamic update module, visual twin modeling module, real-time rendering simulation module, and intelligent decision support module are connected to the input of the model integration module. The model framework module is divided into a model integration module and a model training module.

[0037] The central processing module is interconnected with the 3D visualization module, rendering effect verification module, decision support module, model output feedback module, and visualization simulation prediction module.

[0038] The visualization model layer supports the fusion architecture of graph neural networks and large AI models, and can use a federated learning framework to achieve cross-subject collaborative training. The data layer, visualization model layer and visualization application layer are suitable for building digital twin models in the fields of industrial manufacturing, smart cities and energy and environmental protection.

[0039] The method for rapidly generating digital twin models based on large AI models includes the following steps: S1: Multi-source data acquisition and visualization preprocessing: Collect multi-modal data from the target physical entity and its operating environment, including structural CAD data, real-time sensor data, historical operational data, image and video data, and scene environment data. Clean, align, normalize, and label the multi-modal data, and convert it into a unified serialized or embedded representation format suitable for AI visualization modeling.

[0040] S2: Large Model Selection and Visualization Adaptation: Based on the type and data characteristics of the target system, a pre-trained general-purpose AI large model is selected as the basic architecture. The AI ​​large model includes, but is not limited to, Transformer-based sequence models, VisionTransformer, and multimodal large models. By introducing visualization domain adaptation modules, 3D rendering attention mechanisms, and scene knowledge injection interfaces, the basic model is fine-tuned and enhanced to enable it to have the ability to visualize digital twin modeling and automatic rendering.

[0041] S3: Automatic generation of visual digital twin models: Input the preprocessed multimodal data into the adapted AI large model, and the model automatically extracts 3D geometric features, physical rendering attributes, dynamic behavior logic, and real-time interaction relationships. The AI ​​large model outputs a complete visual digital twin model file that can be directly loaded and rendered, including 3D mesh structure, material texture, dynamic rendering rules, and state visualization equations.

[0042] S4: Visualization Model Optimization and Rendering Verification: Render simulation verification of the generated visualization model is performed using real running screens and state data. The consistency error between the model visualization effect and the actual physical entity screen is calculated. Based on the error feedback, online learning or incremental training strategies are used to jointly optimize the AI ​​large model and the visualization digital twin model to improve the model visualization accuracy, dynamic rendering smoothness and real-time adaptability.

[0043] S5: Visualization Model Deployment and Real-time Update: Deploy the optimized visualization digital twin model to the digital twin platform to achieve real-time synchronization of physical entities and virtual models, 3D visualization simulation prediction, and interactive decision support. When the physical system undergoes structural changes or new data accumulates to a set threshold, the model visualization automatic update mechanism is triggered, calling the AI ​​large model to regenerate or locally correct the rendered screen and 3D structure.

[0044] The AI ​​big model has a built-in "human-machine-environment" visualization collaborative modeling module, which simultaneously models the visualized behavior of operators, the operating status of equipment, and the rendering effects of the external environment, realizing a global system-level visualized digital twin. The method of steps S1-S5 supports small sample visualization modeling. When the initial data is insufficient, knowledge distillation technology is used to transfer prior rendering and modeling knowledge from existing visualized digital twin models of similar scenarios to assist in the generation of new visualization models.

[0045] Example 1: Visual Digital Twin Modeling of Industrial Equipment The system of this invention is deployed for a certain type of wind turbine generator: Data acquisition: Collect CAD structural data of the unit, real-time data from the SCADA system (wind speed, rotational speed, temperature, vibration), real-time images of the inspection, and videos of the equipment's appearance.

[0046] Data preprocessing: Real-world images are converted into visual modeling sequences, dynamic image features are extracted from videos, sensor data is bound to the visual status, and real-time image synchronization is achieved by aligning with timestamps.

[0047] Model selection and adaptation: A multimodal large model is selected as the base model, and a 3D rendering loss function and physical constraints are added to guide the model to learn the wind turbine structure, materials, and dynamic operation visualization effects.

[0048] Model generation: After inputting data, the AI ​​big model outputs a complete visualized digital twin model that includes a 3D geometric model, real-time status rendering, fault visualization and early warning, and operation and maintenance interactive interface.

[0049] Verification and Deployment: Run 3D visualization simulation in the digital twin platform to render the wind turbine's operating status in real time. The visualization screen is 98% consistent with the real equipment. After the model is deployed, it supports real-time screen alerts and 3D interactive operation and maintenance.

[0050] Example 2: Visualized Digital Twin of Urban Traffic Intersections Data sources: real-time video stream from intersection cameras, data from geomagnetic detectors, traffic light control logs, and weather and environmental footage.

[0051] Modeling process: VisionTransformer is used to process the video stream and extract the 3D trajectory of traffic flow and the intersection scene structure; the signal status is analyzed in conjunction with the visualization rendering module; and the AI ​​large model automatically generates a 3D visualization simulation model of traffic flow.

[0052] Results: The model can render traffic flow at intersections in real time, simulate the visual evolution of congestion under different signal timing schemes, and support dynamic optimization of traffic lights and synchronous updates of the visualization scene.

[0053] This method first collects and standardizes the multi-source heterogeneous data of physical entities for visualization, then selects an appropriate AI large model and injects visualization scene knowledge and rendering constraints. The AI ​​large model automatically completes the end-to-end generation from multimodal data to a complete visualized digital twin model. After rendering simulation verification and online image optimization, it is deployed to the twin platform, supporting automatic iterative updates of the visualized model after changes in the physical system. The system consists of a three-layer architecture: data layer, model layer, and application layer. The data layer is responsible for multi-source data collection, cleaning, and storage, serving visualization modeling. The model layer includes an AI large model engine, a visualization adaptation module, and a training and optimization unit. The application layer provides visualization model generation, 3D rendering display, and interactive deployment management functions.

[0054] It should be noted that, in this document, relational terms such as first and second (number one, number two), etc., are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0055] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A rapid generation system for digital twin models based on AI large-scale models, comprising a data layer, a visualization model layer, and a visualization application layer, characterized by: The data layer connects to the visualization model layer, which in turn connects to the visualization application layer. The visualization model layer includes an AI model building module, a multimodal integration module, a model training module, a dynamic update module, a visualization twin modeling module, a real-time rendering and simulation module, an intelligent decision support module, a model integration module, and a model framework module. It is equipped with an AI large model engine, a visualization adapter, and a training optimization unit. The visualization application layer includes a 3D visualization display module, a rendering effect verification module, a decision support module, a model output feedback module, a visualization simulation and prediction module, and a central processing module, providing model generation, visualization, and deployment management functions.

2. The rapid generation system for digital twin models based on AI large models according to claim 1, characterized in that: The outputs of the multimodal integration module, dynamic update module, visual twin modeling module, real-time rendering simulation module, and intelligent decision support module are connected to the input of the model integration module. The model framework module is divided into a model integration module and a model training module.

3. The rapid generation system for digital twin models based on large AI models according to claim 1, characterized in that: The central processing module is interconnected with the 3D visualization display module, the rendering effect verification module, the decision support module, the model output feedback module, and the visualization simulation prediction module.

4. The rapid generation system for digital twin models based on large AI models according to claim 1, characterized in that: The visualization model layer supports the fusion architecture of graph neural networks and large AI models, and can use a federated learning framework to achieve cross-subject collaborative training. The data layer, visualization model layer and visualization application layer are suitable for building digital twin models in the fields of industrial manufacturing, smart cities and energy and environmental protection.

5. A method for rapid generation of digital twin models based on large AI models, characterized by: Specifically, the following steps are included: S1: Multi-source data acquisition and visualization preprocessing: Collect multi-modal data from the target physical entity and its operating environment, including structural CAD data, real-time sensor data, historical operation data, image and video data, and scene environment data. Clean, align, normalize, and label the multi-modal data, and convert it into a unified serialization or embedded representation format suitable for AI visualization modeling. 6.S2: Large Model Selection and Visualization Adaptation: Based on the type and data characteristics of the target system, a pre-trained general-purpose AI large model is selected as the basic architecture. The AI ​​large model includes, but is not limited to, Transformer-based sequence models, VisionTransformer, and multimodal large models. By introducing visualization domain adaptation modules, 3D rendering attention mechanisms, and scene knowledge injection interfaces, the basic model is fine-tuned and enhanced to enable it to have the ability to visualize digital twin modeling and automatic rendering. 7.S3: Automatic generation of visual digital twin model: Input the preprocessed multimodal data into the adapted AI large model, and the model automatically extracts 3D geometric features, physical rendering attributes, dynamic behavior logic, and real-time interaction relationships. The AI ​​large model outputs a complete visual digital twin model file that can be directly loaded and rendered, including 3D mesh structure, material texture, dynamic rendering rules, and state visualization equations. 8.S4: Visualization Model Optimization and Rendering Verification: Render simulation verification of the generated visualization model is performed using real running screens and state data. The consistency error between the model visualization effect and the actual physical entity screen is calculated. Based on the error feedback, online learning or incremental training strategies are used to jointly optimize the AI ​​large model and the visualization digital twin model to improve the model visualization accuracy, dynamic rendering smoothness and real-time adaptability. 9.S5: Visualization Model Deployment and Real-time Update: Deploy the optimized visualization digital twin model to the digital twin platform to achieve real-time synchronization of physical entities and virtual models, 3D visualization simulation prediction, and interactive decision support. When the physical system undergoes structural changes or new data accumulates to a set threshold, the model visualization automatic update mechanism is triggered, calling the AI ​​large model to regenerate or locally correct the rendered screen and 3D structure.

10. The method for rapid generation of digital twin models based on large AI models according to claim 5, characterized in that: The AI ​​big model has a built-in "human-machine-environment" visual collaborative modeling module, which simultaneously models the visual behavior of operators, the operating status of equipment, and the rendering effects of the external environment, to achieve a global system-level visual digital twin.

11. The method for rapid generation of digital twin models based on large AI models according to claim 5, characterized in that: The method described in steps S1-S5 supports small-sample visualization modeling. When initial data is insufficient, it uses knowledge distillation technology to transfer prior rendering and modeling knowledge from existing visualization digital twin models of similar scenarios to assist in the generation of new visualization models.

Images