Digital twin modeling method, device and equipment and storage medium

Abstract

The embodiment of the invention provides a digital twin modeling method and device, equipment and a storage medium, and the method comprises the steps: obtaining first modeling scene description information inputted in a natural language, the first modeling scene description information comprising description information of a to-be-modeled object and space environment description information of the to-be-modeled object; generating cue words according to the first modeling scene description information, grammar description information of a preset markup language and digital twin model information corresponding to the to-be-modeled object; the cue word is input into a machine learning model to obtain second modeling scene description information output by the machine learning model in a markup language, and the second modeling scene description information comprises digital twin model information corresponding to the to-be-modeled object and space environment description information; and rendering a digital twin page corresponding to the second modeling scene description information by adopting a rendering engine with markup language analysis capability. According to the scheme, the efficiency of digital twin modeling is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a digital twin modeling method, device, equipment and storage medium. Background Art

[0002] Digital Twin modeling is a technology that digitalizes the real physical world. Through Digital Twin modeling, information such as the operating conditions, performance, and behaviors of real objects (such as production equipment, etc.) in the real physical world can be presented in a digital form, thereby realizing the simulation, analysis, and optimization of the real physical world.

[0003] 3D visualization is a technical means in Digital Twin modeling. Through 3D visualization technology, the results of Digital Twin modeling can be presented to users in a more intuitive and realistic manner.

[0004] Currently, in the process of Digital Twin 3D modeling, it often relies on professional 3D modeling tools and requires a relatively high labor cost. For example, in the application scenario of Digital Twin 3D modeling of a factory, since the modeling involves Digital Twins of various objects such as factories, production lines, and equipment, the modeling process often requires professional designers or engineers to participate. Based on on-site surveying and mapping, or various CAD formats, 3D general formats, Digital Surface Model (DSM for short), and dense 3D point clouds, etc., a 3D Digital Twin model is generated, and the user experience is not good. Summary of the Invention

[0005] Embodiments of the present invention provide a digital twin modeling method, device, equipment and storage medium to improve the efficiency of Digital Twin 3D modeling.

[0006] In a first aspect, embodiments of the present invention provide a digital twin modeling method, and the method includes:

[0007] Obtain first modeling scenario description information input in natural language, where the first modeling scenario description information includes description information of an object to be modeled and description information of the spatial environment of the object to be modeled;

[0008] Generate a prompt word according to the first modeling scenario description information, the syntax description information of a preset markup language, and the digital twin model information corresponding to the object to be modeled; wherein, the syntax description information includes markup formats of the modeling object and the spatial environment to which the modeling object belongs;

[0009] Input the prompt into a machine learning model to obtain second modeling scenario description information output by the machine learning model in a markup language, where the second modeling scenario description information includes digital twin model information corresponding to the object to be modeled and the spatial environment description information;

[0010] Use a rendering engine with the ability to parse markup language to render a digital twin page corresponding to the second modeling scenario description information.

[0011] In a second aspect, an embodiment of the present invention provides a digital twin modeling device, where the device includes:

[0012] An acquisition module, configured to acquire first modeling scenario description information input in natural language, where the first modeling scenario description information includes description information of the object to be modeled and spatial environment description information of the object to be modeled;

[0013] A processing module, configured to generate a prompt according to the first modeling scenario description information, syntax description information of a preset markup language, and digital twin model information corresponding to the object to be modeled; where the syntax description information includes markup formats of the modeling object and the spatial environment to which the modeling object belongs; input the prompt into a machine learning model to obtain second modeling scenario description information output by the machine learning model in a markup language, where the second modeling scenario description information includes digital twin model information corresponding to the object to be modeled and the spatial environment description information;

[0014] A rendering module, configured to use a rendering engine with the ability to parse markup language to render a digital twin page corresponding to the second modeling scenario description information.

[0015] In a third aspect, an embodiment of the present invention provides a digital twin modeling method, where the method includes:

[0016] Receive a request triggered by a client device by invoking a digital twin modeling service provided by the cloud, where the request includes first modeling scenario description information input by the user in natural language, and the first modeling scenario description information includes description information of the object to be modeled and spatial environment description information of the object to be modeled;

[0017] Generate a prompt according to the first modeling scenario description information, syntax description information of a preset markup language, and digital twin model information corresponding to the object to be modeled; where the syntax description information includes markup formats of the modeling object and the spatial environment to which the modeling object belongs;

[0018] Input the prompt into a machine learning model to obtain second modeling scenario description information output by the machine learning model in a markup language, where the second modeling scenario description information includes digital twin model information corresponding to the object to be modeled and the spatial environment description information;

[0019] Use a rendering engine with markup language parsing capabilities to render a digital twin page corresponding to the second modeling scenario description information;

[0020] Feed the digital twin page back to the client device for display.

[0021] In a fourth aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a communication interface; wherein, executable code is stored on the memory, and when the executable code is executed by the processor, the processor can at least implement the digital twin modeling method as described in the first aspect or the third aspect.

[0022] In a fifth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored, and when the executable code is executed by a processor of an electronic device, the processor can at least implement the digital twin modeling method as described in the first aspect or the third aspect.

[0023] In the embodiments of the present invention, in the process of performing digital twin modeling on a certain scenario to generate a corresponding digital twin page, in order to improve the efficiency of digital twin modeling, after the user describes the modeling scenario in a natural language manner, the machine learning model (such as a large language model, etc.) performs a normalization expression process on the modeling scenario described in natural language, and describes the modeling scenario in a markup language with a specific markup format. Specifically, in the modeling process, first, the first modeling scenario description information input in natural language is obtained, where the first modeling scenario description information includes the description information of the object to be modeled and the spatial environment description information of the object to be modeled. After that, according to the first modeling scenario description information, the syntax description information of the preset markup language, and the digital twin model information corresponding to the object to be modeled, a prompt is generated; where the syntax description information includes the markup formats of the modeling object and the spatial environment to which the modeling object belongs. Then, the prompt is input into the machine learning model to obtain the second modeling scenario description information output by the machine learning model in the markup language. The second modeling scenario description information includes the digital twin model information corresponding to the object to be modeled and the spatial environment description information, and both the object to be modeled and the spatial environment to which the object to be modeled belongs are uniformly marked in the corresponding markup formats. Since the markup language has a specific markup format and is not as complex and variable as natural language, the machine can understand the markup language more accurately. In practical applications, after converting the first modeling scenario description information input in natural language into the second modeling scenario information described in the markup language, regardless of how the natural language is described, a rendering engine with the ability to parse the markup language can quickly and accurately parse the scenario information corresponding to the modeling scenario from the second modeling scenario information and render the digital twin page corresponding to the second modeling scenario description information. This solution realizes digital twin modeling based on natural language, and by defining a markup language with a specific markup format, converting the first modeling scenario information described in natural language into the second scenario description information described in the markup language, the efficiency of digital twin modeling is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0025] Figure 1 Schematic diagram of the hardware execution environment of a digital twin modeling method provided by an embodiment of the present invention;

[0026] Figure 2 Schematic diagram of the cloud computing environment of a digital twin modeling method provided by an embodiment of the present invention;

[0027] Figure 3 Schematic diagram of the application of a digital twin modeling method provided by an embodiment of the present invention;

[0028] Figure 4 Flowchart of a digital twin modeling method provided by an embodiment of the present invention;

[0029] Figure 5 Schematic diagram of a digital twin page provided by an embodiment of the present invention;

[0030] Figure 6 Flowchart of the training of a large language model provided by an embodiment of the present invention;

[0031] Figure 7 Flowchart of a prompt generation method provided by an embodiment of the present invention;

[0032] Figure 8 Another schematic diagram of the application of a digital twin modeling method provided by an embodiment of the present invention;

[0033] Figure 9 Another flowchart of a digital twin modeling method provided by an embodiment of the present invention;

[0034] Figure 10 Another schematic diagram of a digital twin page provided by an embodiment of the present invention;

[0035] Figure 11 Schematic diagram of the structure of a digital twin modeling device provided by an embodiment of the present invention;

[0036] Figure 12 For Figure 11 Schematic diagram of the structure of an electronic device corresponding to the digital twin modeling device provided by the embodiment shown. Detailed implementation manners

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention fall within the scope of protection of the present invention.

[0038] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0039] The following will describe in detail some embodiments of the present invention with reference to the accompanying drawings. Without conflict among the embodiments, the following embodiments and the features in the embodiments can be combined with each other. In addition, the step timings in the following method embodiments are only examples and are not strictly limited.

[0040] First, the terms or concepts involved in the embodiments of the present invention will be explained:

[0041] Digital Twin modeling is a technology that digitalizes the real physical world and usually involves two processes: data collection and data display. In the data collection stage, information such as the operating conditions, performance, and behavior of physical objects can be obtained by directly connecting to physical objects, or data interaction can be carried out with physical objects through proprietary communication protocols such as the communication protocol corresponding to a Programmable Logic Controller (PLC) or the Message Queuing Telemetry Transport (MQTT) protocol to obtain information such as the operating conditions, performance, and behavior of physical objects. In the data display stage, based on the information of physical objects obtained in the data collection stage, two-dimensional or three-dimensional models of physical objects (i.e., digital twin models of physical objects) can be constructed through various two-dimensional or three-dimensional modeling tools to digitalize physical objects. Based on the digitalized physical objects, the physical world can be simulated, analyzed, and optimized.

[0042] A markup language is a computer text encoding that combines text and other information related to the text to show details about the document structure and data processing. Among them, the other information related to the text includes: the structure of the text and the identification information corresponding to the text, etc. Generally speaking, markup languages are used to achieve the typesetting of text and the differential identification of different contents in the text, so that the overall content of the text is clear and hierarchical, facilitating reading. For example, Markdown, as a lightweight markup language, has concise typesetting syntax, enabling users to pay more attention to the content itself more quickly, and has the advantages of simplicity, efficiency, readability, and writability.

[0043] A machine learning model mainly builds a mathematical model and uses the mathematical model to simulate or learn human behavior. Generally, the machine learning model obtained through training can be applied to various scenarios such as computer vision processing, natural language processing, and speech recognition processing. In this embodiment, the machine learning model is used to convert the modeling scenario information described in natural language into scenario description information described in markup language. In the specific implementation process, the machine learning model includes, but is not limited to, large language models (LLMs), feed forward neural network language models (FFNNLMs), recurrent neural network language models (RNNLMs), long short-term memory (LSTM) models, recurrent neural networks (RNNs), Transformer models, etc.

[0044] Among them, the large language model is a language model constructed by a deep neural network containing more than tens of billions of weights, and usually uses self-supervised learning methods to be trained through a large amount of unlabeled text. Based on the large language model, human-computer interaction based on natural language can be realized, such as various tasks from understanding to generation, such as question answering, classification, summarization, translation, chatting, etc.

[0045] A prompt is an input instruction used to guide or stimulate a machine learning model such as a large language model to complete a specific task. Usually, the prompt contains the context related to the input information, which can help the machine learning model better understand the intention of the input and generate the correct output. In this embodiment, the prompt is specifically used to help the machine learning model more accurately understand the modeling scenario description information input in natural language and accurately output the modeling scenario description information in markup language, that is, to achieve accurate language conversion.

[0046] An embedding vector is a vector composed of real numbers and can be used to represent various objects such as text, music, and video. In the field of natural language processing, through embedding, high-dimensional data (such as text, audio, etc.) is mapped to a low-dimensional digital space (i.e., the embedding vector). On the one hand, context semantic information can be extracted from natural language. On the other hand, the extracted context information can be vectorized and converted into a data representation form that the machine learning model can process, so as to perform vector operations such as calculating cosine similarity later.

[0047] In actual modeling, for digital twin modeling of various types of physical scenarios, in addition to digital modeling of physical objects in the physical scenario, it also includes modeling of the physical environment in which the physical objects are located. Given the diversity of physical objects and the physical environment, after the data collection of the physical scenario is completed, a relatively large amount of human cost is still required in the data display stage to construct the digital twin model.

[0048] Taking the physical scenario of a factory as an example, in the process of digital twin modeling of a factory, since the deployment of workshops, production lines, and equipment in the factory is flexible and changeable, and the equipment structure is complex, in the data display stage of digital twin modeling, professional designers or engineers are usually required to generate the digital twin model corresponding to the factory based on on-site surveying and mapping, or various CAD formats, three-dimensional general formats, Digital Surface Model (DSM for short), and dense three-dimensional point clouds, etc. In the subsequent use process, when the actual layout of the factory changes, if you want to adjust the layout of the digital twin model of the factory, it also needs to be completed by professional personnel, and the user experience is not good.

[0049] To solve at least one of these problems, in the embodiments of the present invention, a digital twin modeling method based on a machine learning model is proposed. This solution decouples digital twin modeling into two parts: language conversion and model rendering, realizes digital twin modeling based on natural language, and realizes a standardized modeling scenario description by converting the first modeling scenario information described in natural language into the second scenario description information described in a markup language by defining a markup language with a specific markup format, which has universality. Through this automated digital twin modeling method, the efficiency of digital twin modeling is improved, and the technical operation difficulty of digital twin modeling is reduced. In addition, in the embodiments of the present invention, through markup language conversion, a standardized modeling scenario information description method is provided, and users can adjust the content in the digital twin page by directly adjusting the modeling scenario description information described in the markup language, with low adjustment operation difficulty and convenient operation.

[0050] Generally speaking, in the language conversion stage, "machine learning model" and "markup language" are introduced to realize the conversion of natural language into markup language. Specifically, according to the obtained first modeling scenario information input in natural language, the digital twin model information corresponding to the object to be modeled, and the syntax description information of the preset markup language, the machine learning model converts the first modeling scenario information and the digital twin information corresponding to the object to be modeled into the second modeling scenario information, where the second modeling scenario information is a markup language. Since the markup language has a specific markup format and is not as complex and changeable as natural language, the machine can understand the markup language more accurately.

[0051] In the model rendering stage, a "markup language parser" is introduced. Through a rendering engine with the ability to parse markup languages, it can quickly and accurately parse the scene information corresponding to the scene to be modeled from the second modeling scene information, that is, the digital twin information corresponding to the object to be modeled and the spatial environment description information to which the object to be modeled belongs, so as to efficiently render a digital twin page corresponding to the second modeling scene description information.

[0052] The digital twin modeling solution provided by the embodiments of the present invention will be introduced and described below.

[0053] For ease of understanding, in the following embodiments, a large language model is taken as an example of the machine learning model for illustration.

[0054] Figure 1 It is a schematic diagram of the hardware execution environment of a digital twin modeling method provided by the embodiments of the present invention. As Figure 1 shown, the hardware execution environment of this digital twin modeling method can be composed of a client device 101 and / or a server device 102, and the client device 101 is communicatively connected to the server device 102.

[0055] The client device 101 can be a certain type of terminal device for independent use, such as a smart phone, a tablet computer, a PC, etc., or can also be two or more terminals used in combination, such as a virtual reality device 101a and a smart phone 101b. The server device 102 can be a server of a content provider or a cloud server of a cloud service provider.

[0056] Optionally, when the client device 101 consists of a virtual reality device 101a and a smart phone 101b, the execution process of the above digital twin modeling method can be: the user inputs the modeling scene description information to the smart phone 101b in natural language, and the smart phone 101b generates a corresponding digital twin page based on the input modeling scene description information and transmits it to the virtual reality device 101a to display the digital twin page on the screen presented by the virtual reality device 101a. At this time, it can be understood that the virtual reality device 101a and the smart phone 101b are communicatively connected, and the above digital twin page is transmitted based on this communication connection.

[0057] Or, optionally, after the user inputs the modeling scene description information to the smart phone 101b, the smart phone 101b can transmit the modeling scene description information to the server device 102 to complete the generation of the corresponding digital twin page through the server device 102, and feedback the generated digital twin page to the smart phone 101b for display, or the smart phone 101b transmits the digital twin page to the virtual reality device 101a for display.

[0058] In practical applications, the above-mentioned server device 102 can be an independent physical server or a physical server cluster maintained by a content provider, or a cloud server maintained by a cloud service provider - referred to as a computing node. Figure 2 The figure is a schematic diagram of a cloud computing environment for a digital twin modeling method provided by an embodiment of the present invention. In the Figure 2 cloud computing environment shown, it may include several ( Figure 2 such as 201-1, 201-2,... shown in the figure) computing nodes (cloud servers) deployed distributively. Each computing node has processing resources such as computing and storage. In the cloud computing environment, multiple computing nodes can be organized to provide a certain service. Of course, a single computing node can also provide one or more services, such as Figure 2 services A, B, C, and D shown in the figure. The way to provide the service in the cloud computing environment can be to provide a service interface externally, and the client device calls the service interface to use the corresponding service. The service interface includes forms such as a Software Development Kit (SDK) and an Application Programming Interface (API).

[0059] The above services are deployed according to various virtualization technologies supported by the cloud computing environment, such as virtualization technologies based on virtual machines and containers. Taking the virtualization technology based on containers as an example, several containers corresponding to a service can be assembled into a container group (pod). For example, Figure 2 service B shown in the figure can be configured with one or more pods. Each pod can include a proxy and one or more containers. One or more containers in the pod are used to process requests related to one or more corresponding functions of the service, and the proxy in the pod is used to control network functions related to the service, such as routing and load balancing.

[0060] During the operation process, when executing requests from the client device, it may be necessary to call one or more services in the cloud computing environment. When executing one or more functions of a service, it may be necessary to call one or more functions of another service. As Figure 2 shown, after service A receives a request sent by the client device, it can call service B, and service B can request service D to execute one or more functions.

[0061] In the above cloud computing environment, an embodiment of the present invention provides an application schematic diagram of a digital twin modeling method as Figure 3 shown.

[0062] In Figure 3In a cloud computing environment, a digital twin modeling service and corresponding service interfaces are provided. A client device calls the service interface to trigger a request to the digital twin modeling service. The request includes modeling scenario description information input by the user in natural language, and the modeling scenario description information includes description information of the object to be modeled and spatial environment description information of the object to be modeled. In response to the request, the cloud computing environment determines a computing node that provides the digital twin modeling service for responding to the request, and uses the processing resources in the computing node to generate a digital twin page corresponding to the modeling scenario information, so as to feedback the digital twin page to the client device for display.

[0063] The following combines Figure 3 and Figure 4 to introduce in detail the execution process of the digital twin modeling method provided by the embodiments of the present invention. The digital twin modeling method can be executed either by the above-mentioned client device or by the computing node in the above-mentioned cloud computing environment.

[0064] Figure 4 is a flowchart of a digital twin modeling method provided by an embodiment of the present invention. As Figure 4 shown, the method includes the following steps:

[0065] 401. Obtain first modeling scenario description information input in natural language. The first modeling scenario description information includes description information of the object to be modeled and spatial environment description information of the object to be modeled.

[0066] 402. Generate a prompt word according to the first modeling scenario description information, the syntax description information of a preset markup language, and the digital twin model information corresponding to the object to be modeled; wherein, the syntax description information includes the markup formats of the modeling object and the spatial environment to which the modeling object belongs.

[0067] 403. Input the prompt word into a large language model to obtain second modeling scenario description information output by the large language model in the markup language. The second modeling scenario description information includes digital twin model information corresponding to the object to be modeled and spatial environment description information.

[0068] 404. Use a rendering engine with markup language parsing ability to render a digital twin page corresponding to the second modeling scenario description information.

[0069] Refer to Figure 3, in this embodiment, the modeling process of digital twin modeling is decoupled into two stages: a language conversion stage and a model rendering stage. Among them, the language conversion stage is used to perform a normative expression on the modeling scenario description information input by the user, that is, to convert the first modeling scenario description information input in natural language into the second modeling scenario description information described in markup language through a large language model. The model rendering stage is used to automatically generate a digital twin page corresponding to the modeling scenario based on the normative expression of the modeling scenario information (i.e., the second modeling scenario description information).

[0070] Among them, in the above language conversion stage, the application of a large language model is involved. It can be understood that in order to make the large language model adapt to a specific application scenario and improve the performance of the large language model in a specific task, it is usually necessary to pre-train the large language model. To facilitate the understanding of the digital twin modeling method provided in this embodiment, in this embodiment, we first focus on the application of the large language model in the digital twin modeling process, and the training process of the large language model will be described in detail in subsequent embodiments.

[0071] As Figure 3 shown, in the language conversion stage, generally speaking, it is necessary to convert the first modeling scenario information input in natural language into the second modeling scenario description information described in markup language.

[0072] To achieve this conversion process, it is necessary to pre-define a markup language. Specifically, the grammar description information corresponding to the markup language is defined. Generally speaking, it is necessary to define what markup formats are used to identify different objects (such as: text objects, picture objects, table objects, etc.) in the markup language. In this embodiment, the modeling scenario includes modeling objects and the spatial environment to which the modeling objects belong. Correspondingly, the language description information of the markup language includes: the markup formats of the modeling objects and the spatial environment to which the modeling objects belong.

[0073] In an optional embodiment, the markup language and the grammar description information of the markup language can be customized by the user based on usage requirements, that is, to define a brand-new markup language and the grammar description information of the markup language; or, based on an existing markup language, define its corresponding grammar description information. Taking Markdown (a lightweight markup language) as an example, it is possible to customize what different markup formats are used to identify the first-level headings, second-level headings, etc. in Markdown. For example, the first-level headings are identified by blue dots, and the second-level headings are identified by yellow horizontal lines, etc. If the spatial environment to which the modeling object belongs is configured as a first-level heading, then the spatial environment to which the modeling environment belongs is identified by blue dots. Similarly, if the modeling object is configured as a second-level heading, then the modeling object is identified by a yellow horizontal line.

[0074] In this embodiment, in addition to the markup language, it is also necessary to pre-obtain the digital twin models corresponding to various modeling objects, that is, to establish a digital twin model library. Optionally, the pre-constructed digital twin models can be classified according to scenarios. For example, for those corresponding to industrial scenarios, they can include digital twin models corresponding to various production equipment, etc. Thus, when performing digital twin modeling, the digital twin model corresponding to the object to be modeled can be obtained from the digital twin model library, and by arranging the digital twin model of the object to be modeled, a digital twin page corresponding to the modeling scenario can be generated.

[0075] In the specific digital twin modeling process, when digital twin modeling of a certain scenario is required, first, describe the scenario in natural language, that is, input the first modeling scenario description information in natural language. Among them, the first modeling scenario information includes: description information of the object to be modeled (such as: equipment name, equipment operation parameters, etc.), and spatial environment description information of the space to which the object to be modeled belongs (such as: the area, function, etc. of each of the multiple regions included in the spatial environment).

[0076] Then, based on the description information of the object to be modeled in the first modeling scenario description information, obtain the digital twin model information corresponding to the object to be modeled from the pre-obtained digital twin models; and generate a prompt for inputting into the large language model according to the first modeling scenario description information, the syntax description information of the preset markup language, and the digital twin model information corresponding to the object to be modeled. Among them, by generating the prompt, the large language model can better understand the semantics expressed by the first modeling scenario description information.

[0077] After that, input the prompt into the large language model so that the large language model outputs the second modeling scenario description information in the markup language according to the information corresponding to the prompt, completing the conversion of the first modeling scenario information input in natural language into the second modeling scenario description information described in the markup language. Among them, the second modeling scenario description information includes: digital twin model information corresponding to the object to be modeled and spatial environment description information.

[0078] Since the markup language has a predefined standardized markup format, which is more conducive to machine understanding, a corresponding markup language parser can also be set to match the markup language for understanding the second modeling scenario description information in this embodiment. Optionally, the markup language parser can be built into the rendering engine so that the rendering engine has the markup language parsing ability.

[0079] In practical applications, during the model rendering phase, after the rendering engine obtains the second modeling scene description information, on the one hand, it parses the digital twin model information corresponding to the object to be modeled in the second modeling scene information, and on the other hand, it parses the marking formats of the object to be modeled and the spatial environment to which the object to be modeled belongs, so as to determine the cascading relationship between the object to be modeled and the spatial environment, as well as the deployment location of the digital twin model corresponding to the object to be modeled, and generate a digital twin page corresponding to the modeling scene.

[0080] Furthermore, the actual physical information of the object to be modeled in the physical space can be configured for the digital twin model corresponding to the object to be modeled in the digital twin page, such as: device parameters, the current operating state of the device, etc., so that users can obtain the actual operating conditions of the object to be modeled based on the digital twin model in the digital twin page.

[0081] Optionally, the digital twin model in the digital twin page can also establish a two-way data transmission channel with the object to be modeled in the physical space, so that not only can the information of the object to be modeled in the physical space be displayed through the digital twin model, but also the state of the object to be modeled in the physical space can be controlled through the digital twin model.

[0082] For ease of understanding, this embodiment takes the physical scenario of a factory as an example for illustration. It can be understood that a factory can be divided into production lines, workshops, equipment, etc., and there is a cascading relationship between them. For example: Factory A contains 2 production lines, Factory B has 3 workshops, Workshop C contains 4 production lines, and Production Line D contains 2 devices, etc.

[0083] Suppose the user wants to perform digital twin modeling on 2 production lines and inputs the first modeling scene description information in natural language: "I have 2 production lines, each production line has a generator, and production line 2 has an induction cooker". Among them, the generator and the induction cooker are the objects to be modeled, and the 2 production lines (including production line 2) are the spatial environment.

[0084] If the preset markup language is Markdown, and the markup format of the modeling object in the syntax description information of Markdown is "--", and the markup format of the spatial environment to which the modeling object belongs is "●", then after generating a prompt word based on the first modeling scene description information, the syntax description information of Markdown, and the digital twin model information corresponding to the object to be modeled (generator and induction cooker), the second modeling scene description information output by the large language model in Markdown can be expressed as:

[0085] "● Production Line 1

[0086] -- Generator

[0087] ● Production Line 2

[0088] —— Generator

[0089] —— Induction cooker”

[0090] Among them, the digital twin model information of the generator is associated with the generator, and the digital twin model information of the induction cooker is associated with the induction cooker.

[0091] After the rendering engine with Markdown parsing ability obtains the second modeling scene description information, it is parsed that there are a total of 2 production lines, namely production line 1 and production line 2, and there is no nested relationship between the production lines. Production line 1 and production line 2 are parallel; the two production lines involve a total of 3 devices. Production line 1 includes 1 generator, and production line 2 includes 1 generator and 1 induction cooker.

[0092] Based on the cascading relationship between the production lines and devices parsed above, the position coordinates of each device relative to the origin can be determined. Further, according to the digital twin model information corresponding to each device parsed, the digital twin models corresponding to each device are respectively deployed at the position coordinates corresponding to the device to obtain the digital twin pages of 2 production lines, as Figure 5 shown.

[0093] Figure 5 It is a schematic diagram of a digital twin page provided by an embodiment of the present invention. It should be noted that, Figure 5 In the digital twin page shown, the digital twin models corresponding to the generator and the induction cooker are only a schematic representation. In actual applications, their corresponding digital twin models are closer to their real appearance in the physical world.

[0094] In the physical scenario of the factory, after generating the digital twin page, through the integration of information technology (IT) and operation technology (OT), attribute data or index data related to the device can be bound to the digital twin model corresponding to the device in the digital twin page, so as to realize the management and analysis of the devices in the factory, realize intelligent production and management, and improve the production efficiency and quality of the devices.

[0095] In summary, this embodiment provides a digital twin modeling method, through which digital twin modeling based on natural language can be achieved. In the digital twin modeling process provided in this embodiment, the modeling scenario information is described in natural language, with relatively low professional requirements for users. Users can conveniently and quickly input the modeling scenario description information (i.e., the first modeling description scenario information). Subsequently, without the need for secondary operations by the user, it can automatically generate a prompt for inputting into the large language model based on the first modeling scenario description information, the syntax description information of the preset markup language, and the digital twin model information of the object to be modeled, so that the large language model outputs the second modeling scenario description information in the preset markup language based on the prompt, completing the normalization process of natural language. Finally, based on the second modeling scenario description information with a specific markup format, the rendering engine can quickly and accurately parse the cascading relationship between the object to be modeled and the spatial environment to which the object to be modeled belongs in the modeling scenario, perform position layout on the digital twin modeling corresponding to the object to be modeled, and render a digital twin page corresponding to the second modeling scenario description information, thereby improving the modeling efficiency of digital twin modeling. In addition, since the digital twin model information corresponding to the object to be modeled has been pre-configured, in different modeling scenarios, the same object to be modeled can reuse the pre-configured digital twin model information without having to regenerate the corresponding digital twin model every time modeling is performed, thus also saving the modeling time of digital twin modeling and improving the digital twin modeling efficiency.

[0096] The above embodiment introduced the digital twin modeling method. Next, the training process of the large language model used in digital twin modeling will be described.

[0097] Figure 6 The following is a training flowchart of a large language model provided by an embodiment of the present invention. As Figure 6 shown, it includes at least the following steps:

[0098] 601. Obtain the prior knowledge information of the target domain, the digital twin model dataset information of the target domain, and the syntax description information of the preset markup language.

[0099] Among them, the target domain matches the application domain corresponding to the first modeling scenario description information. For example, the target domain is the same as the application domain corresponding to the first modeling scenario description information, or the target domain includes the application domain corresponding to the first modeling scenario description information.

[0100] 602. Fine-tune and train the large language model according to the prior knowledge information, the digital twin model dataset information, and the syntax description information.

[0101] In this embodiment, the purpose of fine-tuning the large language model is to enable the large language model to have better language conversion (from natural language to markup language) performance in the target domain.

[0102] Among them, the prior knowledge information of the target domain includes: industry terms, common knowledge, etc. of the target domain. Taking the industrial production domain as an example of the target domain, the corresponding prior knowledge information can include industry terms in the industrial production domain, such as: the concepts of production line, workshop, process, equipment, etc.

[0103] It can be understood that based on the prior knowledge information of the industrial production domain, further, the association relationships (which can also be understood as cascade relationships, dependency relationships, or subordination relationships, etc.) between different industry terms can be refined. For example: a factory can have workshops under it, and corresponding production lines can be set up under the workshops. Whether it is the prior knowledge information of the target domain or the association relationships obtained based on the prior knowledge information, they are all beneficial for the large language model to perceive the target domain to better execute tasks.

[0104] The digital twin model dataset information of the target domain refers to the set of digital twin models of the modeling objects in the target domain. For example: the set of digital twin models corresponding to various different production equipment in the industrial production domain. The digital twin models of the modeling objects in the target domain can be pre-constructed before digital twin modeling. Optionally, the name of the modeling object can be associated and stored with the resource path of the corresponding digital twin model, and the digital twin model dataset can be formed in the form of "modeling object name - model resource link".

[0105] The syntax description information of the preset markup language can refer to the introduction in the foregoing embodiment, and will not be elaborated in this embodiment.

[0106] As an optional fine-tuning training method, after obtaining the prior knowledge information of the target domain, the digital twin model dataset information of the target domain, and the syntax description information of the preset markup language, through the embedding encoding layer (Embedding layer) in the large language model, the prior knowledge information, the digital twin model dataset information, and the syntax description information can be processed for embedding vector encoding to obtain the corresponding vector database.

[0107] Among them, the vector database includes the embedding vectors corresponding to different digital twin models, the embedding vectors corresponding to the syntax description information, and the embedding vectors related to the spatial environment extracted from the prior knowledge information.

[0108] Among them, the embedding vectors related to the spatial environment extracted from the prior knowledge information can be understood as the embedding vectors corresponding to the association relationships refined from industry terms. For example: the embedding vectors corresponding to the association relationship "factory-workshop-production line", or the embedding vectors corresponding to the association relationship "factory: workshop, equipment", etc.

[0109] In this embodiment, in the pre-training stage, by fine-tuning the large language model, a vector database corresponding to the target domain is obtained. The vector database contains all the embedding vectors corresponding to the target domain. When performing digital twin modeling on any modeling scenario in the target domain, the target embedding vector matching the modeling scenario can be queried from the vector database. Based on the target embedding vector, the large language model can better understand the first modeling scenario description information and more accurately convert the first modeling scenario description information into the second modeling scenario description information. It can be understood that the improvement of the language conversion accuracy rate is beneficial to ensuring the accuracy of the finally generated digital twin page.

[0110] It should be noted that the training process of the above large language model can also be applied to other machine learning models except the large language model in this embodiment. The training process of the large language model is only an exemplary illustration of the machine learning model training process in this embodiment.

[0111] Based on the above fine-tuning training process of the large language model, the embodiment of the present invention provides a Figure 7 flowchart of a prompt word generation method as shown in Figure 7 shown, Figure 4 Step 402 in

[0112] 701. Determine the first embedding vector corresponding to the first modeling scenario description information through the embedding encoding layer in the large language model.

[0113] 702. Query from the vector database to obtain a second embedding vector that matches the first embedding vector. The second embedding vector includes the embedding vector of the digital twin model that matches the object to be modeled, the embedding vector corresponding to the syntax description information, and the embedding vector that matches the spatial environment description information of the object to be modeled.

[0114] 703. Generate a prompt word according to the second embedding vector and the first modeling scenario description information.

[0115] In this embodiment, in combination with Figure 8 to Figure 7 shown prompt word generation method is described. Figure 8 is an application schematic diagram of another digital twin modeling method provided by the embodiment of the present invention.

[0116] As shown in Figure 8As shown, first, the first modeling scenario description information input in natural language is input into the embedding encoding layer of the large language model to perform word segmentation and embedding vector encoding processing on the first modeling scenario description information, generating corresponding first embedding vectors. Among them, the first embedding vectors include embedding vectors related to the object to be modeled and embedding vectors related to the spatial environment of the object to be modeled.

[0117] After that, for example, by calculating the similarity (such as cosine similarity) between the first embedding vector and the embedding vectors in the vector database, the embedding vectors in the vector database with a similarity greater than the set similarity threshold to the first embedding vector can be screened out as the second embedding vectors. Among them, the corresponding second embedding vectors include: the embedding vectors of the digital twin model matching the object to be modeled, the embedding vectors corresponding to the syntactic description information, and the embedding vectors matching the spatial environment description information of the object to be modeled.

[0118] The following elaborates on the acquisition process of the second embedding vectors.

[0119] In practical applications, the expressions of natural language are often diverse. For the same object to be modeled, the expressions of different users may vary. For example, for the modeling object of "generator", some users will express it completely as "generator" in the first modeling scenario information, while some users will express it as "motor". By vectorizing the first modeling scenario information and calculating the similarity with the vectors in the vector database, the embedding vectors of the digital twin models matching the object to be modeled can be more comprehensively screened out from the embedding vectors corresponding to different digital twin models included in the vector database. For example, even if the user's expression is "motor", the embedding vectors of the digital twin model corresponding to the modeling object with a certain similarity to "motor", that is, the embedding vectors of the digital twin model corresponding to "generator", can be queried from the vector database, so as to ensure that the digital twin model of the object to be modeled "motor" can be obtained.

[0120] Similarly to the object to be modeled, the spatial environment to which the object to be modeled belongs, described by the user in natural language, is often not standardized. For example, the association relationships between spatial environments in the first modeling scenario description information are chaotic, etc. By vectorizing the first modeling scenario information and calculating the similarity with the vectors in the vector database, it is possible to screen out the embedding vectors that match the spatial environment description information of the object to be modeled from the embedding vectors related to the spatial environment extracted from the prior knowledge information contained in the vector database. For example, assuming that the embedding vectors related to the spatial environment contained in the vector database are the embedding vectors corresponding to the subordinate relationship of "factory-workshop-production line-equipment", then whether the user expresses "equipment is configured in the factory" or "production lines are included in the factory" in the first modeling scenario description information, the embedding vectors corresponding to "factory-workshop-production line-equipment" can be matched through the embedding vectors of "factory", "production line", and "equipment", so as to obtain the correct association relationship between the spatial environments to which the object to be modeled belongs.

[0121] Generally, to ensure the accuracy of language conversion, the embedding vector corresponding to the syntactic description information in the second embedding vector is the embedding vector corresponding to all the syntactic description information in the vector database.

[0122] Finally, generate a prompt for inputting into the large language model based on the first modeling scenario description information and the second embedding vector.

[0123] In this embodiment, by calculating the similarity between vectors and screening out the second embedding vector from the vector database, on the one hand, the processing pressure on the large language model can be reduced, that is, the large language model does not need to perform language conversion based on all the information in the target domain; on the other hand, enough embedding vectors related to the first embedding vector can be obtained from the vector database to serve as prior knowledge for the large language model to perform language conversion, helping the large language model better understand the first modeling scenario description information and ensuring that the large language model more accurately converts the first modeling scenario description information into the second modeling scenario description information based on the prompt.

[0124] In the foregoing embodiment, the language conversion process of converting the first modeling scenario information into the second modeling scenario information based on the large language model is specifically described. Next, the model rendering process will be specifically introduced.

[0125] Figure 9 It is a flowchart of another digital twin modeling method provided by an embodiment of the present invention. As Figure 9 shown, it at least includes the following steps:

[0126] 901. Obtain the first modeling scenario description information input in natural language, where the first modeling scenario description information includes the description information of the object to be modeled and the spatial environment description information of the object to be modeled.

[0127] 902. Generate a prompt based on the first modeling scenario description information, the syntax description information of the preset markup language, and the digital twin model information corresponding to the object to be modeled; wherein, the syntax description information includes the markup formats of the modeling object and the spatial environment to which the modeling object belongs.

[0128] 903. Input the prompt into the large language model to obtain the second modeling scenario description information output by the large language model in the markup language. The second modeling scenario description information includes the digital twin model information corresponding to the object to be modeled and the spatial environment description information. Among them, the digital twin model information includes the loading link of the digital twin model.

[0129] 904. Use a rendering engine with the ability to parse the markup language. Based on the spatial environment description information of the object to be modeled included in the second modeling scenario description information, determine the corresponding scene graph and the position of the scene graph in the digital twin page, so as to render the scene graph in the digital twin page.

[0130] 905. Use a rendering engine with the ability to parse the markup language. Based on the loading link of the digital twin model corresponding to the object to be modeled included in the second modeling scenario description information, load the digital twin model, determine the position of the digital twin model in the scene graph, so as to render it in the scene graph.

[0131] Among them, the specific implementation processes of steps 901 to 903 can refer to the foregoing embodiments, and will not be elaborated in this embodiment.

[0132] In this embodiment, the process of the rendering engine rendering the digital twin page can be disassembled into: a spatial environment rendering stage and an object-to-be-modeled rendering stage. Among them, the spatial environment rendering stage is used to render the scene graph corresponding to the spatial environment into the digital twin page to ensure that the scene graph can be displayed in a certain way in the digital twin page, for example: the scene graph is displayed in the central area of the digital twin page. The object-to-be-modeled rendering stage is used to render the digital twin model corresponding to the object to be modeled in the scene graph.

[0133] Generally, to ensure the normal rendering of the scene graph and the digital twin page, usually the rendering engine communicates with the rendering resource manager through a communication interface to obtain static resources such as scenes, materials, textures, and cameras required for rendering, as well as the model file corresponding to the digital twin model, for example: a model file in GLTF format. Among them, the model file of the rendering resource manager is dynamically loaded based on the current digital twin model to be rendered.

[0134] In the space environment rendering stage, since the space environment to which the object to be modeled belongs may be divided into multiple different space regions, when performing digital twin modeling, it is necessary to reasonably plan the layout of the scene graph corresponding to the space environment on the digital twin page based on the distribution of different space regions in the physical space to ensure the display effect of the digital twin page.

[0135] Still taking Figure 5 the situation shown as an example, Figure 5 the space environment description information of the object to be modeled included in the corresponding second modeling scene description information indicates that the space environment to which the object to be modeled belongs consists of two parts, namely production line 1 and production line 2, and production line 1 and production line 2 are in a parallel relationship. Based on this, taking the horizontal center dividing line of the digital twin page as a reference, it can be determined that the scene graphs corresponding to production line 1 and production line 2 are symmetrically distributed on both sides of the horizontal center dividing line. After determining the positions of the scene graphs corresponding to production line 1 and production line 2 in the digital twin page, the rendering engine renders the scene graphs corresponding to production line 1 and production line 2 into the digital twin page based on the static resources such as the pre-acquired scenes and materials. It should be noted that Figure 5 only an exemplary illustration of a position distribution situation of the scene graph in the digital twin page is given, and it is not limited to this.

[0136] In addition, Figure 5 in the scene graphs of production line 1 and production line 2 are both blank rectangular areas. In practical applications, optionally, the scene graph can also be set as a grid area, or, according to the actual texture information of the space environment in the physical space, the corresponding scene graph can be rendered. This embodiment does not limit the display form of the scene graph.

[0137] In this embodiment, the digital twin model information includes the loading link of the digital twin model. Based on this loading link, the model file (such as a glb file) used to generate the corresponding digital twin model can be obtained.

[0138] In the rendering stage of the object to be modeled, the rendering engine loads the model file of the digital twin model of the object to be modeled according to the loading link of the digital twin model corresponding to the object to be modeled in the second modeling scene description information, and renders the digital twin model at the corresponding position in the scene graph based on the model file and static resources such as materials and textures.

[0139] For example, in Figure 5In the schematic scenario, the rendering engine loads the model files of the generator and the induction cooker respectively according to the loading links of the digital twin models corresponding to the generator and the induction cooker. After determining the positions corresponding to the generator in the scene graphs of production line 1 and production line 2 respectively, and the position of the induction cooker in the scene graph of production line 2, based on the model files of the generator and the induction cooker respectively, as well as static resources such as materials, textures, and cameras, the two generators and one induction cooker are respectively rendered to the corresponding positions in the scene graphs of production line 1 and production line 2.

[0140] In an optional embodiment, in order to improve the rendering efficiency of the digital twin model, if it is determined that a certain digital twin model has been loaded, the already loaded digital twin model is reused for rendering. For example, Figure 5 In the schematic scenario, both production line 1 and production line 2 contain generators. When rendering the digital twin model corresponding to the generator, the rendering engine only needs to load the model file of the digital twin model of the generator once.

[0141] In this embodiment, the step-by-step rendering of the scene graph corresponding to the spatial environment and the digital twin model corresponding to the model to be rendered can ensure the display effect of the digital twin model in the digital twin page by controlling the display position of the scene graph in the digital twin page, that is, fully displaying the digital twin model from a better perspective.

[0142] It can be understood that to ensure the consistency between the digital twin page and the real physical scene, when the physical scene changes, the digital twin page needs to be updated synchronously accordingly. In this embodiment, the digital twin page can be updated by adjusting the second modeling scene description information described in the markup language.

[0143] As an optional specific implementation manner, the adjusted third modeling scene description information can be determined first based on the user's adjustment operation on the second modeling scene description information; then, a rendering engine with the ability to parse the markup language is used to render the digital twin page corresponding to the third modeling scene description information as the updated digital twin page.

[0144] In an optional embodiment, in order to ensure the accuracy of the rendering engine's parsing of the third modeling scene description information, after determining the third modeling scene description information, the third modeling scene description information can be pre-parsed to verify whether its syntax structure is correct. When it is determined that the syntax structure of the third modeling scene description information is correct, it is transmitted to the rendering engine with the ability to parse the markup language.

[0145] In another alternative embodiment, to improve the rendering efficiency of the digital twin model, when rendering the digital twin page corresponding to the third modeling scene description information, if it is determined that the corresponding digital twin model has been loaded based on the loading link of the digital twin model corresponding to the object to be modeled included in the third modeling scene description information, the already loaded digital twin model is reused for rendering.

[0146] For ease of understanding, for example, still taking the Figure 5 scenario shown as an example, assume that an induction cooker is added to production line 1 in the physical scene. Then, the Figure 5 corresponding second modeling scene information:

[0147] "● Production line 1

[0148] —— Generator

[0149] ● Production line 2

[0150] —— Generator

[0151] —— Induction cooker"

[0152] is modified to the following third modeling scene information:

[0153] "● Production line 1

[0154] —— Generator

[0155] —— Induction cooker

[0156] ● Production line 2

[0157] —— Generator

[0158] —— Induction cooker"

[0159] After that, a rendering engine with markup language parsing capabilities is used to render the digital twin page corresponding to the third modeling scene description information as the updated digital twin page, as Figure 10 shown. Figure 10 This is another schematic diagram of the digital twin page provided by the embodiment of the present invention.

[0160] Among them, since the digital twin models corresponding to the generator and the induction cooker have been loaded when rendering the digital twin page corresponding to the second modeling scene information, when rendering the digital twin page corresponding to the third modeling scene information, the already loaded digital twin models corresponding to the generator and the induction cooker can be reused for rendering, without repeated loading, thereby saving the loading time of the digital twin model and improving the rendering efficiency.

[0161] The digital twin modeling device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art can understand that these devices can all be configured by using commercially available hardware components through the steps taught by this solution.

[0162] Figure 11 The following is a schematic structural diagram of a digital twin modeling device provided by an embodiment of the present invention, as Figure 11 shown. The device includes: an acquisition module 11, a processing module 12, and a rendering module 13.

[0163] The acquisition module 11 is used to acquire first modeling scene description information input in natural language, and the first modeling scene description information includes description information of the object to be modeled and description information of the spatial environment of the object to be modeled.

[0164] The processing module 12 is used to generate a prompt word according to the first modeling scene description information, the grammar description information of the preset markup language, and the digital twin model information corresponding to the object to be modeled; wherein, the grammar description information includes the markup formats of the modeling object and the spatial environment to which the modeling object belongs; input the prompt word into the machine learning model to obtain second modeling scene description information output by the machine learning model in the markup language, and the second modeling scene description information includes the digital twin model information corresponding to the object to be modeled and the spatial environment description information.

[0165] The rendering module 13 is used to render a digital twin page corresponding to the second modeling scene description information by using a rendering engine with markup language parsing ability.

[0166] Optionally, the device further includes a model training module, which is used to acquire prior knowledge information of the target field, the digital twin model dataset information of the target field, and the grammar description information, and the application field corresponding to the first modeling scene description information matches the target field; fine-tune and train the machine learning model according to the prior knowledge information, the digital twin model dataset information, and the grammar description information.

[0167] Optionally, the model training module is specifically used to perform embedded vector encoding processing on the prior knowledge information, the digital twin model dataset information, and the grammar description information through the embedded encoding layer in the machine learning model to obtain a corresponding vector database, and the vector database includes embedded vectors corresponding to different digital twin models, embedded vectors corresponding to the grammar description information, and embedded vectors related to the spatial environment extracted from the prior knowledge information.

[0168] Optionally, the processing module 12 is specifically configured to determine a first embedding vector corresponding to the first modeling scenario description information through an embedding encoding layer in the machine learning model; query a second embedding vector matching the first embedding vector from a vector database, where the second embedding vector includes an embedding vector of a digital twin model matching the object to be modeled, an embedding vector corresponding to the syntax description information, and an embedding vector matching the spatial environment description information of the object to be modeled; and generate a prompt word according to the second embedding vector and the first modeling scenario description information.

[0169] Optionally, the digital twin model information includes a loading link of the digital twin model. The rendering module 13 is specifically configured to determine a scene graph corresponding to the spatial environment description information and the position of the scene graph on the digital twin page based on the spatial environment description information of the object to be modeled included in the second modeling scenario description information, so as to render the scene graph on the digital twin page; load the digital twin model based on the loading link of the digital twin model corresponding to the object to be modeled included in the second modeling scenario description information; and determine the position of the digital twin model in the scene graph for rendering in the scene graph.

[0170] Optionally, the rendering module 13 is further configured to determine a third modeling scenario description information obtained after adjustment based on an adjustment operation of the user on the second modeling scenario description information; and render a digital twin page corresponding to the third modeling scenario description information by using a rendering engine with markup language parsing capabilities.

[0171] Optionally, the rendering module 13 is further specifically configured to, if it is determined that the digital twin model has been loaded based on the loading link of the digital twin model corresponding to the object to be modeled included in the third modeling scenario description information, reuse the already loaded digital twin model for rendering.

[0172] Figure 11 The device shown can execute the steps introduced in the foregoing embodiments. For the detailed execution process and technical effects, refer to the descriptions in the foregoing embodiments, which will not be elaborated here.

[0173] In a possible design, the above Figure 11 The structure of the digital twin modeling device shown can be implemented as an electronic device, as Figure 12 shown. The electronic device may include: a memory 21, a processor 22, and a communication interface 23. Among them, executable code is stored on the memory 21. When the executable code is executed by the processor 22, the processor 22 can at least implement the digital twin modeling method provided in the foregoing embodiments.

[0174] In an alternative embodiment, the electronic device for executing the digital twin modeling method provided by the embodiments of the present invention can be any user terminal, such as a mobile phone, a laptop computer, a PC, or an Extended Reality (XR) device. XR is a collective term for various forms such as virtual reality and augmented reality.

[0175] In addition, the embodiments of the present invention provide a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the digital twin modeling method provided in the foregoing embodiments.

[0176] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative effort.

[0177] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of adding a necessary general hardware platform, and of course, it can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence, or the part that contributes to the prior art can be embodied in the form of a computer product. The present invention can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A digital twin modeling method, characterized in that, Including: Obtain first modeling scenario description information input in natural language, where the first modeling scenario description information includes description information of an object to be modeled and description information of the spatial environment of the object to be modeled; Generate a prompt according to the first modeling scenario description information, the syntax description information of a preset markup language, and the digital twin model information corresponding to the object to be modeled; wherein, the syntax description information includes the markup formats of the modeling object and the spatial environment to which the modeling object belongs; Input the prompt into a machine learning model to obtain second modeling scenario description information output by the machine learning model in the markup language, where the second modeling scenario description information includes the digital twin model information corresponding to the object to be modeled and the spatial environment description information; Use a rendering engine with markup language parsing ability to render a digital twin page corresponding to the second modeling scenario description information.

2. The method according to claim 1, wherein The method further includes: Obtain prior knowledge information of the target domain, the digital twin model dataset information of the target domain, and the syntax description information, where the application domain corresponding to the first modeling scenario description information matches the target domain; Fine-tune and train the machine learning model according to the prior knowledge information, the digital twin model dataset information, and the syntax description information.

3. The method according to claim 2, wherein The fine-tuning and training of the machine learning model according to the prior knowledge information, the digital twin model dataset, and the syntax description information includes: Through the embedding encoding layer in the machine learning model, perform embedding vector encoding processing on the prior knowledge information, the digital twin model dataset information, and the syntax description information to obtain a corresponding vector database, where the vector database includes embedding vectors corresponding to different digital twin models, embedding vectors corresponding to the syntax description information, and embedding vectors related to the spatial environment extracted from the prior knowledge information.

4. The method according to claim 3, wherein The generating a prompt according to the first modeling scenario description information, the syntax description information of a preset markup language, and the digital twin model information corresponding to the object to be modeled includes: Determine a first embedding vector corresponding to the first modeling scenario description information through the embedding encoding layer in the machine learning model; Query from the vector database to obtain a second embedding vector that matches the first embedding vector, where the second embedding vector includes the embedding vector of the digital twin model that matches the object to be modeled, the embedding vector corresponding to the syntax description information, and the embedding vector that matches the spatial environment description information of the object to be modeled; Generate a prompt according to the second embedding vector and the first modeling scenario description information.

5. The method according to claim 1, characterized in that, The digital twin model information includes the loading link of the digital twin model; The rendering a digital twin page corresponding to the second modeling scenario description information includes: Based on the spatial environment description information of the object to be modeled included in the second modeling scenario description information, determine a scene graph corresponding to the spatial environment description information and the position of the scene graph in the digital twin page, so as to render the scene graph in the digital twin page; Load the digital twin model based on the loading link of the digital twin model corresponding to the object to be modeled included in the second modeling scenario description information; Determine the position of the digital twin model in the scene graph for rendering in the scene graph.

6. The method according to claim 5, characterized in that, The method further includes: Based on the adjustment operation of the user on the second modeling scenario description information, determine the third modeling scenario description information obtained after adjustment; Use a rendering engine with markup language parsing ability to render a digital twin page corresponding to the third modeling scenario description information.

7. The method according to claim 6, characterized in that, The rendering of the digital twin page corresponding to the third modeling scenario description information includes: If it is determined that the digital twin model has been loaded based on the loading link of the digital twin model corresponding to the object to be modeled included in the third modeling scenario description information, reuse the already loaded digital twin model for rendering.

8. A digital twin modeling method, characterized in that, Includes: Receive a request triggered by a client device by invoking a digital twin modeling service provided by the cloud. The request includes first modeling scenario description information input by the user in natural language, and the first modeling scenario description information includes description information of the object to be modeled and spatial environment description information of the object to be modeled; Generate a prompt according to the first modeling scenario description information, the syntax description information of the preset markup language, and the digital twin model information corresponding to the object to be modeled; wherein, the syntax description information includes the markup formats of the modeling object and the spatial environment to which the modeling object belongs; Input the prompt into a machine learning model to obtain second modeling scenario description information output by the machine learning model in markup language. The second modeling scenario description information includes the digital twin model information corresponding to the object to be modeled and the spatial environment description information; Use a rendering engine with markup language parsing ability to render a digital twin page corresponding to the second modeling scenario description information; Feed back the digital twin page to the client device for display.

9. An electronic device, characterized in that, Includes: A memory, a processor, and a communication interface; wherein, executable code is stored on the memory. When the executable code is executed by the processor, the processor executes the digital twin modeling method according to any one of claims 1 to 7.

10. A non-transitory machine-readable storage medium, characterized in that, Executable code is stored on the non-transitory machine-readable storage medium. When the executable code is executed by the processor of an electronic device, the processor executes the digital twin modeling method according to any one of claims 1 to 7.

Images