A somatic intelligent twin construction method, device and medium

Abstract

This application discloses a method, device, and medium for constructing an embodied intelligent twin, mainly relating to the field of intelligent twin technology, to address the problems of existing solutions lacking means of representing and understanding the characteristics and time-varying states of the entity itself, and having insufficient deep perception and semantic understanding capabilities for tasks and scenarios. The method includes: acquiring real-time structured corpus of the corresponding device ontology in the current task operation; adding current mechanism data and global state data to the real-time structured corpus and storing it in a preset knowledge corpus; extracting knowledge ontology triples from the preset knowledge corpus; inputting the knowledge ontology triples into a preset base model; using preset rule-based knowledge ontology and preset parsing prompts, outputting inference results; and using the current mechanism data, global state data, and inference results as planning decision inputs to a task decision planning module to generate task control instructions.

Description

Technical Field

[0001] This application relates to the field of intelligent twin technology, and in particular to a method, device and medium for constructing an embodied intelligent twin. Background Technology

[0002] Currently, autonomous operating equipment has become a powerful intelligent tool for replacing human labor in heavy, delicate, and harsh environments, reshaping the global industrial landscape, improving social efficiency, and driving innovation in human lifestyles. As a core technological direction for enhancing the autonomous operation capabilities of equipment, embodied intelligence has received widespread attention from academia and industry in recent years. Embodied intelligent twin systems aim to integrate environmental perception, task planning, and decision-making execution, enabling operating equipment to understand task objectives and operate autonomously in complex and dynamic environments.

[0003] However, embodied intelligent devices for complex tasks still face the following challenges: First, during task planning and decision-making, embodied intelligent devices generally lack the means to represent and understand their ontological characteristics and time-varying states, making it difficult to apply relevant information as key constraints in the planning and decision-making process. Second, the devices lack sufficient deep perception and semantic understanding of tasks and scenarios, making it difficult to transform constraints such as environmental geometry, interaction relationships, and safety rules into computable and reasonable representations, and to combine industry knowledge to perform professional analysis of tasks, thereby forming high-quality planning and decision-making inputs. Summary of the Invention

[0004] This application provides a method, device, and medium for constructing an embodied intelligent twin, in order to solve the problems of existing solutions lacking means to represent and understand its ontological characteristics and time-varying states, and having insufficient deep perception and semantic understanding capabilities for tasks and scenarios.

[0005] Firstly, this application provides a method for constructing an embodied intelligent twin, the method comprising: Establish the connection relationship between the central computing unit of the embodied intelligent twin and the corresponding multimodal sensors, device body, and edge controller of the intelligent twin; The mechanism parameters of multimodal sensors, device body, and edge controller are obtained, and a key element space of the mechanism model is constructed. Based on the key element space of the mechanism model, the current mechanism data is obtained using the mechanism model. The mechanism model includes: intelligent twin geometric model, physical mechanism model, device behavior model, and rule model. Multimodal sensor data is collected using multimodal sensors; based on preset node and edge features, the multimodal sensor data is converted into a running state diagram; based on the data model, a graph attention mechanism is used to process the running state diagram. Layered graph convolutions are used to obtain global state data. Obtain the real-time structured corpus of the corresponding device body in the current task, add the current mechanism data and global state data to the real-time structured corpus, and store it in the preset knowledge corpus; Extract knowledge ontology triples from real-time structured corpora; input the knowledge ontology triples into a preset base model, and output the reasoning results using preset rule-based knowledge ontology and preset parsing prompts; The task decision planning module takes current mechanism data, global state data, and inference results as inputs for planning decisions and generates task control instructions.

[0006] In one implementation of this application, establishing a connection between the embodied intelligent twin central computing unit and the corresponding multimodal sensors, device body, and edge controller of the intelligent twin specifically includes: Configure the central computing unit with a multi-protocol hardware interface to access feedback signals from multimodal sensors, the device body, and the edge controller; connect the multimodal sensors to the sensing interface of the central computing unit through a unified wiring method; connect the operating device body and the edge controller to the interface of the central computing unit through reserved standard cable routing; Based on the communication interface requirements of multimodal sensors, device bodies, and edge controllers, a standardized communication protocol parsing script suitable for multiple devices is obtained; through the standardized communication protocol parsing script, the various types of data uploaded by the interface are formatted and uniformly converted, and mapped into standard data frames within the central computing unit; The connection path is configured as a bidirectional interactive link that includes status uplink and command downlink. The status data link is responsible for transmitting real-time data from multimodal sensors, the device body, and the edge controller to the central computing unit after parsing. This data is used to update the global status data, mechanism model parameters, and semantic reasoning results in the twin. The command distribution link parses the task control commands generated by the task decision planning module. Through the control adaptation and protocol conversion unit, the unified task control commands are converted into execution signals for the device body and distributed to the device body.

[0007] In one implementation of this application, the mechanistic parameters of the multimodal sensor, the device body, and the edge controller are obtained, and a key element space of the mechanistic model is constructed, specifically including: Based on the structure of the mobile equipment itself and the mobile delivery task requirements of the machine tool assembly workshop, a key element space for the mechanism model is constructed: ; in, This represents the set of mechanism parameters of the mobile operation equipment. , , These represent subsets of mechanistic parameters for the perception layer, execution layer, and control layer, respectively; for any layer Its mechanistic parameters are expressed in a unified structured form as follows: ,in For geometric parameters, These are physical property parameters. These are behavioral feature parameters. These are the rule / constraint parameters.

[0008] In one implementation of this application, multimodal sensor data is acquired using a multimodal sensor; based on preset node features and edge features, the multimodal sensor data is converted into a running state diagram, specifically including: Data is acquired using a multimodal sensor. , in For the device body's multimodal sensor at time The observation vector, As a dimension, The sampling time for the current task execution process, multimodal sensor It should include at least: triaxial acceleration, triaxial angular velocity, triaxial magnetic field, and attitude angle quaternion; Data preprocessing is performed on multimodal sensor data, and resampling and sliding window slicing are applied to multimodal sensor data with different sampling frequencies to form standardized sample sequences. ;in, Indicates the first Multimodal sensor data in one window For the device body action / control input sequence within the window, For monitoring labels, Indicates the total number of windows; Standardized multi-source data sequences are constructed into a graph structure, and time intervals are defined. The running status diagram is as follows: ;in, For a moment A set of nodes, nodes Represents preset node elements, This is the set of edges corresponding to the edge elements.

[0009] In one implementation of this application, a graph attention mechanism is used to process the runtime state graph based on a data model. Layered graph convolutions obtain global state data, specifically including: Introducing a graph attention mechanism, defining nodes For nodes Attention score: , in, Represents a node At any moment No. Hidden state / embedding vector of the layer Representing neighboring nodes The corresponding hidden state; For the edge The edge feature vector is used to characterize the node. With nodes Relationship type and strength; The scores were then normalized to obtain the first... Layer attention weights: , in, For attention scoring function, The edge feature vector; Representing neighboring nodes For nodes In the Attention weights of layers; Perform AGNN convolutional updates, defining the message function as follows: Then the node The The hidden state is updated to: , in, For activation functions; This is a message mapping function; Finish After layer graph convolution update, a high-dimensional representation of the node set is obtained. Construct a graph-level representation and obtain time intervals. Graph-level representation of the twin state: , in, Representing the global state data of the embodied intelligent twin at any given moment. This represents the graph-level readout function.

[0010] In one implementation of this application, the real-time structured corpus of the corresponding device ontology in the current task is obtained, the current mechanism data and global state data are added to the real-time structured corpus and stored in a preset knowledge corpus, and knowledge ontology triples are extracted from the preset knowledge corpus, specifically including: Acquire and aggregate structured corpus data, including equipment status data, operating mechanisms, equipment tasks, operating specifications, and safety rules, during the mobile delivery tasks of the machine tool assembly workshop, to form a domain knowledge corpus. : , in, The status data corpus includes status information of the device itself, its location, the work object, the environment, and the task execution. The corpus covers operational mechanisms, including geometric and topological relationships, kinematics / dynamics, and mechanistic knowledge of control mechanisms. The equipment-specific task corpus includes task definitions and objectives, task cases, resources and constraints, task processes, and task knowledge related to process requirements. This includes rule-based corpora covering safety operating procedures, system operation rules, and machine tool assembly rules. Add the corresponding mechanism data and global state data to the structured corpus; For any structured corpus The knowledge extraction process in Natural Language Processing is defined as extracting "entity-relationship-attribute" information as follows: , in, For text encoders These represent "entity-relationship-attribute" information respectively. These represent the processes of extracting information from "entity-relationship-attribute"; After information extraction, entity, relation, and attribute information are organized into device ontology task knowledge triples, forming a set containing entity information. Relationship information With attribute information Knowledge ontology triples: , , , in, An object triple representing an entity-relationship-entity; This represents a data triple consisting of an entity, an attribute, and an attribute value. To get the value for the attribute; The extracted set of triples Indexing and standardizing are performed according to a unified model, and the data is stored in a pre-defined knowledge corpus.

[0011] In one implementation of this application, a knowledge ontology triple is input into a preset base model, and a reasoning result is output using a preset rule-based knowledge ontology and preset parsing prompts, specifically including: Acquire raw rule knowledge, including expert experience rules, historical task rules, and existing specifications. : , in, It is a set of expert experience rules, including experience in the path and passage of mobile operation equipment, experience in the operation and maintenance of actuators, and experience in the safe execution of control links; It is a set of historical task rules, including task flow and process parameter correspondence rules, similar task migration rules, and resource scheduling rules; This is the existing set of rules, including safety regulations, operating procedures, and equipment operation constraints. Using the original rule knowledge, a preset rule knowledge ontology corresponding to the mobile device ontology delivery operation is constructed, and the rule weights in the preset rule knowledge ontology are dynamically updated based on the memory-forgetting model. Using AI models as open-source generative base models Then, a lightweight model is obtained through distillation. ; The adapted model The pre-defined knowledge corpus and pre-defined rule-based knowledge ontology corresponding to the knowledge ontology triples are jointly encapsulated into a knowledge-enhanced reasoning interface: , in, Preset parsing prompts, As a result of reasoning, These are a pre-defined knowledge corpus and a pre-defined rule-based knowledge ontology for mobile device ontology delivery operations. This represents the inference execution function.

[0012] In one implementation of this application, current mechanism data, global state data, and inference results are used as planning decision inputs to a preset control model to generate task control instructions, specifically including: The device's intrinsic mechanism data and global state data are constrained and fused in a unified state space to obtain a fused state. The device body is in a fusion state Together with the inference result 'a', they serve as inputs to the pre-defined control model for planning decisions: , in, For task planning and decision functions, For output task control commands.

[0013] Secondly, this application provides an embodied intelligent twin construction device, the device including: a processor; And a memory that stores executable code, which, when executed, causes the processor to perform an embodied intelligent twin construction method as described above.

[0014] Thirdly, this application provides a non-volatile computer storage medium storing computer instructions thereon, which, when executed, implement an embodied intelligent twin construction method as described above.

[0015] As can be seen from the above technical solutions, this application has the following advantages: By establishing connections between the embodied intelligent twin central computing unit and multimodal sensors, the device itself, and edge controllers, and constructing a key element space for the mechanistic model based on mechanistic parameters, including geometric models, physical mechanism models, device behavior models, and rule models, the system achieves a structured representation of the device's characteristics and real-time state. This mechanism transforms the device's physical attributes, motion constraints, and operating rules into computable mechanistic data, enabling the task planning module to directly perform constraint reasoning based on the device's own dynamic characteristics. This improves the accuracy of the planning decision-making process in perceiving the device's state and the consistency of its response, solving the decision-making disconnect problem caused by the lack of ontological representation in traditional systems.

[0016] By collecting data through multimodal sensors and constructing an operational state graph based on preset node and edge elements, and combining graph attention mechanisms and layered graph convolution algorithms to perform global state modeling of the graph structure, the system achieves semantic fusion and spatial relationship modeling of multi-source heterogeneous information in complex operating environments. This method transforms environmental geometry, interaction object relationships, and dynamic constraints into a computable graph representation, enabling the device to autonomously identify key interaction nodes and potential conflict paths in the task scenario. This enhances the deep perception capability of the operating environment and compensates for the weak semantic understanding of traditional systems in unstructured scenarios.

[0017] By fusing mechanistic data and global state data into a real-time structured corpus of the device ontology, and extracting knowledge ontology triples to input into a pre-defined base model, the system achieves collaborative reasoning between industry knowledge and real-time state by combining rule-based knowledge ontology and parsing prompts. This mechanism explicitly encodes implicit safety rules, process specifications, and operational logic into a reasonable semantic network, enabling the task decision-making module to perform compliance judgments and strategy optimizations based on professional semantics. This improves the industry adaptability and logical rigor of the planning output, providing an interpretable and traceable decision input foundation for complex tasks. Attached Figure Description

[0018] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is a flowchart of a method for constructing an embodied intelligent twin provided in an embodiment of this application.

[0020] Figure 2 This is a schematic diagram of the mechanism model of an embodied intelligent twin provided in an embodiment of this application.

[0021] Figure 3 This is a schematic diagram of an embodied intelligent twin data model provided in an embodiment of this application.

[0022] Figure 4 This is a schematic diagram of the structure of an embodied intelligent twin provided in an embodiment of this application.

[0023] Figure 5 This is a schematic diagram of the internal structure of an embodied intelligent twin construction device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Those skilled in the art should understand that the embodiments described below are merely preferred embodiments of this disclosure and do not imply that this disclosure can only be implemented through these preferred embodiments. These preferred embodiments are merely used to explain the technical principles of this disclosure and are not intended to limit the scope of protection of this disclosure. Based on the preferred embodiments provided by this disclosure, all other embodiments obtained by those skilled in the art without creative effort should still fall within the scope of protection of this disclosure.

[0026] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0027] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0028] The embodiment provides a method for constructing an embodied intelligent twin, such as Figure 1 As shown in the embodiments of this application, the method mainly includes the following steps: Step 110: Establish the connection relationship between the embodied intelligent twin central computing unit and the corresponding multimodal sensors, device body, and edge controller of the intelligent twin.

[0029] In some embodiments, this step may specifically be as follows: Configure the central computing unit with a multi-protocol hardware interface to access feedback signals from multimodal sensors, the device body, and the edge controller; connect the multimodal sensors to the sensing interface of the central computing unit through a unified wiring method; connect the operating device body and the edge controller to the interface of the central computing unit through reserved standard cable routing; Based on the communication interface requirements of multimodal sensors, device bodies, and edge controllers, a standardized communication protocol parsing script suitable for multiple devices is obtained; through the standardized communication protocol parsing script, the various types of data uploaded by the interface are formatted and uniformly converted, and mapped into standard data frames within the central computing unit; The connection path is configured as a bidirectional interactive link that includes status uplink and command downlink. The status data link is responsible for transmitting real-time data from multimodal sensors, the device body, and the edge controller to the central computing unit after parsing. This data is used to update the global status data, mechanism model parameters, and semantic reasoning results in the twin. The command distribution link parses the task control commands generated by the task decision planning module. Through the control adaptation and protocol conversion unit, the unified task control commands are converted into execution signals for the device body and distributed to the device body.

[0030] It should be further explained that establishing the connection between the embodied intelligent twin central computing unit and multimodal sensors, the device itself, and the edge controller enables unified access of heterogeneous industrial equipment at the physical and protocol layers. This eliminates data breakpoints caused by incompatible communication protocols, allowing sensors and control units from different manufacturers or eras to continuously and stably upload real-time status data through standardized interfaces. By using pre-built protocol parsing scripts to uniformly convert multi-source heterogeneous data into structured standard data frames, the computational redundancy and parsing error rate of the central computing unit during data preprocessing are reduced, improving the determinism of data throughput. The establishment of a bidirectional interactive link allows the twin to update its geometric model, physical mechanism parameters, and operating status diagram in real time based on uplink data, ensuring synchronization between digital mapping and the physical entity. Simultaneously, control commands generated by the decision module can be accurately mapped to executable control signals of the device itself through the protocol conversion unit, forming a closed-loop control path from digital space to physical execution. This avoids delays and command distortion caused by manual intervention or intermediate protocol conversion, enhancing the system's response consistency and control reliability under complex operating conditions.

[0031] Step 120: Obtain the mechanistic parameters of the multimodal sensor, device body, and edge controller, and construct the key element space of the mechanistic model; based on the key element space of the mechanistic model, use the mechanistic model to obtain the current mechanistic data.

[0032] The mechanism model includes: intelligent twin geometric model, physical mechanism model, device behavior model, and rule model.

[0033] The steps involve acquiring the mechanistic parameters of the multimodal sensor, the device body, and the edge controller, and constructing the key element space of the mechanistic model, such as... Figure 2 As shown, a mechanism model for constructing a robot's embodied intelligent twin is built from four dimensions: robot geometry, physics, behavior, and rules. The specific process can be as follows: Based on the structure of the mobile equipment itself and the mobile delivery task requirements of the machine tool assembly workshop, a key element space for the mechanism model is constructed: ; in, This represents the set of mechanism parameters of the mobile operation equipment. , , These represent subsets of mechanistic parameters for the perception layer, execution layer, and control layer, respectively; for any layer Its mechanistic parameters are expressed in a unified structured form as follows: ,in For geometric parameters, These are physical property parameters. These are behavioral feature parameters. These are the rule / constraint parameters.

[0034] Then, using the intelligent twin's geometric model, physical mechanism model, device behavior model, and rule model, corresponding model data is generated.

[0035] It should be further explained that the mechanistic parameters of the embodied intelligent twin central computing unit are associated with those of multimodal sensors, the device itself, and the edge controller. A key element space composed of four types of models—geometric, physical, behavioral, and rule-based—is constructed to ensure that the internal state expression of the digital twin has physical interpretability and structural consistency. The geometric model defines the spatial topology through the dimensions of the device itself, joint coordinates, and kinematic chain parameters. The physical mechanism model describes the dynamic response based on material properties, inertial parameters, and torque balance equations. The device behavior model is driven by the control cycle, command response delay, and state transition logic provided by the edge controller. The rule-based model embeds structured conditions such as process constraints, safety thresholds, and operational logic. The parameters of the four types of models are mapped to a computable structured data domain in a unified format, ensuring that the generation process of mechanism data does not depend on external assumptions and is derived only from measured parameters and preset physical laws. This provides an initial state benchmark that conforms to engineering reality for the subsequent construction of the operating state diagram and the input of the graph neural network, avoiding simulation drift and decision failure caused by missing model parameters or semantic ambiguity.

[0036] Step 130: Collect multimodal sensor data using a multimodal sensor; convert the multimodal sensor data into a running state diagram based on preset node and edge features; and, based on the data model, use a graph attention mechanism to process the running state diagram. Layered graph convolution is used to obtain global state data.

[0037] It should be noted that multimodal sensor data is collected through multimodal sensors; based on preset node and edge features, the multimodal sensor data is converted into an operational status diagram, which can be specifically as follows: Data is acquired using a multimodal sensor. , in For the device body's multimodal sensor at time The observation vector, As a dimension, The sampling time for the current task execution process, multimodal sensor It should include at least: triaxial acceleration, triaxial angular velocity, triaxial magnetic field, and attitude angle quaternion; Data preprocessing is performed on multimodal sensor data, and resampling and sliding window slicing are applied to multimodal sensor data with different sampling frequencies to form standardized sample sequences. ;in, Indicates the first Multimodal sensor data in one window For the device body action / control input sequence within the window, For monitoring labels, Indicates the total number of windows; Standardized multi-source data sequences are constructed into a graph structure, and time intervals are defined. The running status diagram is as follows: ;in, For a moment A set of nodes, nodes Represents preset node elements, This is the set of edges corresponding to the edge elements.

[0038] Among them, based on the data model, a graph attention mechanism is used to process the running state graph. Layered graph convolutions obtain global state data, specifically including: Introducing a graph attention mechanism, defining nodes For nodes Attention score: , in, Represents a node At any moment No. Hidden state / embedding vector of the layer Representing neighboring nodes The corresponding hidden state; For the edge The edge feature vector is used to characterize the node. With nodes Relationship type and strength; The scores were then normalized to obtain the first... Layer attention weights: , in, For attention scoring function, The edge feature vector; Representing neighboring nodes For nodes In the Attention weights of layers; Perform AGNN convolutional updates, defining the message function as follows: Then the node The The hidden state is updated to: , in, For activation functions; This is a message mapping function; Finish After layer graph convolution update, a high-dimensional representation of the node set is obtained. Construct a graph-level representation and obtain time intervals. Graph-level representation of the twin state: , in, Representing the global state data of the embodied intelligent twin at any given moment. This represents the graph-level readout function.

[0039] As a supplementary example, construct an embodied intelligent twin data model, such as Figure 3 As shown, a data model of an embodied intelligent twin of a mobile robot can be constructed through steps such as multimodal data acquisition and preprocessing, standardized storage, and structured modeling based on AGNN graph neural networks, thereby realizing the perception and unified representation of the mobile robot's body and task status.

[0040] Step 140: Obtain the real-time structured corpus of the corresponding device ontology in the current task, add the current mechanism data and global state data to the real-time structured corpus, and store them in the preset knowledge corpus. Extract the knowledge ontology triples from the preset knowledge corpus. Input the knowledge ontology triples into the preset base model, and output the reasoning results using the preset rule knowledge ontology and preset parsing prompts.

[0041] Specifically, this involves acquiring the real-time structured corpus of the corresponding device ontology in the current task, adding the current mechanism data and global state data to the real-time structured corpus, and storing it in a preset knowledge corpus. Knowledge ontology triples are then extracted from the preset knowledge corpus. Specifically, this can be done as follows: Acquire and aggregate structured corpus data, including equipment status data, operating mechanisms, equipment tasks, operating specifications, and safety rules, during the mobile delivery tasks of the machine tool assembly workshop, to form a domain knowledge corpus. : , in, The status data corpus includes status information of the device itself, its location, the work object, the environment, and the task execution. The corpus covers operational mechanisms, including geometric and topological relationships, kinematics / dynamics, and mechanistic knowledge of control mechanisms. The equipment-specific task corpus includes task definitions and objectives, task cases, resources and constraints, task processes, and task knowledge related to process requirements. This includes rule-based corpora covering safety operating procedures, system operation rules, and machine tool assembly rules. Add the corresponding mechanism data and global state data to the structured corpus; For any structured corpus The knowledge extraction process in Natural Language Processing is defined as extracting "entity-relationship-attribute" information as follows: , in, For text encoders, These represent "entity-relationship-attribute" information respectively. These represent the processes of extracting information from "entity-relationship-attribute"; After information extraction, entity, relation, and attribute information are organized into device ontology task knowledge triples, forming a set containing entity information. Relationship information With attribute information Knowledge ontology triples: , , , in, An object triple representing an entity-relationship-entity; This represents a data triple consisting of an entity, an attribute, and an attribute value. To get the value for the attribute; The extracted set of triples Indexing and standardizing are performed according to a unified model, and the data is stored in a pre-defined knowledge corpus.

[0042] Specifically, the knowledge ontology triples are input into a preset base model, and the inference results are output using preset rule-based knowledge ontology and preset parsing prompts, including: Acquire raw rule knowledge, including expert experience rules, historical task rules, and existing specifications. : , in, It is a set of expert experience rules, including experience in the path and passage of mobile operation equipment, experience in the operation and maintenance of actuators, and experience in the safe execution of control links; It is a set of historical task rules, including task flow and process parameter correspondence rules, similar task migration rules, and resource scheduling rules; This is the existing set of rules, including safety regulations, operating procedures, and equipment operation constraints. Using the original rule knowledge, a preset rule knowledge ontology corresponding to the mobile device ontology delivery operation is constructed, and the rule weights in the preset rule knowledge ontology are dynamically updated based on the memory-forgetting model. Using AI models as open-source generative base models Then, a lightweight model is obtained through distillation. ; The adapted model The pre-defined knowledge corpus and pre-defined rule-based knowledge ontology corresponding to the knowledge ontology triples are jointly encapsulated into a knowledge-enhanced reasoning interface: , in, Preset parsing prompts, As a result of reasoning, These are a pre-defined knowledge corpus and a pre-defined rule-based knowledge ontology for mobile device ontology delivery operations. This represents the inference execution function.

[0043] It should be further explained that by fusing the real-time structured corpus of the equipment ontology in the current task with mechanistic data and global state data and storing it in a pre-set knowledge corpus, the structured alignment of industrial operation data and domain knowledge is achieved, giving the corpus content a searchable and associative semantic foundation. By defining an "entity-relationship-attribute" extraction framework, knowledge ontology triples conforming to industrial knowledge expression standards are extracted from unstructured or semi-structured corpora, ensuring the uniformity and computability of knowledge representation. The triples and the pre-set rule knowledge ontology are jointly input into the base model, and the model is guided to reason within the rule constraints using pre-set parsing prompts, avoiding outputs that conflict with actual process logic generated by the model's free generation. The rule knowledge ontology dynamically adjusts rule weights through a memory-forgetting model, enabling the reasoning process to adaptively focus on high-confidence rules based on task history and operating status. The distillation results of the lightweight model and the knowledge base are jointly encapsulated into a reasoning interface, reducing reasoning latency and improving the feasibility of deployment at the edge. This allows the system to output diagnostic and decision suggestions that conform to operating specifications based on historical experience and real-time status without relying on cloud computing power.

[0044] Step 150: Input the current mechanism data, global state data and reasoning results into the task decision planning module to generate task control instructions.

[0045] This step can be specifically described as follows: Consistently constraining and fusing the device's intrinsic mechanism data and global state data within a unified state space to obtain a fused state; The device body is in a fusion state Together with the inference result 'a', they serve as inputs to the pre-defined control model for planning decisions: , in, For task planning and decision functions, For output task control commands.

[0046] It should be further explained that this step inputs the current mechanism data, global state data, and inference results into the task decision planning module. By constraining and fusing multi-source data within a unified state space, the physical rationality and semantic integrity of the decision input are ensured, avoiding control logic conflicts caused by data heterogeneity or time synchronization issues. The fused state, as the sole input source of the control model, eliminates instruction deviations caused by information loss or format mismatch in intermediate links in traditional hierarchical decision-making, improving the determinism of control signal generation. The inference results are embedded in the decision-making process as semantic constraints, enabling the control model to make adaptive adjustments within a limited range based on verified rule knowledge and historical experience when executing tasks, rather than relying on fixed strategies, thus enhancing the system's fault tolerance under unexpected operating conditions. The preset control model directly outputs structured task control instructions based on this fused input. Its output format strictly matches the execution protocol of the device itself, requiring no additional parsing or mapping, shortening the response path from decision to execution, and ensuring that the instructions can be directly parsed at the edge controller level and drive the actuator, forming an end-to-end closed-loop control path.

[0047] Based on the above description, as a specific embodiment, such as Figure 4 As shown in the diagram, this application provides a structural schematic of an embodied intelligent twin, which is used to achieve integrated autonomous operation of "perception-planning-decision" for complex tasks, including: The central computing unit 100 is used to run the integrated intelligent twin model of the operating equipment and its functional modules. It carries the core computing of the mechanism model, data model, and knowledge model, and supports the twin system functions such as multimodal data perception, task scenario construction, task analysis and planning, and decision optimization execution. In addition, this unit connects and communicates with the sensor system, edge controller, and the operating equipment itself for data interaction. The data acquired and generated by the above physical units are uniformly accessed and processed by the central computing unit and then distributed to the integrated intelligent twin model and the "perception-planning-decision" functional modules for application.

[0048] The work equipment body 101 serves as the physical carrier for task execution, and is used to receive control commands issued by the edge controller and complete the work actions.

[0049] The multimodal sensor system 102 includes visual sensors, mechanical sensors, motion sensors, and other multimodal sensors. It is used to collect multi-source perception information of "operating equipment-operating object-operating environment" and upload it to the central computing unit of the embodied intelligent twin through a data interface. The central computing unit parses, stores, and standardizes the uploaded data and then sends the processing results to the integrated twin model and twin application module to carry out functional applications. It also serves as input data for embodied fusion perception and twin state update evolution.

[0050] The edge controller 103 is used to receive skill tasks and control commands from the task decision and execution module of the embodied intelligent twin of the working equipment, parse and issue control commands and schedule execution, and output information such as control status, execution feedback and task completion status, which are then processed by the central computing unit and transmitted to the embodied fusion perception module.

[0051] The embodied intelligent twin integrated model 104 includes a mechanism model, data model, and operation domain knowledge model of multiple elements of the operating equipment, forming an integrated twin that integrates "mechanism-data-knowledge". It provides state data, mechanism constraints, domain knowledge, etc. for the embodied fusion perception module, task planning module, and skill decision execution module.

[0052] The embodied fusion perception module 105 obtains multimodal heterogeneous state information collected by the multimodal sensor system from the central computing unit, performs spatial alignment, temporal alignment and deep "time-space" fusion perception on the multimodal heterogeneous information, constructs a semantic spatiotemporal task map for the operation scenario, and provides semantic scene, task and constraint information to the twin task planning and decision execution module.

[0053] Task planning module 106 obtains the twin's state, mechanistic constraints, and task domain knowledge from the embodied intelligent twin integrated model, and obtains embodied perception results from the embodied fusion perception module. It applies the generative knowledge model in the embodied intelligent twin integrated model to construct a task prompting thought chain, completes task parsing, sub-task decomposition and planning evaluation, and generates an executable skill task sequence and control instructions.

[0054] The task decision-making and execution module 107, after obtaining the skill task sequence and control instructions from the task planning module, uses it to adjust and optimize task decisions by combining the planning results with real-time perception feedback, and then sends the optimized control instructions to the edge controller to operate the work equipment. Simultaneously, the module monitors the equipment's execution status and feedback information, driving the twin's evolution and performing dynamic execution optimization.

[0055] Based on the same inventive concept, embodiments of this application also provide a device for constructing an embodied intelligent twin. For example... Figure 5 As shown, the device includes: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to perform an embodied intelligent twin construction method as described in the above embodiments.

[0056] Specifically, the server establishes connections between the embodied intelligent twin central computing unit and the corresponding multimodal sensors, device bodies, and edge controllers of the intelligent twin; it acquires the mechanistic parameters of the multimodal sensors, device bodies, and edge controllers, and constructs a key element space for the mechanistic model; based on the key element space of the mechanistic model, it uses the mechanistic model to obtain current mechanistic data; the mechanistic model includes: the intelligent twin's geometric model, physical mechanistic model, device behavior model, and rule model; it collects multimodal sensor data through multimodal sensors; based on preset node elements and edge elements, it converts the multimodal sensor data into an operational state graph; based on the data model, it uses a graph attention mechanism to process the operational state graph. Layered graph convolution is used to obtain global state data; the corresponding device ontology is acquired in real-time structured corpus in the current task, and the current mechanism data and global state data are added to the real-time structured corpus and stored in a preset knowledge corpus. Knowledge ontology triples are extracted from the preset knowledge corpus; the knowledge ontology triples are input into a preset base model, and the inference results are output using preset rule knowledge ontology and preset parsing prompts; the current mechanism data, global state data and inference results are used as planning decisions input to the task decision planning module to generate task control instructions.

[0057] In addition, embodiments of this application also provide a non-volatile computer storage medium storing executable instructions, which, when executed, implement the above-described method for constructing an embodied intelligent twin.

[0058] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing an embodied intelligent twin, characterized in that, The method includes: Establish the connection relationship between the central computing unit of the embodied intelligent twin and the corresponding multimodal sensors, device body, and edge controller of the intelligent twin; The mechanism parameters of multimodal sensors, device body, and edge controller are obtained, and a key element space of the mechanism model is constructed. Based on the key element space of the mechanism model, the current mechanism data is obtained using the mechanism model. The mechanism model includes: intelligent twin geometric model, physical mechanism model, device behavior model, and rule model. Multimodal sensor data is collected using multimodal sensors; based on preset node and edge features, the multimodal sensor data is converted into a running state diagram; based on the data model, a graph attention mechanism is used to process the running state diagram. Layered graph convolutions are used to obtain global state data. Obtain the real-time structured corpus of the corresponding device ontology in the current task, add the current mechanism data and global state data to the real-time structured corpus and store them in the preset knowledge corpus, extract the knowledge ontology triples from the preset knowledge corpus, input the knowledge ontology triples into the preset base model, and output the reasoning results using the preset rule knowledge ontology and preset parsing prompts. The task decision planning module takes current mechanism data, global state data, and inference results as inputs for planning decisions and generates task control instructions.

2. The method for constructing an embodied intelligent twin according to claim 1, characterized in that, Establishing connections between the embodied intelligent twin's central computing unit and the corresponding multimodal sensors, device body, and edge controllers, specifically including: Configure the central computing unit with a multi-protocol hardware interface to access feedback signals from multimodal sensors, the device body, and the edge controller; connect the multimodal sensors to the sensing interface of the central computing unit through a unified wiring method; connect the operating device body and the edge controller to the interface of the central computing unit through reserved standard cable routing; Based on the communication interface requirements of multimodal sensors, device bodies, and edge controllers, a standardized communication protocol parsing script suitable for multiple devices is obtained; through the standardized communication protocol parsing script, the various types of data uploaded by the interface are formatted and uniformly converted, and mapped into standard data frames within the central computing unit; The connection path is configured as a bidirectional interactive link that includes status uplink and command downlink. The status data link is responsible for transmitting real-time data from multimodal sensors, the device body, and the edge controller to the central computing unit after parsing. This data is used to update the global status data, mechanism model parameters, and semantic reasoning results in the twin. The command distribution link parses the task control commands generated by the task decision planning module. Through the control adaptation and protocol conversion unit, the unified task control commands are converted into execution signals for the device body and distributed to the device body.

3. The method for constructing an embodied intelligent twin according to claim 1, characterized in that, Obtain the mechanistic parameters of the multimodal sensor, device body, and edge controller, and construct the key element space of the mechanistic model, specifically including: Based on the structure of the mobile equipment itself and the mobile delivery task requirements of the machine tool assembly workshop, a key element space for the mechanism model is constructed: ； in, This represents the set of mechanism parameters of the mobile operation equipment. , , These represent subsets of mechanistic parameters for the perception layer, execution layer, and control layer, respectively; for any layer Its mechanistic parameters are expressed in a unified structured form as follows: ,in For geometric parameters, These are physical property parameters. These are behavioral feature parameters. These are the rule / constraint parameters.

4. The method for constructing an embodied intelligent twin according to claim 1, characterized in that, Multimodal sensor data is acquired through multimodal sensors; Based on preset node and edge features, the multimodal sensor data is converted into an operational status diagram, specifically including: Data is acquired using a multimodal sensor. ， in For the device body's multimodal sensor at time The observation vector, As a dimension, The sampling time for the current task execution process, multimodal sensor It should include at least: triaxial acceleration, triaxial angular velocity, triaxial magnetic field, and attitude angle quaternion; Data preprocessing is performed on multimodal sensor data, and resampling and sliding window slicing are applied to multimodal sensor data with different sampling frequencies to form standardized sample sequences. ;in, Indicates the first Multimodal sensor data in one window For the device body action / control input sequence within the window, For monitoring labels, Indicates the total number of windows; Standardized multi-source data sequences are constructed into a graph structure, and time intervals are defined. The running status diagram is as follows: ;in, For a moment A set of nodes, nodes Represents preset node elements, This is the set of edges corresponding to the edge elements.

5. The method for constructing an embodied intelligent twin according to claim 1, characterized in that, Based on the data model, a graph attention mechanism is used to analyze the running state graph. Layered graph convolutions obtain global state data, specifically including: Introducing a graph attention mechanism, defining nodes For nodes Attention score: ， in, Represents a node At any moment No. Hidden state / embedding vector of the layer Representing neighboring nodes The corresponding hidden state; For the edge The edge feature vector is used to characterize the node. With nodes Relationship type and strength; The scores were then normalized to obtain the first... Layer attention weights: ， in, For attention scoring function, The edge feature vector; Representing neighboring nodes For nodes In the Attention weights of layers; Perform AGNN convolutional updates, defining the message function as follows: Then the node The The hidden state is updated to: ， in, For activation functions; This is a message mapping function; Finish After layer graph convolution update, a high-dimensional representation of the node set is obtained. Construct a graph-level representation and obtain time intervals. Graph-level representation of the twin state: ， in, Representing the global state data of the embodied intelligent twin at any given moment. This represents the graph-level readout function.

6. The method for constructing an embodied intelligent twin according to claim 1, characterized in that, Obtain the real-time structured corpus of the corresponding device ontology in the current task, add the current mechanism data and global state data to the real-time structured corpus, and store it in a preset knowledge corpus. Extract knowledge ontology triples from the preset knowledge corpus, specifically including: Acquire and aggregate structured corpus data, including equipment status data, operating mechanisms, equipment tasks, operating specifications, and safety rules, during the mobile delivery tasks of the machine tool assembly workshop, to form a domain knowledge corpus. : ， in, The status data corpus includes status information of the device itself, its location, the work object, the environment, and the task execution. The corpus covers operational mechanisms, including geometric and topological relationships, kinematics / dynamics, and mechanistic knowledge of control mechanisms. The equipment-specific task corpus includes task definitions and objectives, task cases, resources and constraints, task processes, and task knowledge related to process requirements. This includes rule-based corpora covering safety operating procedures, system operation rules, and machine tool assembly rules. Add the corresponding mechanism data and global state data to the structured corpus; For any structured corpus The knowledge extraction process in Natural Language Processing is defined as extracting "entity-relationship-attribute" information as follows: ， in, For text encoders These represent "entity-relationship-attribute" information respectively. These represent the processes of extracting information from "entity-relationship-attribute" respectively; After information extraction, entity, relation, and attribute information are organized into device ontology task knowledge triples, forming a set containing entity information. Relationship information With attribute information Knowledge ontology triples: ， ， ， in, An object triple representing an entity-relationship-entity; This represents a data triple consisting of an entity, an attribute, and an attribute value. To get the value for the attribute; The extracted set of triples Indexing and standardizing are performed according to a unified model, and the data is stored in a pre-defined knowledge corpus.

7. The method for constructing an embodied intelligent twin according to claim 1, characterized in that, Input the knowledge ontology triples into the preset base model, utilize the preset rule-based knowledge ontology and preset parsing prompts, and output the reasoning results, specifically including: Acquire raw rule knowledge, including expert experience rules, historical task rules, and existing specifications. : ， in, It is a set of expert experience rules, including experience in the path and passage of mobile operation equipment, experience in the operation and maintenance of actuators, and experience in the safe execution of control links; It is a set of historical task rules, including task flow and process parameter correspondence rules, similar task migration rules, and resource scheduling rules; This is the existing set of rules, including safety regulations, operating procedures, and equipment operation constraints. Using the original rule knowledge, a preset rule knowledge ontology corresponding to the mobile device ontology delivery operation is constructed, and the rule weights in the preset rule knowledge ontology are dynamically updated based on the memory-forgetting model. Using AI models as open-source generative base models Then, a lightweight model is obtained through distillation. ; The adapted model The pre-defined knowledge corpus and pre-defined rule-based knowledge ontology corresponding to the knowledge ontology triples are jointly encapsulated into a knowledge-enhanced reasoning interface: ， in, Preset parsing prompts, As a result of reasoning, These are a pre-defined knowledge corpus and a pre-defined rule-based knowledge ontology for mobile device ontology delivery operations. This represents the inference execution function.

8. The method for constructing an embodied intelligent twin according to claim 1, characterized in that, Using current mechanism data, global state data, and inference results as inputs to the planning and decision-making pre-defined control model, task control instructions are generated, specifically including: The device's intrinsic mechanism data and global state data are constrained and fused in a unified state space to obtain a fused state. The device body is in a fusion state Together with the inference result 'a', they serve as inputs to the pre-defined control model for planning decisions: ， in, For task planning and decision functions, For output task control commands.

9. A device for constructing an embodied intelligent twin, characterized in that, The device includes: processor; And a memory having executable code stored thereon, which, when executed, causes the processor to perform a method for constructing an embodied intelligent twin as described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, It stores computer instructions, which, when executed, implement a method for constructing an embodied intelligent twin as described in any one of claims 1-8.

Images