Robot system modeling

Abstract

The present invention relates to a modeling apparatus for modeling a robotic system, the model comprising instances of at least a subset of a set of defined types interconnected by links, each instance being a description of an element of the robotic system and links between the instances representing relationships therebetween, and validating the model against validation rules by determining whether the model satisfies the validation rules, the validation rules comprising a definition of one or more directed acyclic graphs of the defined types interconnected by parent-child relationships, the defined types corresponding to nodes and the parent-child relationships corresponding to directed edges of the one or more directed acyclic graphs, each defined type defining type rules that an instance of that type must satisfy to be valid, each parent-child relationship between the pair of types defining the types as parent type and child type with respect to each other according to the corresponding edge directions, and each child type inheriting the type rules of each of the corresponding parent types according to the multiple parent-child relationships.

Description

[Technical Field]

[0001] This disclosure relates, in general terms, to robot system modeling.

[0002] In particular, this disclosure relates to a modeling apparatus for modeling a robotic system, a multi-apparatus modeling system comprising first and second such apparatuses, and related methods, computer programs and storage media. Modeling may include controlling the robotic system. [Background technology]

[0003] A robot can be thought of as an autonomous or semi-autonomous machine capable of automatically performing a complex series of actions. A robot may be guided by an external control device or an embedded controller. One useful example is a robotic manipulator (collectively a robotic arm) fitted with a gripper.

[0004] Robotic systems (systems containing one or more robots) are being used for an increasing number and range of tasks. This is particularly evident in fields such as nuclear power, mining, petrochemical processing, and underwater operations, resulting in more complex robotic facilities being deployed, maintained, and expanded over longer periods than ever before. Furthermore, system requirements are frequently new or changed due to unstructured, experimental, or unknown operating conditions, necessitating expansion and adaptation.

[0005] It is desirable to model robotic systems and, for example, describe or represent those robotic systems to external systems, and / or control those robotic systems.

[0006] While existing modeling frameworks enable robust integration of complex robotic systems, they unfortunately lack highly efficient maintainability and scalability when faced with changing requirements and obsolescence issues, such as over decades. [Overview of the project]

[0007] From the above perspective, it is desirable to provide improved robot system modeling.

[0008] According to a first aspect of this disclosure, a modeling apparatus for modeling a robotic system is provided. The modeling device is a modeler for modeling the robot system using a model, the model comprising multiple instances of at least a subset of a set of defined types interconnected with a plurality of links, each instance being a description of an element of the robot system, the plurality of links between the plurality of instances comprising a modeler representing relationships between the instances, and a verification unit for verifying the model against a plurality of verification rules by determining whether the model satisfies a plurality of verification rules, the plurality of verification rules comprising the definition of one or more directed acyclic graphs of the plurality of defined types interconnected with a plurality of parent-child relationships, the plurality of defined types corresponding to a plurality of nodes, the plurality of parent-child relationships corresponding to a plurality of directed edges of the one or more directed acyclic graphs, each defined type defining a plurality of type rules that an instance of that type must satisfy for it to be valid, each parent-child relationship between a pair of types defining those types as parent and child types to each other according to the corresponding edge direction, and each child type inheriting the plurality of type rules of each of the corresponding parent types according to the plurality of parent-child relationships.

[0009] Such systems offer advantages in terms of maintainability, scalability, and interoperability. For example, in such a system of parent and child types, where each child type inherits multiple type rules from each of its corresponding parent types, rule changes in a parent type efficiently affect its child types (and any grandchild types, etc.). Furthermore, a particular subset of a parent type (i.e., a particular child type) can be updated with respect to its rules without affecting its parent type or other child types of its parent types.

[0010] When morphological rules are expressed in validation rules, explicit assumptions about the structure of a given system can be made by enforcing a combined structure expressed in the ontology of syntactic meaning and type, thereby promoting interoperability.

[0011] A second aspect of the present disclosure provides a multi-device modeling system comprising first and second modeling devices configured to communicate with each other. Each of the first and second modeling devices is the modeling device described in the first aspect, wherein the first modeling device models the first robot system using a first model of the first robot system, and the second modeling device models the second robot system using a second model of the second robot system, and the multi-device modeling system is configured to determine interoperability between the first model and the second model by a process of verifying the first model in light of the verification rules of the second modeling device, and / or a process of verifying the second model in light of the verification rules of the first modeling device, and / or a process of determining compatibility between at least a first model-specific portion of the one or more directed acyclic graphs of the first modeling device and a second model-specific portion of the one or more directed acyclic graphs of the second modeling device.

[0012] Such multi-device modeling systems are particularly advantageous in terms of interoperability. Since the first and second modeling devices each validate their models against their respective validation rules, interoperability between the first and second models can be determined by determining compatibility between at least the first model-specific portion of one or more directed acyclic graphs of the first modeling device and the second model-specific portion of one or more directed acyclic graphs of the second modeling device. Furthermore, compatibility can be determined in an efficient manner because the types are defined by directed acyclic graphs that have rule inheritance.

[0013] A third aspect of the present disclosure provides a method for modeling a robotic system. The method includes the steps of: modeling the robotic system using a model, wherein the model comprises multiple instances of at least a subset of a set of defined types interconnected by a plurality of links, each instance being a description of an element of the robotic system, and the plurality of links between the multiple instances representing relationships between them; and verifying the model against a plurality of verification rules by determining whether the model satisfies a plurality of verification rules, wherein the verification rules are definitions of one or more directed acyclic graphs of the plurality of defined types interconnected with a plurality of parent-child relationships, where the plurality of defined types correspond to a plurality of nodes, the plurality of parent-child relationships correspond to a plurality of directed edges of the one or more directed acyclic graphs, each defined type defines type rules of that type that instances of that type must satisfy for to be valid, each parent-child relationship between a pair of types defines those types as parent and child types to each other according to the corresponding edge direction, and each child type inherits the plurality of type rules of each of the corresponding parent types according to the plurality of parent-child relationships.

[0014] According to a fourth aspect of this disclosure, a computer program is provided which, when executed on a computer, causes the computer to perform the method of the third aspect described above.

[0015] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided on which the computer program of the fourth aspect described above is stored.

[0016] A sixth aspect of the present disclosure provides a series of methods including first and second methods, each of which is a method according to the third aspect described above, the first method modeling the first robot system using a first model of the first robot system, the second method modeling the second robot system using a second model of the second robot system, the series of methods further including determining interoperability between the first model and the second model by the steps of: verifying the first model in light of the verification rules of the second method, and / or verifying the second model in light of the verification rules of the first method, and / or determining compatibility between at least a first model-specific portion of the one or more directed acyclic graphs of the first method and a second model-specific portion of the one or more directed acyclic graphs of the second method.

[0017] According to a seventh aspect of this disclosure, a set of computer programs is provided that, when executed on one or more computers, cause one or more computers to perform a set of methods of the sixth aspect described above.

[0018] According to an eighth aspect of this disclosure, a computer-readable storage medium is provided which stores the series of computer programs of the seventh aspect described above.

[0019] According to a ninth aspect of the present disclosure, a robot system including the modeling device according to the first aspect described above or the multi-device modeling system according to the second aspect described above is provided. Corresponding method aspects, computer program aspects, and storage medium aspects are also contemplated.

[0020] The features of one aspect may be applied to another aspect, and vice versa.

[0021] According to the aspects described above, the limitations of the existing technology are solved. In particular, the limitations of maintainability, scalability, and interoperability in existing robot system modeling technologies are solved.

[0022] These and other aspects will become apparent by referring to the embodiments described below.

Brief Description of the Drawings

[0023] Here, the embodiments will be described merely as examples with reference to the following drawings. [Figure 1] FIG. 1A is a schematic diagram of a robot system for implementing the present invention. FIG. 1B is a schematic diagram of a robot system for implementing the present invention. [Figure 2] FIG. 2 is a schematic diagram of a modeling device for implementing the present invention. [Figure 3] FIG. 3 is a schematic diagram of a model of a robot system. [Figure 4] FIG. 4 is a simplified view of a part of the model of FIG. 3 and is useful for understanding morphological rules. [Figure 5] FIG. 5 is a simplified view of a part of the model of FIG. 3 and is useful for understanding morphological rules. [Figure 6] FIG. 6 is a schematic diagram of a model part and is also useful for understanding morphological rules. [Figure 7] FIG. 7 presents a simplified directed acyclic graph of an exemplary defined type. [Figure 8]Figure 8 shows a broader range of directed acyclic graphs of the defined type than the directed acyclic graph in Figure 7. [Figure 9] Figure 9 is a schematic diagram of an instance data unit, showing its structured format. [Figure 10] Figure 10 is a schematic diagram of a type data unit, showing its structured format. [Figure 11] Figure 11 is a schematic diagram of a multi-device system comprising a first and a second modeling device. [Figure 12] Figure 12 is a schematic diagram of the model-specific type graph. [Figure 13] Figure 13 shows two type graphs that are compatible with each other. [Figure 14] Figure 14 also shows two type graphs that are compatible with each other. [Figure 15] Figure 15 shows two type graphs that are incompatible with each other. [Figure 16] Figure 16 also shows two type graphs that are incompatible with each other. [Figure 17] Figure 17 is a schematic diagram that is useful for understanding how the modeling apparatus in Figure 2 may operate in a series of loop cycles. [Figure 18] Figure 18 is a schematic diagram of a method for carrying out the present invention. [Figure 19] Figure 19 is a block diagram of a computing device that may be used to implement any of the devices or methods described herein. [Modes for carrying out the invention]

[0024] The following description outlines exemplary embodiments of the present disclosure. Further exemplary embodiments and implementations will be apparent to those skilled in the art. Furthermore, those skilled in the art should recognize that various equivalent technologies may be applied instead of, or in conjunction with, the embodiments discussed below, and all such equivalents should be considered to be encompassed by the present disclosure.

[0025] As an introduction, the challenges faced in developing control system software capable of supporting applications such as remote nuclear power plant operation can be categorized into four main areas: interoperability, maintainability, scalability, and performance. The embodiments disclosed herein aim to solve the long-term maintainability and scalability problems through the use of standardized self-describing data representations and associated communication protocols.

[0026] A high level of interoperability is desirable for a control system to support the various custom interfaces required when integrating hardware from various suppliers, while simultaneously reducing the effort involved in integration. The standard interfaces within the middleware provided by the embodiments disclosed herein will demonstrate that the various components of the system are interoperable at a basic level, namely in the discovery and exploration of data and functionality within the system.

[0027] Furthermore, the high level of modularity in the design and structure implemented by the framework architecture allows for the replacement of system components while minimizing the impact on the rest of the system. This modularity further allows for the addition of additional functionality without modifying existing components of the system. The ability to extend the capabilities of the system while minimizing the impact on components is achieved by a highly cohesive, loosely coupled, and granular architecture.

[0028] Figure 1A is a schematic diagram of a robot system 10A comprising a robot 20 and a robot controller 30.

[0029] In this example, robot 20 is shown as a robotic manipulator 22 (together a robotic arm) to which a gripper 24 is attached. The arm comprises several joints 26, and the gripper similarly comprises joints 28. The movement of robot 20 may be controlled under the control of a robotic controller 30, particularly under the control of joints 26 and 28. Naturally, robotic arms 22 and 24 are simple examples of robots, and it will be understood that the techniques disclosed herein may be applied to any type of robot.

[0030] In some cases, the combination of robot 20 and robot controller 30 itself may be referred to as a robot. Furthermore, although only one robot 20 is shown in robot system 10A for simplification, robot system 10A may consist of multiple robots, and these robots may be of different types. Thus, the models of robot systems considered herein may, in some cases, correspond to models of robot systems consisting of multiple robots, and these robots may be of different types. Similarly, references to robots herein may, in some cases, be considered references to multiple robots, and these robots may be of different types.

[0031] The robot controller 30 is shown to include a modeling device 100, the details of which will be described later. Thus, the robot system 10A embodies the present invention. In some configurations, the modeling device 100 can model the robot system in the sense of representing or describing the robot system. In some configurations, the modeling device 100 can control the robot system 10A, in particular the robot 20. The modeling device 100 does not, of course, need to model itself, and therefore, the modeling device may be considered separate from the robot system 10A and as part of a larger system comprising the robot system 10A.

[0032] Figure 1B is a schematic diagram of robot system 10B, which is the same as robot system 10A except that the modeling device 100 is provided separately from the robot 20 and the robot controller 30 and is communicated with them. As previously stated and as indicated by the solid and dashed arrows, in some configurations the modeling device 100 may simply model the robot system in the sense of representing or describing the robot system, and in some configurations the modeling device 100 may additionally control the robot system, in particular the robot 20.

[0033] The modeling device 100 does not, of course, need to model itself, and therefore, the modeling device 100 may be considered separate from the robot system 10A or 10B and as part of a larger system comprising the robot system 10A or 10B. In other words, the modeling device 100 can be considered to model the robot system 10A or 10B, but not model itself.

[0034] Figure 2 is a schematic diagram of the modeling device 100. As an example of its application, the modeling device 100 is considered to be used for modeling robot system 10A (as one of the robot systems 10A and 10B).

[0035] The modeling apparatus 100 comprises a modeler 120, a model verification unit 140, and verification rules 160 (however, the rules may be stored outside the modeling apparatus 100 and thereby accessible). The modeling apparatus 100 may be considered a computer-aided device, or may include a computing device, and may be implemented, for example, on a computer, in software, in hardware, or as a combination of software and hardware.

[0036] The modeler 120 is configured to model the robot system 10A. In this regard, the modeler 120 can be considered to control, execute, handle, or maintain the model. The modeler 120 can be considered to execute, operate, or implement the model. The model may be considered to simulate, emulate, represent, or describe the robot system 10A.

[0037] In this regard, refer to Figure 3, which is a schematic diagram of Model 200 of the robot system 10A, in particular the robot 20 and / or robot controller 30. For the sake of simplification, we will assume that the robot manipulator 22 has one joint 26 (instead of two joints 26 as in Figure 1A) and the gripper 24 has one joint 28.

[0038] Model 200 includes Link 220 (represented by arrows interconnecting boxes) and interconnected instances 210 of defined types (represented by boxes), each instance being a description of an element of robot 20 or robot controller 30, or simply an element of robot system 10A. Model 200 represents or describes the current state and / or target state of robot system 10A. Not all instances 210 and all links 220 are denoted with reference numerals, simply to avoid making Figure 3 overcrowded, and similar considerations apply to other figures.

[0039] Links 220 between instances 210 represent the relationships (including the flow of information) between them. Each instance displays its name (in bold) and, below it, the defined type (not in bold) that defines that instance. In this context, an instance can be considered an example or occurrence of a data unit, item, or block of that type.

[0040] As shown in Figure 3, Model 200 defines a directed graph (or simply a graph) of its instance 210, Link 220 is a directed link, Instance 210 corresponds to a node, and Link 220 corresponds to an edge of the directed graph.

[0041] Furthermore, it should be noted that at this point, some instances of instance 210 are represented by boxes with narrow borders, while others are represented by boxes with wide borders. Instances 210 represented by narrow borders are descriptive instances, while instances 210 represented by wide borders are functional (or active) instances.

[0042] The difference between descriptive and functional types will be explained in detail later, but for the purposes of this discussion, each instance of a descriptive type describes the current state and / or target state of an element of the robot system 10A, while each instance of a functional (or active) type describes how to manipulate the data of interconnected instances of the descriptive type in order to derive information describing the robot system and / or to model the activities within the robot system 10A.

[0043] Each link is “owned” by one of the instances 210 to which it interconnects, and the direction of link 220 represents, as is evident, an “input relationship” or “output relationship” between the interconnecting instances 210 with respect to the instance 210 that owns the link 220. The arrowhead (arrowhead) in the box represents the owner of the relationship; that is, the box containing the arrowhead is the owner of the input or output relationship and is responsible for enforcing the associated rules.

[0044] The direction of a link, if it points toward the instance that owns the link, represents the input relationship between the interconnected instances to the instance that owns the link. The direction of a link, if it points toward the instance that owns the link, represents the output relationship between the interconnected instances to the instance that owns the link.

[0045] For example, if the owner of a link is a functional instance, that link can represent an "input relationship" or "output relationship" between the interconnected instances 210. For instance, in such a link, where the direction is toward the owner instance, the other instance functions as (or is an input to) the owner instance. Similarly, in such a link, where the direction is toward the owner instance, the other instance functions as an output to the owner instance.

[0046] In other words, for at least one link whose direction points toward its functional owner instance, input information about that owner instance is represented in other instances of that link or read from other instances of that link via that link. Similarly, for at least one link whose direction points toward its functional owner instance, output information about that owner instance is represented in other instances of that link or written to other instances of that link via that link.

[0047] On the other hand, if the owner of a link is a descriptive instance, the link may represent an “ownership relationship” or “structural relationship” between the interconnected instances 210. For example, the owner instance of such a link can be understood to “contain” or “have” the other instances of that link.

[0048] To give a concrete example, returning to Figure 3, the topmost instance 210 is named "Robot Arm," its type is "Arm," and it essentially represents the entire robot 20. This instance is then separately connected to another instance 210 named "KUKA LBR iiwa" of the type "Serial Manipulator," and to another instance 210 named "Robotiq 2-Finger Gripper" of the type "Gripper."

[0049] The KUKA LBR iiwa is an example of a convenient serial manipulator currently available from KUGA AG. LBR stands for "Leichtbauroboter" (German for lightweight robot), and iiwa stands for "Intelligent Industrial Work Assistant." Similarly, the Robotiq 2-Finger Gripper is another example of a convenient gripper currently available from Robotiq.

[0050] These instances 210 are descriptive instances, and the links 220 that interconnect them represent a “has a” or “contains a” relationship (i.e., ownership or structural relationship) defined with respect to the owner of those links. There is an input relationship defined for these links with respect to the top-level instance 210 of type “arm,” named “robot manipulator,” which represents an ownership or structural relationship. Thus, the arm is understood to have or contain a serial manipulator and gripper. The serial manipulator and gripper are substantially sub-components of the arm, or the arm is substantially an assembly of (or comprising) a serial manipulator and gripper. This relationship is clearly illustrated by Figure 4, a simplified diagram of part of Model 200 (instance names are omitted).

[0051] Similarly, looking at Figure 3, both the serial manipulator and the gripper “have” a rotation axis in their respective joints, which is named “joint” and is shown by instance 210 of type “rotation axis”. However, instance 210 named “KUKA LBR iiwa” is linked to two instances 210 named “joint”, and the same is true for instance 210 named “Robotiq 2-finger gripper”.

[0052] In each case, one of the rotation axis instances functions as an input instance where the data relates to the current state of the robot system 10A (specifically, the relevant joint 26 or 28), and the other of those rotation axis instances functions as an output instance where the data relates to the desired, requested, or target state of the robot system 10A (specifically, the relevant joint 26 or 28). This relationship is shown in Figure 5, which is a simplified diagram of part of Model 200 (instance names are omitted). Nevertheless, in these links between the rotation axis instances and the serial manipulator and gripper instances, input relationships (representing ownership or structural relationships) to the serial manipulator and gripper instances are defined.

[0053] Focusing on Figure 5, the instance connected below the rotation axis instance in Figure 3 is simplified as a single function instance 210 of the general-purpose function type "[Function]" (the reason will become clear later). Therefore, for example, the rotation axis of the serial manipulator may have a current position represented by the data of the "input" rotation axis instance 210 and be assigned a target position represented by the data of the "output" rotation axis instance 210. Then, the single function instance 210 of the general-purpose function type "[Function]" may be configured to take the current position of the serial manipulator's rotation axis as input and perform a process (simulating the desired activity) that generates the target position of the serial manipulator's rotation axis as output. The reason for the dashed arrows in the links will be ignored here but will be explained later.

[0054] It is important to note here that the functional type "[Function]" instances shown in Figure 5 do not necessarily need to exist, nor do they need to exist for the corresponding functional type instances of Model 200 in Figure 3 (and for Cartesian concept instances that show some of those functional instances interconnected). For example, Model 200 may represent or describe the robot system 10A without attempting to control it. For example, considering the "KUKA LBR iiwa" instance of type "Serial Manipulator" in Figure 3, one of its rotation axis instances functions as an input instance where the data relates to the current state of the robot system 10A, while the other of those rotation axis instances functions as an output instance where the data relates to the desired, requested, or target state of the robot system 10A, as previously described. However, the data of those instances may be substantially "read" from the robot system 10A so that the model describes the current state and the requested or target state provided by the robot controller 30 (not the modeling device 100) that performs its control.

[0055] The upper half of Figure 6 shows a model section containing instances of "Input," "Output," and "Processing," illustrating the input-output relationship between a functional instance (which performs processing) and a descriptive input / output instance (which describes the state of that element of the robot system 10A). Here, the general functional type "[Function]" is replaced with a concrete example of the type "Processor" (which performs processing), and the descriptive input / output instance is shown as an instance of the high-level type "Concept," in which case it represents pure data describing the aspects of the robot system (rather than, for example, physical components). The reasons for the dashed and multi-arrow links are ignored here, but will be explained later.

[0056] This model part and the gripper part of the simplified model in Figure 5 can be related back to the gripper part of Model 200 in Figure 3. In particular, instance 210, named "Robotiq 2-Finger Gripper," is named "Joint" and is linked to two instances 210 of type "Rotation Axis," which are connected to a functional instance named "Gripper Controller" of type "Rotation Axis Controller." The left rotation axis instance functions as an input to the rotation axis controller instance, and the right rotation axis instance functions as an output. Thus, the rotation axis controller instance can, for example, use the current orientation of the gripper's rotation axis (represented by the left rotation axis instance) as an input, and then output the target orientation of the gripper's rotation axis (represented by the right rotation axis instance) (based on several control inputs described later).

[0057] The lower half of Figure 6 shows a concrete example of the model shown in the upper half, which is more closely related to the serial manipulator portion of Model 200 in Figure 3. In this example, the input instance is an instance of the axis concept (e.g., representing the current orientation of the axis), and the output instance is an instance of the Cartesian concept (e.g., representing the current position of the tip of that axis). The functional instance in this example is a forward kinematics functional instance, which effectively calculates the current position of the axis tip based on its orientation. As those familiar with control theory will recognize, the purpose of forward kinematics in this case is to derive the tip position of the manipulator using its joint / axis position.

[0058] Now, returning to the serial manipulator portion of Model 200 itself in Figure 3, instance 210 named "KUKA LBR iiwa" is linked to two instances 210 named "Joint" of type "Rotation Axis". The left rotation axis instance is linked to a tip position instance named "Tip Position" (an input instance describing the current state) via a forward kinematics instance named "KUKA FK". Similarly, the right rotation axis instance is linked to another tip position instance named "Tip Position" (an output instance describing the target state) via an inverse kinematics instance named "KUKA IK".

[0059] The two tip position instances are linked to a functional instance of type "Cartesian Controller" named "Manipulator Controller". The left tip position instance functions as an input, and the right tip position instance functions as an output. Thus, the Cartesian Controller instance can take the current tip position of the manipulator's rotation axis (represented by the left tip position instance) as input and output the target tip position of the manipulator's rotation axis (then represented by the right tip position instance) (based on several control inputs described later). The forward kinematics instance effectively calculates the current position of the axis tip based on the input current orientation, as described above, and similarly, the inverse kinematics instance effectively calculates the target orientation of the axis based on the input target axis tip position.

[0060] Returning to Figure 2, the model validation unit 140 is configured to validate (check or evaluate the validity of) the model 200 against the validation rules 160 by determining whether the model 200 satisfies, fulfills, or conforms to all or at least some of those rules.

[0061] Validation rule 160 includes the definition of one or more directed acyclic graphs (or simply graphs, or directed graphs, or acyclic graphs) of defined types interconnected with parent-child relationships, where the defined types correspond to nodes and the parent-child relationships correspond to directed edges of one or more directed acyclic graphs. Next, each defined type defines the type rules of that type that must be satisfied for an instance of that type to be valid. Furthermore, each parent-child relationship between a pair of types defines those types as parent and child types corresponding to each other according to the corresponding edge direction.

[0062] The effect of a parent-child relationship is that each child type inherits (or has, includes, defines, absorbs, or takes over) each rule of its parent type. That is, each child type is interpreted to have or include each rule of its parent type (by derivation or inheritance from its parent type, or by reference to its parent type) without needing to explicitly define (i.e., repeat) those rules themselves.

[0063] In this regard, refer to Figure 7, which shows a simplified directed acyclic graph 300 of parent-child relationships 320 (represented by arrows connecting boxes) and interconnected exemplary defined types 310 (represented by boxes). The directed acyclic graph 300 can be considered simplified in that all defined types are descriptive types. Therefore, for consistency with the description instance 210 shown earlier, descriptive types are represented by boxes with narrow borders.

[0064] The directed acyclic graph 300 begins with a root type (or base type) that represents rules common to all types. This root type is then shown to be the parent type for two child types: the radiation-resistant equipment type (i.e., equipment resistant to damage or malfunction caused by high levels of radiation) and the manipulator type. In effect, these latter two types are different types or subsets of the root type. Therefore, both the radiation-resistant equipment type and the manipulator type inherit the rules of their parent type, in this case the root type (i.e., they are considered to have these rules within their own rules).

[0065] Similarly, the manipulator type is shown to be the parent type of two child types: the serial manipulator type and the parallel manipulator type. In effect, the latter two types are different types or subsets of the manipulator type. Therefore, both the serial manipulator type and the parallel manipulator type inherit the rules of their parent type, in this case the manipulator type (i.e., they are considered to contain the rules of the manipulator type within their own rules) (and, through inheritance higher up in the graph, they also inherit the rules of the root type).

[0066] For completeness, both the radiation-resistant equipment type and the serial manipulator type are shown to be parent types with respect to their child type, i.e., the radiation-resistant serial manipulator type. In effect, the radiation-resistant serial manipulator type is a specific type or subset of both the radiation-resistant equipment type and the serial manipulator type. Therefore, the radiation-resistant serial manipulator type inherits (i.e., is considered to have in its own rules) the rules of each of its parent types, in this case the radiation-resistant equipment type and the serial manipulator type (and also inherits the rules of the radiation-resistant equipment type, manipulator type, and root type through higher-level inheritance in the graph).

[0067] Therefore, validation rule 160 enables multiple inheritance of rules by type, as defined by the directed acyclic graph. Each child type defines the type rules it inherits from each parent type by referencing each parent type, rather than literally repeating those rules within itself. This allows, for example, changes to the rules in a parent type to efficiently affect all of its child types (and grandchild types, etc.). Each child type defines its own type rules within itself, beyond the type rules it inherits from each parent type. This allows updating the rules of a particular subset of a parent type (i.e., a particular child type) without affecting the parent type or its other child types. When multiple inheritance is implemented, at least one child type has at least two parent types. Multiple generations may be represented hierarchically (e.g., grandparent-parent-child, as in Figure 7).

[0068] Referring to Figure 8, a more complete (or larger or broader) directed acyclic graph 400 is shown, where defined types 310 (represented by boxes) are interconnected by parent-child relationships 320 (represented by arrows connecting the boxes). Directed acyclic graph 400 can be considered more complete than directed acyclic graph 300 in that it consists of both functional and descriptive types and comprises enough defined types (e.g., a set of defined types) to support the validation of model 200 in Figure 3 (in this example, consisting of instances of a subset of the set of defined types). To maintain consistency with the previously shown instance 20, descriptive types are represented by narrow-bordered boxes, and functional types (or active types) are represented by wide-bordered boxes.

[0069] Similar to directed acyclic graph 300, directed acyclic graph 400 begins with a root type (or base type) which may express rules common to all types. If both functional and descriptive types exist, this root type is represented by both narrow-bordered and wide-bordered boxes. This root type is shown to be the parent type for four descriptor types, namely manipulator, concept, subsystem, and HID (Human Interface Device) types, and two functional types, namely processor and processor coordinator types.

[0070] The manipulator type is the parent type for four child types: serial manipulator, 1-degree-of-freedom manipulator, gripper, and parallel manipulator.

[0071] The serial manipulator type is the parent type of a specific serial manipulator type known as "KUKA LBR iiwa," as mentioned above. Therefore, the "KUKA LBR iiwa" type is understood to be a subset (or subtype) of the serial manipulator type. Of course, "KUKA LBR iiwa" is just one example of a serial manipulator type.

[0072] As previously shown, the 1-degree-of-freedom manipulator type and the gripper type are parent types of the 2-finger gripper type, which implements multiple inheritance. The gripper type is also a parent type of the 3-finger gripper type. Therefore, the 2-finger gripper type is understood to be a subset (or subtype) of both the 1-degree-of-freedom manipulator and the gripper type, while the 3-finger gripper type is a subset of the gripper type alone.

[0073] The conceptual type, which represents pure data describing the configuration of a robotic system, is the parent type of three child types: the axis conceptual type, the Cartesian conceptual type, and the digital I / O conceptual type. The axis conceptual type is the parent type of two child types: the linear axis conceptual type and the rotary axis conceptual type. The subsystem and HID types are the parent types of the arm type and the HTC Vive handset type, respectively. Therefore, the HTC Vive handset type is a specific type of HID corresponding to the HTC Vive handset currently available from HTC. Of course, the HTC Vive handset is just one example of an HID type.

[0074] The processor type is the parent type of four subtypes: the forward kinematics type, the inverse kinematics type, the controller type, and the conceptual coordinator type. The forward kinematics type and the inverse kinematics type are the parent types of the KUKA forward kinematics type and the KUKA inverse kinematics type, respectively, and represent functionalities specific to KUGA AG products, and are therefore provided as convenient examples. The controller type is the parent type of two subtypes: the axis controller type and the Cartesian controller type. The axis controller type is the parent type of two subtypes: the rotary axis controller type and the linear axis controller type.

[0075] Finally, the processor coordinator type, concept coordinator type, and controller type are the parent types of the processor selector type, concept selector type, and Cartesian controller type, respectively.

[0076] Incidentally, inverse kinematics and forward kinematics, different types of controllers and conceptual (data) coordinators are all classified as processor types, but the processor coordinator type is associated only with the processor selector type within the provided hierarchy, for example. Basically, the kinematics module deals with the motion of an object and the forces required to provide that motion.

[0077] Of course, directed acyclic graph 400 is considered to be more complete (or larger or more extensive) than directed acyclic graph 300, but it is understood that any number of defined types may be represented by other, optionally more complete (or larger or more extensive) directed acyclic graphs. Also, although directed acyclic graph 400 is presented as a single directed acyclic graph with a set of defined types (including descriptive and functional types), it is possible to represent defined types with multiple directed acyclic graphs, such as one for descriptive types (e.g., similar to directed acyclic graph 300) and one for functional types. Therefore, it is understood that the validation rules include the definition of one or more directed acyclic graphs for defined types, and this disclosure is interpreted accordingly. If a defined type is defined by a combination of two or more directed acyclic graphs, those graphs may have their own distinct root types or they may have a common shared root type.

[0078] In this configuration, validation rule 160 also includes morphological rules that define the interconnections that, for at least one specific defined type, an instance of that specific defined type must have with other instances of at least a subset of the set of defined types within the model (such as model 200) for the model to be valid. Such morphological rules may not be provided in some configurations.

[0079] Morphology is, of course, the study of shape and structure. Within this framework or structure, ontologs (defined types and their instances) are used to construct a common structure of domain-specific information, which is then distributed and reused to make explicit assumptions. Thus, ontologs for robotics and control systems are used to provide semantic meaning to the constituent elements of robotics and control systems, represented by instances of types and the types themselves (including their hierarchies). Morphology, on the other hand, is used to provide a set of rules for enforcing syntactic meaning and a binding structure to the constituent types represented in the ontology, enabling successful operation between distributed constituents, allowing explicit assumptions to be made about the structure of a given system, and promoting interoperability.

[0080] The defined types provided in an ontology define not only functionality and structure, but also the data they represent and their external interfaces. The relationships between constituent elements in the context of an ontology define connections to other constituent elements. A type can define not only the relationships an element must have (to be considered of that particular type), but also the number of constituent elements related to it (minimum, maximum, or absolute value), and the specific types that related constituent elements must have. As a result of these relational rules, a system forms a specific morphology that is consistent across all systems using common types. These morphologies tend to fall into one of two distinct groups: structural morphologies, as shown in Figure 4, and behavioral morphologies, as shown in Figure 6.

[0081] Structural morphology is used to describe how the physical counterparts of descriptive instances are actually connected, or to construct multiple granular descriptive instances beneath each other (for example, to represent nesting relationships).

[0082] For example, Figure 4 shows an arm, which is an example of how relationships can be used to describe a physical assembly. In this example, the arm physically comprises a serial manipulator and a gripper. This means that a robotic component is considered a robotic arm (see, for example, Model 200 in Figure 3) if it consists of a serial manipulator (e.g., KUKA LBR iiwa) and a gripper (e.g., Robotiq 2-finger gripper). This is achieved using two exemplary relationship rules. The first rule is that an arm must have a serial manipulator type input relationship, and the second rule is that an arm must have a gripper type input relationship. This means that an instance of an arm type cannot be added to a model like Model 200 without a serial manipulator type instance, a gripper type instance, and the relationships (links) configured to connect them. This makes models with this framework and its component structure very discoverable and navigable, as they can recognize and explore these structures.

[0083] Similarly, an HTC Vive handset can be described by combining one Cartesian concept with one or more digital I / O concepts (represented by many-to-one symbols) (through relationships). These components can be used to describe the position of the HTC Vive handset in 3D space (provided by the tracking system) and the state of its buttons. Both of these relationships are inputs to an instance of the HTC Vive handset type, since the state of the individual components must first be evaluated before the state of the modeled HTC Vive handset can be fully known. As with the arm example in Figure 4, these relational rules allow for the creation of recognizable and expected structures within a model like Model 200, which can be used to contextualize groups and derive semantic meaning.

[0084] In Figure 4, the morphological rules of the arm structure are considered to be visually illustrated. The arrowhead (arrowhead) within the box indicates ownership of the rule. The box containing the arrowhead is the owner of the morphological rule and is responsible for implementing the associated rule.

[0085] Behavioral morphology is created when descriptive instances and functional (active) instances are combined, and is therefore useful when considering controlling a robotic system, rather than simply representing a robotic system such as robotic system 10A (recall the solid and dashed arrows in Figure 1B).

[0086] The upper half of Figure 6 shows a processor example as a simple example illustrating how descriptive and functional instances can be used together. This example uses the basic types of concepts and processors and has no particular functional purpose, but is used to establish a template that instances of a particular type can follow. The processor type is shown as defining the first and third of the following three relationships or morphological rules: 1) An instance of a processor type can have one or more input relationships (indicated by dotted lines and one-to-many symbols) of a conceptual type; 2) An instance of a processor type can have one or more input relationships of a processor type; and 3) An instance of a processor type must have at least one output relationship (indicated by solid lines and one-to-many symbols) of a conceptual type. These three relationship rules exist for behavioral reasons and are there to ensure that instances of processor types function. The first rule is to provide input to the processor. This may not be necessary if the processor is an "open loop" and therefore optional. The second rule is to allow for subprocesses that must first complete in order for the processor to function. This is also optional as it may not be necessary (not shown in Figure 6). The third rule is to provide output to the processor. This is essential because if there is no output, the processor does not contribute to the system state and is therefore redundant.

[0087] Therefore, it is understood that such morphological rules define the interconnections that instances of a particular defined type must have with other instances of the defined type within the model for the model to be valid. Such morphological rules may be defined in the type rules for each type, or they may be defined separately, and may not be provided in some configurations. An example of such rules is shown above, but of course, many other interconnected groups of instances of a type may be defined in the rules. Figure 5 shows such an example, where the robot component is considered a robot arm if it consists of a serial manipulator and a gripper (as described above), each having a rotation axis, and each rotation axis has input and output instances which may be connected by functional instances (indicated by dashed arrows).

[0088] It should be noted that function is either obvious or implicit in morphology, defined by morphological rules. For example, the forward kinematics example at the bottom of Figure 6 is a less abstract example of the processor morphology shown at the top. In this case, the overarching conceptual type is inherited by the more specific axial and Cartesian conceptual types, but since both the axial and Cartesian concepts are inherited from the conceptual type (see Figure 8), this modification still satisfies the morphological rules established by the underlying processor type. Thus, child types can inherit morphological rules from their parent types. Since a forward kinematics instance takes one or more axial conceptual instances (in this case, joint positions) as input and produces one Cartesian conceptual instance (in this case, tip position) as output, the purpose of forward kinematics to derive the tip position of a manipulator using joint positions is obvious in morphology. Similar to the structural morphologies described earlier, these behavioral morphologies help create structures that are consistent, discoverable, and navigable within an overall framework.

[0089] Returning to Figure 3, it can be recalled that each node represents an instance 210 of the defined type 310. In this configuration, each instance is implemented as an instance data unit that optionally defines a link to another instance of model 200.

[0090] Generally, each instance data unit (sometimes called a simplex) contains a defined type indication of what it is an instance of, along with typed data. Each instance data unit may further contain definitions of the status of one or more supported commands. Conveniently, instance data units can have the same structured format as one another.

[0091] Taking into account optional active control, an instance data unit 500 may be associated with an extension data unit (not shown) that defines tasks performed to manipulate data of that instance or interconnected instances. The combination of an instance data unit 500 and an extension data unit may itself be considered an extension or extension data unit (extended to the instance data unit 500), but it may be advantageous for the instance data units 500 to have the same structured format as each other, regardless of whether active control is implemented.

[0092] More specifically, in this configuration, instance data units use the same format for their internal data representation and have the same external interface to one another. This data representation can be used as part of a communication protocol to enable distributed components of a single control system to exchange data without prior knowledge of each other. This means that a model, such as Model 200 (for example, acting as a control system), can grow to incorporate new hardware and control features without modifying other distributed components.

[0093] The instance data unit 500 facilitates functionality within a control system, including defining data, describing input / output relationships, invoking functionality, representing a system (or part thereof), controlling active elements (i.e., controllers), or communicating with other instance data units within the system.

[0094] Figure 9 is a schematic diagram of the instance data unit 500, showing its structured format. A typical instance data unit 500, “s” m ∈ S, is a self - contained unit of information and is a tuple (or an ordered, defined list). s m = <type m ,id m ,data m ,rlsp m ,cmd m ,cmdbox m > Here, ·type m corresponds to the type in Figure 9 and is a unique identifier that references the type from the type graph (e.g., the directed acyclic graph 400). The instance data unit 500 must satisfy all verification rules in the referenced type (and its ancestors by inheritance as described above), or for the type represented. ·id m corresponds to the path and ID in Figure 9 and is a unique identifier that identifies the instance data unit 500. The path can enable the grouping / organization of the instance data units 500. ·data m corresponds to the data in Figure 9 and contains an identifier / value pair of data items of a predetermined data type, defined by the verification rules in the referenced type or for the type represented. The value may be, for example, an int, float, bool, or string value, or may include an array. ·rlsp mThis corresponds to the relationship in Figure 9, defining one or more links 210 to other instance data units 500 as identifier / path pairs to retrieve information or invoke commands (therefore defining an entire model such as model 200). Links can be defined as arrays and can have direction as described above (e.g., representing an input / output relationship). Links also have association types that represent (and implement morphological rules for) the morphological relationships mentioned earlier. ·cmd m This corresponds to the commands in Figure 9 and defines one or more commands and their availability, called identifier / availability pairs. A command can be thought of as a function that can be invoked to perform a task or change. A command can be declared as a list of response parameters sent with the invocation and can return response parameters upon completion, and the parameters can have data types and can be arrays. • cmdbox m This corresponds to the command mailbox in Figure 9 and is a standardized interface for exchanging commands. The command mailbox can contain a queue of commands sent to instance data unit 500 (by other instance data units). Each command may have a unique number or identifier, request parameter values, response parameter values, and a status (e.g., pending, sent, received, successful, failed, etc.).

[0095] As mentioned above, when considering active control, the instance data unit 500 may be associated with an extension data unit (not shown) that defines tasks to be performed to manipulate the data of the instance or interconnected instances.

[0096] In such cases, the tuple can be thought of as being extended (separately from the instance data unit 500) as defined below. • task mThis defines the function that is executed in each loop cycle of the entire system (which will be explained in more detail later).

[0097] Alongside the instance data unit 500, there is also the type data unit 600 (sometimes called a simplex). Figure 10 is a schematic diagram of the type data unit 600, showing its structured format.

[0098] One or more directed acyclic graphs of a defined type (see, for example, directed acyclic graph 400) comprises a type data unit 600 for each defined type 310, each of which defines the defined type and optionally defines parent-child relationships for each of its parent types for each defined type other than the root defined type corresponding to the root node of one or more directed acyclic graphs.

[0099] Generally, each type data unit 600 comprises an instruction for its defined type 310 and a definition of at least one data type for which data type data of an instance of the defined type must exist. Each type data unit 600 may further comprise a definition of how the status of one or more supported commands must be represented within an instance 210 of the defined type 310. Advantageously, the type data units 600 may have the same structured format as one another.

[0100] Furthermore, the structured format of the instance data unit 500 may correspond to the structured format of the type data unit 600, as can be seen by comparing Figures 9 and 10. This results in a standardized (but extensible) data unit or block, guaranteeing standardized data exchange between blocks. Standardized information exchange is guaranteed or made possible by a standardized interface. Because data units of a given structure can be used at various levels of the inheritance hierarchy, not only standardization but also extensibility in the form of polymorphism becomes possible. This is one of the advantages of using an ontology-type system.

[0101] As shown in Figure 10, the ID portion is a unique identifier that identifies the type 310. The data rule portion includes identifier / definition pairs, with each definition being a rule that enforces that the instance data unit 500 (of that type) must include data items of a specified data type and, optionally, a minimum and / or maximum array size. The relation rule portion similarly includes identifier / definition pairs, with each definition being a rule that enforces that the instance data unit 500 (of that type) must define one or more links (of a specific data type) with, optionally, a minimum and / or maximum array size, and corresponds to the morphological rule. The command rule portion includes identifier / definition pairs, with each definition being a rule that includes a list of request and / or response parameter definitions. Each parameter definition enforces that the instance data unit 500 (of that type 310) must define commands with parameters of a specific data type with, optionally, a minimum and / or maximum array size.

[0102] The parent type portion can link types 310 together in a directed acyclic graph, such as the directed acyclic graph 400. As mentioned above, child types inherit the rules of their parent types and implement multiple inheritance, so one or more parent types are allowed for each child type. All rules, including those inherited from ancestor types, must be satisfied by the instance data unit 500 (i.e., instance 210) of that type 310.

[0103] As described above, the modeling device 100 can be used to represent the robot, but it cannot control it. However, the modeling device 100 can also be used to actively control the robot 20 (or robot system 10A). For example, such active control may be applied in the active functionality of the "[Function]" instance (which may represent a combination of multiple instances) in Figure 5, whose output becomes the request or target data of the "Output" rotation axis instance, which is then applied to control the robot 20 (or robot system 10A). Of course, in specific applications, any complex active functionality can be implemented.

[0104] In this regard, the modeling device 100 may include an interface for connection between the modeling device 100 and the robot system 10A, and the modeling device 100 is configured to receive data (e.g., signals or data signals which may be called control signals) from the robot 20 via the interface after the model 200 has been verified, update the model based on the received data, and / or output data to the robot 20 (or robot system 10A) via the interface in order to control the robot 20 (or robot system 10A) based on the model 200. For example, if the modeling device 100 is employed to represent or describe the robot 20 (or robot system 10A) rather than to control the robot 20 (or robot system 10A), the model 200 may consist only of descriptive instances, or only of descriptive instances and instances of a subset of available functional types (e.g., including forward kinematics types to discover information about the tip position, i.e., to describe the tip position, without controlling the robot 20 (or robot system 10A)).

[0105] In particular, looking back at Figures 3, 8, 9, and 10, it will be understood that by using standardized instance data unit 500 (Figure 9) and type data unit 600 (Figure 10) with defined type ontologs (see Figure 8), common building blocks can be adopted for the model (see Figure 3) at a granularity level related to the minimum number of reusable units.

[0106] The concepts behind the self-descriptive distributed data model are built upon instances 210 with defined types 310, associated with software ontology and morphological techniques. The ontology provides semantic meaning to the components of the robot and control system, while morphological rules associate syntax with the components represented within the system.

[0107] An ontology (represented by defined types and associated type rules) ensures the common and consistent use of domain knowledge and allows for the explicit expression of domain assumptions. For example, in a multi-device system of modeling devices 100 (sometimes referred to as agents in this context), if validation rules (type rules, and optionally morphological rules) are shared before execution, models (such as model 200) can be validated in a way that ensures interoperability between models across modeling devices 100. At runtime, modeling devices 100 can make explicit assumptions using the knowledge represented in the shared ontology (and shared models), thereby ensuring consistent reuse of distributed domain knowledge across modeling devices 100.

[0108] Validation rules (type models or rules, and optionally morphological rules) can be distributed, for example, at runtime, eliminating the need for prior knowledge. Furthermore, validation rules can be fully customized for specific purposes rather than using predetermined types. If two modeling devices 100 (agents) need to interoperate, the types within their type models must not contradict each other, but the entire models do not need to be identical.

[0109] Consider the following example: Agent A declares an axis type and includes position, velocity, and acceleration data. Agent B also declares an axis type but states that it includes position, velocity, and torque data. These two agents are not interoperable because they do not agree on the definition of an axis type.

[0110] However, consider this next example: Agent A declares an axis type as before. Agent B declares an axis type with a definition that matches Agent A's, but also declares another type that inherits from axis and includes torque data, which we call axis with torque. These two agents are interoperable because they do not conflict in the definition of any of the types, but Agent B can extend the axis type to include the data it needs. This not only means that Agent B can add data to the system, but also that Agent A can use an instance of axis with torque in any interface designed for an instance of axis type, since one inherits from the other, and therefore must include all the necessary information from axis type.

[0111] Figure 11 is a schematic diagram of a multi-device system 700 consisting of first and second modeling devices 100-1 and 100-2 configured to communicate with each other. Each of the first and second modeling devices 100-1 and 100-2 can be considered a modeling device 100, as in Figure 2. Of course, the multi-device system 700 may comprise two or more modeling devices 100 configured to communicate with each other.

[0112] For the sake of clarity, the modeler 120, model verification unit 140, and verification rules 160 in Figure 2 are not explicitly shown in the modeling devices 100-1 and 100-2. Instead, we will focus on the model 200 in Figure 3 and the type graph 400 in Figure 8.

[0113] Therefore, the first modeling apparatus 100-1 is shown to include a first model 200-1 corresponding to model 200 and a first type graph 400-1 corresponding to type graph 400. Similarly, the second modeling apparatus 100-2 is shown to include a second model 200-2 corresponding to model 200 and a second type graph 400-2 corresponding to type graph 400.

[0114] The first modeling device 100-1 may be considered here as a first agent or a first cluster, and similarly, the second modeling device 100-2 may be considered here as a second agent or a second cluster. Thus, for example, the first modeling device 100-1 controls a first model 200-1 which is represented as a first robot system 10A-1 (not shown), and the second modeling device 100-2 controls a second model 200-2 which is represented as a second robot system 10A-2 (also not shown).

[0115] As shown in Figure 11, the first and second modeling devices 100-1 and 100-2 are equipped with first and second communication interfaces 800-1 and 800-2, respectively, and are therefore configured to communicate with each other. Here, the first modeling device 100-1 and the second modeling device 100-2 are in a publisher-subscriber relationship, with the first modeling device 100-1 being the publisher and the second modeling device 100-2 being the subscriber. In this sense, the first modeling device 100-1 can be considered a "local" device, and the second modeling device 100-2 can be considered a "remote" device.

[0116] In its role as a subscriber, the second modeling device 100-2 is configured to determine interoperability between the first model 200-1 of the first robot system 10A-1 and the second model 200-2 of the second robot system 10A-2. For this purpose, the first and second modeling devices 100-1 and 100-2 communicate via the first and second communication interfaces 800-1 and 800-2, respectively, so that the second modeling device 100-2 (acting as a subscriber) retrieves the first model 200-1 (which describes the first robot system 10A-1) and the first type graph 400-1 from the first modeling device 100-1 (acting as a publisher and making the first model 200-1 and the first type graph 400-1 available), as shown in Figure 11.

[0117] As mentioned above, the first model 200-1 can be "extended" in some configurations to enable active control of the first robot system 10A-1. However, as explained in relation to Figure 9, such active control does not need to be represented in the instance data unit 500 (and, in some configurations, is not advantageously represented). In such cases, the first model 200-1 may be considered as the extended model 200-1EX (optional, shown in parentheses in Figure 11), while the non-extended first model 200-1 (not the extended model 200-1EX) may be communicated to the second modeling device 100-2 to simply describe the first robot system 10A-1 to the second modeling device 100-2. With the non-extended model being the model to communicate along the instance data unit 500, the advantages of standardized data units and interfaces are enjoyed in terms of rapid interoperability between multi-device systems (i.e., one modeling device does not need to worry about active control performed by another modeling device).

[0118] Of course, similarly, the second model 200-1 may also be the extended model 200-2EX in some configurations.

[0119] As explained earlier, each of the first and second modeling devices 100-1 and 100-2 is configured to verify its model against a verification rule by determining whether the model satisfies that verification rule, as shown in Figure 11. Specifically, the first modeling device 100-1 is configured to verify the first model 200-1 (or 200-1EX) against the first type graph 400-1 (representing the verification rule of the device), and the second modeling device 100-2 is configured to verify the second model 200-2 (or 200-2EX) against the second type graph 400-2 (representing the verification rule of the device). In this way, the first and second modeling devices 100-1 and 100-2 can operate independently of each other according to the above teaching.

[0120] However, the multi-device modeling system 700 is further configured to determine interoperability between the first model 200-1 and the second model 200-2 by verifying the first model 200-1 against the verification rules of the second modeling device 100-2, and / or verifying the second model 200-2 against the verification rules of the first modeling device 100-1, and / or determining compatibility between at least the first model-specific portion of one or more directed acyclic graphs 400-1 of the first modeling device 100-1 and the second model-specific portion of one or more directed acyclic graphs 400-2 of the second modeling device 100-2.

[0121] In the example shown in Figure 11, the second modeling apparatus 100-2 is configured to determine interoperability by determining compatibility between the first type graph 400-1 and the second type graph 400-2, as shown.

[0122] The first and second type graphs 400-1 and 400-2 can be considered, like type graph 400 in Figure 8, as complete “device-level” type graphs for their respective modeling devices. In fact, this is a way to determine “complete” interoperability between two modeling devices (and therefore their models). However, a more efficient way to determine interoperability may be to do this to the extent necessary to make the first and second models 200-1 and 200-2 work (and to “assume” or assume that they satisfy or meet other rules (i.e., rules that are unaffected and therefore do not need to be tested or matched)).

[0123] For this purpose, the first type graph 400-1 may be considered a first model-specific portion of one or more directed acyclic graphs of the first modeling apparatus 100-1, and may be a portion of one or more directed acyclic graphs of the first modeling apparatus 100-1 that includes each root-defined type corresponding to each root node, all defined types having multiple instances in the first model 200-1, and each other defined type connected between them via parent-child relationships. Figure 12 is a schematic diagram of such a model-specific first type graph 400-1(MS), and it can be seen that it does not include all the types of the type graph 400 in Figure 8 (and therefore may be considered not to represent all the rules of the type graph 400 in Figure 8). For example, the type graph 400 in Figure 8 may be considered to define a set of defined types (and their associated rules), and the type graph 400-1(MS) in Figure 12 may be considered to define a subset of the set of defined types (and their associated rules). In some configurations, the set itself may be a subset of a larger set of defined types. The first modeling apparatus 100-1 may be configured to generate a first model eigenpart of one or more directed acyclic graphs of the first modeling apparatus 100-1.

[0124] The second type graph 400-2 may also be considered as a second model-specific portion 400-2(MS) of one or more directed acyclic graphs of the second modeling apparatus 100-2, and is understood to be a part of one or more directed acyclic graphs of the second modeling apparatus 100-2, comprising each root-defined type corresponding to each root node, all of the defined types having multiple instances in the second model 200-2, and each of the other defined types connected between them via parent-child relationships. The second modeling apparatus 100-2 itself may be configured to generate a second model-specific portion of one or more directed acyclic graphs of the second modeling apparatus 100-2.

[0125] It is worth noting that the structured formats of the instance data unit 500 and the type data unit 600 may be identical or substantially identical between the first and second modeling devices 100-1 and 100-2. This enables standardized (but extensible) data units or blocks throughout the multi-device system 700, ensuring standardized data exchange between blocks and between the first and second modeling devices 100-1 and 100-2.

[0126] The multi-device modeling system 700 is configured to determine compatibility if at least a first model-specific portion of one or more directed acyclic graphs of the first modeling device 100-1 and at least a second model-specific portion of one or more directed acyclic graphs of the second modeling device 100-2 have substantially identical defined types in both of those portions, or define at least the same type rules for each other, or do not define conflicting rules or conflicting hierarchies for defined types.

[0127] In principle, either the first modeling apparatus 100-1 or the second modeling apparatus 100-2 can determine compatibility between a first model-specific portion of one or more directed acyclic graphs of the first modeling apparatus 100-1 and a second model-specific portion of one or more directed acyclic graphs of the second modeling apparatus 100-2. However, in the case of Figure 11, the second modeling apparatus 100-2 (acting as a subscriber) determines compatibility between the first and second type graphs 400-1, 400-2 (which are apparatus-level or model-specific graphs).

[0128] Incidentally, taking the first modeling apparatus 100-1 as an example, the apparatus can validate its model 200-1 against those validation rules by determining whether the model satisfies those rules, using a subset of the validation rules corresponding to the model-specific graph 400-1(MS) in Figure 12, rather than against all of the validation rules corresponding to the apparatus-level graph 400 in Figure 8. In other words, other rules, i.e., rules that are not affected and therefore do not need to be tested or checked, may be satisfied, assumed to be satisfied, or considered to be satisfied.

[0129] Considering compatibility, Figure 13 shows two type graphs that are compatible with each other because they have the same defined types. Figure 14 shows two type graphs that are compatible with each other because the defined types in both type graphs are the same. It is true that only type A is common to both graphs, but the other types shown do not conflict between the two graphs.

[0130] On the other hand, Figure 15 shows two type graphs that are incompatible because types B and B' are not substantially identical (not the same as each other), and therefore the rules for types D and E may differ between the two type graphs. Figure 16 also shows two type graphs that are incompatible because types G and G' are not substantially identical (not the same as each other). Here, types G and G' are effectively identified as the same type (and therefore both are represented by the same letter), but are assumed to be different in some respects because G' inherits from C and H, and G inherits only from C. The two type graphs are compatible if G' is a completely different type from type G, such as type F.

[0131] Returning to Figure 2, Figure 17 is a schematic diagram of a high-level method 900 that is useful for understanding that the modeling apparatus 100 may operate in a series of loop cycles 910.

[0132] The operation in each loop cycle is triggered by a tick signal (synchronization signal) that can be synchronized with the internal clock signal of the modeling device 100. Therefore, in each loop cycle 910, the tick signal is awaited (920), and then the update is read (930).

[0133] Reading an optional update (930) may include reading data from a remote modeling device such as modeling device 100-2 and / or reading data from the modeling robot system 10A. In this way, the current state data values ​​can be entered into the description instance 210 of the model 200.

[0134] If the modeling device 100 controls the robot system 10A and the model 200 has a functional (active) instance 210, the tasks associated with the relevant instance 210 may be performed in an order determined by the relationships between the instances of the model (940).

[0135] Next, any updates can be written (950). This may include writing data to a remote modeling device such as a modeling device 100-2 and / or writing data to the robot system 10A being modeled in order to control the robot system 10A.

[0136] Figure 18 is a schematic diagram of high-level methods 960 and 970 for carrying out the present invention, and is also useful for understanding the embodiments disclosed herein.

[0137] Method 960 can be used with modeling apparatus 100, or with either modeling apparatuses 100-1 and 100-2. Taking modeling apparatus 100 as an example, Method 960 is a method for modeling a robot system 10A, comprising the steps of: modeling using model 200, wherein model 200 includes instances 210 of at least a subset of the defined set of types 310 interconnected with link 220 as described above; and verifying model 200 by determining whether model 200 satisfies verification rule 160.

[0138] Method 970 may be used by a multi-device system 700 comprising first and second modeling devices 100-1 and 100-2, or either modeling devices 100-1 or 100-2. As shown, Method 970 includes an instance of Method 960 for each modeling device. For example, Method 960-1 in the upper part of Figure 18 may correspond to the first modeling device 100-1, and Method 960-2 in the lower part may correspond to the second modeling device 100-2.

[0139] In addition, Method 970 includes determining interoperability between a model of a first robot system and a model of a second robot system, as described in relation to Figure 11. This may include verifying model 200-1 of the first robot system against the verification rules of the first modeling apparatus 100-1 or Method 960-1, and / or verifying model 200-2 of the second robot system against the verification rules of the second modeling apparatus 100-2 or Method 960-2, and / or determining compatibility between at least a first model-specific portion of one or more directed acyclic graphs of the first modeling apparatus 100-1 or Method 960-1 and a second model-specific portion of one or more directed acyclic graphs of the second modeling apparatus 100-2 or Method 960-2.

[0140] In summary, as can be seen from Figures 11 and 18, the present invention can be embodied as a multi-device modeling system comprising first and second modeling devices configured to communicate with each other. Of course, the examples of first and second modeling devices constituting the system are merely convenient examples, and any number of modeling devices (e.g., two, three, four, or more) may be provided in the system.

[0141] Each of the first and second modeling devices is for modeling a robotic system and may be configured to model the robotic system using a model and to validate the model against those validation rules (see, for example, Figures 8 and 10) by determining whether the model satisfies those validation rules. A model (see, for example, Figure 3) comprises instances (see, for example, Figure 9) of at least a subset of a defined set of types interconnected by links, each instance being a description of an element of the robotic system, and the links between instances representing the relationships between them. Here, as previously stated, the robotic system may be considered to be one or more robots, or comprising one or more robots.

[0142] For each of the first and second modeling apparatuses, as shown in Figure 8 as an example, the verification rules may include the definition of one or more directed acyclic graphs of defined types interconnected by parent-child relationships, where the defined types correspond to nodes and the parent-child relationships correspond to the directional edges of one or more directed acyclic graphs. Each defined type (see, for example, Figure 10) defines the type rules of that type that an instance of that type must satisfy in order to be valid, and each parent-child relationship between a pair of types defines those types as parent and child types to each other according to the direction of the corresponding edges, and each child type inherits the respective type rules of its parent type according to the parent-child relationship.

[0143] Referring to Figure 11, in the example, the first modeling device can model the first robot system using the first model of the first robot system, and the second modeling device can model the second robot system using the second model of the second robot system.

[0144] For each of the first and second modeling devices, the definition of one or more directed acyclic graphs of a defined type may comprise a type data unit (see, for example, Figure 10) for each defined type, each of which defines the defined type and defines the parent-child relationships to each of its parent types for each defined type other than the root defined type corresponding to the root node of one or more directed acyclic graphs. Furthermore (see, for example, Figure 10), each type data unit may include an indication of the defined type and a definition of at least one data type that the data of an instance of the defined type must be for it to be valid. Furthermore, a defined type may include (at least) a descriptive type, each instance of which describes the current state and / or target state of that element of the robot system. Furthermore, each instance may be an instance data unit (see, for example, Figure 9) that defines each of the links to one or more other instances of the model, each instance data unit including an indication of the defined type that it is an instance of and typed data. Furthermore, validation rules may include morphological rules that define the interconnections that instances of a particular defined type must have with other instances of that defined type in the model for the model to be valid, for at least one specific defined type to work.

[0145] For both the first and second modeling apparatus, type data units may have the same structured format as each other, instance data units may have the same structured format as each other, and the structured format of the instance data units may correspond to the structured format of the type data units.

[0146] In this regard, instance data units can have the same structured format as one another, in the sense that they have at least to some extent the same constituent parts (individual or separated sections, fields, encapsulated parts). For example, as seen in Figure 9, all instance data units may have “Path,” “ID,” “Type,” “Data,” and “Relationship” parts, even if the contents of these parts may differ from instance data unit to instance data unit. For example, as seen in Figure 10, all type data units may have “ID,” “Parent Type,” “Data Rule,” and “Relationship Rule” parts, even if the contents of these parts may differ from type data unit to type data unit. Furthermore, the structured format of an instance data unit may correspond to the structured format of a type data unit, in the sense that both formats define constituent parts (sections, fields, encapsulated parts), and the constituent parts of a type data unit define the rules that the corresponding constituent parts of an instance data unit must satisfy for the instance data unit to be valid. For example, referring to Figures 9 and 10, the “Data Rule” part of a type data unit for a given type can define the rules that the “Data” part of an instance data unit of that type must satisfy for that instance data unit to be valid. Similarly, the "relationship rules" portion of a type data unit for a given type can define the rules that the "relationship" portion of an instance data unit of that type must satisfy in order for that instance data unit to be valid.

[0147] As a result, data units that can be communicated within and between modeling devices are standardized in terms of structure / format so that they can be read and interpreted / understood by both modeling devices. Modeling devices may be configured to interpret / understand instance data units and type data units based on their known or predetermined standardized structure / format. Thus, how and in what form information is represented may be known across the modeling devices of the system, and as a result, an instance data unit of one modeling device may be read and understood / interpreted by any other modeling device in the system. Similarly, a type data unit of one modeling device may be read and understood / interpreted by any other modeling device in the system.

[0148] Simply put, modeling devices communicate information internally and with each other using the same data structure standards (information representation standards, communication standards, or communication languages). A data structure standard can be thought of as specifying a set of defined or labeled fields and defining how information is organized or represented. That is, an instance data unit conforms to (or conforms to, satisfies, or fulfills) a data structure standard or definition known to or accessible by the modeling device(s). Similarly, a type data unit conforms to (or conforms to, satisfies, or fulfills) a data structure standard or definition known to or accessible by the modeling device(s). Therefore, it is known which parts of a data unit to look at to find specific information and how that information is represented; in other words, a data unit is understandable by the modeling device. Each type data unit expresses rules (in the way defined by) a data structure standard that enables instance data units of the same type to be checked for validity and interpreted / understood (processed / given meaning in relation to a robotic system). Therefore, if a given modeling device has a corresponding type data unit, it can interpret / understand an instance data unit. In a sense, a data structure standard is defined in part the same way for all modeling devices (e.g., so that all type data units can understand / interpret) and in part extended by type data units for the type graph (e.g., see Figure 8) (e.g., so that instance data units can be understood / interpreted if they belong to a type defined in that type graph). Furthermore, the type data units of a given modeling device collectively define a tree structure of types (e.g., see Figure 8) or hierarchy that utilizes inheritance to represent the rules of a given type in an efficient way and can be built over time (by adding new types that inherit from existing types) without affecting how existing types are used.

[0149] Part of ensuring the existence of this common data structure standard includes determining interoperability. A multi-device modeling system may be configured to determine interoperability between a first model and a second model by verifying the first model against the verification rules of a second modeling device, and / or verifying the second model against the verification rules of a first modeling device, and / or determining compatibility between at least (or all) first model-specific portions of one or more directed acyclic graphs of the first modeling device and at least second model-specific portions of one or more directed acyclic graphs of the second modeling device.

[0150] Therefore, one modeling device can understand communications (instance data units and optionally type data units) from other modeling devices, and thus can understand the configuration and state of the robotic systems of the other modeling devices.

[0151] For example, a second modeling device may be configured to obtain a first model from a first modeling device. If the first model satisfies the verification rules of the first modeling device and is determined to be compatible (between the type graph or related parts thereof), or if the first model is determined to satisfy the verification rules of the second modeling device, the second modeling device may be configured to generate (and optionally output in a format understandable to a human user) a representation of the first robot system based on the first model, representing the current state of the first robot system (e.g., structure / configuration and / or operating state).

[0152] The output representation of a robotic system (for example, for a human user) may include a graphical representation such as a table of values ​​(e.g., a labeled table) or a 2D or 3D representation (e.g., a 2D or 3D computer graphics model). If such a representation is updated over time, the representation may take the form of an updated graphical representation such as an updated table of values, one or more update traces (e.g., values ​​plotted against time), or a dynamic 2D or 3D representation (e.g., a dynamic or animated 2D or 3D computer graphics model).

[0153] As an example, Figure 11 shows that the model 200-1 (instance data unit) of the robot system of modeling device 100-1 is verified against the type graph 400-1 (type data unit). If it is found that the type graph 400-1 is compatible with (or valid against) the type graph 400-2 of modeling device 100-2, then modeling device 100-2 can understand the communication (instance data unit) from modeling device 100-1, and as a result, can understand the configuration and state of the robot system of modeling device 100-1. In effect, modeling device 100-2 can reconstruct a local model corresponding to model 200-1 based on the communication (instance data unit) from modeling device 100-1, and generate a meaningful representation of the robot system of modeling device 100-1.

[0154] The “current state” (and similarly, “future state”) of a robotic system (such as a robot) can be understood as defining the general internal state of the robotic system and consisting of configuration and operating states. Considering such a robotic system as a whole (or considering a part of it), the configuration may include, for example, the elements that comprise it, how those elements are connected, and the functionality of those elements. Operating states may include technically meaningful states such as position, orientation, velocity, torque, power, fault conditions, and behavior—of course, these are merely examples. Thus, the current state can include dynamic and / or static information that defines the technical characteristics of the robotic system. General internal states of a robotic system, as considered here, may be thought to include operating modes, technical conditions, or events related to the system's internal functions. Some of these states change dynamically and are automatically detected. The presentation of such states may prompt a user (or other system) to interact with the robotic system, for example, to avoid technical malfunctions.

[0155] The current state of the robot system can, of course, change / update over time. In an example, the first modeling device may be configured to read data from the first robot system and, based on the read data, populate at least one instance of the descriptive type of the first model with the current state data values. The second modeling device may then be configured to obtain the first model having the current state data values ​​and generate a representation of the first robot system based on the first model having the current state data values. In this way, the information (which may be presented to the user) may relate to the prevailing internal state in the technical system (robot system) and enable the user to operate the technical system appropriately.

[0156] The current state of the robot system may be monitored from time to time or periodically, for example, in a continuous cycle. In the running example, the first modeling device may be configured to read data from the first robot system in a continuous cycle and populate at least one instance of the descriptive type of the first model with current state data values ​​based on the data read, and the second modeling device may be configured to output a representation of the first robot system based on the first model updated in a given cycle.

[0157] Of course, the use of a common data structure standard has advantages not only between modeling devices but also within a single modeling device. As in the example in Figure 3, a single modeling device can model a robot system with multiple different robots from different manufacturers, for example (KUKA and Robotiq in Figure 3). The corresponding exemplary type graph in Figure 8 represents both robots that have some overlap or commonality between types (for example, both the KUKA LBR iiwa and the Robotiq 2-finger gripper have grandparent type manipulators), so both robots can be understood / represented (or controlled) in the same way, as they have the same level of versatility where the types overlap (i.e., are the same), in this case as manipulators. For example, both robots in Figure 3 have a rotation axis modeled as an instance of the rotation axis type, and therefore can be understood / represented (or even controlled) in the same way with that level of versatility (even though they have rotation axes of different specific types at lower levels of the type graph). In effect, if the model-specific parts of the type graph overlap at least partially, or are identical to each other (beyond the root type) for two robots, there exists a level of generality (e.g., manipulator) that allows the two robots to be understood / represented (or controlled) in the same way, even if there are non-overlapping parts (e.g., serial manipulator vs. gripper) that indicate the two robots may be understood / represented (or controlled) in different ways at a lower level.

[0158] Similar to reading the current state of the robot system described above, the robot system can be controlled (or written to) based on a model over time, thereby changing and updating the current state of the robot system over time. In the example, the first modeling device may be configured to write data to (control) the first robot system. The second modeling device may be configured to take a first model having current state data values ​​and future state data values ​​and to generate a representation showing how the first robot system is being controlled.

[0159] In summary, the technologies described herein provide solutions not only for the requirements of safely integrating complex robotic systems, but also for long-term maintainability and scalability, and the ability to expand the system with minimal effort. This is advantageous, for example, in the scenario of the nuclear industry. By abstracting the hardware, control system implementation, and operator interface, incompatibility issues arising from hardware changes are reduced, the reusability and reconfigurability of the control solution are improved, and a consistent operator interface can be provided regardless of the hardware configuration.

[0160] Therefore, the control system framework aims for long-term maintainability and scalability. To achieve this, the system infrastructure implements two concepts: 1) low coupling between components, and 2) high coupling of very fine-grained modular components. A building block methodology (instance data units and type data units) is provided. Required system functionality is provided by combining multiple common plug-and-play components (instance data units and type data units). In some configurations, instance data units and type data units use the same structure for internal data representation and have the same external interface (i.e., a standardized interface). This data representation can be used as part of a communication protocol to enable distributed components of a control system (multiple clusters, agents, modeling devices) to exchange data without prior knowledge of each other. This means that the control system can grow and incorporate new hardware and control functions without modifying other distributed components, while minimizing the impact on the local system.

[0161] A self-descriptive distributed data model consists of a collection of instance data units. Each instance data unit contains information about how it connects to other instance data units, the functionality it offers, and its associated types. These types form a software ontology that includes rules about the syntactic information stored in each instance data unit (type rules) and morphological rules for creating a standardized structure. The inherited nature of the ontology provides semantic meaning to various control system components. Within the framework, the ontology is used to build a common structure of domain-specific information, which is then used to distribute and reuse components and make explicit assumptions. In addition, morphology provides syntactic meaning and is used to create structures between components using the types represented in the ontology. These structures are used to standardize distributed components, enable explicit assumptions about the contents of a given system, and promote interoperability.

[0162] This system may be considered to use only one ontology model, i.e., a type model ontology. This system model (e.g., Figure 3) is a system model that features a type model ontology (e.g., Figure 8) to organize the necessary information, rather than the ontology itself.

[0163] Connections between instance data units are described using “relationships.” A defined type can define not only the relationships it must have (to be considered that particular type), but also how many (minimum, maximum, or absolute) instance data units must be associated with it and their associated types. These relation rules produce systems with consistent specific forms across systems using the same type. These forms tend to fall into one of two distinct groups: structural (e.g., a robotic arm consisting of a serial manipulator and a gripper) and behavioral (e.g., an inverse kinematics solver requiring a Cartesian position input and outputting several joint angles). These relation rules result in enhanced interoperability, and they are used to explicitly assume integrated components.

[0164] Self-descriptive data representations are favorably used here. Standardized but extensible data interfaces (defined by instance data units and type data units) are developed to provide interoperability, and semantic meaning is self-descriptive by the components through defined types associated with software ontologs for the robot and control system components. Software morphologies are implemented and used to provide syntactic meaning to support structural interpretation in data exchange between these interfaces. The robot and control system knowledge structures are distributed among agents at runtime.

[0165] The use of standardized interfaces, combined with encapsulation that employs low coupling between components and high coupling of fine-grained modular components, helps achieve modularity and ease of testing. Quality and maintainability requirements can then be met through lifecycle management processes and effective component-based development techniques. Systems or devices as described herein can be run with a constant memory footprint, minimizing runtime allocation and memory leaks.

[0166] Figure 19 is a block diagram of a computing device 1000 that embodies the present invention and can be used to implement any of the devices or methods described herein. The computing device 1000 comprises a processor 993 and memory 994. Optionally, the computing device also includes a network interface 997 for communication with other computing devices, such as first and second modeling devices 100-1 and 100-2.

[0167] For example, an embodiment may consist of a network of such computing devices. Optionally, the computing devices may also include one or more input mechanisms such as a keyboard or mouse 996, and one or more display devices such as monitors 995.

[0168] Each component can be connected to one another via bus 992.

[0169] Memory 994 may include computer-readable media, and the term may refer to one or more media configured to carry computer-executable instructions or to have data structures stored thereon (e.g., centralized or distributed databases and / or associated caches and servers). Computer-executable instructions may include instructions and data that are accessible by, for example, a general-purpose computer, a dedicated computer, or a dedicated processing device (e.g., one or more processors) and cause them to perform one or more functions or operations. Accordingly, the term “computer-readable storage medium” may include any medium capable of storing, encoding, or carrying a set of instructions for machine execution and causing a machine to perform one or more of the methods of the Disclosure. Accordingly, the term “computer-readable storage medium” is interpreted to include, but is not limited to, solid-state memory, optical media, and magnetic media. Such computer-readable storage media may include, but are not limited to, non-transient computer-readable storage media such as random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, and flash memory devices (e.g., solid-state memory devices).

[0170] The processor 993 is configured to control the computing device and perform processing operations, for example, by executing code stored in memory to implement various different functions of the modules and units described herein. Memory 994 stores data read from and written by the processor 993. As referred to herein, the processor may include one or more general-purpose processing units, such as a microprocessor or a central processing unit. The processor may include a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing another instruction set, or a processor implementing a combination of instruction sets. The processor may also include one or more dedicated processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor. In one or more embodiments, the processor is configured to execute instructions for performing the operations and steps discussed herein.

[0171] The display unit 997 can display a representation of data stored by the computing device, and can also display a cursor, and dialog boxes and screens that enable interaction between the user and programs and data stored on the computing device. The input mechanism 996 may enable the user to input data and commands to the computing device to, for example, generate or create a model 200, or use the model 200 to discover the state of the robot system 10A, and / or use the model 200 to control the robot system 10A. The input mechanism 996 may include a user interface configured to cause the modeling device 100 to add, delete, modify, or control the model 200 and / or verification rules 160 based on user input.

[0172] The network interface (network I / F) 997 may be connected to a network such as the Internet, and can connect to other such computing devices via the network. The network I / F 997 may control data input and output with other devices via the network. Other peripheral devices such as microphones, speakers, printers, power supplies, fans, enclosures, scanners, and tracker balls may also be included in the computing device.

[0173] Of course, this embodiment is described in relation to the modeling (and even control) of robotic systems. However, it will be understood that the modeling described herein may be applied to other types of systems, such as any system for which the ability to describe (or control) its current state, structure / configuration, and available commands is beneficial / necessary. In this regard, references to robotic systems may be considered as references to systems or controllable systems. Similarly, references to robots may be interpreted as references to components of a system or controllable components of a controllable system. This disclosure will be understood accordingly.

[0174] This disclosure extends to computer programs that, when executed on a computer, cause the computer to perform any of the apparatus or methods described herein. Such computer programs may be stored on a computer-readable medium.

[0175] Those skilled in the art will recognize that some aspects of the above-described apparatus and methods can be embodied as processor control code on a non-volatile carrier medium such as a disk, CD or DVD-ROM, or programmed memory such as read-only memory (firmware), or on a data carrier such as an optical or electrical signal carrier. For example, the modeling apparatus 100 may be implemented as a processor that operates based on processor control code.

[0176] In some applications, such embodiments are implemented on a DSP (Digital Signal Processor), ASIC (Application-Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array). Thus, the code may include conventional program code or microcode, or, for example, code for configuring or controlling an ASIC or FPGA. The code may also include code for dynamically configuring reconfigurable devices, such as a reprogrammable logic gate array. Similarly, the code may include code in a hardware description language, such as Verilog® or VHDL. As those skilled in the art will understand, the code may be distributed among multiple coupled components communicating with one another. Where appropriate, such embodiments may be implemented using code executed on a field (re)programmable analog array or similar device to configure analog hardware.

[0177] The embodiments described above are illustrative, not limiting, of the invention, and it should be noted that those skilled in the art can design many alternative embodiments without departing from the scope of the appended claims. The words “comprising” do not preclude the existence of elements or steps other than those described in the claims, and the indefinite articles “a” or “an” do not preclude plurals, and one feature or other unit may perform the function of multiple units described in the claims. No reference number or label in the claims should be construed as limiting its scope.

[0178] This disclosure includes all changes, substitutions, modifications, alterations, and modifications to the exemplary embodiments herein as will be understood by those skilled in the art. Similarly, where necessary, the appended claims include all changes, substitutions, modifications, alterations, and modifications to the exemplary embodiments herein as will be understood by those skilled in the art. Furthermore, any reference in the appended claims to a device or system, or component of a device or system, that is adapted, positioned, capable, configured, activated, operable, or operable to perform a particular function, includes that device, system, or component, insofar as it is adapted, positioned, capable, configured, activated, operable, or operable, regardless of whether it or its particular function is activated, turned on, or unlocked. Thus, modifications, additions, or omissions can be made to the systems, devices, and methods described herein without departing from the scope of this disclosure. For example, components of systems and devices may be integrated or separated. Furthermore, the operation of the systems and apparatus disclosed herein may be performed by more, fewer, or other components, and the methods described may include more, fewer, or other steps. Furthermore, the steps may be performed in any suitable order. Where used herein, “each” refers to each member of a set or each member of a subset of a set.

[0179] Exemplary embodiments are shown in the drawings and described below, but the principles of this disclosure may be implemented using any number of techniques, whether currently known or not. This disclosure should not be limited in any way to the exemplary implementations and techniques shown in the drawings and described above.

[0180] Unless otherwise specified, the items depicted in the drawings are not necessarily drawn to scale.

[0181] All examples and conditional language described herein are intended for educational purposes to help readers understand the disclosure and the concepts to which the inventors contribute to the advancement of the art, and are construed as not being limited to such specifically described examples and conditions. While embodiments of this disclosure have been described in detail, it should be understood that various changes, substitutions, and modifications can be made to this specification without departing from the spirit and scope of this disclosure.

[0182] While specific advantages have been listed above, various embodiments may include, omit, or include all of the listed advantages. In addition, other technical advantages will become readily apparent to those skilled in the art after reviewing the aforementioned drawings and description.

[0183] This disclosure extends to the following numbered paragraphs, and / or the present invention is at least partially defined by the following numbered paragraphs.

[0184] (Note 1) A modeling device for modeling robot systems, A modeler for modeling the robot system using a model, wherein the model comprises multiple instances of at least a subset of a set of defined types interconnected with a plurality of links, each instance being a description of an element of the robot system, and the plurality of links between the plurality of instances representing the relationships between the instances. The system includes a verification unit that verifies the model in light of the multiple verification rules by determining whether the model satisfies the multiple verification rules, The plurality of validation rules include the definition of one or more directed acyclic graphs of the plurality of defined types interconnected with a plurality of parent-child relationships, the plurality of defined types corresponding to a plurality of nodes, the plurality of parent-child relationships corresponding to a plurality of directed edges of the one or more directed acyclic graphs, each defined type defines a plurality of type rules of that type that must be satisfied for an instance of that type to be valid, and each parent-child relationship between a pair of types defines those types as parent and child types to each other according to the corresponding edge direction. A modeling apparatus characterized in that each child type inherits the plurality of type rules of each of the corresponding plurality of parent types according to the plurality of parent-child relationships.

[0185] (Note 2) Each subtype is, The child type inherits from each of the corresponding parent types by referencing each of the corresponding parent types, and / or The modeling apparatus according to Appendix 1, characterized in that it defines each type rule for the child type other than the multiple type rules that the child type inherits from each of the multiple parent types it corresponds to.

[0186] (Note 3) At least one child type has at least two parent types, and / or The modeling apparatus according to Appendix 1 or 2, characterized in that at least one child type has at least one parent type, and that itself (the at least one parent type) is a child type having at least one (another) parent type, and optionally, that itself (the at least one other parent type) is a child type having at least one (yet another) parent type.

[0187] (Note 4) The plurality of verification rules include a plurality of morphological rules that define a plurality of interconnections that, for at least one particular defined type, an instance of the particular defined type must have together with a plurality of other instances of a plurality of defined types in the model for the model to be valid, The modeling apparatus according to any of the preceding appendices, wherein optionally, for at least one defined type, the corresponding multiple type rules include the corresponding multiple morphological rules.

[0188] (Note 5) A modeling apparatus according to any of the above notes, characterized in that the model defines a directed graph of multiple instances of the model, the multiple links are multiple directed links, the multiple instances correspond to multiple nodes, the multiple links correspond to multiple edges of the directed graph, each link is owned by one of the multiple instances to which each link is interconnected, and the direction of the multiple links represents an input or output relationship between the interconnected multiple instances with respect to the instance that owns the link.

[0189] (Note 6) The directions of the plurality of links represent the input relationships between the interconnected plurality of instances to the instance that owns the link, and / or The modeling apparatus according to Appendix 5, characterized in that the direction of the plurality of links represents the output relationship between the interconnected plurality of instances with respect to the instance that owns the link, when the direction points away from the instance that owns the link.

[0190] (Note 7) For at least one link whose direction points toward the owner instance of that link, other instances of that link are inputs to the owner instance and / or The modeling apparatus according to Appendix 5 or 6, characterized in that, for at least one link whose direction points away from the owner instance of that link, the other instances of that link are outputs to the owner instance.

[0191] (Note 8) For at least one link whose direction points toward the owner instance of that link, the input information of the owner instance is represented in other instances of that link, or read from other instances of that link via that link, and / or The modeling apparatus according to any one of appendices 5 to 7, characterized in that, for at least one link whose direction points away from the owner instance of that link, the output information of the owner instance of that link is represented in other instances of that link or written to other instances of that link beyond that link.

[0192] (Note 9) A modeling apparatus according to any one of Notes 5 to 8, characterized in that, for at least one link whose direction points toward the owner instance of that link, the ownership information and / or containing information of the owner instance of that link is represented in other instances of that link.

[0193] (Note 10) The above defined types include multiple descriptive types and optionally include at least one functional type. Each instance of the descriptive type describes the current state and / or target state of the corresponding element of the robot system, and optionally, each of the states is an operating state or situation or mode, and optionally, The modeling apparatus according to any of the preceding appendices, characterized in that each instance of a functional type describes a method for manipulating data of interconnected instances of a descriptive type in order to derive information describing the robot system and / or to model the activities within the robot system, and optionally, the data is functional data or operational data.

[0194] (Note 11) The modeling apparatus according to any of the above notes, characterized in that each instance is an instance data unit that optionally defines each of a plurality of links corresponding to one or more other instances of the model.

[0195] (Note 12) Each instance data unit includes an indication of the defined type for which each instance data unit is an instance, and typed data. The modeling apparatus according to Appendix 11, wherein each instance data unit optionally further includes a definition of the status of one or more supported commands.

[0196] (Note 13) The plurality of instance data units have the same structured format as each other, and / or The plurality of instance data units are known or accessible by the modeling apparatus and conform to a data structure standard that defines the format of the plurality of compliant instance data units, and / or The modeling apparatus according to Appendix 11 or 12, characterized in that each of the plurality of instance data units includes a plurality of (individual or separated) parts defined by a data structure standard known to or accessible by the modeling apparatus.

[0197] (Note 14) The modeling apparatus according to any one of Notes 11 to 13, characterized in that at least one instance data unit is associated with an extension data unit that defines a task to be performed to manipulate the data of the instance or interconnected instances.

[0198] (Note 15) The definition of one or more directed acyclic graphs of the above-mentioned multiple defined types includes a type data unit for each defined type, The modeling apparatus according to any of the above appendices, characterized in that each type data unit defines the defined type, and optionally defines a parent-child relationship for each of the multiple parent types corresponding to each defined type other than the root defined type corresponding to the root node of the one or more directed acyclic graphs.

[0199] (Note 16) Each type data unit includes an indication of the type defined for that type data unit, and a definition of at least one data type which data type must be for the data of an instance of the defined type to be valid. Optionally, each type data unit may further include a definition of how the status of one or more supported commands must be represented in an instance of the defined type for it to be valid. The modeling apparatus described in Appendix 15, characterized by the features described herein.

[0200] (Note 17) Multiple type data units have the same structured format as each other, and / or The modeling apparatus according to Appendix 15 or 16, characterized in that the plurality of type data units are known to or accessible by the modeling apparatus and conform to a data structure standard that defines the format of the type data units to which the apparatus conforms.

[0201] (Note 18) The structured format of the plurality of instance data units corresponds to the structured format of the plurality of type data units, and / or Each of the aforementioned plurality of type data units includes a plurality of (individual or separated) parts as defined by the data structure standard. Each type data unit defines, in the manner defined by the data structure standard, the rules that multiple instance data units of the same type must satisfy in order to be valid and optionally interpreted, understood, or processed by the modeling device, and / or The modeling apparatus according to Appendix 17, characterized in that each type data unit defines or expresses, in each of the multiple individual or separated parts of the type data unit as defined by the data structure standard, the rules that the corresponding (individual or separated) parts of multiple instance data units of the same type, as defined by the data structure standard, must satisfy in order to be valid and optionally interpreted, understood, or processed by the modeling apparatus, in a manner defined by the data structure standard.

[0202] (Note 19) Each instance of the above model is a description of the robot elements of the robot system and / or the robot controller of the robot system, and / or The modeling apparatus according to any of the appendices, characterized in that the model represents the current state and / or target state of at least a part of the robot system, the robot of the robot system, or the robot controller of the robot system.

[0203] (Note 20) The modeling device is provided with an interface for connection between the modeling device and the robot system, and the modeling device optionally verifies the model based on the interpretation / understanding of the plurality of type data units and the plurality of instance data units based on the data structure standard, The interface receives data from the robot system, updates the model based on the received data, optionally outputs or generates a representation of the robot system based on the updated model, and / or Data is output to the robot system via the interface, the robot system is controlled based on the model, optionally the operation of the robot system is controlled, or the robot system is guided, or the movement of the robot system is controlled, optionally the control is active control. Optional, The aforementioned data is functional data that defines the current state / behavior or future state / behavior of at least one element of the robot system, and / or The aforementioned data defines the operating state or operating mode of at least one element of the robot system, and / or The aforementioned data defines the technical state or technical mode of at least one element of the robot system, and / or The aforementioned data defines the current or future configuration or performance or activity or behavior of at least one element of the robot system, and / or The modeling apparatus according to any of the preceding appendices, characterized in that the data defines the position, orientation, activity, or motion of at least one element of the robot system.

[0204] (Note 21) The one or more directed acyclic graphs are directed acyclic graphs at the device level. The aforementioned modeling apparatus is Generate one or more model-specific directed acyclic graphs of the defined type represented within the model, The one or more model-specific directed acyclic graphs are part of the one or more device-level directed acyclic graphs, each root-defined type corresponding to each root node, all of the defined types having multiple instances within the model, and each other defined type connected between them via the multiple parent-child relationships. The model is validated against the plurality of validation rules defined in the one or more model-specific directed acyclic graphs, and optionally, the model is determined to be valid by assuming that each validation rule defined in the one or more device-level directed acyclic graphs but not defined in the one or more model-specific directed acyclic graphs satisfies these rules. A modeling apparatus as described in any of the above appendices, characterized by the features described above.

[0205] (Note 22) The modeling apparatus according to any of the above notes, characterized in that the modeling apparatus is or comprises a computing apparatus, a computer implementation apparatus, or a computer-aided apparatus.

[0206] (Note 23) Modify the above model and / or the above multiple verification rules, optionally The modeling apparatus according to any of the above appendices, characterized in that, following the above modification, the model is verified in light of the plurality of verification rules by determining whether the model satisfies the plurality of verification rules.

[0207] (Note 24) The above change The process involves modifying the multiple type rules of one of the multiple defined types, optionally the model containing instances of the multiple type rules of one of the multiple defined types, or a specific defined type inheriting multiple type rules from the multiple type rules of one of the multiple defined types, wherein the model contains instances of the specific defined type, or inherits multiple type rules from a predetermined defined type, wherein the model contains instances of the predetermined defined type, and the process involves modifying the multiple type rules. A process of adding a new defined type to one or more directed acyclic graphs, along with at least one parent-child relationship to an existing defined type of the one or more directed acyclic graphs, wherein the new defined type is interconnected with the parent-child relationship to a particular existing defined type such that it inherits the type rules of that particular existing defined type, and the model includes instances of that particular existing defined type. A process to remove unnecessary defined types from one or more directed acyclic graphs, along with each of the parent-child relationships of the unnecessary defined types to existing defined types in the one or more directed acyclic graphs, and optionally remove any defined types that inherit multiple rules of any defined type from the unnecessary defined types, wherein the unnecessary defined types are interconnected with at least one parent-child relationship to a specific existing defined type such that they inherit the specific existing defined type rules, and the model includes instances of the specific existing defined type. The process of modifying an instance of a defined type within the aforementioned model, The process of adding an instance of a defined type to the model, along with at least one link to an existing instance of the model, The modeling apparatus according to Appendix 23, characterized by including at least one (or any combination) of: a process of removing an instance of a defined type from the model, along with at least one link to an existing instance of the model.

[0208] (Note 25) If it is determined that the model satisfies the verification rules, the model is determined to be valid, and / or A modeling apparatus according to any of the above appendices, characterized in that if it is determined that the model does not satisfy the verification rules, it is determined that the model is not valid.

[0209] (Note 26) A modeling apparatus according to any of the above notes, characterized in that it comprises a user interface configured to cause the modeling apparatus to add, remove, modify, use or control the model and / or the verification rules based on user input.

[0210] (Note 27) A multi-device modeling system comprising first and second modeling devices configured to communicate with each other, wherein each of the first and second modeling devices is a modeling device described in any of the above notes. The first modeling apparatus models the first robot system using the first model of the first robot system, and the second modeling apparatus models the second robot system using the second model of the second robot system. The aforementioned multi-device modeling system is A process of verifying the first model in light of the verification rules of the second modeling apparatus, and / or A process of verifying the second model in light of the verification rules of the first modeling apparatus, and / or A multi-device modeling system characterized in that it is configured to determine interoperability between the first model and the second model by performing a process to determine compatibility between at least a first model-specific portion of the one or more directed acyclic graphs of the first modeling device and a second model-specific portion of the one or more directed acyclic graphs of the second modeling device.

[0211] (Note 28) The first model-specific portion of the one or more directed acyclic graphs of the first modeling apparatus is a part of the one or more directed acyclic graphs of the first modeling apparatus that includes each root-defined type corresponding to each root node, all of the defined types having instances in the first model, and each other defined type connected between them via the parent-child relationships, The multi-device modeling system according to Appendix 27, characterized in that the second model-specific portion of the one or more directed acyclic graphs of the second modeling device is a part of the one or more directed acyclic graphs of the second modeling device, which includes each root-defined type corresponding to each root node, all of the defined types having instances in the second model, and each other defined type connected between them via the plurality of parent-child relationships.

[0212] (Note 29) At least the first model-specific portion of one or more directed acyclic graphs of the first modeling device and at least the second model-specific portion of one or more directed acyclic graphs of the second modeling device, The defined types in both the first and second model-specific parts are substantially identical, and / or The defined types in both the first and second model-specific parts define at least the same type rules to each other, and / or The first model-specific part and the second model-specific part do not define conflicting rules or conflicting hierarchies of defined types. A multi-device modeling system according to appendix 27 or 28, characterized in that it is configured to determine compatibility in certain cases.

[0213] (Note 30) The first modeling apparatus is configured to generate the first model-specific portion of the one or more directed acyclic graphs of the first modeling apparatus, The second modeling apparatus is configured to generate the second model-specific portion of the one or more directed acyclic graphs of the second modeling apparatus, The multi-device modeling system according to any one of appendices 27 to 29, characterized in that either the first modeling device or the second modeling device is configured to determine compatibility between the first model-specific portion of the one or more directed acyclic graphs of the first modeling device and the second model-specific portion of the one or more directed acyclic graphs of the second modeling device.

[0214] (Note 31) The multi-device modeling system according to any one of Notes 27 to 30, characterized in that the second modeling device is configured to obtain the first model-specific portion of the one or more directed acyclic graphs of the first modeling device from the first modeling device and to determine the compatibility between the first model-specific portion of the one or more directed acyclic graphs of the first modeling device and the second model-specific portion of the one or more directed acyclic graphs of the second modeling device.

[0215] (Note 32) The multi-device modeling system according to Note 31, characterized in that the first modeling device is a publisher device configured to make available to a subscriber device a first model-specific portion of the one or more directed acyclic graphs of the first modeling device, and the second modeling device is the subscriber device.

[0216] (Note 33) A multi-device modeling system comprising first and second modeling devices configured to communicate with each other, Each of the first and second modeling devices is for modeling robot systems, A modeler for modeling the robot system using a model, wherein the model comprises multiple instances of at least one subset of a set of defined types interconnected with multiple links, each instance being a description of an element of the robot system, and the multiple links between the multiple instances representing the relationships between them, The system includes a verification unit that verifies the model in light of the verification rules by determining whether the model satisfies a plurality of verification rules, The verification rule comprises a definition of one or more directed acyclic graphs of one or more defined types interconnected with a plurality of parent-child relationships, wherein the defined types correspond to a plurality of nodes, the plurality of parent-child relationships correspond to a plurality of directed edges of the one or more directed acyclic graphs, each defined type defines type rules of that type that must be satisfied for an instance of that type to be valid, each parent-child relationship between a pair of types defines those types as parent and child types to each other according to the corresponding edge direction, and each child type inherits the respective type rules of the plurality of corresponding parent types according to the plurality of parent-child relationships, The first modeling apparatus models the first robot system using the first model of the first robot system, and the second modeling apparatus models the second robot system using the second model of the second robot system. For each of the first and second modeling devices, The definition of one or more directed acyclic graphs of the plurality of defined types includes a type data unit for each defined type, each type data unit defines the defined type, and for each defined type other than the root defined type corresponding to the root node of the one or more directed acyclic graphs, defines the parent-child relationships to each of the corresponding plurality of parent types. Each type data unit includes an indication of the corresponding defined type and a definition of at least one data type that the data of which must be for the data of an instance of the defined type to be valid. The aforementioned defined types include descriptive types, Each instance of the descriptive type describes the current state and / or target state of the corresponding element of the robot system. Each instance is an instance data unit that defines each of a plurality of corresponding links to one or more other instances of the model, Each instance data unit contains an indication of the defined type for which it is an instance, and typed data. The aforementioned set of validation rules includes, for at least one specific defined type, morphological rules that define the interconnections that an instance of that specific defined type must have together with multiple other instances of multiple defined types within the model for the model to be valid. For both the first and second modeling apparatuses, The aforementioned multiple data units have the same structured format as each other. The aforementioned multiple instance data units have the same structured format as each other. The structured format of the plurality of instance data units corresponds to the structured format of the plurality of type data units, The multi-device modeling system enables interoperability between the first model and the second model. A process of verifying the first model in light of the verification rules of the second modeling apparatus, and / or A process of verifying the second model in light of the verification rules of the first modeling apparatus, and / or The determination is made by a process that determines compatibility between at least a first model-specific portion of the one or more directed acyclic graphs of the first modeling device and a second model-specific portion of the one or more directed acyclic graphs of the second modeling device, A multi-device modeling system characterized in that the second modeling device acquires the first model from the first modeling device, and if it is determined that the first model satisfies the verification rules of the first modeling device and is compatible, or if it is determined that the first model satisfies the verification rules of the second modeling device, it is configured to generate / output a representation of the first robot system that represents at least one of the structure, functionality, and current state of the first robot system based on the first model.

[0217] (Note 34) The first modeling apparatus is configured to read data from the first robot system and to populate at least one instance of the descriptive type of the first model with current state data values ​​based on the data read, The multi-device modeling system according to Appendix 33, characterized in that the second modeling device is configured to acquire the first model having the current state data values ​​and to generate / output the representation of the first robot system based on the first model having the current state data values.

[0218] (Note 35) The first modeling apparatus is configured to read data from the first robot system in a continuous cycle and to input the current state data value into at least one instance of the descriptive type of the first model based on the data read. The multi-device modeling system according to appendix 33 or 34, characterized in that the second modeling device is configured to generate / output a representation of the first robot system based on the first model which has been updated in a predetermined cycle.

[0219] (Note 36) A method for modeling a robot system, A step of modeling the robot system using a model, wherein the model includes multiple instances of at least a subset of a set of defined types interconnected by multiple links, each instance being a description of an element of the robot system, and the multiple links between the multiple instances representing the relationships between them. The process includes a step of verifying the model in light of the multiple verification rules by determining whether the model satisfies the multiple verification rules, The verification rule is a definition of one or more directed acyclic graphs of a plurality of defined types interconnected with a plurality of parent-child relationships, wherein the plurality of defined types correspond to a plurality of nodes, the plurality of parent-child relationships correspond to a plurality of directed edges of the one or more directed acyclic graphs, each defined type defines type rules for which an instance of that type must satisfy in order to be valid, each parent-child relationship between a pair of types defines those types as parent and child types to each other according to the corresponding edge direction, and each child type inherits the plurality of type rules of each of the corresponding parent types according to the plurality of parent-child relationships.

[0220] (Appendix 37) A computer program that, when executed on a computer, causes the computer to perform the method described in Appendix 36.

[0221] (Note 38) A computer-readable storage medium containing the computer program described in Note 37.

[0222] (Appendix 39) A series of methods comprising the first and second methods, each of which is the method described in Appendix 36, The first method models the first robot system using a first model of the first robot system, and the second method models the second robot system using a second model of the second robot system. The aforementioned series of methods further provides interoperability between the first model and the second model. A step of verifying the first model in light of the verification rules of the second method, and / or A step of verifying the second model in light of the verification rules of the first method, and / or A series of methods characterized by including a step of determining compatibility between at least a first model eigenpart of the one or more directed acyclic graphs of the first method and a second model eigenpart of the one or more directed acyclic graphs of the second method.

[0223] (Appendix 40) A series of computer programs that, when executed on one or more computers, cause one or more computers to perform the series of methods described in Appendix 39.

[0224] (Appendix 41) A computer-readable storage medium characterized by storing the series of computer programs described in Appendix 40.

[0225] (Note 42) A robotic system including a modeling device as described in any of Notes 1 to 26, or a multi-device modeling system as described in any of Notes 27 to 35.

[0226] (Appendix 43) The robot system according to Appendix 42, characterized in that it includes at least one robot modeled by each model.

[0227] Variations of the disclosed embodiments will be understood and effective by those skilled in the art when carrying out the principles and techniques described herein, based on a review of the drawings, disclosures, and appended claims. In the claims, the words “comprising” do not exclude other elements or steps, and the indefinite articles “a” or “an” do not exclude plurals. The mere fact that certain means are described in different dependent claims does not imply that combinations of these means cannot be used advantageously. No reference numeral in the claims should be construed as limiting the scope.

Claims

1. A multi-device modeling system comprising a first modeling device and a second modeling device configured to communicate with each other, wherein each of the first modeling device and the second modeling device is a modeling device for modeling a robot system, A modeler for modeling the robot system using a model, wherein the model comprises multiple instances of at least a subset of a set of defined types interconnected with a plurality of links, each instance being a description of an element of the robot system, and the plurality of links between the plurality of instances represent the relationships between the plurality of instances. The system includes a verification unit that verifies the model in light of the multiple verification rules by determining whether the model satisfies the multiple verification rules, The plurality of validation rules include the definition of one or more directed acyclic graphs of the plurality of defined types interconnected with a plurality of parent-child relationships, the plurality of defined types corresponding to a plurality of nodes, the plurality of parent-child relationships corresponding to a plurality of directed edges of the one or more directed acyclic graphs, each defined type defines a plurality of type rules of that type that must be satisfied for an instance of that type to be valid, and each parent-child relationship between a pair of types defines those types as parent and child types to each other according to the corresponding edge direction, Each child type inherits the respective type rules of the corresponding parent types according to the parent-child relationships. The first modeling apparatus models the first robot system using a first model of the first robot system, and the second modeling apparatus models the second robot system using a second model of the second robot system. The aforementioned multi-device modeling system is A process of verifying the first model in light of the verification rules of the second modeling apparatus, and / or A process of verifying the second model in light of the verification rules of the first modeling apparatus, and / or The system is configured to determine interoperability between the first model and the second model by determining compatibility between at least a first model-specific portion of the one or more directed acyclic graphs of the first modeling device and a second model-specific portion of the one or more directed acyclic graphs of the second modeling device. A multi-device modeling system characterized by the following features.

2. Each subtype is, The child type inherits from each of the corresponding parent types by referencing each of the corresponding parent types, and / or The multi-device modeling system according to claim 1, characterized in that it defines each type rule for the child type other than the multiple type rules that the child type inherits from each of the multiple parent types it corresponds to.

3. At least one child type has at least two parent types, and / or The multi-device modeling system according to claim 1 or 2, characterized in that at least one child type has at least one parent type, the at least one parent type is a child type having at least one other parent type, and optionally, the at least one other parent type is a child type having at least one yet another parent type.

4. The plurality of verification rules of the first modeling apparatus include a plurality of morphological rules that define a plurality of interconnections that an instance of the particular defined type must have together with a plurality of other instances of a plurality of defined types in the first model for the first model to be valid, for at least one particular defined type. The multi-device modeling system according to any one of claims 1 to 3, wherein optionally, for at least one defined type, the corresponding multiple type rules include the corresponding multiple morphological rules.

5. A multi-device modeling system according to any one of claims 1 to 4, characterized in that the first model and / or the second model define a directed graph of multiple instances of the model, the multiple links are multiple directed links, the multiple instances correspond to multiple nodes, the multiple links correspond to multiple edges of the directed graph, each link is owned by one of the multiple instances to which each link is interconnected, and the direction of the multiple links represents an input or output relationship between the multiple interconnected instances with respect to the instance that owns the link.

6. The aforementioned multiple defined types include multiple descriptive types and optionally include at least one functional type. Each instance of the descriptive type describes the current state and / or target state of the corresponding element of the first robot system, and optionally, The multi-device modeling system according to any one of claims 1 to 5, characterized in that each functional instance describes a method for manipulating data of interconnected descriptive instances in order to derive information describing the first robot system and / or to model activities within the first robot system.

7. Each instance is an instance data unit that optionally defines each of a plurality of links corresponding to one or more other instances of the first model, The multi-device modeling system according to any one of claims 1 to 6, wherein optionally, multiple instance data units have the same structured format as one another.

8. The definition of one or more directed acyclic graphs of the aforementioned plurality of defined types includes a type data unit for each defined type, The multi-device modeling system according to any one of claims 1 to 7, characterized in that each type data unit defines the defined type and optionally defines a parent-child relationship for each of the corresponding parent types for each defined type other than the root defined type corresponding to the root node of the one or more directed acyclic graphs.

9. The multi-device modeling system according to claim 8, characterized in that multiple type data units have the same structured format as one another.

10. The first modeling apparatus is Modify the plurality of verification rules of the first model and / or the first modeling apparatus, Following the above modification, the first model is verified in light of the plurality of verification rules by determining whether the first model satisfies the plurality of verification rules. A multi-device modeling system according to any one of claims 1 to 9, characterized in that it is configured in such a way.

11. The aforementioned change is, The process involves modifying the multiple type rules of one of the multiple defined types, optionally the multiple type rules of one of the multiple defined types in which the first model includes an instance, or the multiple type rules of one of the multiple defined types in which a specific defined type inherits multiple type rules, wherein the first model includes an instance of the specific defined type, or the multiple type rules of one of the multiple defined types in which a predetermined defined type inherits multiple type rules, wherein the first model includes an instance of the predetermined defined type, and the process involves modifying the multiple type rules. A process of adding a new defined type to one or more directed acyclic graphs, along with at least one parent-child relationship to an existing defined type of the one or more directed acyclic graphs, wherein the new defined type is optionally interconnected with the parent-child relationship to a particular existing defined type such that it inherits the type rules of that particular existing defined type, and the first model includes instances of that particular existing defined type, A process to remove unnecessary defined types from one or more directed acyclic graphs, along with each of the parent-child relationships of the unnecessary defined types to existing defined types in the one or more directed acyclic graphs, and optionally remove any defined types that inherit multiple rules of any defined type from the unnecessary defined types, wherein the unnecessary defined types are interconnected with at least one parent-child relationship to a specific existing defined type such that they inherit the specific existing defined type rules, and the first model includes instances of the specific existing defined type. The process of modifying an instance of a defined type in the first model, The process of adding an instance of a defined type to the first model, along with at least one link to an existing instance of the first model, The multi-device modeling system according to claim 10, comprising at least one of the following: a process of removing an instance of a defined type from the first model, along with at least one link to an existing instance of the first model.

12. Each instance of the first model is a description of the robot elements and / or the robot controller of the first robot system, and / or The multi-device modeling system according to any one of claims 1 to 11, characterized in that the first model represents the current state and / or target state of at least a part of the first robot system, the robot of the first robot system, or the robot controller of the first robot system.

13. The first modeling apparatus comprises an interface for connection between the first modeling apparatus and the first robot system, and the first modeling apparatus, after verifying the first model, The interface receives data from the first robot system, updates the first model based on the received data, and / or The multi-device modeling system according to any one of claims 1 to 12, characterized in that it is configured to output data to the first robot system via the interface and to control the first robot system based on the first model.

14. At least the first model-specific portion of one or more directed acyclic graphs of the first modeling apparatus and at least the second model-specific portion of one or more directed acyclic graphs of the second modeling apparatus are, The defined types in both the first and second model-specific parts are substantially identical, and / or The defined types in both the first and second model-specific parts define at least the same type rules to each other, and / or The first model-specific part and the second model-specific part do not define conflicting rules or conflicting hierarchies of defined types. It is configured to determine compatibility in certain cases. A multi-device modeling system according to any one of claims 1 to 13.

Images