Robot digital representation and conversion implementation method and system based on industrial foundation class

Abstract

Computer-implemented methods and systems are provided for adjusting a robot-centric data format to a building-centric data format to represent a robot. The building-centric data format represents the robot with a plurality of links representing non-deformable components of the robot and a plurality of joints describing relationships between the links. The method comprises the following steps: extracting robot information from the robot-centered data format; creating a robot container to model the robot as an unitary-partial structure; converting the plurality of links and the plurality of joints from the robot-centered data format to the building-centered data format on the basis of the extracted robot information; filling the robotic container with the transformed links and the transformed joints; and constructing the building-centric data format using the robotic container filled with the transformed links and the transformed joints.

Description

Technical Field

[0001] The present invention generally relates to the technical field of data format conversion, and more particularly to a computer-implemented method and system for converting a robot-centric data format, such as the Unified Robot Description Format (URDF), into a building-centric Industry Foundation Classes (IFC) data format, thereby enabling the digital representation of robots. Background Art

[0002] Robotics has the potential to enhance the productivity of the construction industry by replacing humans in dangerous, repetitive, and physically demanding construction tasks. Although its potential has long been widely recognized, large-scale construction robot applications have only recently become feasible. This progress is due to collective breakthroughs in multiple technologies such as artificial intelligence (AI), intelligent sensing, and cybernetics. Initial evidence suggests that construction robotics has been remarkably effective in addressing long-standing issues such as low productivity, inadequate safety management, and inconsistent quality control. Traditional construction methodologies are expected to soon reach their limits, and the widespread adoption of robots in the construction environment is imminent.

[0003] Digital Robot Representation (DRR) is crucial for robot development and its application in fields such as construction. It enables virtual testing of different robot configurations and their adaptability to the external environment without the need to physically manufacture the robot. This facilitates the exchange, reuse, and cross-departmental and multidisciplinary collaboration and communication of key robot information. The most common DRR method is based on the Unified Robot Description Format (URDF). The URDF format provides a concise framework for digitally representing the composition, geometry, and kinematic and dynamic properties of a robot. It provides a practical abstraction of a robot from the perspective of robotics engineering and reduces the cost and resource consumption of robot development.

[0004] However, existing URDF-based DRRs are not easily compatible with established tools and business processes in the Architecture, Engineering, Construction and Operation (AECO) domain. From a business perspective, the goal of AECO activities is to design, build, and maintain the built environment, while the inherent goal of robotics is to design and develop automated systems capable of operating independently. These different business goals lead to different expectations for DRRs. The inward focus of robotics makes existing DRRs prioritize information such as robot kinematics, dynamics, and contact interaction simulation, while AECO professionals are more concerned with the impact of robot integration on project productivity, cost, and spatial design. From an implementation perspective, there are significant differences in the data formats used in the robotics and AECO domains. Although URDF is very popular in the field of robotics engineering, few building design and project management software solutions support its parsing. Today's AECO business workflows are mainly based on Building Information Modelling (BIM), which uses Industry Foundation Classes (IFC) as its underlying data format. Therefore, a language barrier between the two fields of robotics engineering and AECO requires a format conversion method to transform URDF into IFC. Summary of the Invention

[0005] An object of the present invention is to provide a data format conversion method to solve the above technical problems.

[0006] According to a first aspect of the present invention, there is provided a computer-implemented method for adjusting a robot-centric data format to a building-centric data format to represent a robot. The robot-centric data format represents a robot using a plurality of links representing non-deformable parts of the robot and a plurality of joints describing the relationships between the links. The method includes: extracting robot information from the robot-centric data format; creating a robot container to model the robot as a whole-part structure; transforming the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format based on the extracted robot information; filling the robot container with the transformed links and the transformed joints; and constructing the building-centric data format using the robot container filled with the transformed links and the transformed joints.

[0007] According to a second aspect of the present invention, there is provided a computer-implemented system for adjusting a robot-centric data format to a building-centric data format to represent a robot. The robot-centric data format represents a robot using a plurality of links representing non-deformable parts of the robot and a plurality of joints describing the relationships between the links. The system includes a processor configured to: extract robot information from the robot-centric data format; create a robot container to model the robot as a whole-part structure; transform the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format based on the extracted robot information; populate the robot container with the transformed links and the transformed joints; and construct the building-centric data format using the robot container populated with the transformed links and the transformed joints. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Embodiments of the present invention are described in more detail below with reference to the drawings, in which:

[0009] Figure 1 a flowchart of a computer-implemented method for adjusting a robot-centric data format to a building-centric data format to represent a robot according to an embodiment of the present invention is shown; and

[0010] Figure 2 a robot representation defined in the context of a building environment is shown.

[0011] Figure 3 how an exemplary robot TurtleBot can be conceptualized under the principles of URDF is shown.

[0012] Figure 4 the IFC entities involved and how they are connected to form a digital representation of the robot are shown.

[0013] Figure 5 an exemplary pseudocode for converting a URDF visual representation of a link to an IFC representation is shown.

[0014] Figure 6 how the placement of a joint is specified by referring to the coordinate system of its previous joint is shown.

[0015] Figure 7A how the tree structure of the IFC representation successfully replicates the "whole-part" structure of a robot arm is shown; Figure 7B how the IFC representation visually mimics its physical counterpart from a geometric and topological perspective is shown; Figure 7C is shown Figure 7B a close-up view of the properties of the robot in the IFC representation; Figure 7D is shownFigure 7B A close - up view of the link properties in the IFC representation; and Figure 7E shows Figure 7B A close - up view of the joint properties in the IFC representation.

[0016] Figures 8A to 8C Respectively show the resulting IFC representations for different robot types.

[0017] Figure 9 Shows an example of the internal space design adaptation facilitated by Building RobAvatar.

[0018] Figure 10 Shows an example of importing the IFC robot representation into the MEP design of Revit.

[0019] Figure 11 Shows an example of conflict detection and immediate design adjustment achieved by the IFC robot representation.

[0020] Figure 12 Shows the IFC Avatar of TurtleBot adopted in the BIM environment to support facility management tasks.

[0021] Figure 13 Shows an exemplary application and related inspection results of the digital twin of a facility inspection robot based on the IFC - verified Building RobAvatar.

[0022] Figure 14 Shows how to make full use of the integration achieved by the IFC RoboAvatar to remotely control the robot. Detailed implementation mode

[0023] In the following description, the details of the present invention are set forth as preferred embodiments. Those skilled in the art will appreciate that modifications including additions and / or substitutions can be made without departing from the scope and spirit of the present invention. Specific details may be omitted so as not to obscure the present invention; however, this disclosure is written to enable those skilled in the art to practice the teachings herein without undue experimentation.

[0024] Figure 1 Shows a flowchart of a computer - implemented method for adjusting a robot - centered data format to a building - centered data format to represent a robot according to an embodiment of the present invention. The robot - centered data format represents a robot using a plurality of links representing non - deformable parts of the robot and a plurality of joints describing the relationships between the links. As shown, the method includes the following steps:

[0025] S102: Extract robot information from the robot - centered data format;

[0026] S104: Create a robot container to model the robot as a whole-part structure;

[0027] S106: Transform the plurality of linkages and the plurality of joints from the robot-centered data format to the building-centered data format based on the extracted robot information;

[0028] S108: Fill the robot container with the transformed linkages and the transformed joints; and

[0029] S110: Construct the building-centered data format using the robot container filled with the transformed linkages and the transformed joints.

[0030] In some embodiments, the robot-centered data format is based on URDF, and the building-centered data format is based on IFC. A computer-implemented method for adjusting a URDF file to an IFC file is provided for representing a robot and its avatar in the context of a building and for standardizing and extending according to domain-specific requirements in the AECO domain.

[0031] The most recent authoritative attempt to formalize a general RoboAvatar is the Core Ontology for Robotics and Automation (CORA), which views a robot as an agent device composed of other devices. This constructivist view holds that a robot is "the sum of its parts" rather than an inseparable whole. CORA uses a dichotomy to divide "entities" into "physical entities" (entities with a position in space-time) and "abstract entities" (entities without a position in space-time). Clearly, a robot and its components are both physical entities, with the former classified as "agents" and "devices" in CORA and the latter classified as "devices". In addition to its physical existence, a robot exhibits a series of properties that can be described by "attributes" under "abstract" entities. CORA (including its further extensions) only defines some basic abstract concepts, such as position, orientation, and design. According to the matter-attribute dichotomy, a robot R is the sum of its physical existence S and the attributes A it possesses, expressed mathematically as:

[0032] R = {S, A}

[0033] The purpose of robot representation is to determine the mapping from the physical robot R to its description model F:

[0034] f i : R → F i (i ∈ {ModelView})

[0035] where i refers to a specific model view, defined as describing the angle or perspective of the robot. From the building perspective, the mapping function f b () includes a material mapping function and an attribute mapping function as shown below:

[0036]

[0037] Figure 2 shows the robot representation defined in the context of a built environment or a building RoboAvatar. Mathematically, the building RoboAvatar is F b = f b (R), which is similar to the real robot R and consists of the representation of the material and attributes as follows:

[0038]

[0039] For the material representation it should reflect the constructivist nature of the robot as "the sum of its parts". This will allow for motion analysis, which is necessary for applications such as layout planning and clash detection in the AECO domain. The mathematical expression is as follows:

[0040]

[0041] where s i is the material of the robot components and N part is the number of robot components.

[0042] For the attribute representation it should describe basic geometric and high-level semantic information to meet the industrial requirements of the AECO domain. Specifically, it should include a productivity parameter set P, a mechanical property set M, a topological description set T, a capability description set C, and a geometric property set G.

[0043] Table 1 summarizes the descriptions, specific indices, scopes of work, and use cases of the attribute sets in the built environment.

[0044] Table 1. Overview of the robot description attributes. Note: 1 An inexhaustive list of example attributes under a specific set; 2 The scope of work of a specific attribute, where R, P, and B represent the robot, robot components, and both.

[0045] The basic principle of URDF is to conceptualize a robot as a combination of links and joints, as exemplified by the TurtleBot in Figure 3 . A link is an undeformable part of a robot that is connected by joints. A joint represents how two links move relative to each other, which effectively defines the position of the links in space. There are six common types of joints, namely revolute, continuous, prismatic, fixed, floating, and planar. "Fixed" means that all degrees of freedom are locked and the joint cannot move, while "continuous" means the ability to rotate around an axis without limit. Conceptualizing a robot as links and joints reflects the constructivism of the physical matter of a robot being composed of parts.

[0046] URDF uses a tree structure to describe a robot in the Extensible Markup Language (XML). RoboAvatar based on URDF models the part-whole nature of a robot via its link and joint system and encodes a series of robot attributes into joints or links. This enables applications such as kinematic and dynamic simulation and production line design in robotics and manufacturing.

[0047] In step S102, a given URDF file can be parsed so that its encoded robot information can be retrieved or extracted. Since the URDF file format is essentially an XML file, it can be parsed by reading elements with different semantic meanings (e.g., <robot> 、 <link> 、 <joint>) and having different attributes (e.g., <joint>The tags of the name and type of the label are used to perform the parsing of URDF. For the mass and number of components of the robot, since they are not explicitly expressed in URDF, they need to be extracted by adding up the masses and numbers of all the links. For the remaining properties (e.g., productivity and capabilities), they are specified as "none" because URDF does not provide such information.

[0048] The extracted robot information is used for the generation of the robot container and the transformation (or conversion) of the links and joints in steps S104 and S106. The robot container generated in step S104 is basically a bucket to be filled with the transformed (or converted) links and joints. The robot container can be modeled by IfcElementAssembly in IFC, but there is no clear visual representation and placement. Therefore, the main task is simply to transform the extracted robot properties.

[0049] After creating the robot container, in step 106, the links and joints are transformed into IFC representations. First, the physical entities of the links and joints are created (i.e., IfcBuildingElementProxy and IfcVirtualElement). Then, the properties of the elements extracted from URDF are transformed and represented in IFC.

[0050] Figure 4 Shows the IFC entities involved and how they are connected to form RoboAvatar. IFC follows the principle that an entity consists of its physical substance (via IfcObject) and the properties of its representation (via IfcProperty) plus the relationships that connect them (via IfcRelationship). The physical substance of the robot is composed of the components connected in sequence. By referring to the URDF-based representation, it is decided to represent the robot physical substance in IFC through three types of objects (i.e., robot, link, and joint).

[0051] The robot as a whole is an assembly of components. Although it exhibits some unique properties that cannot be described by individual components, it does not have a clear geometry but is formed by the geometric representations of the individual components together. The appropriate IFC entity for modeling the robot will be IfcElementAssembly. IfcRelAggregates is used to connect the robot assembly with its components.

[0052] The link is a physically non-deformable component of the robot. The IFC provides several alternative ways to represent the link, such as IfcElementComponent, IfcDistributionElement, and IfcBuildingElementProxy. Table 2 compares these three entities, indicating that there is no existing entity that can perfectly reflect the nature of the link. In contrast, considering the scalability of IfcBuildingElementProxy, it may be most suitable for link representation.

[0053] Table 2. Comparison of IFC entities for robot link representation.

[0054] The joint is an abstract element that connects different links. This property can be well represented by the IfcVitualElement entity, which is commonly used to provide a hypothetical boundary between elements. IfcRelAssignToProduct is used to assign the links that the joint connects. To indicate the connection order, a joint must be assigned to its parent link, which is the object to which its child links are assigned, as Figure 4 shown.

[0055] The properties of the robot and its components are defined by IfcPropertySet and then assigned to the corresponding objects via IfcDefinesByProperties. It should be noted that some of the properties in Table 1 have been implicitly described in the robot matter modeled by the above IFC. For example, the connectivity (T cn ) and order (T s ) of the topological properties are easily encoded into the joint-link relationship specified by IfcRelAssignToProduct. Regarding the geometric properties, when creating link entities via IfcObjectPlacement and IfcProductRepresentation, the geometric properties have already been specified. Only add properties in addition to the implicitly specified ones. Table 3 summarizes the information on the newly added properties.

[0056] Table 3. Overview of information on newly added robot properties. * Type of matter, where R, L, and J represent robot, link, and joint respectively.

[0057] Each time a new link or joint is successfully transformed, it needs to be added to the robot container. Special attention should be paid to a) the generation of the visual representation of the link, and b) the specification of the placement of both the link and the joint, which will be elaborated in detail below.

[0058] Figure 5 The pseudocode in urdf shows how to transform the URDF visual representation V of the link into the IFC representation V ifc . There are four main types of visual representations in URDF, namely, "box", "cylinder", "sphere", "mesh". Depending on the type of V urdf , different strategies will be used for the transformation. If V urdf belongs to the "box" type (represented by <box>If the type is a "label indicator", its geometry is specified by the length, width, and height of the box. Using these three parameters, a copy of the box can then be generated in IFC. A similar principle applies to the "cylinder" type, where the bottom closed surface is a circle determined by its radius. When V urdf is "sphere", an IfcSphere element can be created based on the sphere radius to represent the corresponding V ifc . The most challenging case is when V urdf is an "irregular-shaped" "mesh" model. A series of formats may have been used to represent the mesh model, such as.stl,.dae, and.obj. Therefore, we first normalize different mesh models to the.stl format using the aspose-3d package and then uniformly load the models via the read_stl_file method of the OCC library. The parsed.stl file is further subdivided and represented in a subdivided form in IFC. It should be noted that some links may not have a visual representation.

[0059] The placement specification clarifies the position and orientation of link / joint placement. URDF and IFC adopt a similar local placement principle when specifying the object orientation. As Figure 6 shown, the placement of a joint is specified by referring to the coordinate system of its previous joint, while the placement of the first joint is assigned relative to the first link of the robot, which is usually a nominal link without a visual representation. By placing the joints in the appropriate positions, the orientation of the links can then be specified relative to the corresponding previous joints. This recursive process can be mathematically represented as follows:

[0060]

[0061] P link (n + 1) = pose link (n + 1)⊥P joint (n), (n ≥ 1) (5)

[0062] where pose⊥P represents the position and orientation of the object specified by the pose in the reference frame determined by P.

[0063] Different from URDF which makes full use of Euler rotations (i.e., roll α, pitch β, and yaw γ) to represent the orientation, IFC uses the normalized orientation vectors of the Z-axis and X-axis of the object (i.e., dir Z and dir X ) to represent its orientation. The mathematical formula for the transformation can be expressed as:

[0064]

[0065] dir X = (cosβcosγ, cosβsinγ, -sinβ) (7)

[0066] To evaluate the effectiveness of the format adjustment method provided by the present invention, the IFC-based representation of the robot RoboAvatar is evaluated in terms of the modeling aspects outlined in Figure 2 . To ensure comprehensiveness, multiple types of robots have been used for evaluation, including low-cost wheeled mobile robots (TurtleBot3 Burger), self-balancing wheel-legged robots (Direct Drive Diablo), and collaborative robotic arms (UR-5e).

[0067] Figures 7A to 7E The obtained building RoboAvatar of the UR-5e verified by IFC is shown, which is visualized in an IFC model viewer (e.g., BIMvision). The IFC representation has successfully replicated the "whole-part" structure of the robotic arm. This is evident from the tree structure in Figure 7A , where the robotic components (links) are represented by IfcBuildingElementProxy and the joints are represented by IfcVirtualElement. All links and joints have been combined to form the robotic assembly. As shown in Figure 7B , the geometry, topology, and other properties have also been effectively modeled. From a geometric and topological perspective, the IFC representation visually mimics its physical counterpart, which is shown in the upper left part of Figure 7B . Regarding other properties (such as productivity, capabilities, and mechanics), they have also been numerically represented and associated with the corresponding components.

[0068] The IFC building RoboAvatars of three distinct robots have been developed and compared with the URDF-based RoboAvatar. The evaluation is carried out from two aspects, namely, (a) the ability to represent the "whole-part" nature of the robotic material, and (b) the ability to describe various sets of robotic attributes. Table 4 presents an overview of the evaluation results. The study found that although the specific models are different, whether it is mobile robots like TurtleBot3 and Diablo, or fixed robotic arms like UR-5e, both IFC and URDF can correctly simulate the "whole-part" nature of the robotic material. However, when it comes to the ability to describe the attributes in Table 1, deviations are observed.

[0069] As shown in Table 4, the IFC-based RoboAvatar can consistently represent five aspects of properties across different robot models, while the traditional URDF format fails to describe the properties in terms of productivity and capabilities. For example, the IFC representation has successfully encoded the productivity properties (e.g., battery limit = 28.9 Wh and navigation speed = 0.22 m / s) and capabilities (e.g., climbing = "false" and locomotion = "wheel") of TurtleBot3 that are not available in the URDF format. The inclusion of these properties has important value for AECO activities. Consider the scenario of using a robot for facility inspection. After knowing the navigation speed and battery limit of the robot, different inspection plans can be better simulated and more accurately evaluated. The results demonstrate the effectiveness of the proposed IFC RoboAvatar in digitally representing the robot information required for built environment applications.

[0070] Table 4. Comparison between URDF- and IFC-based robot representations. * P-prop, M-prop, T-prop, C-prop, and G-prop refer to productivity, mechanical, topological, capability, and geometric properties, respectively.

[0071] Evaluate the performance of the computer-implemented method S100 represented as the RobIFC Transformer. The transformation performance is measured in three aspects, namely accuracy, time, and storage. Specifically, accuracy focuses on evaluating whether the transformer can correctly transform the visual presentation, structure, and core properties of the robot. A four-level grading system is established, where the number of "X"s reflects how many aspects the transformer has correctly transformed, as shown in Table 5.

[0072] Table 5. The established four-level grading system for transformation accuracy evaluation. * A correct transformation means zero tolerance for errors in the corresponding aspect (e.g., if the "properties" aspect is rated as correct, it means that all properties of the robot have been correctly transformed).

[0073] Figures 8A to 8C The resulting IFC representations for different robot types are shown separately, indicating robust transformation accuracy. The transformer has successfully produced the same IFC geometric representation as the URDF-based robot description. In fact, the two representations not only visually mimic each other but also have the same geometry and scale, as is evident from the same annotations of typical dimensions in the URDF and IFC representations. To indicate the accuracy of transforming the robot structure, a series of arrows are used to connect the corresponding robot components in the URDF and the resulting IFC representation. The transformer correctly processes and transforms all robot components (including links and joints). For example, the structure of the Direct Drive Diablo is the most complex, with a total of 11 links and 12 joints described in the original URDF file. After transformation, the same structure is retained, where the IFC-based Diablo representation also consists of 11 links (represented by IfcBuildingElementProxy) and 12 joints (represented by IfcVirtualElement). In Figures 8A to 8C Some key properties for direct comparison are also listed. After conversion to IFC, all properties remain unchanged, such as mass, joint type, its axis of rotation, and movement limits. The results demonstrate the highest level of transformation accuracy, where all three aspects of the visual representation, structure, and core properties have been correctly transformed.

[0074] We conducted a series of tests applying the RobIFCTrans transformer on six robots. The robots vary significantly in their type, geometric appearance, and motion mechanism, which can ensure the comprehensiveness and objectivity of the evaluation. Table 6 summarizes the performance of the RobIFCTrans transformer in terms of accuracy, time, and storage. As discussed above, the transformer can consistently ensure transformation accuracy regardless of the robot type involved. When it comes to efficiency, it is observed that the processing time is proportional to the size of the URDF representation of the robot. It records the longest time consumption (71.9 seconds) during the transformation of UR-5e, which has a model size of 6.1 MB. A more careful analysis of the transformation process reveals that a large portion of the time consumption is allocated to subdividing the mesh geometric representation of the robot. For example, out of the 71.9 seconds consumed in the UR-5e transformation, 68.8 seconds are spent on mesh subdivision. To some extent, the transformation is found to be storage-intensive. It significantly expands the size of the robot representation, as illustrated by TurtleBot3, which requires nearly 8 times the original space to store the resulting IFC file. The storage intensity can be related to the redundant nature of IFC.

[0075] Table 6. Performance evaluation of the URDF-to-IFC transformer.

[0076] Use Cases of Building RoboAvatar

[0077] Indoor Space Design for Robots

[0078] The IFC-based Building RoboAvatar can be seamlessly integrated into existing BIM-based design workflows, making it possible to consider early on how building designs can accommodate the use of construction robots. The use case is illustrated with a wall painting robot. The wall painting robot is a UR-5e mounted on a mobile platform. It has the ability to move on flat surfaces via its "wheel" mobility. The mobile platform has dimensions of 1.0m × 0.9m × 0.8m, and the reach of the UR-5e robot arm is 0.9m.

[0079] Figure 9 Figure A shows the initial design of the indoor space to be painted by the above wall painting robot. The color-coded map indicates the accessibility and reachability of the robot, showing that the robot cannot enter a large part of the space (i.e., S2 and S3). For example, the robot cannot enter S2 because it cannot pass through the stairs connecting S1 and S2. Regarding S3, entrance #1-3 is too narrow for the robot to pass through, as shown in the subfigure in Figure 9 Figure A. Incorporating the painting robot directly enables the initial design to adapt to the needs of robotic construction. As Figure 9 shown in Figure B, entrance #1-3 is widened to allow the robot to enter S3. At the same time, a ramp is added on one side of the stairs, which ensures that the robot can enter S2.

[0080] Figure 9 Although all spaces can now be accessed, it can be seen from Figure 9 Figure B that the robot cannot paint the upper ends of the walls. This is because the reach of the robot arm only allows a painting height of up to 1.7m (robot arm reach of 0.9m plus platform height of 0.8m). Obviously, it is not practical to lower the entire floor height to ensure full painting coverage. Therefore, a commendable solution is to adjust the design of the robot. For example, the mobile platform can be raised from the current 0.8m to 1.3m, and at the same time, the robot arm should be replaced by a robot arm with a larger reach (e.g., a UR-10e with a 1.3m reach). This will increase the painting coverage to up to 2.6m. In this case, all the walls in the space can be appropriately covered, as

[0081] Figure 10 Shows an integrated design example of a mechanical, electrical, and plumbing (MEP) inspection robot. False ceilings are commonly used to store, conceal, and protect essential building facilities (e.g., MEP systems). False ceilings must be inspected and maintained regularly to avoid health and safety hazards such as water accumulation, corrosion, and pest infestation. Existing false ceiling maintenance procedures are labor-intensive and can be both troublesome and dangerous due to poor lighting conditions, limited inspection space, and numerous obstacles. Robots have the potential to assist people in inspection tasks, but false ceiling designs that are incompatible with robots often undermine their performance. Using the IFC-based representation of the inspection robot, we can directly import the robot model into the BIM design software Revit. It enables us to easily implement an integrated design, which is detailed as follows:

[0082] To thoroughly inspect the false ceiling, the robot must traverse into complex pipework to observe objects of interest and detect defects. With both the robot and the MEP model open in the same software environment, Dynamo in Revit is used to retrieve model information (e.g., robot height, MEP clearance, etc.) and check for conflicts. Taking a hospital project as an example, first, the clearance between fixtures and ceiling panels is found and then compared with the robot's bounding box. As a result, it is detected that the positions of 694 pipes need to be adjusted. Based on the above inspection, some suggestions are given to the designers: (i) The 694 pipes detected on the fourth floor can be moved to a higher position to provide a larger clearance; (ii) Rearranging the position of the MEP system can reduce visual obstructions and thus expand the effective field of view of the robot.

[0083] Figure 11 Shows another design example with a robot implemented by the IFC robot representation. It is based on the case of a delivery robot within a hospital. The robot needs to be allowed to reach as many rooms as possible to serve more people. Due to our invention, the models of the delivery robot and BIM for the said project can be integrated into the same design environment based on Revit. In Dynamo of Revit, the designer generates the bounding box of the robot based on RoboAvatar and compares it with the size of the door to perform a size compatibility check, as Figure 11 shown. Doors that are incompatible with the robot size will display a warning and be marked in red. The designer needs to consider the function of the room and the function of the robot to decide whether to enlarge the size of the door.

[0084] The above example shows how the Building RoboAvatar can facilitate the co - adaptation of building and robot designs to allow for the smooth adoption of robotics in construction. Of course, this is a simplified hypothetical scenario. Since real - life designs always involve trade - offs between multiple objectives under a set of constraints, envisioning drastic design changes just to accommodate the use of robots may be too idealistic. However, the situation shows the promise of our Building RoboAvatar in facilitating designers to consider robots as an additional design factor.

[0085] Digital Twin Facility Management Robot

[0086] Another use case for the Building RoboAvatar is the high - fidelity digital twin of facility management robots. As a digital copy that reflects the real - time state of a physical robot, the digital twin is considered a promising way to effectively plan facility management carried out by robots. However, significant effort is required to develop an avatar of a robot that is compatible with the existing IFC - based building environment representation. This either results in a significant workload of reprogramming existing robot descriptions into IFC - based RoboAvatars or in a low - granularity simplified representation (e.g., representing the robot with a box or a point).

[0087] The IFC - based Building RoboAvatar provides a scalable way to generate compatible and high - fidelity robot digital twins. Take the TurtleBot3 used for facility inspection as an example. Users can simply use our transformer to convert its off - the - shelf URDF representation into an undistorted IFC - based avatar, and the transformer will be compatible with mainstream BIM software such as Autodesk's Revit and BIM 360. Figure 12 The IFCAvatar of the TurtleBot in the Revit model of an office space is shown. All details of the robot (e.g., its constituent components and corresponding mechanical properties) are retained in the IFCAvatar. Subsequently, the movement of the physical robot can be directly combined with its avatar for intuitive monitoring during facility inspection.

[0088] Figure 13 Presents the digital twin results of the TurtleBot while inspecting office facilities. When the robot navigates in the office, videos of its surrounding environment are recorded for visual inspection. The robot's state, which includes position (i.e., x, y, and z) and pose (i.e., the yaw angle of the robot and the rotation angles of the robot's wheels), is monitored and sent to a remote server. The digital twin based on the IFC digital robot representation periodically retrieves the robot state data from the server. The retrieved information is used to animate the corresponding joints of the robot digital twin. For example, if the left wheel of the robot rotates 0.5 radians, its digital counterpart will rotate the same angle around the central joint. Figure 13 The line graph in the upper right corner shows the data recorded throughout the inspection process. The IFCRoboAvatar is animated based on the real-time captured data, forming a digital twin of the physical robot. Figure 13 The top view plane graph in the upper left corner shows the movement trajectory of the robot, with three timestamps highlighted in circles. The streaming video captured by the physical camera and the corresponding digital twin are also Figure 13 shown.

[0089] It can be observed that the resulting digital twin has a high level of granularity, capable of not only indicating the robot's movement using a realistic geometric avatar but also capturing the subtle kinematic movements of its components. Figure 13 The significant change in the position of the blue cube at the center of the wheel on the right relative to the red cube clearly demonstrates this. These cubes are added to indicate the wheel movement. The screen recording footage can be found online at: https: / / youtu.be / EHvGLBLpf4U. The high-fidelity digital twin opens up new possibilities for kinematic or dynamic simulation of FM robots in the BIM environment.

[0090] Figure 14 Shows an example of how to make full use of the integration implemented by the IFC RoboAvatar to remotely control the robot. In this web portal, we have applied our invention to integrate the robot and the building model. Since the environmental information is readily available, if the robot is required to perform certain tasks (such as moving to a certain point), simply click on the target location in this model. And then the system can automatically use the building information to plan the robot's movement.

[0091] Building RoboAvatar as a way to achieve interoperability in construction robotics

[0092] With the increasing popularity of construction robotics, the issue of interoperability has emerged, where the data formats used to represent robots and the built environment are mutually incompatible. This has caused limitations and additional costs in the adoption of robotics in construction. This interoperability dilemma is not new, but it occurs whenever disruptive digital technologies are introduced into the AECO industry, as observed in the efforts to integrate BIM with Geographic Information System (GIS). When such a dilemma arises, an interface is needed to facilitate interoperability.

[0093] By laying out the information required to represent robots in the built environment, the Building RoboAvatar has the potential to become an interface that bridges robots and the AECO domain.

[0094] First, the definition of the Building RoboAvatar fully considers the needs of buildings and the basic structure of robots. Two property aspects that are of interest to AECO professionals have been introduced, namely productivity and ability. The former characterizes parameters that are highly relevant to the productivity of robots when performing construction tasks, such as production volume and success rate, which are crucial for overall project planning and scheduling. On the other hand, the ability property defines what a specific robot can do, such as climbing and grasping. This information is helpful for task-level planning or assignment. In addition to the high-level properties, our Building RoboAvatar also inherits the mainstream description methods in existing robot representations. This intentional design of the Building RoboAvatar goes beyond robotics and AECO, providing a unique opportunity to bridge these two domains.

[0095] Second, the Building RoboAvatar has been validated through IFC, the de facto common language of the AECO industry. Without proper validation, no matter how well the RoboAvatar is defined, it will not have much impact in implementation. This is especially true in the construction industry, which uses a large number of proprietary design and project management software programs. We chose IFC, a widely accepted vendor-neutral data format for describing building asset information, to instantiate the Building RoboAvatar. Entities in the IFC format were carefully examined and compared to determine the appropriate model view definition to validate the RoboAvatar. The IFC-based representation ensures compatibility with the mainstream AECO software and workflows that are mainly based on BIM.

[0096] Third, a transformer for URDF-to-IFC conversion is developed, which provides a means to directly utilize many off-the-shelf RoboAvatar tools. Different from existing attempts to convert IFC building environment representations to URDF, our RobIFCTrans transformer views the adoption of robotics from a building-centric perspective. It provides a new approach for robot-oriented design. By converting existing URDF RoboAvatar to IFC representations, our method can directly integrate robot information into existing BIM-based design workflows. Subsequently, the robot-oriented design concept is concretized into something implementable in existing design tools, enabling AECO professionals to explore how building designs can accommodate the introduction of robots.

[0097] Functional units and modules of [devices, apparatuses, systems, compounds, materials, and / or methods] according to embodiments disclosed herein may be implemented using a computing device, a computer processor, or an electronic circuit system, including but not limited to application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, and other programmable logic devices configured or programmed according to the teachings of this disclosure. Computer instructions or software code running on a computing device, a computer processor, or a programmable logic device may be readily made by practitioners in the software or electronics arts based on the teachings of this disclosure.

[0098] All or part of the method according to an embodiment may be executed in one or more computing devices including a server computer, a personal computer, a laptop computer, a mobile computing device (such as a smartphone), and a tablet computer.

[0099] Embodiments may include computer storage media, transient and non-transient memory devices having computer instructions or software code stored therein, the computer instructions or software code being usable to program or configure a computing device, a computer processor, or an electronic circuit system to execute any of the programs of the present invention. The storage media, transient and non-transient memory devices may include (but are not limited to) floppy disks, optical disks, Blu-ray disks, DVDs, CD-ROMs, and magneto-optical disks, ROMs, RAMs, flash memory devices, or any type of medium or device suitable for storing instructions, code, and / or data.

[0100] Each of the functional units and modules according to various embodiments may also be implemented in a distributed computing environment and / or a cloud computing environment, where all or part of the machine instructions are executed in a distributed manner by one or more processing devices interconnected by a communication network, such as an intranet, a wide area network (WAN), a local area network (LAN), the Internet, and other forms of data transmission media.

[0101] Although the present disclosure has been described and illustrated with reference to specific embodiments of the invention, such descriptions and illustrations are not restrictive. The figures may not necessarily be drawn to scale. Due to manufacturing processes and tolerances, differences may exist between the figures in the present disclosure and the actual devices. There may be other embodiments of the present disclosure that are not specifically described. Modifications may be made to adapt a particular situation, material, composition of matter, method, or process to the purposes and scope of the present disclosure. All such modifications are intended to be within the scope of the appended claims. Although the methods disclosed herein have been described with reference to specific operations performed in a particular order, it should be understood that these operations may be combined, sub-divided, or re-ordered without departing from the teachings of the present disclosure to form equivalent methods. Thus, unless specifically indicated herein, the order and grouping of the operations are not limitations.< / box> < / joint> < / joint> < / robot>

Claims

1. A method and system for implementing digital representation and conversion of a robot based on industrial foundation classes, converting a robot-centric data format into a building-centric data format to digitally represent the robot, wherein the building-centric data format uses a plurality of links representing non-deformable parts of the robot and a plurality of joints describing the relationship between the links to represent the robot, the method comprising: extracting robot information from the robot-centric data format; creating a robot container to model the robot as a whole-part structure; transforming the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format based on the extracted robot information; filling the robot container with the transformed links and the transformed joints; as well as The building-centric data format is constructed using the robot container populated with the transformed links and the transformed joints. 2 . The computer-implemented method of claim 1 , wherein extracting robot information from the robot-centric data format comprises parsing the robot-centric data format.

3. The computer-implemented method of claim 1 , wherein extracting robot information from the robot-centric data format comprises: Calculate the total number of parts by counting all the connecting rods; as well as Calculate the robot mass by adding the masses of all links.

4. The computer-implemented method of claim 1 , wherein transforming the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format comprises: Create a corresponding connecting rod type material entity for each connecting rod; specifying the corresponding link type material entity using link properties extracted from the robot-centric data format; Create corresponding joint type material entity; as well as The corresponding joint-type material entity is specified using joint properties extracted from the robot-centric data format. 5 . The computer-implemented method of claim 4 , wherein transforming the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format further comprises generating a visual representation for each link.

6. The computer-implemented method of claim 5, wherein the visual representation is a box-type visual representation; and the corresponding link-type material entity is specified by a length, a width, and a height of the box-type visual representation.

7. The computer-implemented method of claim 5, wherein the visual representation is a cylindrical visual representation; and the corresponding connecting rod-type material entity is specified by a base radius and a height of the cylindrical visual representation.

8. The computer-implemented method of claim 5, wherein the visual representation is a spherical visual representation; and the corresponding link-type material entity is specified by a radius of the spherical visual representation.

9. The computer-implemented method of claim 5, wherein the visual representation is a mesh-type visual representation; and the corresponding link-type material entity is specified by a mesh model of the mesh-type visual representation.

10. The computer-implemented method of claim 1, wherein transforming the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format further comprises: Specify the placement of each link; as well as Specify the placement of each joint.

11. A computer-implemented system for adapting a robot-centric data format to a building-centric data format for representing a robot, wherein the robot-centric data format represents the robot using a plurality of links representing non-deformable components of the robot and a plurality of joints describing relationships between the links, the system comprising a processor configured to: extracting robot information from the robot-centric data format; creating a robot container to model the robot as a whole-part structure; transforming the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format based on the extracted robot information; filling the robot container with the transformed links and the transformed joints; as well as The building-centric data format is constructed using the robot container populated with the transformed links and the transformed joints.

12. The computer-implemented system of claim 11, wherein the processor is further configured to extract the robot information from the robot-centric data format by parsing the robot-centric data format.

13. The computer-implemented system of claim 11 , wherein the processor is further configured to extract the robot information from the robot-centric data format by: Calculate the total number of parts by counting all the connecting rods; and Calculate the robot mass by adding the masses of all links.

14. The computer-implemented system of claim 11, wherein the processor is further configured to transform the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format by: Create a corresponding connecting rod type material entity for each connecting rod; specifying the corresponding link type material entity using link properties extracted from the robot-centric data format; Create corresponding joint type material entities; and The corresponding joint-type material entity is specified using joint properties extracted from the robot-centric data format.

15. The computer-implemented system of claim 14, wherein the processor is further configured to transform the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format by generating a visual representation for each link.

16. The computer-implemented system of claim 15, wherein the visual representation is a box-type visual representation; and the corresponding link-type material entity is specified by a length, a width, and a height of the box-type visual representation.

17. The computer-implemented system of claim 15, wherein the visual representation is a cylindrical visual representation; and the corresponding connecting rod-type material entity is specified by a base radius and a height of the cylindrical visual representation.

18. The computer-implemented system of claim 15, wherein the visual representation is a spherical visual representation; and the corresponding link-type material entity is specified by a radius of the spherical visual representation.

19. The computer-implemented system of claim 15, wherein the visual representation is a mesh-type visual representation; and the corresponding link-type material entity is specified by a mesh model of the mesh-type visual representation.

20. The computer-implemented system of claim 11, wherein the processor is further configured to transform the plurality of links and the plurality of joints from the robot-centric data format to the building-centric data format by: Specify the placement of each link; and Specify the placement of each joint.

Images