Robotic manipulation of material units

Abstract

Systems for and methods of robotic manipulation of material units. A robotic arm and / or a platform provide a movable base. A gripper and / or gantry device can grip and carry the material units. A motorized gimbal device is arranged between the base and the gripper and / or gantry device. The gimbal device stabilizes the material units being carried by the gripper and / or gantry device. The material units can include masonry units and / or sheathing units.

Description

TITLE: ROBOTIC MANIPULATION OF MATERIAL UNITS CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U.S. Provisional Application No.63 / 721,643 filed on November 18, 2024, the entire contents of which are hereby incorporated herein by reference.FIELD

[0002] The present disclosure is directed generally to gimbal devices and robotics.BACKGROUND

[0003] The following paragraphs are not an admission that anything discussed in them is prior art or part of the knowledge of persons skilled in the art.

[0004] United States Patent No. 6,370,837 B1 discloses a system for laying masonry blocks.

[0005] United States Patent No. 6,687,645 B1 discloses a motion and trajectory data generator for a multi-gimbaled rotating platform.

[0006] United States Publication No. 2009 / 0038258 A1 discloses an automated brick laying system for constructing a building from a plurality of bricks.

[0007] United States Patent No. 8,825,208 B1 discloses automated construction machinery and method.

[0008] United States Patent No. 8,965,571 B2 discloses a brick laying system. INTRODUCTION

[0009] The following paragraphs are intended to introduce the reader to the detailed description that follows and not to define or limit the claimed subject matter.

[0010] The present disclosure relates to robotic systems and methods for manipulating material units, which can be masonry units and / or sheathing units forconstruction, either onsite or in factory. In various examples, the systems and methods can achieve: multi block / bricklayer, that places block / bricks level and plum with and without mortar; and / or concrete extruder for mortar delivery on block or bricks. In some examples, the systems and methods can further achieve: floor joist and / or ceiling joists level and plum with nailed into place; lifting of housing frames level and plum while construction workers can nail in place sections of the housing frame, included outside frame and internal frames; siding installation, including plywood or oriented strand board; subfloor installation, including plywood or oriented strand board, with adhesive and can be fastened to joists with nails or screws; and / or roof installation, including materials, and angled roof installation, rafters, trusses, and the fastening of plywood or oriented strand board.

[0011] In some examples, a gripping apparatus and / or a gantry apparatus is mounted on a motorized gimbal. Tools or grippers can be mounted on the gantry slider gantry. The gimbal arm can be attached to a robot crane arm with a counter balancing weight. Cameras can be mounted on the crane end as part of the gimbal. The counter balancing weight can sit on a motorized slider. The crane arm can be mounted to a turret that has a motor that moves the crane arm. The turret can have a motor that rotates the turret. The turret can sit on a cart chassis with wheels. The cart chassis can sit on a monorail with wheel bearings.

[0012] In some examples, a braking system can be used with each gimbal axis, crane arm, and turret.

[0013] In some examples, cameras can be used, and markers, such as Arllco markers or QR codes, can be provided on the robot and other components. In some examples, there can be marker towers and rows, on the towers and rows there are a plurality of cameras and markers. In some examples, markers, such as Arllco markers or QR codes, can be disposed on the material units and provide information related to characteristics of the material units.

[0014] In some examples, robotic control and computer vision control can be implemented. The use of motorized gimbals with PID (proportional-integral-derivative) controllers can be combined to tools and methods for material handling, and material manipulation. In some examples, a level base is not necessary or required. The motorized digital-gyroscope gimbal can correct axis points and stabilize the tools and materials.

[0015] In some examples, cordless technologies can be used, including battery powered power tools. Digital control of tools electronically can be implemented, including taking a tool and connecting to the electronic mechanism. The gantry slider mount can have pressure sensors and wires to communicate with the tools and manage them.

[0016] In some examples, an inertial measurement unit can connect with a computer system, and use artificial intelligence.

[0017] In some examples, tools can be connected electronically to computer and / or microcontroller through a mechanical isolation method, and track the tools to act or turn on the tool for a duration. Tools used can be battery powered and have mechanical switches that close an electric loop and trigger the tool, and this can be connected with mechanical coupler.

[0018] In some examples, glare filtering can be used for the cameras ensuring accurate object size and / or edge detection. In some examples, night vision can be used to see in darkness through image enhancement.

[0019] In some examples, components can be water resistant and temperature controlled to allow for external weather conditions based on the environment being used.

[0020] In some examples, inverse kinematics can be used in place of or with the inertial measurement unit. In some examples, a chassis telehandler can be used. In some examples, the monorail can be motorized. In some examples, a braking system can be used with each gimbal axis.

[0021] In some examples, a nail gun device can be mounted on the gimbal, and / or on the gantry slider. In some examples, a screw driver device can be mounted on thegimbal, and / or on the gantry slider. In some examples, a spraying device can be mounted on the gimbal, and / or on the gantry slider. In some examples, a motorized router can be mounted on the gimbal, and / or on the gantry slider.

[0022] In some examples, a motorized saw can be mounted on the gimbal, and / or on the gantry slider. In some examples, a concrete extruder can be mounted on the gimbal, and / or on the gantry slider. In some examples, a bricks jointer tools can be mounted on the gimbal, and / or on the gantry slider. In some examples, a rotating brush can be mounted on the gimbal, and / or on the gantry slider. In some examples, a wall tie device fastening system can be mounted on the gimbal, and / or on the gantry slider. In some examples, a brick trowel can be mounted on the gimbal, and / or on the gantry slider. In some examples, a vacuum gripper can be mounted on the gimbal, and / or on the gantry slider.

[0023] In some examples, a magnetic attachment connector with sensors can be used. Once placed down in locations, there can be a load cell force signal.

[0024] In some examples, preparing of blocks / bricks to enable the block / brick laying can include: providing blocks / bricks on a conveyor belt, pass through a mortar application device, placing mortar on bricks and ready for robotic arm pickup; with sensors, measuring the deflection; the blocks / bricks are placed on the wall on top of a wall with mortar with the final destination and the bricks being leveled in place. The gripper disengages, the brakes / safety disengage and the grippers.

[0025] Other aspects and features of the teachings disclosed herein will become apparent, to those ordinarily skilled in the art, upon review of the following description of the specific examples of the present disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings included herewith are for illustrating various examples of apparatuses and methods of the present disclosure and are not intended to limit the scope of what is taught in any way. In the drawings:

[0027] FIG. 1 A is a side view of an exemplary system.

[0028] FIG. 1 B is a perspective view of the system.

[0029] FIG. 2A is a perspective view of portions of the system, namely a gripper and / or gantry device, a gimbal device, and a connection.

[0030] FIG. 2B is a side view of the gripper and / or gantry device, the gimbal device, and the connection.

[0031] FIG. 20 is an exploded view of the gripper and / or gantry device, the gimbal device, and the connection.

[0032] FIG. 2D is an exploded view of the gimbal device.

[0033] FIG. 3A is a perspective front view of the gripper and / or gantry device.

[0034] FIG. 3B is a perspective rear view of the gripper and / or gantry device.

[0035] FIG. 3C is an exploded view of the gripper and / or gantry device.

[0036] FIG. 4A is a side view of a gripper mechanism above a brick.

[0037] FIG. 4B is a side view of the gripper mechanism gripping the brick.

[0038] FIG. 40 is perspective view of the gripper mechanism.

[0039] FIG. 5A is a perspective view of a tool and gantry connector.

[0040] FIG. 5B is an exploded view of the tool and gantry connector.

[0041] FIG. 5C is a side view of the tool and gantry connector.

[0042] FIG. 5D is a side view of the tool and gantry connector, positioned at the start of connecting.

[0043] FIG. 5E is a side view of the tool and gantry connector, positioned fully connected.

[0044] FIG. 6A is an exploded side view of portions of the system.

[0045] FIG. 6B is an exploded front view of portions of the system.

[0046] FIG. 7A is a front view of a marker tower.

[0047] FIG. 7 B is a front view of a marker row.

[0048] FIG. 7C is an exploded view of the marker tower.

[0049] FIG. 7D is an exploded view of the marker row.

[0050] FIG. 8A is a perspective view of a wall tie fastener apparatus.

[0051] FIG. 8B is an exploded view of the wall tie fastener apparatus.

[0052] FIG. 9A is a perspective view of a mortar extrusion apparatus.

[0053] FIG. 9B is an exploded view of the mortar extrusion apparatus.

[0054] FIG. 10A is a front view of a jointer apparatus for vertical joints.

[0055] FIG. 10B is a front view of a jointer apparatus for horizontal joints.

[0056] FIG. 10C is a perspective view of the vertical and horizontal jointer apparatuses.

[0057] FIG. 11 A is a side view of a trowel apparatus.

[0058] FIG. 11 B is a perspective view of the trowel apparatus.

[0059] FIG. 12A is a side view of a rotating brush apparatus.

[0060] FIG. 12B is a perspective view of the rotating brush apparatus.

[0061] FIG. 13A is a side view of an adhesive / caulk extrusion apparatus.

[0062] FIG. 13B is an exploded view of the adhesive / caulk extrusion apparatus.

[0063] FIG. 14A is a perspective view of a vacuum gripping apparatus.

[0064] FIG. 14B is an exploded view of the vacuum gripping apparatus.

[0065] FIG. 15A is a perspective view of a screw gun / nail gun apparatus.

[0066] FIG. 15B is an exploded view of the screw gun / nail gun apparatus.

[0067] FIG. 16 is a side view showing a gripper forking into a brick stack.

[0068] FIG. 17 is a top view showing the gripper after forking into the bricks.

[0069] FIG. 18 is a top view showing the bricks aligning with a wall.

[0070] FIG. 19 is a side view showing the gripper with the bricks.

[0071] FIG. 20 is a front view of showing the bricks being placed downwardly.

[0072] FIG. 21 is a perspective view showing use of a wall tie fastener apparatus.

[0073] FIG. 22 is a top view showing use of the wall tie fastener apparatus.

[0074] FIG. 23 is a top view showing use of a mortar extrusion apparatus.

[0075] FIG. 24 is a perspective view showing use of the mortar extrusion apparatus.

[0076] FIG. 25 is a perspective view showing use of a horizontal jointer.

[0077] FIG. 26 is a perspective view showing use of a vertical jointer.

[0078] FIG. 27 is a perspective view showing use of a rotating brush apparatus.DETAILED DESCRIPTION

[0079] Various apparatuses or methods will be described below to provide an example of an embodiment of each claimed invention. No embodiment described below limits any claimed invention and any claimed invention may cover apparatuses and methods that differ from those described below. The claimed inventions are not limited to apparatuses and methods having all of the features of any one apparatus or method described below, or to features common to multiple or all of the apparatuses or methods described below. It is possible that an apparatus or method described below is not an embodiment of any claimed invention. Any invention disclosed in an apparatus or method described below that is not claimed in this document may be the subject matter of another protective instrument, for example, a continuing patent application, and the applicant(s), inventor(s) and / or owner(s) do not intend to abandon, disclaim or dedicate to the public any such invention by its disclosure in this document.

[0080] Referring to FIGS. 1 A and 1 B, an exemplary apparatus is shown generally at reference numeral 10. In the example illustrated, the apparatus 10 includes a robotic arm 12, a gripper and / or gantry device 14, a gimbal device 16, a connection 18, a turret 20, a motorized cart 22, and a monorail 24. Further illustrated are attachable tools on a tool rack 26, a camera and marker row 28, and a camera and marker tower 30.

[0081] Referring to FIGS. 2A and 2B, the connection 18 includes a marker 32 and a camera 34. A gimbal joint 36 can provide for rotation about a vertical axis. A gantry slider 38 is provided for travel on a linear rail 40 with belt drive. In the example illustrated, the gripper and / or gantry device 14 is shown to include four gripper mechanisms 42. A gimbal joint 44 can provide for turning about a first horizontal axis. A gimbal joint 46 can provide for turning about a second horizontal axis, which can be orthogonal to the first horizontal axis. In addition, the gimbal joints can be a combination of orthogonal and non-orthogonal, or non-orthogonal providing for turning of joints for horizontal positioning and / or any angle positioning.

[0082] Referring to FIG. 20, the example illustrated features a housing 48, a computer 50 with motion controller with backup battery, a geared motor 24, unique markers 54, a pillow block bearing 56, a keyed rod 58, thrust bearings 60 between two mounts, drum and drum brake 62, a geared motor 64 to activate brake for horizontal turning axis, a geared motor 66 to activate brake for vertical turning axis, a drum brake 68, a motor housing 70, rod and bearings 72, a linear extrusion 74, a computer 76 with motion controller with backup battery, and a geared motor 78.

[0083] Referring to FIG. 2D, the example illustrated features a computer with motion controller and backup battery 80, a drive shaft 82 that connected to a drum brake, a pillow block bearing 84 for drive shaft, a flanged connector 86 that connects drive shaft to gimbal axis, a gripper or gantry connection 88, an arm axis connection 90, a flanged connector hole 92, and a flanged bearing hole 94.

[0084] Referring to FIGS. 3A and 3B, the gantry slider 38 and linear rail 40 are again shown. A geared motor 96 is connected to drive wheel that connects drive belt thatconnects to slider. A flanged connector 98 connects to drive shaft or to bearing for gimbal axis. A main stabilizing extrusion 100 connects to axis motor on arm and holds arm together.

[0085] Referring to FIG. 30, the gantry slider 38 and the linear rail 42 are shown above a belt drive and idler 102, and the extrusion 100 used as the substructure for linear rail. Fasteners 104 are provided. Further shown are slider blocks 106 that connect rail to individual gripper covers 108. A housing for electronics 110 also connects to the slider blocks 106. The belt drive 102 connects to a drive shaft 112 and a geared motor 114. A linear rail 116 connects to allow for the spreading of grippers with servos. A camera 118 is provided. Linear bearings 120 connects to linear rod 122 which further connects to gripper 42.

[0086] Referring to FIGS. 4A and 4B, a housing 124 for gripper motor and electronics is shown, and each gripper mechanism 42 can have its own inertial measurement unit (IMU) therein. The gripper 42, with forks 126 aligned to holes of a brick 128, is shown in preparation for gripping and picking the brick 128.

[0087] The teachings herein relate to the manipulation of material units, which can include masonry units and / or sheathing units.

[0088] The teachings herein can be applied to manipulate both bricks and blocks, and the term masonry unit is intended to encompass both bricks and blocks. For manipulation of bricks and turning, longer forks and grippers are used to ensure maximum friction and leverage of the brick.

[0089] The teachings herein can also be applied to manipulate various types of sheathing that can be used for constructing subfloors, walls, and roofing, including oriented strand boards (OSB), plywood, drywall, structured insulated panels (SIP), and engineered wood products. The term sheathing unit is intended to encompass any such panels and sheathing.

[0090] The term material unit is further intended to encompass other objects that can be robotically manipulated and used in construction and / or manufacturing, including,for example, various tools, structural beams and joists, fasteners, rebar, shingles, tiles, windows, doors and / or other building materials formed of metal, wood and / or composite materials.

[0091] Referring to FIG. 4C, a geared servo motor 130 is shown connected to a driveshaft 132, that connects to an inner side of a gripping plate 134. An adjustable screw 136 is coupled to a load cell 138 and allows for calibration thereof. The load cell 138 is a contact sensor that uses a strain gauge to measure force when pressing down with a gripper mechanism and / or apparatus to measure the force being used. An extrusion 140 serves as the substrate for the various components. A small geared motor 142 with coupler and ball screw can connect to an adjacent gripper, and can be used to adjust for different bricks sizes, used to adjust space between bricks due to irregular brick sizes, used when picking up bricks from a brick stack, and / or the lateral manipulation allows direct placement from brick stack to final destination.

[0092] Referring to FIGS. 5A and 5B, a tool and gantry connector is shown to include a housing 144 for tools, a connector 146 that is connected to the housing 144, a mating hole 148, connectors for electronics 150, and a gantry belt drive connector 152. The connectors for electronics 150 can have wires meant to provide electrical power and communicate with computer, microcontrollers, motors, motion controllers, sensors, and magnetic locking mechanism. A computer and microcontroller housing 154 is shown beside an electromagnet 156, and a marker 158.

[0093] FIG. 50 shows the tool and gantry connector, positioned above connector. FIG. 5D shows the tool and gantry connector, positioned in the start of connecting. FIG.5E shows the tool and gantry connector, positioned fully connected.

[0094] Referring to FIGS. 6A and 6B, shown again are the robotic arm 12, the turret 20, the motorized cart 22, and the monorail 24. A guide extrusion 84 is also used for mounting a counter balance 162. A crane arm 164 connects to a motor drive shaft with rod and bearings 166, and has a drum brake (not shown) that is controlled by a geared motor. A geared motor 168 moves weight back and forth on linear rail. A turret gearedmotor 170 is for turning the turret left or right. Turret plates 172 compress large through bearings 174. A drum brake 176 is provided for the turret motor 170 and a motor 178 controls the brake 176. The turret 20 further includes a base plate 180. An onboard battery 182 is centered to keep balance. A tow ball 184 is for connecting the chassis to a tow motor. A railed guide connector 186 is coupled to the monorail 24. Wheels 188 convey the cart 22 over a ground surface.

[0095] In some examples, the robotic arm can be omitted, and a platform can be implemented in its place. The motorized gimbal device can be arranged between the platform and the gripper and / or gantry device. The platform can be robotically and / or mechanically moved to a desired starting position and serve as a base for the system.

[0096] Referring to FIGS. 7A, 7B, 7C and 7D, the camera and marker row 28 and the camera and marker tower 30 are shown in further detail. These components can be spaced according to need. These components can be made from extrusions, with power running through the channel of the extrusion. Each can feature a large, stable base that is weighted.

[0097] Referring to FIGS. 8A and 8B, a wall tie fastener apparatus 190 includes a rechargeable fastener 192, a magazine of fasteners 194, a hole 196 for placement of the magazine 194, and a magazine of wall ties 198.

[0098] Referring to FIGS. 9A and 9B, a mortar extrusion apparatus 200 includes a drill 202 that connects with an auger 204, creating pressure to pass mortar in a bucket 206, and the nozzle 208 then pushes the mortar through a shaping nozzle 210. An enclosure 212 has markers 214 used for lining up the extruder.

[0099] Referring to FIG. 10A, a jointer apparatus for vertical joints is shown at reference numeral 216. Referring to FIG. 10B, a jointer apparatus for horizontal joints is shown at reference numeral 218. Jointers 216, 218 are shown side-by-side in FIG. 10C.

[0100] FIGS. 11A and 11 B show a trowel apparatus 220. FIGS. 12Aand 12B show a rotating brush apparatus 222. FIGS. 13A and 13B show an adhesive / caulk extrusionapparatus 224. FIGS. 14A and 14B show a vacuum gripping apparatus 226. FIGS. 15A and 15B show a screw gun / nail gun apparatus 228.

[0101] In some examples, a screw driver device can be mounted on the gimbal, and / or on the gantry slider. In some examples, a spraying device can be mounted on the gimbal, and / or on the gantry slider, and can be configured to spray, for example, paint, clear coat and / or joint compound. In some examples, a sanding / grinding apparatus can be mounted on the gimbal, and / or on the gantry slider. In some examples, a cutting / sawing apparatus can be mounted on the gimbal, and / or on the gantry slider. In some examples, a drilling apparatus can be mounted on the gimbal, and / or on the gantry slider. In some examples, a welding apparatus can be mounted on the gimbal, and / or on the gantry slider. In some examples, a motorized router can be mounted on the gimbal, and / or on the gantry slider.

[0102] With the foregoing features having been described, the following is intended to give an understanding of how such components may be implemented and used within exemplary systems.Artificial intelligence

[0103] Artificial intelligence is intelligence exhibited by machines with computers. It can use machine perception which is a computer system capable of receiving and interpreting data from inputs including live sensors, stored data from databases, and / or computation sources like pre-trained models created from machine learning, methods of deep learning, natural language processing, large language models, and artificial general intelligence.

[0104] Artificial general intelligence is a type of artificial intelligence that can provide cognitive capabilities for action, decision making, and / or two-way communication, and can be accessed by application programming interfaces (APIs). Generally, an API is a connection between computers or between computer programs, which can act as software intermediaries that allow applications to talk to each other. APIs can have specifications, sets of rules, and / or protocols that enable software applications tocommunicate with each other to exchange data, features, functionality and actions. Such methods / protocols for communication include Representational State Transfer (REST), Simple Object Access Protocol (SOAP), Remote Procedure Call (RPC), and chatbots.

[0105] In some examples, artificial general intelligence can break down work, jobs, makes decisions and creates task / actions and navigation design to create actions for automation and manufacturing. The planning and navigation using logical sequential actions and tasks, the spatial management of robots and tools, and problem solving manufacturing issues that arise, with the capability of creating revised or new plans, tasks.

[0106] In some examples, artificial general intelligence can make decisions and creates tasks such as pattern recognition, edge detection, and motion tracking.

[0107] In some examples, artificial general intelligence can do object detection and can be used for manufacturing with image identification, segmentation, and labeling, on images.

[0108] In some examples, artificial general intelligence can make decisions and creates tasks / actions detecting and tracking a moving object in a live video stream.

[0109] In some examples, artificial general intelligence can make decisions and creates tasks / actions for image processing pipelines, facilitating tasks like recognizing tools, materials, components, state of work, and / or analyzing images for defects in manufacturing.

[0110] In some examples, artificial general intelligence can make decisions and creates tasks / actions creating automation systems and used to process video input and trigger actions based on observations.

[0111] In some examples, artificial general intelligence can make decisions and creates tasks / actions creating a robot navigation system stitch images together, real-time maps of the manufacturing environment.

[0112] In some examples, artificial general intelligence can make decisions and creates tasks / actions for image and video processing capabilities integrated into thecomputer guidance system, for pre-processing and real-time analysis. This can include video processing and feature detection, motion detection, i.e. tracking moving objects in a video feed using frame differencing.

[0113] In some examples, artificial general intelligence can make decisions and create tasks / actions for augmented reality application, feature detection is used in identifying real-world objects and overlaying virtual information, enabling accurate placement of augmented reality objects. Combining object detection with frame-by-frame analysis, the system can interpret materials, tools, machines, vehicles and provide real time date for controlling robots.General algorithm

[0114] For the main stage management algorithm, for each task, the main data points and data streams that are tracked are as follows:1. before stage, robot arm get into starting position;2. command received stage, commands are received;3. computer guidance select tools and / or materials, or continue with current tool; 4. computer guidance the robot apparatus to start work, during the work stage;5. computer guidance uses algorithm, patterns, artificial intelligence / machines learning to execute tasks;6. computer guidance to do work ensure work follows the path while tracking current stage; and7. completion stage of task and / or job.How robotic apparatus obtains instructions

[0115] In some examples, a computer guidance system follows a pattern algorithm for home construction and sends this information.

[0116] In some examples, robot with human user, identified jobs and task to the computer guidance system by any or all of the following:1. touch pad;2. markers such as QR codes, used to visually communicate with robotic systems;3. voice commands and voice instruction interface for advance manufacturing using voice recognition software; and / or4. Wi-Fi, Bluetooth, telepresence, satellite, cell network and / or global positioning system (GPS).

[0117] In some examples, artificial intelligence, machine learning, and / or algorithms guide the computer guidance system.Artificial intelligence with direct machine control

[0118] In some examples, Artificial intelligence (Al) is used for construction and construction related manufacturing to directly control devices, machines, vehicles, robots and robotic systems, databases, sensors and computer vision, augmented reality for the task, and APIs.

[0119] The current state of the art of Al that it is used with are artificial general intelligence (AGI) and artificial intelligent agents (Al agents), large language models (LLM), machine learning (ML), and can be used with retrieval-augmented generation (RAG). RAG can be used with Al to access construction and advanced manufacturing information on databases (external to the Al) and knowledge bases, predefined processes, best practices, and / or prioritization of processes. Using this construction contextualized information for data-driven decision-making and for managing many robotic systems for automation.

[0120] The following are exemplary steps:Step 1 : Al creation plans and actions for autonomous construction management; Step 2: Al generates the action from the plans created in Step 1 ;Step 3: action, monitoring, and evaluation cycle (repeats 3a and 3b until completion);Step 3a: Al action stage; andStep 3b: Al evaluates current status of tasks and monitors the results.

[0121] Expanded steps are as follows:1. Al creation plans and actions for autonomous construction management:a. Al gets data for plan from database and knowledge base for construction site and next stage of construction;b. based on data retrieved, Al defines direct control of devices: machines, vehicles, robots and robotic systems, databases, sensors and computer vision, augmented reality for the task, and API;c. Al generates step-by-step actions / tasks for devices;d. Al generates tests and test methods to validate those tasks, checking for the current state of completed and completion of those tasks;e. creates plans for the management of the supply chain and job site materials;andf. determines Al Agents needed for tasks.2. Al generates the action from the plans created in step 1 :a. Al creates the action for the control of devices, machines, vehicles, robots and robotic systems, databases, sensors and computer vision, augmented reality for the task, and API according to step 1 ; andb. Al creates all the testing actions to check the progress until completion. 3. Action, monitoring, and evaluation cycle - repetition of steps 3a and 3b until completion of actions and / or plans:a. Al action stage;i. use the action created in step 2;b. Al evaluates current status of tasks and monitors the results;i. check results for actions and their defined results in real-time during construction; andii. evaluate the status of constructions base the test created in step 2.Marker system

[0122] In some examples, cameras can be used, and markers, such as Arllco markers or QR codes, can be provided on the robot and other components. In some examples, there can be marker towers and rows, on the towers and rows there are a plurality of cameras and markers. In some examples, markers, such as Arllco markersor QR codes, can be disposed on the material units and can provide information related to characteristics of the material units. QR codes and Arllco markers are number systems that have limited size, but can be used in exemplary systems to provide data for:1. identifiers for jobsite;2. identifiers for tool locations;3. job site information;4. item names;5. tool identifiers;6. alignment detection;7. height detection;8. edge detection to draw lines;9. standardized size detection, e.g. based on size of QR codes;10. QR codes size and dimensions for creating lines and shapes;11. retrieve data, video and / or images with a scanner for human review; and / or 12. Al and machine learning to combine all data on QR codes, video, images, to combine sensor data acquired during a specific procedure.Motions controller

[0123] A motions controller contains motor driver circuitry that can send electrical current and voltage to motors to cause motion. Motion controllers usually include a microcontroller or a computer that sends or relays messages / data / instructions from another computer and converts them into signals.Bricks stacking, organization and delivery

[0124] There are multiple types of brick stacks and brick types. Bricks with holes can be picked up with fork grippers. Grippers can pick up bricks up from a stack of bricks. A bricks with no holes may require stack of bricks to be prepared in such a way that the bricks can picked up with a friction / clamp gripper.Magnetic coupler

[0125] In some examples, a magnetic attachment connector can be used and tools are connected electronically to computer or microcontroller through a mechanical isolation method and tracking the tools to act or turn on the tool for a duration. Tools used in this way can be battery powered and have mechanical switches that close an electric loop and trigger the tool, this is connected with mechanical coupler.Marker towers and rows

[0126] In some examples, marker towers are used and / or with marker rows. These can be used to enable navigation, robot orientation, tool orientation, alignment of materials, and / or for the computer to create virtual lines for alignment. Specific uses can include for spacing of materials, job site identification, and / or brick rows alignment Gimbal stabilization

[0127] The gimbal is composed of camera, geared motors, motional controller, brakes, inertial measurement units, grippers, and / or tool(s), or gantry with a slide to attached tools. It is connected to a computer with computer vision software and artificial intelligence software.

[0128] The motorized gimbals with inertial measurement units can be used as a gimbal stabilization system. The stabilization system can isolate the materials, tools, and gantry from the chassis, therefore allowing the gimbal to work independently of any irregularities of uneven ground. The inertial measurement units, data is used to gradually slow down or accelerate the arm and gimbal.

[0129] It can be connected to a computer that determines the angle the gimbal stabilization system has to do at any time during motion and the speed of motion required.

[0130] The gimbal can maintain objects, tools, or the gantry level and plum.

[0131] The computer vision system can keep the objects, tool, or the gantry aligned and when stacking materials aligned vertically along the body and align diagonally across the body.Computer guidance

[0132] In some examples, computer vision software can be implemented on a computer or remote server with cameras. The cameras can be located on the robot and all around a work area allowing for view of the advanced manufacturing area as well as the moving of materials from outside the manufacturing area. The computer vision software can use prebuilt libraries in machine learning methods, artificial intelligence methods, large learning model methods, computer based analysis, algorithms, and built in libraries are part of the computer vision software.

[0133] The computer vision system software is the main software computing and providing instructions to robotic systems, robotic behaviors, gathering feedback from all sensors and storing all the data on a database.

[0134] The cameras are used with the computer vision for live streaming of video and processed by the computer vision live computing information. Another is used to take the livestream data and display live information over the images and to record the live video with embedded information on a hard drive / server. Another is to take pictures with the data and save the data. The data location and information is stored in a database, where it can be retrieved by the same computer vision software or by an external query.

[0135] Videos and photos are taken by a camera stored on a server that has a hard drive, the location of the video and / or photos are references in the database record.

[0136] Every task or action that is executed can be tracked against a master list of tasks. On completion of tasks, an option for the computer can inspect the work using machine learning and artificial intelligence to determine the quality of work and if the state of work is complete. The computer then pulls the data from the master list for the next task and determines the next stage of work and tools of the job to be used.

[0137] The data can be used for creating masks or overlays for use with sending instructions to the motion controller, uses images to create mask / overlay from image and overlays / superimposed / mask over the live videos. It can create paths, to points and returnpaths by obtaining encoder data to create a return path, and reversing the encoder information and returns.Mask and overlay generation

[0138] Masking and drawing virtual lines using computer vision software can allow the computer to send instructions to orient the turret, arm, gimbal, and or gimballed gantry to match the angle in parallel or target a specific point. The gimbal and gimbal gantry can be moved into position to align with the mask created and have the gimbal match the virtual lines created to line up with. The mask can be used to determine where the fasteners or materials go. The mask, overlay and superimposed path can be used to control the angle of the tools, grippers, and / or gantry.

[0139] The robot can align the tools, grippers, gantry with the virtual lines on the mask and the computer can use the motional controller to direct into position, for fastening, the fastener can be lowered to an exact target point on the mask with the correct angle for the tool.

[0140] The location of materials can be identified, for example: bricks, wood, completed panels, plywood, and joists. Information can be obtained by inputs including QR codes or other markers, with locations of structures stored in a database.

[0141] The mask and overlays can be used by the computer to guide the robot, orientation of the gimbal, the robot arm, the turret, and the chassis. They can be used in conjunction with algorithms and / or computer vision, to generate motion control, tool selection and the path of the tools, and the use of tools.

[0142] The process involves the computer obtaining an image information database for use with the current task, jobsite, for the current robot at the current time, and the following steps:1. From the database, obtains an image or multiple images and converts it to a mask for the current task.2. Gets image information database for use with current task.Creates a mask and overlay from the top view and performs calculations on where the path will be based on the task algorithm.The computer calculates the real world size of objects, using Arllco markers and / or QR codes markers information stored. The size of the markers are stored in the database and from the size of the markers the computer can extrapolate the side of an object. Creating a virtual grid system using QR codes where the computer vision system uses specific QR codes to draw virtual lines for compute guidance. The computer vision system takes the mask and overlay, and places it over the realtime / live stream video.The computer vision system uses the algorithm for the task to determine the path, angle, and place materials, use tools and the angle of tools.The computer vision system uses cameras to identify, starting position of tasks, and sends information to the motion control. On what to do, in the order from this point. This is the computer guidance system.The computer guidance system engages the brakes on the motors to lock them when needed, or to slow down the rotation to reduce inertia.The computer guidance system compares live stream information with the overlay and mask, and path, to the overlay / mask of an image and the relative position to the live image. This information to make corrections and adjustments in real time. This includes sensor data and inertial measurement unit data.The computer tracks and records all the live feed over the overlay, masks, and guidance paths and stores this information in a database.QR codes and computer vision replace the use of sensors for placement of materials, endstops, starting points, center lines. The end stops can replace the use of sensors.Computer vision systems and artificial intelligence and machine learning can be used to calibrate tools, toll locations to compensate for motor or gear skipping orbacklash. The computer vision system can observe the location of the arm using QR codes and control the motors through a motion controller and give instructions to it to guide the chassis, arm, gimbal, gripper, or tools to the desired location. 13. QR codes as endstops to calculate movement from a specific point with computer visions with OpenCV.Material management

[0143] For identification and shape detection, machine learning and artificial intelligence software or services can be deployed. The color of material, and if the material has Arllco markers I QR codes, can be used to obtain information from the database. With wood material, bow / crown can be detected. The number of holes in a brick can be determined. Size can be estimated with Arllco markers I QR codes. For example, a brick can be identified by if it is the correct size, if it has holes, and what is the color.

[0144] After identification it is confirmed if the materials meet the specification and are not damaged, for example a brick is the correct size, if it works with the gripper, the wood is too warped, the right color brick and the brick color matches specification.

[0145] Computer guidance system guides the orientation of the material according to placement.Laying bricks or blocks

[0146] When laying bricks, the computer guidance system can send the arm to the bricks or block location to be picked up by the motorized gimbal gripper. Bricks can then be picked up from a brick or block stack. FIG. 16 shows the gripper 42 (FIG. 4A and 4B) forking into a brick stack 230 in direction 232. The stack 230 is shown on a scissor lift with safety rails removed 234.

[0147] The fork-style gripper can be guided by the computer guidance system into the bricks and clamped and fully grips the bricks. The computer guidance system can receive pressure sensor data and visual input from camera(s) to determine if the gripper has forked or encompassed the bricks or blocks and then clamps the bricks or blocks.FIG. 17 shows the gripper after forking into bricks, with circular arrow 236 indicating turning path.

[0148] Using the computer guidance system, the clamped bricks or blocks are then moved above the location they are to be placed. The robot gimballed gripper can then spread or adjust the spacing between the bricks. FIG. 18 shows the bricks aligning with the wall, and the arrows 238 indicating that the bricks 128 will spread.

[0149] The computer guidance system can then use the inertial measurement unit to make the bricks or blocks level and plum based on the final desired orientation of the bricks or blocks. FIG. 19 shows the bricks 128, with arrow 240 indicating turning and axis stabilizing.

[0150] The computer guidance system can arrange the bricks straight horizontally and diagonally. Arranging is the process of aligning the bricks or blocks horizontally across multiple bricks or blocks. Arranging the bricks diagonally, is the process of aligning the bricks or blocks scrolls multiple rows of bricks, arranging ensures all the bricks plum across multiple rows.

[0151] The computer guidance system can lower the bricks or blocks to the desired height and position and press the bricks or blocks into mortar. FIG. 20 shows the bricks 128 being placed in a downward direction 242.

[0152] The computer guidance system can determine the final thickness of the mortar between all bricks and stops when the desired thickness is reached between the bricks. The spacing of the bricks can be measured by the computer guidance system.

[0153] The computer guidance system can check if the bricks are level, plum, arranged horizontally and arranged diagonally. If necessary, the computer guidance system can adjust the placement of bricks or blocks until it is accurate.

[0154] The computer guidance system can release the brick when it is in the correct position.Corners

[0155] In some examples, laying bricks or stone for building the comers of a wall involves the computer system guiding the arm to build connected bricks or stone by weaving courses into the comers.

[0156] For the first course, this can involve placing bricks or blocks down with one end bricks or stones at the end of the wall or desired position to have a corner. The next row is placed adjacent to the end brick or stone that will be the corner with the predetermined spacing. Mortar is placed over the bricks or stone in the desired amount to achieve a result.

[0157] The second course, the bricks or stone are placed in an alternating pattern, from the bottom upward. This is weaving the courses, and weaving the comers. The computer guidance system tracks the placement of bricks or stones, in real time ensuring the corner is weaved accurately.

[0158] The computer guidance system can adjust the bricks or blocks until the angle is achieved. The computer guidance system ranges and ranges diagonal for two intersecting walls of bricks or blocks.

[0159] In some examples, wall ties can be placed to secure brick or block wall to wood frame. In some examples, at a right angle (90 degrees), the computer guidance system can align edges of the bricks or blocks vertically, visually creating a straight line vertically as the edge of the bricks.Wall ties

[0160] In some examples, wall ties can be placed by the robot using the wall tie fastener apparatus. FIGS. 21 and 22 show wall tie installation with the wall tie fastener apparatus 190 (FIGS. 8A and 8B), and with Arllco markers I QR codes 244 applied to oriented strand board (OSB) or plywood 246. The apparatus 190 is moving in direction

[0161] The computer guidance system can track all the bricks or blocks being placed. The wall tie is an “L” shaped metal member that is fastened to a wood frame and placed at a right-angle wall and brick, with at least one fastener placed into the wall securing the wall tie.

[0162] The computer guidance systems algorithm can determine if wall ties are necessary. The computer guidance system can use an algorithm to determine where the wall ties are required to be placed to support the brick or blocks and according to building code.Mortar extruder

[0163] In some examples, the computer guidance system can select the mortar extrusion apparatus to apply mortar on the bricks or blocks. It can guide placement of mortar on top of the bricks or blocks. FIGS. 23 and 24 show the mortar extrusion apparatus 200 (FIGS. 9A and 9B) at the start of mortar placement, and moving in direction 250.Horizontal and vertical jointers

[0164] In some examples, when there is excess mortar, jointers can be used having a blade to clean out excess mortar along the horizontal and / or vertical axes.

[0165] When the laying of bricks or block reaches a predetermined height by the computer, the next stage in the process can be to smooth the joints and to brush the bricks. The gimbal gantry can be moved into position and rotated to pick up the horizontal jointer. This is used for joints of at least one brick horizontally in an up and down motion until the joints are prepared. The next step can be for the gimbal gantry to return the horizontal jointer to the rack and attach the vertical jointer, and the vertical jointers can then go into to position, the brake system locks the jointer in the correct location, using the marker system to align the jointer in the correct path it will follow. The jointer will slide black and forth along the gantry smoothing or preparing the wall.

[0166] FIG. 25 shows the horizontal jointer 218 (FIG. 10B), with arrow 252 showing direction of robot arm movement. FIG. 26 shows the vertical jointer 216 (FIG. 10A), with arrow 254 showing direction of robot arm movement.

[0167] The robot on the main track system can go back and forth and reposition the jointers for each stage in the wall to complete the wall. The robot arm can alternate between horizontal and vertical jointers until the process for the wall or corner is complete.Rotating brush

[0168] In some examples, the robot arm can move the gimballed gantry and replace the jointers with the rotating brush. FIG. 27 shows the rotating brush apparatus 222 (FIGS. 12A and 12B), with arrow 256 indicating direction of spinning.

[0169] The rotating brush can lightly brush the wall back and forth removing mortar or dust from the wall. The brush can only reach an outwardly facing surface of the bricks or blocks and does not brush over the mortar, so as not to disturb the mortar.Adhesion tool

[0170] The robot arm can use the gimballed gantry to attach an adhesive dispensing tool. Guided by the marker rows and a computer masking of the location of the joists, the computer can determine the center of each joist. The computer can then guide the gimbaled gantry over the center with the adhesive attachment and move along the line of the joist, placing adhesive down the center of each joist.Fasteners device

[0171] Fastener devices, such as nail gun, screw gun / screw driver, and / or wall tie fastener can be used. The computer guidance system can guide the tool to the location of interest. The gimballed fastener, once over the area, can be lowered to a starting position. Fasteners can be placed in specific angles and spacing according to the algorithm or pattern. The process can continue until all of the fasteners is complete, based on the current mask / overlay according to the computer guidance software.Subfloor placement and installation

[0172] This process can be used for placing panels on side housing frames (not shown) and can also be used to place drywall walls on a frame as well for interiors. The gimballed gantry can exchange tools for the vacuum gripper for the subfloor placement. The gimballed gantry, with the vacuum gripper, can use the computer system and determine the size of a subfloor piece to be placed.

[0173] The computer system can select the subfloor and place the vacuum gripper on the piece, engage the vacuum, and pick up the subfloor and move it into position, guided by the computer vision system, marker row system, and / or masks, and place the piece in position guided by the arm.

[0174] The brake system can be engaged as necessary to ensure movement or placement. In the event of being off, the arm can adjust and place the subfloor exactly as needed, usually half on the partial of the joint.

[0175] After placing all the subfloors, the robot arm can exchange the gimbaled vacuum arm with a fastening tool, either a screw driver system or a nail gun system, for example.

[0176] The robot vision system, using the marker rows and the computer vision masking, can then guide the fastening tool to fasten the subfloor in place. The gimballed gantry with the fastening tool can move the gantry slider with the fastening to the correct position, placing the fastening at spacing location and angle as determined by the computer system.

[0177] In some examples, adhesive can be applied before placement of the subfloor.

[0178] While the above description provides examples of one or more apparatuses or methods, it will be appreciated that other apparatuses or methods may be within the scope of the accompanying claims.

Claims

CLAIMSI claim:

1. A system for robotic manipulation of material units, comprising:a robotic arm;a gripper and / or gantry device; anda motorized gimbal device arranged between the robotic arm and the gripper and / or gantry device for stabilizing the material units.

2. The system of claim 1 , wherein the gripper and / or gantry device comprises at least one gripper mechanism adapted to grip and carry the material unit.

3. The system of claim 2, wherein the at least one gripper mechanism comprises a geared motor connected to a driveshaft, and the driveshaft connects to an inner side of a gripping plate that is configured to engage an outer surface of the material unit.

4. The system of claim 3, wherein the driveshaft is configured to draw the gripping plate towards the forks for gripping the material unit therebetween, and / or the driveshaft is configured to draw the forks towards the gripping plate to grip the material unit therebetween.

5. The system of claim 3 or 4, wherein the at least one gripper mechanism comprises forks configured to align with and engage holes of the material unit.

6. The system of any one of claims 2 to 5, wherein the at least one gripper mechanism comprises a load cell for weighing and / or measuring force when pressing the material unit being carried.

7. The system of any one of claims 2 to 6, wherein the at least one gripper mechanism comprises an inertial measurement unit.

8. The system of any one of claims 2 to 7, wherein the gripper and / or gantry device comprises a plurality of the gripper mechanisms arranged to carry a row of the material units.

9. The system of any one of claims 1 to 8, wherein the gripper and / or gantry device comprises a gantry slider that is configured to move along a linear rail and is driven with a geared motor and a belt drive and / or a ball screw transmission.

10. The system of claim 9, comprising, for connection with the gantry slider:a wall tie fastener apparatus comprising a rechargeable fastener, a load cell, a magazine of fasteners, and a magazine of wall ties;a mortar extrusion apparatus comprising a rechargeable driver system, a load cell, a bucket, mortar, an auger, and a nozzle for applying mortar on the material units;a jointer apparatus comprising a load cell, a plurality of blades for preparing mortar in joints between the material units;an adhesive / caulk extrusion apparatus comprising a rechargeable gun for applying adhesive / caulk, and a load cell;a vacuum gripping apparatus comprising a rechargeable vacuum and a load cell; and / ora screw gun / nail gun fastener apparatus comprising a rechargeable gun, a magazine of fasteners, and a load cell.

11. The system of any one of claims 1 to 10, wherein the gimbal device comprises first, second and third gimbal joints for turning about first, second and third axes.

12. The system of claim 11, wherein the first, second and third axes are mutually orthogonal.

13. The system of claim 11 or 12, wherein the gimbal device comprises a geared motor and a brake system for each of the gimbal joints.

14. The system of any one of claims 1 to 13, wherein the robotic arm comprises a counterbalance comprising a weight and a geared motor configured to move the weight along a linear rail.

15. The system of any one of claims 1 to 14, wherein the robotic arm is supported by a turret that comprises a geared motor configured to turn the turret side-to-side.

16. The system of claim 15, wherein the turret is supported by a motorized cart, and the motorized cart comprises wheels for driving over a ground surface and / or is configured to drive along a monorail.

17. The system of any one of claims 1 to 16, comprising a plurality of cameras and a plurality of markers that are readable by the cameras.

18. The system of claim 17, wherein at least a portion of the markers are disposed on the material units and provide information related to characteristics of the material units.

19. The system of claim 17 or 18, wherein each of the markers comprises an Arllco marker or a QR code.

20. The system of any one of claims 17 to 19, wherein at least a portion of the cameras and the markers are housed in at least one row or tower.

21. The system of any one of claims 1 to 20, wherein the material units comprise masonry units.

22. The system of any one of claims 1 to 21, wherein the material units comprise sheathing units.

23. A method of robotic manipulation of material units, comprising:moving a robotic arm to a starting position;with a gripper and / or gantry device, gripping at least one of the material units; andwith a motorized gimbal device arranged between the robotic arm and the gripper and / or gantry device, stabilizing the at least one of the material units.

24. The method of claim 23, comprising picking up the at least one of the material units from a stack, moving the at least one of the material units to above a desired location it is to be placed, and lowering the at least one of the material units to its desired final position.

25. The method of claim 23 or 24, comprising controlling a gripper mechanism of the gripper and / or gantry device to grip and carry the material unit.

26. The method of claim 25, comprising:engaging an outer surface of the material unit with a gripping plate;aligning and engaging forks with holes of the material unit; anddrawing the gripping plate towards the forks to grip the material unit therebetween, and / or drawing the forks towards the gripping plate to grip the material unit therebetween.

27. The method of claim 25 or 26, comprising weighing and / or measuring force when pressing the at least one material unit with a load cell of the at least one gripper mechanism.

28. The method of any one of claims 25 to 27, comprising receiving data from an inertial measurement unit of the at least one gripper mechanism.

29. The method of any one of claims 25 to 28, comprising using a plurality of the gripper mechanisms to carry a row of the material units.

30. The method of any one of claims 23 to 29, comprising moving a gantry slider along a linear rail with a geared motor and a belt drive and / or a ball screw transmission.

31. The method of claim 30, comprising, with the gantry slider:installing wall ties;extruding mortar and applying it to at least a first portion of the material units;preparing mortar in joints between the first portion of the material units; applying adhesive / caulk to frames;with a vacuum gripper, picking up and placing at least a second portion of the material units; and / orfastening the second portion of the material units to a frame.

32. The method of any one of claims 23 to 31 , controlling a geared motor and a brake system for each of first, second and third gimbal joints.

33. The method of any one of claims 23 to 32, comprising moving a weight along a linear rail as a counterbalance.

34. The method of any one of claims 23 to 33, comprising supporting the robotic arm by a turret, and turning the turret side-to-side.

35. The method of claim 34, comprising supporting the turret by a motorized cart, and driving the motorized cart over a ground surface and / or along a monorail.

36. The method of any one of claims 23 to 35, comprising using computer vision and guidance software to control at least one of the robotic arm, the gripper and / or gantry device, and the gimbal device.

37. The method of claim 36, comprising reading a plurality of markers with a plurality of cameras.

38. The method of claim 37, comprising disposing at least a portion of the markers on the material units and, with the markers, providing information related to characteristics of the material units.

39. The method of claim 37 or 38, wherein each of the markers comprises an Arllco marker or a QR code.

40. The method of any one of claims 36 to 39, comprising applying at least one of a mask and an overlay with the computer vision and guidance software.

41. The method of any one of claims 36 to 40, comprising receiving information from at least one of a touch pad, markers, voice commands, Wi-Fi, Bluetooth, telepresence, satellite, cell network and global positioning system (GPS).

42. The method of any one of claims 36 to 41 , comprising using at least one of artificial intelligence, machine learning, and algorithms.

43. The method of any one of claims 23 to 42, wherein the material units comprise masonry units.

44. The method of any one of claims 23 to 43, wherein the material units comprise sheathing units.

45. A system for robotic manipulation of material units, comprising:a platform;a gripper and / or gantry device; anda motorized gimbal device arranged between the platform and the gripper and / or gantry device for stabilizing the material units.

46. The system of claim 45, wherein the gripper and / or gantry device comprises at least one gripper mechanism adapted to grip and carry the material unit.

47. The system of claim 45 or 46, wherein the gripper and / or gantry device comprises a tool and gantry connector.

48. The system of any one of claims 45 to 47, wherein the gripper and / or gantry device comprises at least one apparatus for extruding material, and optionally the material consists of mortar, adhesive, caulk and / or cement.

49. The system of any one of claims 45 to 48, wherein the gripper and / or gantry device comprises at least one apparatus for fastening or tying, and optionally the at least one apparatus comprises a nail gun, a screw gun, a screw driver, a wall tie fastener and / or a rebar tying tool.

50. The system of any one of claims 45 to 49, wherein the gripper and / or gantry device comprises at least one vacuum gripping apparatus, and optionally the at least one vacuum gripping apparatus is configured for gripping masonry units, sheathing units, drywall, structured insulated panels, shingles, tiles, windows and / or doors.

51. The system of any one of claims 45 to 50, wherein the gripper and / or gantry device comprises at least one spraying apparatus, and optionally the at least one spraying apparatus is configured for spraying paint, clear coat and / or joint compound.

52. The system of any one of claims 45 to 51 , wherein the gripper and / or gantry device comprises at least one of a trowel apparatus and a brush apparatus.

53. The system of any one of claims 45 to 52, wherein the gripper and / or gantry device comprises at least one sanding / grinding apparatus, at least one cutting / sawing apparatus, at least one drilling apparatus, and / or at least one welding apparatus.

54. The system of any one of claims 45 to 53, wherein the gimbal device comprises first, second and third gimbal joints for turning about first, second and third axes.

55. The system of claim 54, wherein the first, second and third axes are mutually orthogonal.

56. The system of any one of claims 45 to 56, wherein the platform comprises at least one of a robotic platform and a mechanical platform.

57. A method of robotic manipulation of material units, comprising:moving a platform to a starting position;with a gripper and / or gantry device, gripping at least one of the material units; andwith a motorized gimbal device arranged between the platform and the gripper and / or gantry device, stabilizing the at least one of the material units.

58. The method of claim 57, comprising controlling a gripper mechanism of the gripper and / or gantry device to grip and carry the material unit.

59. The method of claim 57 or 58, comprising connecting the gripper and / or gantry device to at least one tool.

60. The method of any one of claims 57 to 59, comprising extruding material with the gripper and / or gantry device, and optionally the material consists of mortar, adhesive, caulk and / or cement.

61. The method of any one of claims 57 to 60, comprising fastening or tying with the gripper and / or gantry device, and optionally using a nail gun, a screw gun, a screw driver, a wall tie fastener and / or a rebar tying tool.

62. The method of any one of claims 57 to 61, comprising vacuum gripping objects with the gripper and / or gantry device, and optionally the objects are masonry units, sheathing units, drywall, structured insulated panels, shingles, tiles, windows and / or doors.

63. The method of any one of claims 57 to 62, spraying with the gripper and / or gantry device, and optionally spraying paint, clear coat and / or joint compound.

64. The method of any one of claims 57 to 63, comprising, with the gripper and / or gantry device, at least one of trowelling, brushing, sanding / grinding, cutting / sawing, drilling and welding.

65. The method of any one of claims 57 to 64, comprising using computer vision and guidance software to control at least one of the platform, the gripper and / or gantry device, and the gimbal device.

66. The method of claim 65, comprising reading a plurality of markers with a plurality of cameras.

67. The method of claim 66, comprising disposing at least a portion of the markers on the material units and, with the markers, providing information related to characteristics of the material units.

68. The method of claim 66 or 67, wherein each of the markers comprises an Arllco marker or a QR code.

69. The method of any one of claims 65 to 68, comprising applying at least one of a mask and an overlay with the computer vision and guidance software.

70. The method of any one of claims 65 to 69, comprising receiving information from at least one of a touch pad, markers, voice commands, Wi-Fi, Bluetooth, telepresence, satellite, cell network and global positioning system (GPS).

71. The method of any one of claims 65 to 70, comprising using at least one of artificial intelligence, machine learning, and algorithms.

72. The method of any one of claims 57 to 71, comprising moving the platform robotically and / or mechanically to the starting position.

73. An apparatus or a method comprising any combination of one or more of the features described above and / or illustrated in the drawings.

Images