Mobility platform for autonomous navigation on construction sites

Abstract

An autonomous tool including a multi-sensor package that enables identification of the tool's position within a construction site so that the tool can navigate to a location where a task is to be performed. The tool may include at least one sensor from the group of a drive system encoder, a stereo camera, an inertial measurement unit, an optical flow sensor, and a LiDAR. The tool may include a holonomic drive that enables at least one tool on a mobility platform to reach the end of the construction site. The platform may be used as part of a system that receives design information regarding the construction site and the task to be performed by the tool, and then generates commands to control the tool to navigate along a path that is generated to include landmarks that are within range of the sensors for a majority of the path and therefore detectable by the sensors for a majority of the path.

Description

[Technical Field]

[0001] Related Applications

[0001] This application claims the benefit under 35 USC § 119(e) of U.S. Provisional Patent Application No. 62 / 843,815, filed May 6, 2019, which is incorporated herein by reference in its entirety.

[0002] Field FIELD OF THE INVENTION

[0002] The disclosed embodiments relate to autonomous positioning of tools or implements on a construction site and related methods of use. [Background technology]

[0003] background

[0003] Several attempts have been made to deploy autonomous or semi-autonomous systems on construction sites that can perform targeted tasks. These conventional systems typically use total station surveying instruments or beaconed navigation systems that require the placement of navigational devices or beacons at known locations on the construction site. Summary of the Invention [Means for solving the problem]

[0004] overview In some embodiments, a method for operating a mobility platform within a construction site based on instructions for one or more tasks to be performed at one or more task locations within the construction site and a design file with at least a two-dimensional representation of the construction site, where the design file includes one or more landmarks within the construction site and the mobility platform includes at least one tooling location, the method includes generating and / or executing, with at least one processor, a path for the mobility platform that relies on the locations of the one or more landmarks and the one or more task locations by iteratively adjusting the path to increase accuracy by compensating for detected errors or increasing the amount of the path within a threshold distance of the one or more landmarks. The method also includes providing task commands to a controller disposed on the mobility platform based on the generated path and the one or more tasks.

[0005] In some embodiments, a mobility platform for navigation at a construction site includes a drive system configured to move the mobility platform in three degrees of freedom, at least one actuator operably coupled to the drive system and configured, upon activation, to move the drive system to correspondingly move the mobility platform in at least one of the three degrees of freedom, and at least one tool operable to perform one or more tasks within the work site. The mobility platform also includes a controller having a motion control unit configured to selectively activate or deactivate the at least one actuator, and a tool control unit configured to selectively activate or deactivate the at least one tool to perform the one or more tasks.

[0006] In some embodiments, a method for operating a mobility platform within a construction site, the platform including a plurality of sensors having operating ranges within which the sensors can detect landmarks, the method includes identifying locations of one or more landmarks within the construction site and identifying one or more tasks to be performed at one or more task locations within the construction site. The method also includes generating a path for the mobility platform that relies on the locations of the one or more landmarks and the one or more task locations by iteratively adjusting the path to increase accuracy by compensating for detected errors or increasing an amount of the path within a threshold distance of the one or more landmarks, where the threshold distance is based on the operating ranges of the sensors.

[0007]

[0007] In some embodiments, a method for operating a mobility platform within a construction site includes identifying locations of one or more landmarks within the construction site, moving the mobility platform along a navigation path based at least in part on the locations of the one or more landmarks, detecting the one or more landmarks as the mobility platform is moved along the path by at least one selected from the group consisting of a stereo camera, an inertial measurement unit, an optical flow sensor, and a LiDAR unit, and correcting the movement of the mobility platform based on information provided by at least one of the drive system encoders, the stereo camera, the inertial measurement unit, the optical flow sensor, and the LiDAR unit.

[0008]

[0008] In some embodiments, a method for operating a mobility platform within a construction site based on instructions for one or more tasks to be performed at one or more task locations within the construction site and a design file with at least a two-dimensional representation of the construction site, wherein the design file includes one or more landmarks within the construction site and the mobility platform includes at least one tool mounting location, the method includes using at least one processor to generate a path for the mobility platform that is dependent on the locations of one or more of the landmarks and one or more task locations by iteratively adjusting the path to increase predicted positioning accuracy of the mobility platform based on predicted error in estimated position from continuous displacement and / or sensing the position of the mobility platform relative to one or more landmarks along the path, and providing task commands to a controller disposed on the mobility platform based on the generated path and the one or more tasks.

[0009]

[0009] A method for operating a mobility platform within a construction site, the platform including a plurality of sensors having an operating range within which the sensors can detect landmarks, the method including identifying the locations of one or more landmarks within the construction site, identifying one or more tasks to be performed at one or more task locations within the construction site, and generating a path for the mobility platform that is dependent on the locations of one or more of the landmarks and one or more task locations by iteratively adjusting the path and / or detecting a position relative to one or more landmarks along the path to increase predicted positioning accuracy of the mobility platform based on predicted error in estimated position from continuous displacement.

[0010]

[0010] It should be appreciated that the above-mentioned concepts, and further concepts described below, may be arranged in any suitable combination, as the present disclosure is not limited in this respect. Furthermore, other advantages and novel features of the present disclosure will become apparent from the following detailed description of various non-limiting embodiments when considered in conjunction with the accompanying drawings.

[0011] BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are not to scale. In the drawings, identical or nearly identical components shown in various figures may be represented by like reference numerals. For clarity, not every component may be labeled in every figure. [Brief explanation of the drawings]

[0012] [Figure 1]

[0012] FIG. 1 is a schematic diagram of an exemplary embodiment of a construction support system comprising a mobility platform for navigation on a construction site. [Figure 2]

[0013] 1 is a schematic top view of an exemplary embodiment of a mobility platform chassis in a first position. [Figure 3]

[0014] 3 is a schematic top view of the mobility platform of FIG. 2 in a second position. [Figure 4]

[0015] FIG. 1 is a functional block diagram of an exemplary embodiment of a system and method for operating a mobility platform at a construction site. [Figure 5A]

[0016] 1 is a graphical representation of an example embodiment of a construction site design file. [Figure 5B]

[0017] 5B is a graphical representation of an example embodiment of landmarks extracted from the design file of FIG. 5A. [Figure 5C]

[0018] 1 is a graphical representation of an example embodiment of constraints for at least one tool extracted from a design file. [Figure 6A]

[0019] 1 is a graphical representation of an example embodiment of landmark spheres of influence utilized in some embodiments of a route optimization method for a mobility platform navigating in a construction site. [Figure 6B]

[0020] 6B is a graphical representation of an example embodiment of a route generated for a mobility platform based on the landmark spheres of influence of FIG. 6A. [Figure 7]

[0021] FIG. 2 is a schematic diagram of an exemplary embodiment of a task server, a mobility control unit of a mobility platform, and a tool control unit. [Figure 8]

[0022] FIG. 2 is a schematic diagram of an exemplary embodiment of a motion control unit for controlling a mobility platform along a route. [Figure 9]

[0023] FIG. 1 is a schematic diagram of an exemplary embodiment of a motion control unit for calibrating one or more sensors of a mobility platform. [Figure 10]

[0024] FIG. 10 is a schematic top view of another exemplary embodiment of a mobility platform chassis. [Figure 11]

[0025] FIG. 1 is a schematic diagram of an exemplary embodiment of a printing system. DETAILED DESCRIPTION OF THE INVENTION

[0013] Detailed Description

[0026] Construction productivity, measured in units of work done per labor hour, has been steadily declining in the United States over the past 50 years. Low productivity, combined with a shortage of craft labor and higher labor costs, is a persistent challenge in the construction industry. While several prior efforts have been made to automate or semi-automate tasks on construction sites, these prior systems require periodic human supervision, are prone to navigation errors, and have limited mobility in tight spaces, all of which limit the ability of such prior systems to perform useful tasks on construction sites. Furthermore, many prior systems require the placement of active devices or beacons to assist with navigation on the work site, complicating the rapid and large-scale use of automated platforms.

[0014]

[0027] In view of the above, the inventors have recognized techniques for the design and operation of a mobility platform that can support a variety of tools and accurately and repeatedly navigate a workspace to enable automated tasks to be performed by the tools. Using one or more techniques described herein, a system that uses a mobility platform to autonomously position tools at a construction job site may increase construction productivity by overcoming one or more shortcomings of previous efforts to automate construction tasks or operations.

[0015]

[0028] According to one aspect, a mobility platform may utilize multiple sensors, which may be used to calculate a comparable position of the mobility platform within a workspace. Some of the multiple sensors may be used in determining a local position, and others may be used in determining a separate global position, resulting in separate, comparable local and global positions being calculated independently. The platform positions determined from the outputs of the independently operating sensors may be compared to calculate a positioning accuracy value, which may be used to modify one or more parameters of the platform as it moves along a path. For example, the platform may include drive system encoders, cameras, stereo cameras, optical flow sensors, inertial measurement units, and LiDAR units, any of which may incorporate information to enhance the accuracy and precision of the mobility platform's automated navigation of a construction site. These sensors may detect existing features (e.g., structures) within the workspace, allowing the mobility platform to be deployed without the need for additional deployment of specific beacons or separate navigation equipment.

[0016]

[0029] According to another aspect, a mobility platform may include a holonomic drive system for the platform to navigate a construction site. The holonomic drive system may enable the mobility platform to move in three degrees of freedom so that a tool attached to the mobility platform can reach the ends of a workspace and perform one or more tasks. In some embodiments, the holonomic drive may enable the mobility platform to move in all directions in three degrees of freedom. In one embodiment, the holonomic drive system includes four independently actuable and independently swivelable wheels, enabling the mobility platform to translate in a plane, rotate about a central axis, or a combination of the two (i.e., three degrees of freedom).

[0017]

[0030] According to yet another aspect, the construction assistance system may include one or more processors capable of generating task commands for controlling a mobility platform. These processors may be programmed to execute a design file processing tool that generates relevant navigation information from standard design files (e.g., .csv, .dwg, .dxf, .dwg, .rvt, .nwd, .ifc, etc.) used in the construction industry. The design files may be processed for existing or anticipated features within the job site, such as survey control points, survey control lines, structural elements, or other structural features, that may be identified as one or more landmarks for use during navigation of the mobility platform. For example, in some cases, such features may relate to structural elements of a building (e.g., load-bearing walls, columns, stairwells, elevator shafts, etc.). The design file processing tool may be implemented on the mobility platform, on a remote server in communication with the mobility platform, or both. In some embodiments, the server may be accessible over the Internet or other network to users, who may upload design files and provide other input regarding tasks that are performed autonomously.

[0018]

[0031] Some task commands may control platform movement. Other task commands may control tool operation. A design file may also be scanned for one or more task locations to execute one or more task commands. Alternatively, locations for task command execution may be received as user input. The outermost task locations may be used as constraints to determine the operational envelope of the mobility platform. A processing tool may generate a path based on one or more landmarks and one or more task locations. The path may be calculated taking into account the locations of the landmarks. Path segments with a large number of landmarks within range of sensors on the mobility platform may be selected in preference to path segments with fewer landmarks within range. Therefore, a system according to exemplary embodiments described herein may allow an architect, construction supervisor, or craftsman to upload a file to a design file processing tool and have the system automatically determine an optimal path to maximize the accuracy and precision of mobility platform navigation.

[0019]

[0032] According to yet another aspect, a mobility platform for navigating a construction site may determine position accuracy and calibrate a controller based on known landmarks, such as control points or control lines. In some embodiments, the mobility platform interprets data from at least one sensor (e.g., an inertial measurement unit, optical flow sensor or mono camera, drive system encoder, stereo camera, odometer, etc.) at high frequency and data from a second at least one sensor (e.g., LiDAR, image recognition, etc.) at low frequency. The high-frequency translated data may be combined to determine an estimated first local position by determining a change in the mobility platform's position relative to its last known position. The low-frequency translated data may be used to determine a second global position of the robotic system relative to known landmarks by identifying the landmarks and establishing the mobility platform's absolute position with respect to a coordinate frame. However, data from the example sensors described herein may be interpreted at any suitable frequency to determine separate local and global positions, as the disclosure is not so limited. In some embodiments, the local position estimate may enable continuous location mapping, while the global positioning estimate may enable discrete location mapping. For example, the mobility platform may utilize odometry from wheels for continuous location mapping for use in the local position estimate, while sensors detecting one or more landmarks may be used for discrete location mapping for use in the global position estimate. The first and second positions may be compared to determine a positioning accuracy value. When the positioning accuracy value falls below a predetermined threshold, the mobility platform may navigate to a known location within the work site (e.g., a landmark such as a control point or control line) and recalibrate the controller to compensate for errors in the high-frequency and / or low-frequency sensors.Such an arrangement allows tuning of the robotic system navigation controller during operation and compensating for errors in the sensors or modifying one or more parameters of the controller (e.g., one of the proportional, integral, or derivative constants when a PID or PIV controller is used). Of course, while the exemplary embodiments herein describe mobility platforms having one or more sensors selected from the group of an inertial measurement unit, an optical flow sensor, a mono camera, a stereo camera, an odometer, and a LiDAR, any suitable sensor may be used to provide position, velocity, and / or acceleration information to the mobility platform, as the disclosure is not so limited.

[0020]

[0033] According to yet another aspect, the automated robotic system may be integrated with human-operated workstations or work stations within the construction site to refine route optimization, calibrate the navigation controller, and potentially enable manual control. A ground station or ground station may communicate with the mobility platform controller and / or a remote server. Once a route is generated by the remote server and / or controller, it may be sent to a graphical user interface at the ground station for review by a human operator. The operator may reject the route and have it recalculated by the controller and / or remote server, manually modify the route, or accept the route. Upon initial or final acceptance of the route, the mobility platform may navigate autonomously along the route. Such an arrangement allows a human operator to check the mobility platform's route before making a move or completing any task, and fine-tune the route for variable conditions at the construction site.

[0021]

[0034] The mobility platform of the example embodiments described herein may be capable of performing various tasks and services by transporting, positioning, and operating automated tools rather than by a human operator. Tasks that may be performed include transforming a digital design into a real-world layout (i.e., accurately marking the location of specific architectural / engineering features on a site), material handling (transporting materials and equipment to the appropriate locations), performing installation or portions of an installation job (e.g., marking mounting points, drilling holes, installing hangers, fabricating materials, preparing equipment, etc.), and / or installing various building systems (e.g., wall systems, mechanical systems, electrical systems, plumbing systems, sprinkler systems, telephone / data systems, etc.). The mobility platform may be fitted with one or more tools, including, but not limited to: marking systems (e.g., printers, brushes, markers, etc.), material handling and manipulation systems (arms, grapples, grippers, etc.), rotary tools (e.g., drills, impact wrenches, saws, grinders, etc.), reciprocating tools (e.g., saws, files, etc.), orbital tools (e.g., sanders, cutters, etc.), impact tools (e.g., hammers, chipping tools, nailers, etc.), and other power tools, including the equipment required to support them (e.g., compressors, pumps, solenoids, actuators, presses, etc.).

[0022]

[0035] Turning to the drawings, certain non-limiting embodiments are described in further detail. It should be understood that the various systems, components, features, and methods described with respect to these embodiments can be used either individually and / or in any desired combination, as the disclosure is not limited only to the specific embodiments described herein.

[0023]

[0036] FIG. 1 is a schematic diagram of one embodiment of a construction assistance system including a mobility platform 10 for navigation on a construction site. As shown in FIG. 1, the system may include one or more computer processors that interpret various types of data. The computer processors may be programmed to perform functions such as retrieving or extracting information about the construction site from design files, receiving inputs that define construction tasks to be performed, determining or executing routes for the mobility platform to travel to perform the tasks, and generating commands for the mobility platform to perform those tasks. The processors may be co-located or distributed across multiple locations. In some embodiments, some processors may be on the mobility platform 10, and other processors may be in one or more remote devices that may be connected to the Internet or other communications network.

[0024]

[0037] As shown in Figure 1, the mobility platform may navigate and operate autonomously or semi-autonomously and may communicate with one or more remote or local devices. In the embodiment of Figure 1, the mobility platform includes various controllers and sensors mounted on a chassis 12 to enable high-precision navigation on a construction site without relying on navigation beacons or equipment located on the construction site. In particular, the robotic system of Figure 1 includes a task server 30 having a motion control unit 32 and a tool control unit 34.

[0025]

[0038] The motion control unit is configured to control a drive system including at least a first wheel 20A driven by a first actuator 22A and a second wheel 20B driven by a second actuator 22B. In some embodiments, the drive system is a holonomic drive system, which in the described embodiment enables the mobility platform to move in three degrees of freedom in all directions, as further described with reference to Figures 2-3.

[0026]

[0039] The tool control unit is configured to control the actuation and / or operation of one or more tools mounted on the mobility platform. The tool control unit may issue one or more commands to the associated tool to perform one or more tasks. In the embodiment shown in FIG. 1 , the mobility platform includes a marking device 50 mounted on a carriage 52 that allows the marker to reach the tip of the mobility platform chassis 12. The tool control unit is configured to control the movement of the marking device on the carriage and deposit ink, powder, or other effective marking material to lay out the construction site according to the design file. In response to commands from the tool control unit, the marking device may mark materials within the construction site, such as walls and floors, pillars, ceilings, etc. In response to commands from the tool control unit, the carriage may position the marking device within the appropriate task location. Other commands from the tool control unit may control marking parameters such as line thickness, color, material, etc.

[0027]

[0040] As shown in FIG. 1 , the mobility platform 10 includes multiple sensors configured to acquire and / or output information about the mobility platform's surroundings so that the mobility platform can navigate autonomously without additional beacons or navigational equipment at the work site. According to the illustrated embodiment, the mobility platform includes one or more stereo cameras 40, optical flow sensors 42, inertial measurement units 44, and LiDAR units 46. As described further below, the information acquired and / or output by each of the sensors may be fused by the task server 30 as the mobility platform navigates the construction site. In some embodiments, information from at least two sensors may be averaged, conditionally selected, compared to find errors, and / or any other suitable operation may be performed. For example, an independent local position may be determined by inertial integration or optical flow, while a global position is generated by landmark recognition by the LiDAR units 46 or camera stereo pairs. Comparison of these independently generated positions may allow the mobility platform to self-test its position accuracy and recalibrate one or more parameters of the task server as it navigates the construction site. Of course, any suitable number or type of sensors may be used, and their data may be fused, combined, or compared to improve the precision and / or accuracy of autonomous navigation on the construction site, as the disclosure is not so limited.

[0028]

[0041] In the embodiment shown in Figure 1, mobility platform 10 also includes additional devices that cooperate with task server 30 to enable the mobility platform to autonomously navigate and perform tasks within a construction site. For example, the mobility platform includes a storage device 36, such as a hard drive, solid state drive, or other memory for storing instructions and other data, as well as a wireless communication device 38 that communicates wirelessly with various local or remote devices via any suitable communication protocol (e.g., satellite, cellular, Wi-Fi, 802.15.4, etc.). Of course, although the mobility platform of Figure 1 communicates wirelessly, any suitable wired communication interface, such as a wired serial port, an Ethernet port, etc., may also be used. The combination of storage and wireless communication allows the mobility platform to send, receive, and store data from one or more external devices, such as a remote server (i.e., cloud server) 100, a remote computer 130, a mobile device 140, or a local workstation 110 (e.g., a portable or handheld device, such as a laptop, tablet, or cell phone, a desktop computer, or any other suitable device within wireless or wired range of the mobility platform and / or within range of a network access point such that the workstation can communicate with or control the mobility platform from a construction site). Such an arrangement may allow files provided from the remote server to be analyzed by the task server to generate paths, tasks, task locations, and other related information that the robotic system can use to autonomously or semi-autonomously perform tasks at a work site.

[0029]

[0042] As described above, the mobility platform 10 of FIG. 1 is configured to communicate with multiple external devices to simplify autonomous navigation and perform one or more tasks. External devices that communicate directly or indirectly with the mobility platform include a remote server 100, a ground station 110, a router 120, a remote computer 130, and a mobile device 140. As part of a cloud computing service, a remote server, which may be located in a data center, is used to manage files used by the robotic system to navigate and perform tasks. That is, the remote server may collaboratively coordinate file management, path generation, path correction, task planning, and any other desired functionality. In some embodiments, path corrections may be collaboratively coordinated on the mobility platform 10. The remote server enables designers, such as contractors, consultants, engineers, and architects, to provide design files and task information that can be used by the mobility platform. As further described with reference to FIG. 4 , the remote server may automatically generate optimal routes for performing tasks at various locations on the construction site by retrieving information from design files such as two-dimensional or three-dimensional drawings or computer-aided design (CAD) files. An engineer, architect, or other remote worker may interface with the remote server from an industry-standard file management platform or by a web interface to which files, which may be on the mobile device 140 or the remote computer 130, respectively, are uploaded. The mobile device graphical user interface 142 or the remote computer graphical user interface 132 may be used to transfer or download files from the remote server and modify the files using a CAD or Building Information Management (BIM) software platform.The file management system utilized by the remote server may include a database for storing drawings, plans, and related data and may also be adapted to provide a user with a revision history of stored files. The remote server may also enable contractors, craftsmen, or other workers available on-site at the construction site to provide feedback on routes, task locations, or control parameters. In particular, the remote server may communicate with a ground station having a ground station graphical user interface 112. The ground station graphical user interface may enable an operator to confirm, modify, or reject navigation and task plans generated on-site by the remote server before the mobility platform begins operating autonomously. In some cases, the ground station may also be used to manually override or manually control the mobility platform. According to the embodiment of FIG. 1, the router 120 may be configured as a modem, satellite, cellular tower, or other suitable interface suitable for cooperatively coordinating data transmission between the remote server, the mobility platform, and / or the ground station.

[0030]

[0043] It should be noted that although a remote server 100 is shown and described with reference to FIG. 1, the disclosure is not so limited and any suitable server or processor may be used, such as a server and processor that is local (e.g., mounted on a mobility platform) or in close proximity to a work site.

[0031]

[0044] FIG. 2 is a schematic top view of one embodiment of a mobility platform 10 including a holonomic drive system that allows the mobility platform to move in three degrees of freedom and reach the extremes of a construction site. The holonomic drive system allows a robotic system to position a tool attached to the mobility platform within an area at the extremes of a construction site, such as at a corner, that would otherwise require multiple movements to reach or be difficult to access. The holonomic drive system of FIG. 2 includes four wheels 20A, 20B, 20C, and 20D, each coupled to a respective actuator 22A, 22B, 22C, and 22D. Each of the four wheels rotates about an independently controlled axis, allowing the wheel to be angled relative to the mobility platform chassis 12. That is, each of the wheels and their respective actuators are coupled to the chassis 12 by axial actuators 26A, 26B, 26C, and 26D, each configured as a servo motor coupled to a respective wheel support 24A, 24B, 24C, and 24D. The axial actuators allow the axis of each wheel to be independently adjusted (i.e., swiveled), allowing the mobility platform to move freely in three degrees of freedom. This arrangement provides complete flexibility for motion in a two-dimensional planar environment and enables the execution of very complex motion patterns to accomplish several tasks. Of course, although independently rotatable wheels are shown in FIG. 2, any suitable holonomic drive system may be utilized, such as for omnidirectional wheels.

[0032]

[0045] According to the embodiment of FIG. 2, the mobility platform 10 provides a robust chassis 12 for mounting various tools or payloads. The chassis 12 is configured as two flat plates secured to one another by axial actuators 26A, 26B, 26C, and 26D and other supports (not shown). The chassis includes multiple rigid attachment points that allow tools or payloads to be modularly attached to the mobility platform. Of course, while the chassis of FIG. 2 is configured as two flat plates, any suitable shape and configuration of chassis may be used, as the disclosure is not so limited.

[0033]

[0046] FIG. 2 is a schematic top view of the mobility platform 10 in a first position, and FIG. 3 is a schematic top view of the mobility platform in a second position, illustrating the degrees of freedom provided by the holonomic drive system. As shown in FIG. 2, the holonomic drive system is shown in a first position, in which the rotational axes of the first wheel 20A and the third wheel 20C are aligned, and the rotational axes of the second wheel 20B and the fourth wheel 20D are aligned and perpendicular to the axes of the first and third wheels. In the configuration shown in FIG. 2, the mobility platform can move in three degrees of freedom by changing the various rotational directions of the wheels about their various axes of rotation, or by changing the rotational axes of the wheels themselves. The holonomic drive system can move the mobility platform along a first axis 28A, a second axis 28B that is perpendicular to the first axis, and can also change the orientation of the mobility platform about a third axis 28C. As shown in FIG. 3 , the wheels are rotated to prompt movement of the mobility platform along second axis 28B. That is, each of the wheel rotation axes has been moved by a respective axis actuator so that first wheel 20A is aligned with additional wheel 20D, second wheel 20B is aligned with third wheel 20C, and the axes are parallel to one another. To reach the state shown in FIG. 3 , the drive system unit of the task server may have generated commands specifying which wheel axis to rotate and the desired magnitude of rotation. Alternatively, the task server may have generated task commands specifying a desired location, and the motion control unit may generate corresponding commands for each of the wheel and axis actuators to control the mobility platform to that location. Of course, the task server, motion control unit, or any other suitable processor or control unit may use any suitable task commands to control the operation of the mobility platform, including combinations of the above task commands, as the disclosure is not so limited.Thus, the mobility platform may simply move along the second axis or rotate about the third axis 28C. Similarly, the wheels may be rotated to facilitate movement of the mobility platform along the first axis 28A shown in FIG. 2. The wheel axes may be adjusted without moving the mobility platform itself from its initial position, allowing the mobility platform to move in any of three degrees of freedom from its initial position. For example, one or more axis actuators may adjust the wheel axes upon command from the motion control unit.

[0034]

[0047] FIG. 4 illustrates a functional block diagram for one embodiment of a system method for operating a mobility platform 10 at a construction site 200. As previously discussed, much of the work performed at a construction site can be defined by architectural and engineering designs. These designs are created by architects, engineers, and contractors and captured in electronic files (i.e., design files). The design files can be stored in industry-standard formats, including CAD drawings, BIM models, and other digital formats (e.g., .dxf, .dwg, .rvt, .nwd, .ifc, etc.), and are organized, managed, and distributed by a local or remote file management system, such as the remote server 100 of FIG. 1. According to the process illustrated in FIG. 4, the remote server and mobility platform of the exemplary embodiment described herein translate the design files into real-world activities or actions and tangible work by the mobility platform. The method of FIG. 4 begins with the conversion (1) of a standardized file, such as a .dwg file, received at the remote server. The file can be received by manual upload or by connecting directly to an existing file management system. Upon receiving the files, the remote server processes the standard format design files and generates relevant information from the design, generating three main files that can be used by the mobility platform: a landmark file (.rgL) containing navigation information for design verification and landmark extraction of relevant features of the environment used for indoor navigation without the need to install sensors or beacons, a drive file (.rgD) that determines the planned path for the mobility platform to be executed in the field during operation, and a tool file (.rgT) that determines the sequence, parameters, and operation definitions of supported tools. The method then includes storing all three proprietary files, as well as copies of the original files, on the remote server so that they are accessible to one or more remote users.

[0035]

[0048] In some embodiments, rather than converting the standard file into three main files, the standard file may be split into two files for use by the mobility platform. One file may be a feature file (.rgF) containing navigation information for relevant features of the environment, used for design validation and indoor navigation without the need to install sensors or beacons. The other file may be a task file (.rgT) that determines both the planned path of the mobility platform to be executed in the field during operation, as well as the sequence, parameters, and operation definitions of the supported tools. These proprietary files may be stored on a remote server so that the files are accessible to one or more remote users.

[0036]

[0049] In a typical method using such a system, deployment of the mobility platform 10 in the field may begin when the mobility platform is positioned at a desired construction site and powered on. An operator then initiates task execution locally (e.g., by switching on the mobility platform), at a ground station, or remotely. Upon receiving a command from the operator to execute a task, the mobility platform may recall from internal or external memory or make a request to a ground station or file management service (2) and download the necessary command file. The mobility platform then performs a landmark extraction procedure (3) and uses the information in the file to identify known features (which are reliably present in both the design and the construction site) and unknown features (which are present on the construction site but not in the design), establish its global coordinate system of reference, and perform initial tuning of its control system. For example, the mobility platform uses one or more sensors to detect one or more features (e.g., control points, control lines, structural elements, floor penetrations, etc.) within the construction site and matches the identified features to those described in the task command file. The landmark extraction procedure involves executing a predetermined motion plan in a landmark data system file (.rgL) or, in some embodiments, a feature data system file (.rgF). During execution of the landmark data system file, sensors on the mobility platform collect data and characterize landmarks from the actual floor plan conditions. At the end of the landmark extraction procedure, the mobility platform collects and characterizes the floor plan and performs sensor calibration by comparing its own odometry readings with the control points or other identified known features or landmarks.The extracted landmark file is modified based on feedback from the mobility platform sensors into a final extracted landmark file (.rgLX) and submitted to a human operator for approval, modification, or rejection.

[0037]

[0050] Once the feature identification process is complete, the extracted landmark file is submitted to a remote server, which adjusts the drive and tool files based on the as-is conditions of the site as perceived by the mobility platform 10's sensors (4). The planning adjustment procedure compares the landmark positions and signatures from the extracted landmark file (.rgLX) with the original landmark data file (.rgL). The comparison occurs in two steps: matching and correction. In the matching step, the remote server recognizes the coordinate system adopted by the extracted landmark file based on the mobility platform's initial position on the floor plan, which may or may not be different from the one used in the drive system landmark data file. If the coordinate systems are different, the remote server matches the data generated by the mobility platform with the original landmark file. Then, in the correction step, the remote server evaluates a transformation function that maps the landmark data system to the generated landmark data. Additionally, the remote server identifies any discrepancies in the files, such as the orientation, position, and location of the landmarks, and defines adjustment parameters to be used in other planning.

[0038]

[0051] The remote server creates an adjusted drive file (.rgDX) and an adjusted tool file (.rgTX) and transmits them to one or more operators for review and approval. The control station (e.g., remote computer, ground station, etc.) prompts one or more operators to review and verify the adjusted path (5). The verification step may be deemed accurate, inconsistent, or incorrect by the operator, and the transformation functions and relative errors found in the drive and tool file adjustment step may be displayed for the operator in a graphic interface at the control station. If the operator deems the adjusted files accurate, the modified drive and tool files are accepted by the operator, and the instructions in the files are executed by the mobility platform, resulting in the desired work product. If the operator deems the adjusted files inconsistent, the operator identifies areas in the plan adjustment that are flawed, such as when the landmarks identified in the plan are correct but the adjusted base drive and / or tool plans are dissimilar. In this case, the operator can fine-tune the adjustment parameters using the control station's graphical user interface, including recalculating the transformation functions and adjustment parameters. If the operator deems the adjusted file incorrect, the operator identifies where the landmark extraction produced incorrect landmarks (which may be due to a faulty or interrupted landmark extraction process). If the adjusted file is incorrect, the operator commands the mobility platform to run the landmark extraction procedure again (3) to readjust the extracted landmark file. If the adjustments are deemed accurate and confirmed, the adjustment parameters and transformation functions are applied to the drive system and tool system data files to generate corrected drive and tool system data files (.rgDX and .rgTX, respectively) that contain suitable information for generating navigation paths for the mobility platform.For example, the corrected drive and tool system data file may include corrected locations of one or more tasks to be performed by a tool, corrected locations of one or more landmarks, or other suitable information. For example, in some embodiments, the corrected drive and tool system data file may include trajectory segments for the mobility platform characterized by one or more of an initial position and velocity, a final position and velocity, and a tool action and task location. The corrected drive and tool system data file may also include information regarding how the mobility platform should be controlled based on the task being performed, or the lack thereof. That is, the corrected drive and tool system data file may include an indicator or other information that the mobility platform should be tightly controlled on a segment (e.g., while marking a floor plan) or that the mobility platform can be loosely controlled (e.g., while traveling between points without performing any tasks). The remote server then transmits the corrected drive and tool files to the mobility platform's task server for execution.

[0039]

[0052] In some embodiments, the mobility platform may not perform the landmark extraction process. Rather, the mobility platform may receive the drive file (.rgD) and tool file (.rgT) and attempt to execute them, making small navigation adjustments on the fly. That is, the mobility platform may use data from one or more sensors to avoid unexpected smaller obstacles without significantly deviating from the path as defined in the overall drive and tool files provided to the mobility platform. In some embodiments, the mobility platform may request the ground station to plan a route deviation to navigate around larger obstacles that may be detected by the mobility platform. In some embodiments, the mobility platform may notify the user that the detected landmarks do not match the expected landmarks, so that the landmark file, drive file, and tool file may be re-planned at the remote server or ground station level.

[0040]

[0053] As shown in FIG. 4, once the final, corrected drive and tool plans are accepted by the operator, the remote server transmits the corrected files to a task server located on the mobility platform (6). The task server is the primary embedded software application responsible for executing the verified task. The task server orchestrates the actions of the motion control unit and the tool control unit. The corrected drive file (.rgDX) is transmitted to the motion control unit (7) for interpretation, while the corrected tool file (.rgTX) is transmitted to the tool control unit (8) for interpretation. The motion control unit (7) contains software applications responsible for the mobility platform's movement, including sensor drivers and data processing units, power management and motion control units, filters, etc. Furthermore, the motion control unit is responsible for managing path execution and obstacle avoidance. Therefore, the motion control unit interprets the corrected drive file received from the remote server and executes the set of instructions therein to move the mobility platform along a path and perform a set of tasks. The tool control unit includes a set of software components used in the operation of a tool carried by the mobility platform for the task the tool is configured to perform. The tool control system controls fine positioning and use of attachments through control of actuators, motors, solenoids, compressors, pumps, presses, and / or any other suitable actuators.

[0041]

[0054] The functional block diagram of FIG. 4 illustrates a system for performing several different functions, as described above. In some embodiments, some of the functions described above may be performed by different components, while other functions may be performed by the same components. In some cases, the described calculations and functions may be handled by different components of the exemplary system. In some embodiments, computationally expensive or non-time-critical processes may be performed by a remote server. For example, generation of landmark files, drive files, and tool files may be performed by a remote server. Correspondingly, computationally inexpensive or time-critical calculations may be handled onboard the mobility platform. For example, minor course adjustments based on sensor data and low-level control of moving components (e.g., motors, wheels, etc.) may be performed onboard the mobility platform. Intermediate-level adjustments that may be less time-sensitive or slightly too computationally expensive for the mobility platform may be performed by a control station (e.g., a ground station). For example, larger course adjustments to drive files and tool files may be performed by the control station. Of course, any suitable number and type of appropriate components may be utilized to perform the functions illustrated in FIG. 4, as the disclosure is not so limited.

[0042]

[0055] 4 illustrates specific communications between various components. However, any of the mobility platform, remote server, ground station, remote computer, or mobile device may communicate directly or indirectly with one another for either automatic adjustment of the drive and tool files or manual operation to adjust the drive and tool files to address discrepancies that may exist between the work site and the various files. Thus, the present disclosure is not limited in this respect.

[0043]

[0056] FIG. 5A shows a graphical representation of data that may be included in a construction site design file 200. The design and construction industry generally uses standard file formats (including, but not limited to, .rvt, .dwg, .pdf, etc.) to exchange information. These files may contain two-dimensional or three-dimensional coordinate data of one or more features of an existing or planned construction site. As shown in FIG. 5A, the file is a two-dimensional model of a high-rise building floor plan. According to exemplary embodiments described herein, a remote server exposes one or more APIs that receive the file, extract relevant information, and convert the extracted data into three possible categories according to their relevance to subprocesses: landmark data (.rgL), drive system data (.rgD), and tooling system data (.rgT). To extract relevant landmark data from the design file, the remote server or another processor may use machine learning or another suitable technique to recognize shapes within the design file and tag the shapes as potential landmark candidates. Drive and tooling system data may be generated based on the design file and task-specific information. For example, in the case of a marking task, the remote server or processor may identify the location of a wall as a feature to be marked, and that information is incorporated into the drive and tooling system data. Landmark data is used to enable, assist, and / or enhance navigation of the mobility platform without the need for a global positioning or beacon system. Information relevant to landmark extraction consists of data sets that can unambiguously characterize the environment. For example, landmarks that may be identified include, but are not limited to, control points or lines used by contractors, structural elements such as columns, elevator shafts, or load-bearing balls, and / or any other architectural or engineering features available during operation. In some embodiments, the remote server may use pattern, shape, or character recognition to identify elements in a design file that may be suitable landmarks to use to navigate the mobility platform.In addition to the landmark data, the remote server may also extract boundary or constraint information that defines an outer operating envelope for the mobility platform when navigating the construction site. These constraints may be based on a combination of identified landmarks and the location of one or more tasks to be performed by the mobility platform.

[0044]

[0057] FIG. 5B illustrates one embodiment of landmarks extracted from the design file 200 of FIG. 5A. According to the embodiment of FIG. 5B, the landmarks include structural elements or other features of the worksite that may be in place prior to use of the mobility platform. In particular, the remote server identifies stairwell 202, columns 204, and load-bearing walls 206. These elements in the floor plan model serve as reference points for the mobility platform's motion control unit. The landmark extraction process may also identify differences between the as-is worksite and the design file, so that one or more tasks and task locations may be identified based on the identified landmarks. As further described with reference to FIGS. 6A-6B, the identified landmarks may at least partially determine a path for the mobility platform, which maximizes the number of landmarks within range of sensors deployed on the mobility platform to enhance the accuracy of the motion control unit. In some embodiments, the remote server may also extract one or more control points located on the floor that can be used by the mobility platform for calibration points for the motion control unit to execute paths. In many cases, control points have traditionally been utilized in the construction industry for positioning and reference purposes during the construction process, i.e., no additional placement of beacons or markers on the construction site may be necessary to successfully navigate the construction site. Therefore, the landmark system data extracted from the design file includes the locations and signatures of all relevant landmarks used to identify features in the environment, the locations and identities of all control points used during navigation, and the paths executed by the mobility platform at the site to perform the identification procedures.

[0045]

[0058] FIG. 5C illustrates one embodiment of constraints 208 for at least one tool extracted from design file 200. Motion constraints for the platform are determined by the actions required by each tool operation. These constraints are extracted from the floor plan by a remote server and provide the exact locations to which the mobility platform needs to deliver its tool. According to the constraints illustrated in FIG. 5C , the mobility platform can be configured with a marking device and configured to map the layout of the construction site. The constraints indicate the correct position of the tool tip at some points in space that are considered when planning the mobility platform's path to optimize navigation accuracy and task completion efficiency. In some cases, constraints or tasks can be embedded directly into the design file, or, as the disclosure is not so limited, in some embodiments, the remote server generates proposed tasks and tool constraints based on a particular 3D or 2D design file.

[0046]

[0059] Traditional autonomous navigation systems, typically small and uncontrolled by humans, rely on a limited amount of information, resulting in drift and navigation inaccuracies over time. In contrast, the mobility platform of the exemplary embodiment herein utilizes at least two sensors to generate a path to complete a task and increases the available data for accurately matching that path with reduced drift by selecting path segments within the sensor range of verified landmarks. A schematic diagram of the method used to optimize the path is shown in FIG. 6A, which shows the extracted landmarks 202, 204, and 206 of FIG. 5B along with the assigned landmark influence range 205. As described above, a remote server can generate a drive system data file (.rgD) from a design file containing information extracted from drawings related to the trajectory to be executed by the mobility platform during operation. This trajectory is determined by a motion planning algorithm that performs path optimization while satisfying tool-specific position constraints, and can be executed on the remote server or on the mobility platform. The optimization algorithm optimizes the accuracy of the navigation system by increasing the information available for one or more sensors onboard the mobility platform. In particular, the route optimization plots a route through the landmark sphere of influence. That is, based on the landmarks identified in the landmark extraction step and onboard sensor parameters and / or coverage of the mobility platform's onboard sensors, the remote server and / or the mobility platform identifies the shape, size, and location of the landmark sphere of influence. The path planning algorithm calculates the mobility platform's trajectory within the construction site while ensuring that location constraints on the floor plan are met, while optimizing the availability of landmarks from which its navigation system can orient itself. The path planning algorithm may be executed on the remote server based on information from the design file and landmark extraction. Alternatively or additionally, the path planning algorithm may also be executed on the mobility platform, as the disclosure is not so limited.

[0047]

[0060] In some embodiments, a route may be generated based on predicted positioning accuracy resulting from navigation based on a comparison of perceived continuous displacements with perceived locations based on the locations of one or more landmarks. Route generation may involve generating multiple candidate routes and assigning a cost to each so that a lower-cost route may be selected. A cost function may be used to assign costs to the candidate routes generated for the mobility platform. Candidate routes with potentially large position errors may be assigned a higher cost by the cost function.

[0048]

[0061] The cost function may consider variables that contribute to a path segment's positioning accuracy, such as the path segment's distance from one or more landmarks, sensor range, and sensor accuracy for calculating the mobility platform's position error. A path segment that relies solely on continuous local positioning (e.g., from a drive encoder, optical flow sensor, etc.) may be assigned a higher cost than a segment of comparable length where the global position can be determined based on one or more recognizable landmarks (e.g., by LiDAR, stereo camera, mono camera, etc.). Alternatively or additionally, other parameters may be used in the cost function, such as a parameter representing the efficiency of completing the path, which may be based on the time it takes the mobility platform to travel the path. In some embodiments, the cost function assigns values ​​based on predicted measurement errors provided by one or more sensors onboard the mobility platform throughout the execution of the path, as well as the relative benefits of the mobility platform's efficiency and speed in completing the path. Costs may be assigned to various segments of a candidate path, and the total cost of the candidate path may be summed based on this cost function. Therefore, potential routes are iteratively generated, the route with the lowest overall cost is selected, and finally offered to the mobility platform for execution.

[0049]

[0062] In some embodiments, path segments within sensor range of landmarks may be relatively less expensive per unit length compared to path segments outside of sensor range of landmarks, allowing navigation to be based on local positioning (e.g., from drive encoders, optical flow sensors, etc.). Therefore, generating a path that provides high predicted positioning accuracy may include iteratively adjusting the path to increase the amount of the path located within a threshold distance of one or more landmarks. An exemplary path generated by such a path generation process is shown in FIG. 6B. In some embodiments, optimizing the generated path may include providing a path that utilizes continuous displacement to navigate to a task location away from one or more landmarks and then back toward the one or more landmarks once the task is completed.

[0050]

[0063] To increase the availability of landmark information, once the path is optimized, the mobility platform 10 navigates the optimal path 212, as shown in FIG. 6B . As described above, the mobility platform may read and execute drive system data files (.rgD, .rgDX) containing an optimal path that the motion control unit may use to move the mobility platform. The mobility platform may also read and execute tool system data files (.rgT, .rgTX) containing a list of actions and tool-specific configurations associated with each location throughout the floor plan. The tool system data files may be used by the tool control unit to operate one or more tools to perform tasks, such as marking, drilling, transporting, hammering, etc., within assigned locations along the optimal path.

[0051]

[0064] FIG. 7 shows a schematic, expanded view of one embodiment of a mobility platform's task server 30, including a motion control unit (MCU) 32 and a tool control unit (TCU) 34. According to the embodiment of FIG. 7, the task server unit is a software application executed by a controller, processor, microprocessor, or other suitable device that cooperatively coordinates the actions of the MCU and TCU. As described above, corrected drive system files are received by the task server from a remote server. These corrected drive system files are delivered in the form of trajectory commands to the MCU's path management unit (PMU) 33, the function of which is further described with reference to FIG. 8. Concurrently, corrected tool system files are received by the task server from the remote server and delivered in the form of task commands to the TCU's tool management unit (TMU) 35. The TMU maintains information about both the tasks performed by various tools and task locations and provides that information to the TCU. The TCU executes the commands to execute the tasks along the optimal path. The TCU listens to the combined odometry readings of the sensor fusion unit and / or time step information from the MCU. At each time step of the task server control algorithm, the mobility platform's current position is compared to a list of location-based commands. If a tool action is defined at this location, a tool control command is sent from the task server to the tool control unit to execute the command. Therefore, the task server collaboratively coordinates the execution of the MCU's and TCU's commands in time and space so that the mobility platform can reliably navigate and perform tasks on the construction site.

[0052]

[0065] FIG. 8 shows one embodiment of a block diagram of the motion control unit (MCU) 32, which moves the mobility platform along a path. As previously described, the motion control unit is an embedded software network responsible for controlling the position and orientation (i.e., pose) of the robotic system within the construction site and utilizes a wide range of sensors to recognize its location within the work site. The MCU's operation is triggered by commands (i.e., received from a task server or associated controller) to execute a given trajectory. These trajectory commands are received by the path management unit (PMU) 33 and consist of a series of spatial locations that need to be reached in sequence. For every location within the trajectory command, the PMU creates a segment command message and sends it to the segment motion control unit (SMOCU) 52. The SMOCU is fed with odometry coupling information and calculates velocity commands, which are sent to the drive system 58 to move the mobility platform. In some embodiments, the SMOCU implements a controller selected from the group consisting of a proportional (P), proportional-integral (PI), proportional-derivative (PD), proportional-integral-derivative (PID), or proportional-integral-velocity (PIV) controller characterized by constants (Kpp, Kpi, Kvp, Kvi). Of course, the SMOCU implements any suitable controller, such as a fuzzy logic controller, as the disclosure is not so limited. According to the embodiment of FIG. 9, the combined odometry information used by the SMOCU is generated from sensor fusion of odometry signals from the stereo camera 46, the inertial measurement unit 44, the optical flow sensor 42 (i.e., a mono camera), and the LiDAR unit 40, all of which are fed through an extended Kalman filter (EKF) 50 and communicated to the SMOCU. While the EKF is shown in FIG. 8, any suitable filter, such as a particle filter, may be utilized as the disclosure is not so limited.Of course, any suitable sensor may be utilized to provide information to the MCU (e.g., wheel rotation encoders, image recognition, etc.), and the information may be fused with information from other sensors using any suitable sensor fusion technique.

[0053]

[0066] As shown in FIG. 8 , the LiDAR unit 40 also provides input to a Motion Quality Evaluation Unit (MOQEVU) 54, which continually evaluates the accuracy of the executed motion segments and provides motion quality data output. To do so, the MOQEVU compares high-frequency odometry combined signal integrals from the EKF 50 with a low-frequency landmark comparison routine 41. That is, positions determined from high-frequency sensors, such as the optical flow sensor 42, the inertial measurement unit 44, and the stereo camera 46, are compared with positions determined independently from landmark recognition by the LiDAR unit or an additional stereo camera. In this way, the higher the correlation between the odometry integrals and the landmark comparison, the higher the quality of the motion system. This motion quality evaluation value is sent to a path learning unit (PLU) 56. Each received tuple (motion segment and motion quality data) is associated with a set of controller parameters in the PLU. Adjusting the parameters in the PLU modifies the SMOCU's output to the drive system. In this way, the learning unit can dynamically find correlations between segment types and refine its set of motion controller parameters for each segment performed. In some embodiments, the information in each tuple can be used to modify the weightings used in the sensor fusion process in the EKF or other filter. That is, one or more signal quality thresholds in the EKF can be used to determine the exclusion of any given data point in a signal, and modifying these thresholds can emphasize selected sensors along a given segment of the path. For example, a LiDAR sensor that is very accurate while the mobility platform is traveling in a straight line may be less accurate when the mobility platform is turning. Following this example, the LiDAR signal quality thresholds may be higher in straight lines and lower along turns.

[0054]

[0067] FIG. 9 illustrates one embodiment of the motion control unit 32 that calibrates one or more sensors of the mobility platform. That is, in addition to landmark optimization, the path planning algorithm also includes a planned or unplanned calibration step, in which the mobility platform may use existing control points on the floor plan to verify its position estimation accuracy and, if necessary, recalibrate one or more parameters of the motion control unit. In the embodiment of FIG. 9, the calibration process algorithm is not only executed at a predetermined step in the drive system data file, but may also be triggered by the motion quality evaluation unit (MOQEVU) 54. A calibration control unit (CACU) 60 obtains information on path quality messages sent by the MOQEVU. Whenever the system's positioning accuracy value falls below a predetermined threshold value, the CACU may send a calibration request to the path management unit 33, which initiates the execution of a calibration segment. In some embodiments, the calibration segment is a segment that directs the mobility platform to the nearest control point. When the mobility platform reaches a target control point, a downward-facing camera, e.g., mono camera 42, verifies the mobility platform's position relative to the control point, and the CACU compares the mobility platform's position generated by sensor odometry with the actual stored position of the target control point. In this way, the CACU generates a set of adjusted control parameters that can be sent to segment motion control unit (SMOCU) 52 to improve navigation accuracy when the SMOCU commands drive system 58 to move the mobility platform.

[0055]

[0068] FIG. 10 is a schematic top view of another embodiment of a mobility platform 1010 including a holonomic drive system that allows the mobility platform to move in either direction in a two-dimensional plane to reach the extremes of a construction site. The holonomic drive system allows a robotic system to position a tool attached to the mobility platform at the extremes of a construction site, i.e., in areas flush with corners, that may otherwise require multiple movements due to being difficult to reach or inaccessible. The holonomic drive system of FIG. 10 includes four wheels 300A, 300B, 300C, and 300D, each coupled to a respective actuator 304A, 304B, 304C, and 304D. Each of the four wheels rotates on an independently rotatable wheel support 302A, 302B, 302C, and 302D. Each of the wheels and their respective actuators is coupled to the chassis 1012 via the rotatable wheel supports 302A, 302B, 302C, and 302D. Each of the wheel supports is coupled to an axial actuator (see, e.g., FIGS. 2-3 ) configured to independently adjust (i.e., swivel) each axis of the wheel 360 degrees, allowing the mobility platform to move freely with three degrees of freedom. This arrangement provides complete motion flexibility in a two-dimensional planar environment and allows for highly complex motion patterns to be performed to accomplish several tasks. Furthermore, the arrangement of FIG. 10 ensures that drive encoders can be utilized to assist in determining the mobility platform's position. Because the wheels swivel independently, slippage between the wheels and the underlying surface can be avoided, thereby increasing the accuracy of the drive encoders for calculating position.

[0056]

[0069] 11 is a schematic diagram of an exemplary embodiment of a printing system 400. As mentioned above, the mobility platform of the exemplary embodiments described herein may have one or more tool kits or tools for performing one or more tasks at a work site. The printing system 400 is a tool kit-like system configured to mark architectural and engineering information, including text, directly onto the unfinished floor surface of a construction site. The printing system may be configured with the equipment and components necessary to attach to the mobility platform (e.g., see FIG. 1) and accurately and precisely mark the floor surface with a durable material that remains visible throughout the construction process.

[0057]

[0070] As shown in FIG. 11 , the printing system 400 includes at least one reservoir 402, at least one air compressor or pump 404, an electronic control system (ECS) 406, and at least one printhead 410, all appropriately interconnected with tubing, hoses, pipes, valves, connectors, wiring, switches, etc., as illustrated in FIG. 11 by a manifold 408. The reservoir(s) 402 hold a sufficient amount of marking fluid for the printing toolkit to operate for a desired working period. The reservoir(s) may be connected to the rest of the printing system, both upstream and downstream, to deliver the marking fluid to the next component needed to control and produce the desired mark. In some embodiments, the reservoir(s) hold marking fluid, such as pigmented ink, from a bulk container of marking fluid in a tank that can be open to the atmosphere and manually filled, but can be pressurized when the reservoir is closed, if desired. In some embodiments, the top of one or more reservoirs is connected to an air compressor or air pump 404 using tubes, hoses, or pipes, allowing the air compressor or air pump to pressurize the headspace above the marking fluid at the top of the reservoir, thus firmly pressurizing the marking fluid, and supplying the marking fluid through ink supply tubes, hoses, or pipes that connect the bottom of the reservoir to one or more of the printheads 410. In some embodiments, the reservoirs may be left open to the atmosphere, with the bottom tubes, hoses, or pipes connected to a pump that can draw fluid from the reservoir and supply it downstream through the ink supply tubes, hoses, or pipes to the printheads.

[0058]

[0071] According to the embodiment of FIG. 11 , each of the printheads 410 is configured to deposit marking fluid onto a printing surface. In some embodiments, the printhead may be formed with ink supply tube connections to a reservoir 402 or pump 404, a manifold that distributes the marking fluid to key components within the printhead, and at least one piezoelectric pump that, when activated, ejects the marking fluid in small increments in the form of droplets. The piezoelectric pump may utilize one or more disks that are naturally flat but, upon activation, deform into one of two positions: a pulling position or a pushing position. In the pulling position, positive pressure of the fluid in the ink supply tubes and manifold urges the marking fluid into the piezoelectric chambers. In the pushing position, droplets are expelled from the piezoelectric chambers and deposited on the floor. In some embodiments, an array of piezoelectric pumps is used, allowing for the simultaneous deposition of droplets in columns, rows, matrices, diagonals, or any combination thereof. Such an array enables the marking of complex shapes and patterns, including text.

[0059]

[0072] In some embodiments, the mobility platform includes a printing system. The printing system may include at least one reservoir, at least one pump, and multiple printheads positioned in an array. The at least one pump and multiple printheads may be configured so that each printhead can dispense small amounts of marking fluid in droplet form. The printing system may also include an electronic control system having a processor configured to execute computer-readable instructions stored in memory. The electronic control system may be configured to command the multiple printheads and the at least one pump to deposit droplets of marking fluid in columns, rows, matrices, diagonals, or any combination thereof. The electronic control system may also communicate with a control unit of the mobility platform to receive position and velocity information and cooperatively coordinate the deposition of the marking fluid. In some embodiments, the mobility platform and printing system enable marking of text or other complex shapes or patterns. In some embodiments, the marking fluid is deposited while the mobility platform is moving.

[0060]

[0073] According to the embodiment of FIG. 11 , the printing capabilities provided by the printing system 400, specifically the ability to print text, enable the printing system 400 to deliver unique digitally replicated information to the unfinished floor of a construction site. When deployed on the mobility platform of the exemplary embodiments described herein, the printing system 400 can mark the intended locations of various building systems, components, and equipment, allowing contractors to precisely install each material. While installation locations are currently marked by hand with dots and lines, the printing system's complex marking capabilities, including text, can be used to distinguish trades, communicate non-intuitive installation instructions (i.e., indicating material size, identifying specific parts or equipment, detailing configuration or orientation, and specifying installation height above the floor), and identify pre-fabricated part numbers. The ability to communicate pre-fabricated part numbers may be desirable as pre-fabricated construction techniques become more prevalent. Therefore, the printing system of FIG. 11 , in cooperation with the mobility platform of the exemplary embodiments described herein, provides the ability to communicate the exact installation location, the exact part number, and the exact installation orientation and configuration, enabling contractors to quickly and correctly install components where intended.

[0061]

[0074] 11 , the ECS 406 can interface with the mobility platform's task controller to receive triggers that initiate specific actions necessary to place a precise marking on the floor. Additionally, the printing system can provide feedback to the mobility platform through the same interface to provide real-time information regarding the printer's performance and status. In this manner, the printing system 400 can be self-contained, automating the process of releasing marking fluid based on some external input regarding the mobility platform's timing, location, or other signals as deemed necessary.

[0062]

[0075] In some embodiments, the electronic control system includes a user interface, an onboard processor, a network connection to the mobility platform, and a connection to location-based sensors 412. The user interface may enable actions to be performed on the system, while the network connection enables automated processes to perform the same functions. The network connection to the mobility platform may also be used to provide location-based sensory data, although other sensors may be integrated instead. In such configurations, both the user interface and the mobility platform network interface are used to perform print-related actions, such as initialization, configuration, initiation, and termination of the printing process. Initialization may include loading predefined print tasks; configuration that defines dynamically assigned text, markings, symbols, and other items to be printed; initiating print system triggers to perform printing; and—whether predictably or urgently—terminating the completion of the printing process. When operating autonomously, a higher-level controller on the mobility platform may initiate each trigger based on timing defined in a task file (.rgT), a driver file, a tool file, or other suitable data structure containing executable instructions. Additionally, the printing system may provide feedback to the mobility platform through execution of the task file, allowing the printing system to provide useful information regarding the status of printing (e.g., percentage complete, missed locations, marking fluid levels, etc.) and assisting the on-board controller in collaboratively coordinating and implementing any further actions required. Feedback may include information such as the current print job and settings that allow the operator to track print performance, fluid levels that allow the mobility platform to know when further printing will no longer be possible, and fault status that alerts both the mobility platform and the operator regarding any malfunctions in the system.

[0063]

[0076] According to the embodiment of FIG. 11 , the self-contained printing system can provide a closed-loop mechanism for printing based on sensor input (e.g., from sensor 412). Because each piezoelectric pump can or cannot eject marking fluid, activation of each pump can be triggered according to a timing sequence based on run time, mobility platform position, mobility platform velocity, or other parameters that can be measured. In some embodiments, control of each pump can be performed without control functions being performed by a computer or processor external to the printing system. Therefore, the printing system can be modular and easily interchangeable with other tool kits or systems that can be utilized with the mobility platform.

[0064]

[0077] The above-described embodiments of the technology described herein may be implemented in any of numerous ways. For example, embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on any suitable processor or group of processors, whether provided on a single computer or distributed among multiple computers. Such processors may be implemented as integrated circuits, comprising one or more processors in integrated circuit components, including commercially available integrated circuit components known in the art, such as CPU chips, GPU chips, microprocessors, microcontrollers, or coprocessors. Alternatively, the processor may be implemented in custom circuitry, such as an ASIC, or in semi-custom circuitry resulting from the construction of programmable logic circuits. In yet another alternative, the processor may be part of a larger circuit or semiconductor device, whether commercially available, semi-custom, or custom. As a specific example, some commercially available microprocessors have multiple cores, one or some of which may constitute a processor. However, the processor may be implemented using circuitry in any suitable format.

[0065]

[0078] Furthermore, it should be appreciated that a computer may come in any of several forms, such as a rack-mounted computer, a desk-chip computer, a laptop computer, or a tablet computer. Additionally, a computer may be embedded in a device not generally considered a computer but having suitable processing capabilities, such as a personal digital assistant (PDA), a smartphone, or any other suitable portable or fixed electronic device.

[0066]

[0079] A computer may also have one or more input and output devices. These devices may be used, among other things, to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of output, and a speaker or other sound-generating device for audible presentation of output. Examples of input devices that may be used for a user interface include a keyboard and pointing devices, such as a mouse, touchpad, and digitizing tablets. As another example, a computer may receive input information by voice recognition or in other audible formats.

[0067]

[0080] Such computers may be interconnected by one or more networks of any suitable form, including as a local area network or a wide area network, an enterprise network, the Internet, etc. Such networks may be based on any suitable technology and may operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks.

[0068]

[0081] Also, the various methods or processes outlined herein may be coded as software executable on one or more processors utilizing any one of a variety of operating systems or platforms. Further, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may also be packaged as executable machine code or intermediate code that runs in a framework or virtual machine.

[0069]

[0082] In this regard, the embodiments described herein may be provided as a computer-readable storage medium (or multiple computer-readable media) (e.g., computer memory, one or more floppy disks, compact disks (CDs), optical disks, digital video disks (DVDs), magnetic tapes, flash memory, circuit configurations on field programmable gate arrays or other semiconductor devices, or other tangible computer storage media) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments described above. As is evident from the examples above, a computer-readable storage medium may retain information for a sufficient time to provide computer-executable instructions in a non-transitory form. Such one or more computer-readable storage media may be transportable, and the program or programs stored thereon may be loaded into one or more different computers or other processors to implement various aspects of the present disclosure as described above. As used herein, the term "computer-readable storage medium" includes only non-transitory computer-readable media that may be considered manufactured (i.e., an article of manufacture) or machine. Alternatively or additionally, the present disclosure may be provided as a computer-readable medium other than a computer-readable storage medium, such as a propagating signal.

[0070]

[0083] As used herein, the terms "program" or "software" refer generally to any type of computer code or set of computer-executable instructions that can be utilized to program a computer or other processor to implement various aspects of the present disclosure, as described above. Furthermore, in accordance with one aspect of this embodiment, it should be recognized that one or more computer programs that, when executed, perform the methods of the present disclosure need not reside on a single computer or processor, but may be distributed modularly across several different computers or processors to implement various aspects of the present disclosure.

[0071]

[0084] Computer-executable instructions may be in many formats, such as program modules, for execution by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0072]

[0085] Additionally, data structures may be stored on computer-readable media in any suitable format. For simplicity of explanation, data structures may be depicted as having fields that are associated with locations within the data structure. Such relationships may in turn be achieved by assigning storage for the fields to locations in the computer-readable media that convey the relationship between the fields. However, any suitable mechanism may be used to establish relationships between information within fields of a data structure, including by using pointers, tags, or other mechanisms that establish relationships between data elements.

[0073]

[0086] Various aspects of the present disclosure may be used alone, in combination, or in various arrangements not specifically described in the above embodiments, and therefore, its application is not limited to the details and arrangements of components set forth in the foregoing description or shown in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects described in other embodiments.

[0074]

[0087] Also, the embodiments described herein may be presented as methods, examples of which have been provided. The acts performed as part of the method may be ordered in any suitable manner. Thus, embodiments may be constructed in which acts are performed in an order different from that described, and may include performing some acts simultaneously, even though the described embodiments show acts as sequential.

[0075]

[0088] Additionally, some actions are described as being performed by a "user" or an "operator." It should be recognized that a "user" or "operator" need not be a single individual, and in some embodiments, actions attributed to a "user" or "operator" may be performed by a team of people and / or an individual in combination with computer-assisted tools or other mechanisms. [Example]

[0076] Example

[0089] In some embodiments, a mobility platform for navigation at a construction site includes a drive system configured to move the mobility platform in three degrees of freedom, at least one actuator operably coupled to the drive system and configured, upon activation, to move the drive system to correspondingly move the mobility platform in at least one of the three degrees of freedom, at least one tool operable to perform one or more tasks within the work site, and a controller having a motion control unit configured to selectively activate or deactivate the at least one actuator and a tool control unit configured to selectively activate or deactivate the at least one tool to perform the one or more tasks.

[0077]

[0090] In some embodiments, the at least one tool is selected from the group consisting of a marker, a gripper, a robotic arm, a rotary tool, a reciprocating tool, a track tool, and an impact tool. In some embodiments, the at least one tool includes a marker system configured to mark the work site with lines and / or text. In some embodiments, the lines and / or text indicate instructions for installing the at least one piece of equipment at the work site.

[0078]

[0091] In some embodiments, the holonomic drive includes four wheels, the at least one actuator includes four actuators, and each of the four actuators is coupled to one of the at least four wheels. In some embodiments, a first wheel and a second wheel of the four wheels are rotatable about a first axis, and a third wheel and a fourth wheel of the four wheels are rotatable about a second axis perpendicular to the first axis. In some embodiments, a first wheel of the four wheels is rotatable about the first axis, a second wheel of the four wheels is rotatable about the second axis, a third wheel of the four wheels is rotatable about a third axis, and a fourth wheel of the four wheels is rotatable about a fourth axis, and the first, second, third, and fourth axes are independently movable.

[0079]

[0092] In some embodiments, the mobility platform further includes at least one sensor selected from the group consisting of a drive system encoder, a stereo camera, an inertial measurement unit, an optical flow sensor, and a LiDAR unit, wherein the at least one sensor is configured to provide navigation coupling information to the motion control unit, wherein the motion control unit selectively activates or deactivates the at least one actuator based on the navigation coupling information.

[0080]

[0093] In some embodiments, the controller further includes a task server configured to control the motion control unit and the tool control unit, where the task server includes a computer storage medium storing task commands for operating the motion control unit and the tool control unit. In some embodiments, the motion control unit is configured as a PID or PIV controller, where the motion control unit selectively activates or deactivates at least one actuator based on the task commands. In some embodiments, the tool control unit is configured to selectively activate or deactivate at least one tool based on the task commands. In some embodiments, the mobility platform further includes a wireless transmitter configured to communicate with a remote server, where the task server is configured to receive task commands from the remote server.

[0081]

[0094] In some embodiments, the controller is configured to process output of the at least one sensor to detect one or more landmarks within the construction site, and the task command includes locations of the landmarks. In some embodiments, the controller is configured to process output of the at least one sensor to detect one or more load-bearing structural elements of the building as landmarks. In some embodiments, the task command guides the mobility platform along a path, and the motion control unit selectively activates or deactivates at least one actuator based on the path, and the path is configured to maximize a number of landmarks detected by the at least one sensor when the at least one actuator is selectively activated or deactivated. In some embodiments, the controller further includes a motion quality evaluation unit configured to compare locations of the landmarks in the task command with the landmarks detected by the at least one sensor.

[0082]

[0095] In some embodiments, the at least one sensor is configured to detect one or more control points located on the work site, the one or more control points are present on the construction site, and the task command includes locations of the one or more control points. In some embodiments, the one or more control points are markers placed on the work site by a surveyor.

[0083]

[0096] In some embodiments, the mobility platform further includes a non-transitory computer-readable medium encoded with computer-executable instructions that, when executed by the motion quality assessment unit, are configured to determine a positioning accuracy value based on a comparison between locations of landmarks in the task command and locations of landmarks detected by the at least one sensor, and when the positioning accuracy value is below a threshold positioning accuracy value, the motion control unit is configured to selectively activate and deactivate at least one actuator to move the mobility platform toward one of the one or more control points. In some embodiments, the controller further includes a calibration control unit configured to adjust one or more parameters of the motion control unit based on the detected one or more control points. In some embodiments, the mobility platform further includes a computer station located at the construction site including a graphical user interface configured to receive operator input defining modifications to the task command, selectively activate or deactivate at least one actuator, and / or selectively activate or deactivate at least one tool.

[0084]

[0097] While the present teachings have been described in conjunction with various embodiments and examples, it is not intended that the present teachings be limited to such embodiments or examples. On the contrary, the present teachings encompass various alternatives, modifications, and equivalents, as will be recognized by those skilled in the art. Accordingly, the foregoing description and drawings are by way of example only.

Claims

1. 1. A method for operating a mobility platform within a construction site based on instructions of one or more tasks to be performed at one or more task locations within the construction site and one or more landmarks within the construction site, the mobility platform including at least one tool mounting location, the method comprising, using at least one processor: generating a path for the mobility platform by iteratively adjusting the path to increase predicted positioning accuracy of the mobility platform as the mobility platform moves along the path that is dependent on the locations of the one or more landmarks and the one or more task locations, the predicted positioning accuracy being based on predicted error in estimated position from continuous displacement and sensing position relative to one or more of the landmarks along the path; and providing task commands to a controller located at the mobility platform based on the generated path and the one or more tasks.

2. 2. The method of claim 1, further comprising generating, by the controller, control signals for moving the mobility platform along the generated path and performing the one or more tasks based on execution of the task commands.

3. The method of claim 1 , wherein the one or more landmarks are control points or control lines.

4. The method of claim 1 , wherein increasing the predicted positioning accuracy comprises iteratively adjusting the path to increase an amount of the path located within a threshold distance of the one or more landmarks.

5. 5. The method of claim 4, wherein the threshold distance is based on detectability of the one or more landmarks by at least one sensor selected from the group consisting of a stereo camera, a mono camera, and a LiDAR unit, and the threshold distance is set to a point where the one or more landmarks can be detected by the at least one sensor.

6. The method of claim 1 , further comprising outputting the route on a graphical user interface.

7. Generating the path includes terminating the iterative adjusting based on a trigger condition; and The method of claim 6 , wherein the trigger condition comprises receiving input from a human user at the graphical user interface that accepts the route.

8. moreover, regenerating the route based on the locations of the one or more landmarks and the one or more task locations based on a trigger condition; and The method of claim 6 , wherein the trigger condition comprises receiving input from a human user at the graphical user interface to reject the route.

9. The method of claim 1 , further comprising controlling, with the controller, the mobility platform to move along the path based on the path and the task commands.

10. 10. The method of claim 9, further comprising detecting the one or more landmarks as the mobility platform is moved along the path with at least one sensor selected from the group consisting of a stereo camera, a mono camera, and a LiDAR unit.

11. The method of claim 10 , further comprising correcting the movement of the mobility platform based on information received from the at least one sensor.

12. The method of claim 11 , wherein correcting the movement of the mobility platform comprises using the information provided by the at least one sensor in feedback control.

13. 11. The method of claim 10, wherein the at least one sensor includes a first sensor, the method further comprising integrating odometry information from the first sensor to calculate a first position of the mobility platform at the construction site.

14. The at least one sensor includes a second sensor, and the method further comprises: continuously detecting the one or more landmarks with the second sensor to calculate a second position of the mobility platform at the construction site; and The method of claim 13 , further comprising comparing the first position and the second position to calculate a positioning accuracy value.

15. moreover: comparing the positioning accuracy value to a positioning accuracy threshold; moving the mobility platform toward one of the one or more landmarks when the positioning accuracy value falls below the positioning accuracy threshold; and The method of claim 14 , further comprising calibrating the first sensor by sensing the one landmark of the one or more landmarks with the second sensor.

16. 16. The method of claim 15, further comprising: modifying one or more parameters of the controller based on the positioning accuracy value to integrate the odometry information from the first sensor to increase the positioning accuracy value above the positioning threshold.

17. Iteratively adjusting the path to increase the accuracy of predicted positioning of the mobility platform includes: generating a plurality of candidate paths; assigning a cost to each of the candidate paths; and The method of claim 1 , comprising selecting a candidate route with the lowest cost from the plurality of candidate routes as the generated route.

18. The method of claim 1 , wherein iteratively adjusting the route to increase accuracy of predicted positioning of the mobility platform is also based on sensing a position relative to one or more landmarks along the route.

19. 1. A method for operating a mobility platform within a construction site, the mobility platform including a plurality of sensors having operating ranges capable of detecting landmarks, the method comprising: identifying the location of one or more landmarks within the construction site; identifying one or more tasks to be performed at one or more task locations within the construction site; and generating a path for the mobility platform by iteratively adjusting the path to increase predicted positioning accuracy of the mobility platform as the mobility platform moves along a path for the mobility platform that is based on the locations of the one or more landmarks and the one or more task locations, wherein the predicted positioning accuracy is based on predicted errors in estimated position from continuous displacement and detecting position relative to one or more of the landmarks along the path.

20. 20. The method of claim 19, wherein increasing the predicted positioning accuracy includes iteratively adjusting the path to increase an amount of the path located within a threshold distance of the one or more landmarks, the threshold distance being based on the operating range of the sensor.

21. The method of claim 19 further comprising outputting the route on a graphical user interface.

22. Generating the path includes terminating the iterative adjusting based on a trigger condition; and 22. The method of claim 21, wherein the trigger condition comprises receiving input from a human user at the graphical user interface that accepts the route.

23. moreover, regenerating the route based on the locations of the one or more landmarks and the one or more task locations based on a trigger condition; and 22. The method of claim 21, wherein the trigger condition comprises receiving input from a human user at the graphical user interface to reject the route.

24. The method of claim 19 , wherein the one or more landmarks are control points or control lines.

25. 20. The method of claim 19, wherein iteratively adjusting the route to increase accuracy of predicted positioning of the mobility platform is also based on locating relative to one or more landmarks along the route.

26. 1. A method for operating a mobility platform within a construction site, comprising: identifying the location of one or more landmarks within the construction site; moving the mobility platform along a navigation path based at least in part on the locations of the one or more landmarks; integrating odometry information from at least one first sensor to calculate a first position of the mobility platform within the construction site; detecting the one or more landmarks with at least one second sensor as the mobility platform is moved along the path to calculate a second position of the mobility platform within the construction site; calculating a difference between the first position and the second position to calculate a positioning accuracy value; comparing the positioning accuracy value to a positioning accuracy threshold; and correcting the movement of the mobility platform based on the positioning accuracy value, wherein correcting the movement of the mobility platform includes moving the mobility platform toward one of the one or more landmarks when the positioning accuracy value falls below the positioning accuracy threshold.

27. 27. The method of claim 26, wherein the at least one second sensor includes a LiDAR unit.

28. 28. The method of claim 27, wherein correcting the movement of the mobility platform comprises using the information provided by the at least one first sensor in feedback control.

29. 27. The method of claim 26, further comprising: modifying one or more parameters based on the positioning accuracy value to integrate odometry information from the at least one first sensor, thereby increasing the positioning accuracy value above the positioning accuracy threshold.

30. The method of claim 26, wherein the one or more landmarks are control points or control lines.

31. 31. The method of any one of claims 26 to 30, wherein the at least one first sensor comprises at least one selected from the group consisting of a drive system encoder, an inertial measurement unit, and an optical flow sensor.

32. 27. The method of any one of claims 1, 19 and 26, wherein the one or more tasks include depositing a marking fluid onto a surface.

Images