How to control a printing robot in the shadows

Abstract

A mobile printing robot system for printing construction layouts. The method addresses a problem that occurs when tracking lock to an absolute positioning device, such as a total station, is lost. In a construction site, when the mobile robot moves into a shadow area behind an obstacle, such as a pillar, line of sight between the mobile robot and the absolute positioning device may be lost. Using its local sensor data, the mobile printing robot can continue printing until it reaches a maximum estimated position. The mobile robot can also provide position updates to the absolute positioning device to assist the absolute positioning device in regaining tracking lock when the mobile robot emerges from the shadow area.

Description

[Technical field]

[0001] Aspects of the present disclosure relate generally to the operation of mobile robots using absolute position sensor systems that require an unobstructed line-of-sight path between the mobile robot and an external component. More particularly, aspects of the present disclosure relate to improving the ability of the robot to perform when the line-of-sight is temporarily obstructed. [Background technology]

[0002] Mobile robots can be used in a variety of applications. One environment in which mobile robots can be used is a construction site, where they can perform a variety of functions. One such application is the printing of drawings, text or other markings on construction surfaces, such as the concrete floors of buildings under construction.

[0003] Current architectural layout involves hiring human crews to lay out building components such as walls, ducts, and wiring, and making physical marks on the floor. Locations are referenced from drawings created by the architect and are usually delivered to the site on large rolls of blueprints or in digital form on a tablet computer. A typical layout process involves measuring distances from known landmarks, such as concrete pillars or control points marked in the concrete by a surveyor, with a tape measure, and marking the lines with a piece of chalk-covered string stretched between two pins.

[0004] Instead of using a tape measure to locate lines, a total station can be used to precisely place line endpoints to within 1-2mm of the desired position, improving both the speed and accuracy of the manual layout process. Summary of the Invention [Problem to be solved by the invention]

[0005] One of the challenges in building such a mobile printing robot is the accuracy with which the robot can make its marks. In construction applications, an accuracy of 1 / 16th of an inch (1.6 mm) or better is typically required to create the construction layout. In some applications, the sensor suite onboard the mobile robot can be augmented with an external sensor (e.g., a total station) to provide a more precise measurement of the mobile robot's position and thus improve its localization accuracy. However, in such situations, site obstacles may get between the robot and the line of sight to the total station while the robot is moving through the construction site. [Means for solving the problem]

[0006] Embodiments of the present disclosure are generally directed to improving the position accuracy of a mobile robot used for printing on a construction surface. The mobile robot uses optical techniques to generate highly accurate position information. In some embodiments, the mobile robot has a retroreflective device attached to a surface of the mobile robot. The mobile robot receives position measurements from an external Absolute Positioning Device (APD), which uses the retroreflective device to determine the position of the mobile robot. The mobile robot uses the received position measurements to use the position information and improve accuracy during navigation and printing operations.

[0007] When the robot moves to a position where it loses Line of Sight (LOS) between the reflector and the APD, the robot loses access to accurate absolute position information from the APD. However, the mobile robot may have a set of local sensors that allow it to estimate its current position. In some embodiments, the mobile robot has an operation mode that is implemented when the APD lock is lost. The mobile robot determines the mobile robot's position accuracy error and continues printing only if the position accuracy for printing is within the desired printing accuracy.

[0008] More generally, the mobile robot may have a state estimator that generates state and error information that takes into account data from a suite of sensors mounted on the robot that indicate limitations related to changes in position and accuracy of the mobile robot. In some embodiments, in response to a loss of APD lock, the mobile robot switches to a non-APD mode of operation. An estimated state and associated error are determined. The mobile robot continues printing until the estimated state error is greater than a threshold.

[0009] In other embodiments, the mobile robot has an operational mode that modifies actions in response to loss of APD lock to make best use of the current estimated accuracy of the position estimate, which in some embodiments includes slowing down the mobile robot when loss of APD lock is detected.

[0010] In some embodiments, the robot takes one or more actions to re-establish LOS between the APD and the robot and restore position estimate accuracy, which in some implementations includes sending the mobile robot's estimated position information to the APD to assist the APD in re-establishing lock when LOS is restored. [Brief description of the drawings]

[0011] The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals are used to refer to similar elements and in which: This application contains at least one color drawing. Copies of this application containing color drawing(s) will be provided by the Office upon request and payment of the necessary fee. [Figure 1A] 1 illustrates a mobile robot printing system according to an embodiment. [Figure 1B] 1 illustrates a mobile robot printing system according to an embodiment. [Figure 1C] 1 illustrates a mobile robot printing system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram of a mobile robot according to an embodiment. [Figure 3A] 1 shows a path for a mobile robot from an area where line of sight to the APD initially exists, but where there are also shadow areas where line of sight is blocked by obstacles, according to an embodiment. [Figure 3B] 1 shows a mobile robot moving in a shadow area where line of sight with the APD no longer exists, according to an embodiment. [Figure 3C] 1 shows a mobile robot operating in a shadowed region with an APD providing instructions / information for pointing the mobile robot and providing for regaining APD lock when line of sight becomes available again, according to an embodiment. [Figure 3D] 1 shows a mobile robot entering an area where it has line of sight with an APD, according to an embodiment. [Figure 4] 4 is a flow chart illustrating a method for recovering APD lock, according to an embodiment. [Figure 5A] We show that a mobile robot starts printing a line in an unshadowed area with line of sight to the APD, and continues printing a line in a shadowed area as long as the position accuracy estimate of the line is within the accuracy tolerance of the line. [Figure 5B] We show that a mobile robot starts printing a line in an unshadowed area with line of sight to the APD, and continues printing a line in a shadowed area as long as the position accuracy estimate of the line is within the accuracy tolerance of the line. [Figure 5C] We show that a mobile robot starts printing a line in an unshadowed area with line of sight to the APD, and continues printing a line in a shadowed area as long as the position accuracy estimate of the line is within the accuracy tolerance of the line. [Figure 6] 1 is a flowchart of a method for printing in shadow areas with loss of APD lock, according to an embodiment. [Figure 7] 4 is a flowchart of a method for printing in shadow areas and recovering APD lock, according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] 1A, 1B, and 1C show an implementation of a mobile robotic printing system for navigating a built environment and marking specific locations using mobile robot technology. A mobile printing robot 104 moves autonomously over a construction surface (e.g., a concrete surface) and prints a building layout onto the floor that corresponds to an architect's blueprint of how the building should be constructed.

[0013] A proper and accurate layout is of utmost importance for any construction project. If the layout is drawn incorrectly, it can lead to costly rework, for example if a wall is placed in the wrong place and has to be torn down and rebuilt. Also, an incorrect layout can lead to legal violations, such as not being able to place a toilet at a specified distance from a wall according to local building codes.

[0014] There are various problems with accurately determining the position of a mobile robot using an absolute positioning device. First, the mobile robot that prints the layout is moving around a construction site. Second, there are also practical limitations to some of the components that can be used to determine absolute position using optical techniques.

[0015] FIG 1A is a high level block diagram of a mobile robotic printing system according to an embodiment. FIG 1B and FIG 1C are perspective views. As shown in FIG 1A, 1B and 1C, an embodiment of a system 100 includes a mobile robot 104, one or more absolute positioning devices (APDs) 106, and an optional portable computing device including a user interface 114, such as a tablet computing device.

[0016] The APD 106 measures the position of the robot using a laser beam 130 to detect the position of a retroreflector that can be implemented in a variety of ways, such as a corner reflector, prism, or other retroreflective device 105 that can be affixed to the robot 104. This is shown in FIG. 1A. For example, the reflector, prism, or similar retroreflective device 105 can be permanently attached to the robot 104, removably attached to the robot 104, or otherwise temporarily fixed. For example, a mounting structure can allow the reflector, prism, or other similar retroreflective device 105 to be attached / detached to a mobile robot.

[0017] In one embodiment, the APD used in this robotic printing system is a total station typically used on construction sites, such as those manufactured by Trimble, Leica, Hilti, TopCon, or other companies. Alternatively, the APD may be a laser tracker. More generally, the APD may comprise any device capable of determining position using a laser beam and a laser tracker and measuring the angle relative to and distance to a target. As another example, the APD may utilize other methods, such as time of flight and vision.

[0018] The laser of the APD is shown in solid line in Figure 1A. In one embodiment, the measured position determined by the APD is communicated to the robot via a wireless link 120, as shown in Figure 1C. This link may use an existing network 102, such as WiFi, or it may be through a direct communication channel using a wireless or optical connection between the APD 106 and the robot 104. One advantage of a direct communication channel over WiFi is that it can minimize potential network delays that may occur in locations with heavy WiFi network traffic.

[0019] For communicating position data to the mobile robot 104, a low latency and low variable latency communication link is desirable. This is because there are various trade-offs between the speed of the mobile robot, the desired printing accuracy, and the impact of latency in receiving position information that is useful for improving the printing accuracy of the mobile robot 104. For example, the mobile robot 104 moves at a speed of 1 m / s. A communication delay of 1 ms would then correspond to a distance traveled by the mobile robot of 1 mm. If the goal is to achieve a printing line accuracy of 1 mm, in this example, the time delay in receiving the position information would be an ideal time delay for controlling the movement of the mobile robot 104.

[0020] The communication link 120 can use an existing network such as WiFi. However, Wi-Fi can have a large latency in some locations, such as urban areas with a lot of WiFi communication near construction sites. A long range (LoRa) communication protocol is also a possibility. LoRa uses spread spectrum technology to minimize interference. A local direct optical link is also a possibility, such as using a separate laser beam (or light emitting diode) to communicate location information over a direct link. A direct local radio frequency link is also possible. In any case, a variety of different network communication technologies or a direct communication channel using a wireless or optical connection can be used as the communication link between the APD 106 and the robot 104.

[0021] The mobile robot has a travel system. Although omnidirectional travel is also possible, the mobile robot may have a preferred longitudinal forward direction for printing purposes. As shown in FIG. 1A, there may be a longitudinal axis 190. Thus, as the mobile robot prints the layout, the longitudinal axis of the mobile robot changes its orientation relative to the APD.

[0022] As shown in FIG. 2, one embodiment of the mobile robot 104 includes a controller 290, one or more sensors 202, which may be an IMU, an accelerometer, a gyroscope, or a compass, a mobile base 204 including features such as motors and wheels for moving the mobile robot, a communication device 206, a reflector 208, a printing system 216, a visual odometry module 218, an obstacle detector 220, a BIM module 222, a battery 224, and a cliff detector 226.

[0023] In one embodiment, the communication device 206 implements a wireless communication link with the APD 106 and / or the optional mobile device 114 .

[0024] General information about the lines to be drawn can be obtained from Building Information Models (BIM) or architectural drawings. A version of this associated BIM / drawing information may be stored in the controller 290 along with information corresponding to the obstacle map. For example, the obstacle map may include features derived from the BIM / drawing information plus additional obstacle information collected at the construction site.

[0025] The controller 290 may include one or more modules to address some of the issues associated with using a particular type of retroreflective device 208. For example, the controller 290 may be designed to support a single type of retroreflective device 101, or more generally, may be designed to configure the controller to operate with a particular type of retroreflective device. Depending on the type of retroreflective device, various optimizations appropriate for the type of retroreflective device selected may be utilized, as described in more detail below. These include optimization modules such as a reflector type configuration algorithm 291, an optimization module for reflector orientation initialization 292, an optimization module for course orientation initialization 292, an optimization module for reflector locking algorithm 294, an optimization module for reflector angle adjustment algorithm 295, and an optimization module for correction / calibration optimization 297.

[0026] The optimization modules 292, 293, 294, 295 and 297 may be implemented as computer program code executing on a processor of the controller 290. The optimization modules address various limitations and shortcomings associated with particular reflector types.

[0027] As described in more detail below, the one or more optimization modules can take into account the optical characteristics of individual retroreflective device types and other factors related to using APDs to monitor the position of the mobile robot. The mobile robot can optionally include one or more modules that generate an alert 298 if position accuracy cannot be guaranteed within a desired range throughout the print job. As another example, an alert may be generated if a lock between the APD and the reflector cannot be maintained. Factors related to position accuracy 299 can also be monitored as needed.

[0028] For example, in a printing application, a mobile robot moves through a construction site. The mobile robot may move behind an obstacle that causes the reflector lock to be temporarily lost. Also, as the robot moves through the site, other factors may affect the accuracy of the position measurement, such as radial distance, azimuth angle, and polar angle changing during the print job, as described in more detail below. If tight tolerances on print accuracy are desired, it may be desirable to issue a warning to the operator or to abort the print.

[0029] Generally, many of the sensors on a robot make measurements relative to the robot's current position or state. For example, wheel odometry measures the incremental motion relative to the robot's current position as the wheels move. Another example is a gyroscope, which measures the relative angular rate of change of the robot along three axes. Other position sensing options include visual odometry. Using these types of sensors to estimate the robot's position generally results in position estimates that move away from the robot's true position over time. This is due to the fact that all estimates are relative to the current state and build on these errors, so any small errors will accumulate over time.

[0030] This is why an APD or other absolute sensor is important to accurately estimate the robot's position over time. With an APD or other absolute sensor, correction back to the building's absolute reference frame is possible. However, once the robot moves into a position where it loses line of sight (LOS) between the reflector and the APD, it can no longer accurately assess its position in an absolute sense. In this situation, it is possible to continue to estimate the robot's position using only relative sensors (providing an estimate based on the relative position change since the last known absolute position measurement), but the estimate will move away from the true position over time.

[0031] In current robotic systems for moving and printing, if LOS to the APD is lost (also known as loss of APD lock), the operation is interrupted or paused to prevent the robot from operating if the accuracy is not sufficient. These systems rely on the operator to correct the situation, re-establish LOS, and re-establish the lock between the APD and the robot's reflector. However, this is a time-consuming procedure that interrupts functionality. For example, a work site may have multiple supports and possibly other objects such as pallets. Thus, when performed in a conventional manner, a single layout print job may experience multiple instances of loss of LOS to the APD. This, in a conventional system, may result in multiple instances of the print operation being interrupted or paused, requiring the robot operator to intervene and re-establish the lock between the APD, reflector, and robot. The cumulative loss of productivity and additional operator labor can be substantial.

[0032] In one embodiment, the mobile robot has a normal operation mode that begins after a human operator assists in establishing an APD lock. For example, on a construction site, a human operator can first position the mobile robot in an area of ​​the construction site that has an APD and LOS, and the human operator can take steps to ensure that the APD is first locked onto the mobile robot before the mobile robot moves through the construction site and begins printing lines of a layout.

[0033] In one embodiment, the mobile robot includes a feature that enables it to automatically re-establish lock with the APD without operator assistance, such that if the mobile robot moves into a shadow region where there is no LOS between the mobile robot and the APD, the mobile robot performs at least one action to assist in re-establishing APD lock.

[0034] In another embodiment, if the mobile robot loses APD lock, it continues to estimate its position, but estimates the current estimation accuracy and uses the estimation error to take action when performing any critical operations in a construction environment, such as printing.

[0035] To illustrate these ideas, FIG. 3A shows an example of a robot and APD interaction in one implementation. An obstacle 304 is shown, which may be a pillar, but more generally an obstacle such as a pallet of construction materials that blocks the LOS path from the mobile robot's reflector to the APD. FIG. 3A is a top-down view of a construction site floor showing the robot, the APD, and the obstacle 304, which may be a pillar. In the example shown, the robot passes behind the pillar and comes out on the other side. In this particular example, the robot moves from one location to another and is not printing. In the figure, the robot 104 is moving through a space, such as a construction site. The space also includes an APD 106, such as a total station or laser tracker, for measuring the robot's position.

[0036] The APD measures the robot's position by bouncing a laser beam 130 off a reflector mounted on the robot's body. Note that to perform this measurement, the laser beam needs a line-of-sight LOS path that is not blocked by obstacles.

[0037] The APD has a feedback control system that keeps the laser beam locked to the reflector as it moves through the construction space. This lock requires a continuous line of sight because the locking feedback mechanism gets an estimate of the position error by looking at the reflected beam. If this line of sight is broken, the beam will no longer follow the robot and will typically remain at the last measured position of the reflector.

[0038] Also on site is a concrete pillar 304. In the illustrated situation, the robot follows a particular navigation path 312, indicated by a curved line with an arrow. While following this path, the robot will cross an area 306 on the floor where the pillar 304 blocks the laser from reaching the robot. If the robot is positioned anywhere in this area with its reflector, the LOS will be blocked and absolute position measurements will not be possible. The area in the pillar's shadow will be referred to as the shaded area 306.

[0039] In the situation shown in the figure, two problems arise. First, in the shadow region, the mobile robot's state estimation relies entirely on the APD data to prevent drift and to properly estimate the robot's position. Without this absolute position data, the robot cannot know its position accurately while passing through this shadow region. Second, when the robot enters the shadow region, the APD loses lock on the mobile robot's reflector. As a result, the APD (according to conventional practice) will no longer follow the robot as it passes through the space. For example, if the APD points a laser beam toward the mobile robot's retroreflector, when the APD loses lock, it will not be able to change the pointing direction of the laser beam toward the mobile robot. The combination of these two problems makes it impossible with conventional techniques for a mobile robot to pass a pillar and continue moving accurately on the other side without operator intervention.

[0040] FIG. 3B shows a situation where the robot moves to a shadowed region 306 and the APD loses sight of the reflector. The robot continues on its path, but the laser beam of the APD is still pointing to the last known point where a measurement could be obtained from the reflector, and is no longer pointing to the robot. In the particular embodiment shown, the mobile robot continues moving even if the robot stops receiving data from the APD. The mobile robot may have a state estimator that generates state and error information that takes into account data from a set of sensor inputs. Thus, the information on the current state and its associated error indicates the current position and position error. The position error may be compared, for example, to a maximum position error for printing. So, for example, in the shadowed region, printing may continue until the state estimate and its associated error correspond to a position error that is no larger than the maximum position error. At the point where APD data is no longer available, the robot continues to make an estimate of the current state, but without using APD data. This may be achieved by switching to a different type of state estimator, or by a state estimator that considers APD measurements as an optional data source.

[0041] Because the APD cannot measure the position of the mobile robot's reflector while in the shadow area, the APD does not know the current reflector position. When the mobile robot enters the shadow area, the LOS from the reflector to the APD is lost. As a result, the APD loses lock on the reflector and stops tracking the reflector's position. Also, when the robot moves to a point where the reflector is visible again, the APD is unable to reacquire the reflector. So, in the example of FIG. 3B, the robot has moved, but the APD's laser beam 130 remains fixed on the corner of the pillar. Typically, this situation requires an operator to intervene now to re-establish the reflector lock so that navigation and printing can resume.

[0042] However, even if the APD lock is lost, the robot is still in communication with the APD since the communication channel between the devices is a wireless link. In one embodiment, the robot re-establishes the lock itself by using an estimate of its internal state to generate information to command the APD to point in the correct direction. For example, in response to detecting the loss of the APD signal, the mobile robot can send an instruction to the APD to adjust its position. The instruction also includes information for the APD to determine how much to adjust its pointing position to be close enough to the reflector's position to regain the APD lock when the mobile robot emerges from the shadow region and LOS is restored. This is illustrated in FIG. 3C, where the robot has progressed a little further along its intended path.

[0043] Using a state estimator and on-board sensors, the robot estimates its current position. Knowing its own position and the position of the APD, the robot can calculate the direction in which the APD should point to properly lock onto the robot's reflector. In the situation shown in the figure, the robot has just commanded the APD over the radio channel to point towards the robot's reflector. Therefore, the APD is pointing towards the robot's reflector. However, the robot is still located behind the pillar and the LOS is still blocked.

[0044] A mobile robot typically has no way of knowing where LOS will be restored. It is theoretically possible for a mobile robot to perform advanced image analysis techniques to determine where LOS will be restored. However, a mobile robot typically has no way of knowing where LOS will be restored. As a result, in one embodiment, the mobile robot continuously sends position updates to the APD. At each point, the APD attempts to lock onto the robot (using the position updates sent by the mobile robot), and if lock is not established, the robot retries. Depending on the capabilities of the APD and the wireless channel, the robot attempts updates at an appropriate frequency; for example, once every 5 seconds if the APD is slow to respond. However, updates can be sent more or less frequently, depending on the details of the implementation.

[0045] Figure 3D is the final example where the APD has re-established lock with the robot. In this case, the robot has moved beyond the pillar into an area where LOS can be re-established. At this point the command to re-establish lock is successful and lock is re-acquired. At this point the APD data is re-incorporated into the state estimation and any errors that accumulated while the robot was moving behind the pillar are removed from the robot's state estimate.

[0046] Figure 4 shows a supplemental flow chart that describes how the behavior depicted in Figure 3 is implemented. Beginning at step 402, the robot begins moving through the site. This navigation may be any type of movement the robot may perform, including point-to-point navigation or line following behavior, and may or may not involve other functions the robot may perform, including printing while moving, or scanning while moving.

[0047] While the robot is moving through the scene, it is constantly estimating the complete state of the robot, including its position and angular orientation. While doing so, it checks the state of the APD data sources to determine if there is a possibility of APD data loss as a result of LOS being blocked. This is shown in step 404. If there is no loss of APD data, it continues moving until navigation is finished and no further steps shown in the flowchart are performed. However, if APD data is lost, it switches the operation mode to a predefined state that is determined to be suitable for operation without APD in step 406. This includes modifying the way the state estimator operates to make best use of the remaining data sources available to the robot. It also includes modifying the travel speed of the robot.

[0048] One reason for varying the drive speed is to help the APD regain lock on the reflector target. Some APD devices cannot lock on targets moving at high speeds. Another example of a similar mode is to periodically stop the APD to allow it to acquire lock for APD devices that cannot lock on moving targets of any speed. Another reason for varying the robot speed is to maximize accuracy. For example, state estimation may perform best when the robot is moving slowly, since the wheel odometry is more accurate. Alternatively, it may be better when the robot is moving faster, since drift in relative sensor measurements is often time-based. In the most general case, there may be various tradeoffs between accuracy of position estimation in shadow regions and quickly regaining APD lock when the robot emerges from the shadow region. In some embodiments, these tradeoffs may be optimized for a particular robot design and typical robot operating conditions. For example, some construction sites may have common types of instructions, such as common pole sizes. Some types of APDs may be used at certain construction sites. In some embodiments, the characteristics of the APD, the BIM, and available information about the mobile robot may be used to identify optimal changes in driving speed or driving behavior when LOS is present.

[0049] Once the operational mode has been changed to account for the fact that the APD data is no longer available, the robot starts sending position updates to the APD in step 408. This step can be done in many different ways. If the APD's position updates are fast and there is sufficient bandwidth on the wireless communication channel between the APD and the robot, the robot can send updates continuously. However, the robot can also choose to send periodic updates to the APD (e.g. every 5 seconds). The robot can also change speed or stop when an update is sent to give the APD a chance to lock onto the robot if necessary. If the lock is not successful, the robot will increase its speed or continue its navigation path after a certain time.

[0050] After each update, the robot waits to see if the APD lock has been re-established in step 410. This may be by receiving a lock signal or other message from the APD, or simply by seeing if the APD resumes transmitting position information. If lock has not been re-established, the robot continues to send updates until it regains lock. Once lock is regained, the robot returns to normal operating mode in step 412. This step is the reverse of step 406, so that the state estimator resumes using the APD data and the robot speed returns to normal if these parameters have changed. Finally, once the mode has returned to normal mode, the system returns to step 404 to wait for loss of APD data.

[0051] There are many variations on the described methodology, which can take into account the characteristics of the APD, the ability to reacquire lock, and the quality of the robot's position. For example, after losing the APD, the error in the robot's state estimate will start to drift. If the robot can estimate the magnitude of that state error, it can make intelligent decisions about how to act. For example, if the error is getting too large, it may choose to retrace its steps and exit the shadow rather than continue on and lose it completely. It can also drive back to where it came from, or to any other point that is not in the shadow.

[0052] In another example, if the robot's position estimate becomes too inaccurate, the robot may stop and request that the APD search with a scanning motion over the search area until a lock is reacquired, or the robot may send a notification to the operator if it cannot reacquire a lock on the APD.

[0053] Additionally, to achieve a lock, the robot can have the APD point to a future location instead of the current location. For example, the robot can predict how long it will take for the APD to physically move and point to a new location. It can instruct the APD to point an inch ahead of where the robot is, and by the time the APD has the laser pointed there, it can intercept the robot.

[0054] 5A shows a second embodiment in which a robot is instructed to print a straight line, but in order to print the line, the robot must travel within a shadow area for part of the print. In the figure, the robot 104 is tracked by the APD 106 using the laser beam 130. The line that the robot is being asked to print is shown by the dotted line 502. In order to print the line, the robot must enter the shadow area 306 created by the concrete pillar 304.

[0055] In this scenario, the robot is unaware that it will lose the pending LOS that occurs when the pillars and lines are printed. So the robot starts printing the line as normal, but eventually loses LOS data when the robot's reflector enters the shadow area. The robot monitors the APD data while printing. As with the previous embodiment, if APD data is lost due to LOS or other issues, the robot changes its operating mode to adapt to the change in sensor data. This may include changing the robot's speed or changing the operating mode of the state estimator to more accurately estimate APD data in the absence of it.

[0056] To ensure line continuity, printing does not stop when the robot operation mode changes. Even without APD data, the robot continues printing and estimating its current position. This scenario is shown in Figure 5B, where the robot has traveled into the shadow area but continues printing the line. However, it has traveled a short distance into the shadow area and there is still part of the line to be printed.

[0057] However, to ensure accurate printing, the robot creates a new type of estimation: it continuously estimates the quality of its state estimation. That is, it not only estimates the robot's pose, including its position and heading, but also estimates the accuracy or certainty of that estimate. In this way, it estimates how much the robot's estimated position has drifted from the robot's actual position. In one example, this accuracy estimate can be as simple as knowing that the estimate tends to drift by 1 mm every 10 seconds, or it can be a more sophisticated estimate that takes into account many dynamic factors.

[0058] With this information, the robot can continually determine if the accuracy of its current state estimation is sufficient to continue printing. If the accuracy becomes too low, the robot will stop printing. In the example of traveling 1mm every 10 seconds, the robot might have a printing accuracy requirement that it prints within 1mm of the correct position on the ground. In this case, after 10 seconds of operation, once the estimation accuracy reaches 1mm, the robot will stop printing to ensure that the information printed on the ground does not exceed the 1mm requirement.

[0059] This final situation is shown in Figure 5C, where the robot travels further into the shadow area, but at some point stops printing, realizing that the accuracy is no longer sufficient and turning off the printer. The remainder of the line that was not printed is shown in dotted lines.

[0060] Note that there are many similar scenarios, but the basic concept of making an estimate of accuracy and printing only if the accuracy is high enough always holds. For example, the line to be printed may start in a shadow area. In this case, printing will not start until the robot loses APD lock, and printing will only start if the accuracy of the state estimate is within the acceptable line quality of the system. Printing will finish as shown in the example diagram.

[0061] Note that this concept can be easily combined with the previous concept to allow the robot to print within the shadow area for as far as it can accurately print, and then emerge from the shadow for the APD to lock back to the robot. Note that the precision required for the APD to lock back to the robot may be much less than the precision required for printing. For example, the APD's position may be able to lock onto the robot within 25mm, but the precision required for printing may be 1mm. So the robot may travel a much longer distance in the shadow than it can safely print.

[0062] FIG. 6 is a flowchart that complements the robot navigation shown in FIGS. 5A-C. In step 602, the robot begins moving and printing a line. While the robot is printing the line, it is continuously estimating its complete state, including its position and angular orientation. While doing so, it checks the state of the APD data source to determine if there is a loss of APD data as a result of an obstructed LOS. This is shown in step 604. If there is no loss of APD data, it continues printing until the line is finished and no further steps shown in the flowchart are performed.

[0063] However, if the APD data is lost, the operation mode switches to a predefined state determined to be suitable for operation without the APD in step 606. This may involve modifying the way the state estimator operates to make the most of the remaining data sources available to the robot. It may also involve modifying the speed at which the robot is traveling. In this new state, the robot monitors the accuracy of its state estimate and determines how much error there is in the estimate as shown in step 608. As printing continues, the state is monitored and checked against a threshold in step 610, and if the threshold is exceeded the robot stops printing to prevent the ground from being printed too inaccurately.

[0064] FIG. 7 illustrates how the two techniques can be combined and used together to seamlessly operate in shadowed areas. In step 702, printing begins. Initially, there is an APD lock established when the robot was in a non-shadowed area. In step 704, the mobile robot waits for loss of APD lock. That is, the mobile robot continuously monitors the APD lock and detects loss of the APD data signal from the APD. In step 706, the mobile robot switches to a different operating mode in response to detecting loss of APD lock. As previously mentioned, the mobile robot may switch to using a state estimator that does not include the APD signal. Alternatively, the mobile robot may have a state estimator that the APD signal inputs as required. In step 708, the state is estimated along with the associated error. This state estimate and error correlate to changes in position and position accuracy. The mobile robot essentially starts with its last known APD position data and makes an estimate for its current position and associated position accuracy (without the benefit of an APD update). In block 408, the mobile robot sends a position update to the APD. This may be instructions on where the mobile robot should point the APD. (Or a custom APD can be implemented that requests a position update from the mobile robot in response to loss of APD lock). In step 410, a determination is made if APD lock has been reacquired. In step 709, a determination is made if printing is finished. In step 710, a determination is made if the position / state error is greater than a threshold. If YES, printing is stopped. If not, the sub-process continues at step 708. The method shown in the flowchart of FIG. 7 works to continue printing in the shadow areas as long as practical and also regain APD lock.

[0065] One aspect illustrated by FIG. 7 is the reduction of lost downtime traditionally associated with a mobile robot stopping printing in a shadow area and a human operator having to restore an APD lock.

[0066] It is important to note that these techniques can also be employed on robots where the APD sensor is installed on the robot itself. For example, the robot may have a total station, laser tracker or other absolute positioning device mounted on the robot. This APD can track a reflector or other target installed in the field. In this case, a radio link to the APD is no longer needed, but there will be a loss of LOS if an obstacle comes between the robot and the reflecting target. The techniques described here still apply. The robot estimates what state it would be in the absence of APD data and uses that data to repoint the APD to the target and reacquire lock. Similarly, the robot can continue printing in the absence of APD data, estimating the accuracy of its state and aborting printing if the accuracy is too low.

[0067] Also, note that while shadows from obstacles are one way of losing APD information, more generally, APD information can be lost for other reasons, such as a temporary loss of the radio signal between the mobile robot and the APD. Another possibility is that a person walks across the construction site and temporarily blocks the APD's laser beam.

[0068] Although the example of a mobile printing robot for printing layouts at a construction site has been described, it will be appreciated that the technique can be applied more generally to other types of construction robots used at construction sites, for example a mobile construction robot can perform other position-based construction tasks that require precise positioning.

[0069] In the above description, for the purposes of explanation, numerous specific details are set forth. However, it will be apparent that the disclosed technology can be practiced without any subset of these specific details. In other instances, structures and devices are shown in block diagram form. For example, the disclosed technology is described in some embodiments above with reference to a user interface and specific hardware.

[0070] References herein to "one embodiment," "some embodiments," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least some embodiments of the disclosed technology. The appearance of the phrase "in some embodiments" in various places in this specification are not necessarily all referring to the same embodiment.

[0071] Some of the above detailed descriptions have been presented in terms of processes and symbolic representations of operations on data bits within a computer memory. A process can generally be thought of as a self-consistent sequence of steps leading to a result. The steps can involve physical manipulations of physical quantities. These quantities take the form of electrical or magnetic signals that can be stored, transferred, combined, compared, and otherwise manipulated. These signals can be referred to as being in the form of bits, values, elements, symbols, characters, terms, numbers, or the like.

[0072] These and similar terms can be associated with the appropriate physical quantities and can be considered as labels applied to these quantities. Unless otherwise specifically stated, as is clear from the preceding discussion, it will be understood that throughout this specification, discussions utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" can refer to the actions and processes of a computer system or similar electronic computing system that manipulate and transform data represented as physical (electronic) quantities in the computer system's registers and memory into other data that are similarly represented as physical quantities in the computer system's memory or registers or other such information storage, transmission or display devices.

[0073] The disclosed technology also relates to apparatus for performing the operations herein, which apparatus may be specially constructed for the required purposes, or may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer.

[0074] The disclosed technology can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment containing both software and hardware elements, hi some embodiments, the technology is implemented in software, including but not limited to firmware, resident software, microcode, etc.

[0075] Furthermore, the disclosed technology may take the form of a computer program product accessible from a non-transitory computer usable or computer readable medium providing program code for use by or in connection with a computer or any instruction execution system. As used herein, a computer usable or computer readable medium may be any device that can store, communicate, propagate or transport a program including for use by or in connection with an instruction execution system, apparatus or device.

[0076] A computing system or data processing system suitable for storing and / or executing program code includes at least one processor (e.g., a hardware processor) coupled directly or indirectly to memory elements through a system bus. The memory elements may include local memory employed during the actual execution of the program code, bulk storage, and cache memory that provides temporary storage of at least some of the program code to reduce the number of times the code must be retrieved from bulk storage during execution.

[0077] Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I / O controllers.

[0078] Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the currently available types of network adapters.

[0079] Finally, the processes and displays presented herein may not be inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will appear from the description that follows. In addition, the disclosed technology has not been described with reference to any particular programming language. It will be understood that a variety of programming languages ​​may be used to implement the teachings of the technology described herein.

[0080] The foregoing description of embodiments of the present technology and technology has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the present technology and technology to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The scope of the present technology and technology is not intended to be limited by this detailed description. The present technology and technology can be implemented in other specific forms without departing from its spirit or essential characteristics. Similarly, the specific naming and division of modules, routines, features, attributes, methodologies and other aspects is not required or critical, and mechanisms for implementing the present technology and technology or its features may have different naming, division and / or form. Furthermore, the modules, routines, features, attributes, methodologies and other aspects of the present technology can be implemented as software, hardware, firmware, or any combination thereof. Also, wherever a module, which is an example of a component, is implemented in software, the component may be implemented as a stand-alone program, as part of a larger program, as multiple separate programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and / or any and all other manners known now or in the future in computer programming. Furthermore, the techniques and technology are not limited to implementation in any particular programming language, or in any particular operating system or environment. Accordingly, the disclosure of the techniques and technology is illustrative and not restrictive.

Claims

1. 1. A method of operating a mobile robot, comprising: establishing an initial line-of-sight (LOS) path with an absolute positioning device (APD); receiving, by the mobile robot, APD position information and using the APD position information for navigation of the mobile robot; estimating, by the mobile robot, a relative motion of the mobile robot using a set of sensors of the mobile robot in response to loss of the APD position information; and sending, by the mobile robot, at least one position update to the APD to assist the APD in regaining APD tracking lock on the mobile robot; A method comprising:

2. The method of claim 1 , wherein the transmitting step includes the mobile robot transmitting at least one position update to assist the APD in pointing toward the mobile robot.

3. 2. The method of claim 1, wherein the transmitting step comprises wirelessly transmitting the at least one position update over a wireless network connection that does not require line of sight between the mobile robot and the APD.

4. The method of claim 1 , wherein each position update corresponds to a predicted future position of the mobile robot for interception by the APD.

5. The method of claim 1 , wherein each position update is based on a current position of the mobile robot and further takes into account a response time of the APD to change its pointing direction.

6. 2. The method of claim 1, wherein the transmitting step includes the step of periodically transmitting location updates to the APD until APD lock is restored.

7. 2. The method of claim 1, further comprising the step of regaining the tracking lock between the mobile robot and the APD by pointing to the APD using position update information provided by the mobile robot.

8. The method of claim 1 , wherein the mobile robot is a mobile printing robot for printing a construction layout.

9. 9. The method of claim 8, further comprising the step of, in response to the loss of the APD tracking lock, printing in a shadow area based on an estimated position using latest APD information and the relative motion of the mobile robot.

10. 10. The method of claim 9, wherein the mobile robot determines a position error associated with the relative motion and determines to stop printing if the position error exceeds a preselected maximum position error.

11. 2. The method of claim 1, wherein estimating the relative motion of the mobile robot includes utilizing a state estimator to estimate a current state of the mobile robot, state information that can be used to directly or indirectly determine the relative motion of the mobile robot.

12. 10. The method of claim 1, further comprising the step of altering a speed or pausing movement of the mobile robot to assist the APD in regaining APD tracking lock.

13. 1. A method of operating a mobile robot across a construction surface, comprising: in response to the mobile robot identifying a loss of absolute positioning information from an absolute positioning device (APD), estimating, by the mobile robot, a relative motion of the mobile robot of a construction surface relative to the APD using a sensor suite and a state estimator; estimating an uncertainty in the state estimation performed by the state estimator while the mobile robot moves across the construction surface without accessing the APD, the uncertainty in the state estimation being indicative of a position error; and making a decision to execute at least one location-based service based on the maximum location error being less than or equal to a threshold location error; A method comprising:

14. The method of claim 13, wherein the mobile robot is a mobile printing robot for printing a construction layout.

15. 14. The method of claim 13, further comprising, in response to the loss of APD tracking lock, printing in a shadow area based on an estimated position using updated APD information and the relative motion of the mobile robot.

16. 15. The method of claim 14, wherein the mobile robot of the location-based service is printing, and the mobile printing robot prints if the maximum position error is less than or equal to the threshold position error, and the mobile printing robot stops printing if the position error exceeds a preselected maximum position error.

17. 15. The method of claim 14, wherein a shadow condition corresponds to an object or person interrupting line of sight from the mobile robot to the APD.

18. 15. The method of claim 14, further comprising the step of altering a speed of the mobile robot in a shadow region to reduce the position error.

19. 1. A method for tracking a moving printing robot using an absolute positioning device (APD) that requires a line-of-sight (LOS) path, comprising: establishing an initial LOS path and APD tracking lock between the mobile robot and the APD; receiving, by the mobile robot, APD position information and using the APD position information for navigation of the mobile robot; generating, by the mobile robot, an estimated position of the mobile robot based on relative motion of the mobile robot using a suite of sensors of the mobile robot in response to a loss of the APD tracking lock in a shadow region lacking LOS between the mobile robot and the APD; performing printing by the mobile robot in the shadow region based on the estimated position of the mobile robot if an estimated position error associated with the estimated position is not greater than a maximum position error for printing; sending at least one position update by the mobile robot to the APD to assist the APD in regaining APD tracking lock on the mobile robot; and recovering an APD tracking lock and using the APD position information for navigation of the mobile robot; A method comprising:

20. A mobile printing robot The running system, A printer, and a controller configured to receive absolute position information from an absolute positioning device that tracks the moving printing robot; Equipped with the controller is configured to adjust operation of the mobile printing robot in response to a loss of line of sight (LOS) between the mobile robot and the APD resulting in a loss of APD tracking lock; The controller: A mobile printing robot configured to perform at least one of: 1) generating relative position information and providing position updates to the APD for the APD to regain APD tracking lock; and 2) generating relative position information for printing of the mobile robot during loss of APD tracking lock if a position error associated with an estimated position is not greater than a maximum position error.

Images