Robotic frontloading algorithms for large objects
Patent Information
- Application Number
- JP2024520979
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-10-08
- Filing Date
- 2022-10-11
- Publication Date
- 2024-11-06
Smart Images

Figure 2024540846000001_ABST
Abstract
Description
[Technical field]
[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 253812, filed October 8, 2021, and U.S. Provisional Patent Application No. 63 / 253867, filed October 8, 2021, the contents of each of which are incorporated by reference in their entirety into this application. [Background technology]
[0002] Objects underfoot are not only a nuisance but also a safety hazard. Thousands of people are injured each year due to falls at home. Although clutter on the floor can be dangerous, many people have limited time to deal with clutter in their homes. Automated cleaning or robots may represent an effective solution.
[0003] However, there are various challenges in how to effectively capture and contain some items for transport to a suitable storage or placement location. Objects that are relatively large compared to the containment area cannot simply be swept aside. A set of small, lightweight objects may scatter or roll on first contact, and capturing them one at a time can waste both time and the robot's energy. Highly deformable objects may simply slip or clump away from a rigid capture mechanism. While some objects can be neatly stacked with care, if they are simply dropped and allowed to land, they can become unpleasantly scattered and chaotically stacked.
[0004] Thus, there is a need for pick, store, transport and place algorithms that take into account the geometry and capabilities of the robot's components as well as the potential difficulties associated with particular types of objects. Summary of the Invention [Means for solving the problem]
[0005] In one aspect, a method includes receiving a starting position and attributes of a target object to be lifted by a robot. The robot includes a robot control system, a shovel, grabber arms with grabber pads, and at least one wheel or one track for the robot's locomotion. The method includes determining an object separation strategy that includes using at least one of a reinforcement learning based strategy including rewards and penalties, a rule based strategy, and a dependency on observations, current object state, and sensor data. The method also includes executing the target object separation strategy to separate the target object from other objects. The method includes determining a pick-up strategy, the pick-up strategy including an approach path of the robot to the target object, a gripping height for initial contact with the target object, a gripping pattern for movement of the grabber pad while capturing the target object, and holding positions of the grabber pad and the shovel to secure the target object in a containment area on the robot for transport, the containment area including at least two of the grabber pad arm, the grabber pad, and the shovel. The method also includes executing the pick-up strategy, the pick-up strategy including extending the grabber pad outwardly and forwardly relative to the grabber pad arm and raising the grabber pad to the gripping height, approaching the target object via the approach path and stopping when the target object is located between the grabber pads, executing the gripping pattern to capture the target object within the containment area, and verifying that the target object is within the containment area. With the target object being within the containment area, the method includes applying pressure to the target object with the grabber pad to securely hold the target object within the containment area, and raising at least one of the shovel and the grabber pad holding the target object to the holding position.If the target object is not within the containment area, the method also includes modifying the pick-up strategy using at least one of a different reinforcement learning based strategy, a different rule-based strategy, and a different dependence on observations, current object states, and sensor data, and executing the modified pick-up strategy.
[0006] In one aspect, the robotic system includes a robot including a shovel, a grabber pad arm with a grabber pad, at least one wheel or one track for robot locomotion, a processor, and a memory storing instructions that when executed by the processor enable operation and control of the robot. The robotic system also includes a base station. The robotic system also includes a number of bins for storing objects. The robotic system also includes a robot control system. The robotic system also includes logic that enables the robot and the robotic system to perform the disclosed actions. [Brief description of the drawings]
[0007] To easily identify a particular element or description of an operation, the most significant digit(s) of a reference number refers to the number of the figure in which that element is first introduced. [Figure 1A] FIG. 1A illustrates an embodiment of a robot 100 according to one embodiment. [Figure 1B] FIG. 1B illustrates an embodiment of a robot 100 according to an embodiment. [Figure 1C] FIG. 1C illustrates an embodiment of a robot 100 according to an embodiment. [Figure 1D] FIG. 1D illustrates an embodiment of a robot 100 according to one embodiment. [Figure 2A] FIG. 2A illustrates a lowering shovel position and a lowering grabber position 200a of the robot 100 according to one embodiment. [Figure 2B] FIG. 2B illustrates a lowering shovel position and a lowering grabber position 200b of the robot 100 according to one embodiment. [Figure 2C]FIG. 2C illustrates an elevation shovel position and an elevation grabber position 200c of the robot 100 according to one embodiment. [Figure 2D] FIG. 2D illustrates the robot 100 with the grabber pad 200d extended according to one embodiment. [Figure 2E] FIG. 2E shows the robot 100 with the grabber pad 200e retracted according to one embodiment. [Figure 3A] FIG. 3A illustrates a lowering shovel position and an upper grabber position 300a of the robot 100 according to one embodiment. [Figure 3B] FIG. 3B illustrates the robot 100 in a lowering shovel position and an upper grabber position 300b according to one embodiment. [Figure 3C] FIG. 3C illustrates an elevation shovel position and an elevation grabber position 300c of the robot 100 according to one embodiment. [Figure 4A] FIG. 4A illustrates a lowering shovel position and a lowering grabber position 400a of the robot 100 according to one embodiment. [Figure 4B] FIG. 4B illustrates the robot 100 in a lowering shovel position and an upper grabber position 400b according to one embodiment. [Figure 4C] FIG. 4C illustrates an elevation shovel position and an elevation grabber position 400c of the robot 100 according to one embodiment. [Diagram 5] FIG. 5 illustrates a forward drop position 500 of the robot 100 according to one embodiment. [Figure 6] FIG. 6 illustrates a robot 600 according to one embodiment. [Figure 7] FIG. 7 illustrates a robot 700 according to one embodiment. [Figure 8] FIG. 8 illustrates a robot 800 according to one embodiment. [Figure 9] FIG. 9 illustrates a robot 900 according to one embodiment. [Figure 10] FIG. 10 illustrates a robot 1000 according to one embodiment. [Figure 11] FIG. 11 illustrates a robot 1100 according to one embodiment. [Figure 12]FIG. 12 illustrates a robot 1200 according to one embodiment. [Figure 13] FIG. 13 illustrates a robot 1200 according to one embodiment. [Figure 14] FIG. 14 shows a robot according to an embodiment. [Figure 15] FIG. 15 illustrates a robot 1500 according to one embodiment. [Figure 16] FIG. 16 illustrates a robot 1600 according to one embodiment. [Figure 17] FIG. 17 illustrates a robot 1700 according to one embodiment. [Figure 18A] FIG. 18A illustrates a robot 1800 according to one embodiment. [Figure 18B] FIG. 18B illustrates a robot 1800 according to one embodiment. [Figure 19] FIG. 19 illustrates one embodiment of a robotic control system 1900 for implementing the system components and process steps described herein. [Figure 20] FIG. 20 illustrates a sensor input analysis 2000 according to one embodiment. [Figure 21] FIG. 21 illustrates a primary navigation, collection and arrangement process 2100 according to one embodiment. [Figure 22] FIG. 22 illustrates strategy steps for a separation strategy, pick-up strategy, and drop strategy 2200 according to one embodiment. [Diagram 23] FIG. 23 illustrates a process for determining an action from a policy 2300 according to one embodiment. [Figure 24] FIG. 24 illustrates a placement process 2400 portion of the disclosed algorithm according to one embodiment. [Diagram 25] FIG. 25 illustrates a capture process 2500 portion of the disclosed algorithm according to one embodiment. [Figure 26] FIG. 26 illustrates a first portion of a process diagram 2600 according to one embodiment. [Figure 27A] FIG. 27A illustrates a process for stackable objects 2700 according to one embodiment. [Figure 27B] FIG. 27B illustrates a process for stackable objects 2700 according to one embodiment. [Figure 27C] FIG. 27C illustrates a process for stackable objects 2700 according to one embodiment. [Figure 27D] FIG. 27D illustrates a process for stackable objects 2700 according to one embodiment. [Figure 28A] FIG. 28A illustrates a process for a large, highly deformable object 2900 according to one embodiment. [Figure 28B] FIG. 28B illustrates a process for a large, highly deformable object 2900 according to one embodiment. [Figure 28C] FIG. 28C illustrates a process for a large, highly deformable object 2900 according to one embodiment. [Figure 28D] FIG. 28D illustrates a process for a large, highly deformable object 2900 according to one embodiment. [Figure 29A] FIG. 29A illustrates a process for a large, highly deformable object 2900 according to one embodiment. [Figure 29B] FIG. 29B illustrates one aspect of the subject matter according to one embodiment. [Figure 29C] FIG. 29C illustrates a process for a large, highly deformable object 2900 according to one embodiment. [Figure 29D] FIG. 29D illustrates a subject embodiment according to one embodiment. [Figure 30A] FIG. 30A illustrates a process for a small, easily scattering object 3000 according to one embodiment. [Figure 30B] FIG. 30B illustrates a subject aspect according to one embodiment. [Figure 30C] FIG. 30C illustrates the process for a small, easily scattering object 3000 according to one embodiment. [Figure 30D] FIG. 30D illustrates a subject embodiment according to one embodiment. [Diagram 31] FIG. 31 illustrates a robotic system 3100 according to one embodiment. [Diagram 32]FIG. 32 illustrates a robotic process 3200 according to one embodiment. [Diagram 33] FIG. 33 illustrates another robotic process 3300 according to one embodiment. [Diagram 34] FIG. 34 shows a state space map 3400 of a robotic system according to one embodiment. [Diagram 35] FIG. 35 illustrates a robot control algorithm 3500 for a robotic system according to one embodiment. [Diagram 36] FIG. 36 illustrates a robot control algorithm 3600 for a robotic system according to one embodiment. [Figure 37] FIG. 37 illustrates a robot control algorithm 3700 according to one embodiment. [Figure 38] FIG. 38 illustrates a robotic system 3800 according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] An embodiment of a robotic system is disclosed in which a robot is operated to navigate an environment using cameras to map the type, size and location of toys, clothing, obstacles and other objects. The robot includes a neural network that identifies object type, size and location based on input from sensing systems such as data from an inertial measurement unit (IMU), lidar, odometry and actuator force feedback sensors, as well as images from a front camera, a rear camera, front and rear left and right stereo cameras or other camera configurations. The robot selects a specific object to pick up, performs a path plan and navigates to a point adjacent and facing the target object. An actuated grabber pad arm moves other objects out of the way and operates the grabber pad to move the target object to the shovel to be delivered. While the robot navigates to the next location in the planned path, such as a placement destination, the shovel tilts slightly upwards and, if necessary, the grabber pad can close in front to keep the object in place.
[0009] In some embodiments, the system may include a robotic arm that can reach and grab elevated objects and move them down to the shovel. Some embodiments may also utilize a companion "portable elevator" robot to lift the main robot onto a countertop, table, or other elevated surface and then lower it to the floor. Some embodiments may utilize an up / down vertical lift (e.g., a scissor lift) to change the height of the shovel when dropping items onto a bin, shelf, or other high or elevated location.
[0010] In some embodiments, -Right and left rotating brush on the actuator arm that pushes the object into the shovel; Actuated grippers that grasp objects and move them to the shovel; A rotating wheel with flaps that pushes objects into the shovel from above; one servo or other actuator to lift the front shovel into the air and another actuator to tilt the shovel forward and down to drop the object into the container; A variation of the scissor lift that lifts the excavator and gradually tilts it backwards as the height increases; A ramp on the bin with a hinged front shovel that allows the robot to simply push the item up the ramp and the object will drop into the bin by gravity at the top of the ramp; A storage bin on the robot for additional carrying capacity so that objects can be pushed up a ramp into the bin instead of using a front shovel and the bin tilts up and back like a dump truck to drop the items into the container; One or more of the following components may also be used:
[0011] The robotic system may be utilized for automatic cleaning of surfaces where items left on the surface are automatically dumped into a bin on a regular schedule. In one particular embodiment, the system may be utilized to automatically clean a child's play area (e.g., at a home, school, or business) where toys and / or other items are automatically returned to bins specific to different types of objects after the child is done playing. In other particular embodiments, the system may be utilized to automatically pick up clothes from the floor and organize the clothes into a laundry basket for washing or to automatically pick up trash from the floor and place it, for example, by type, into a box or recycling bin. In general, the system may be deployed to efficiently pick up a wide variety of different objects from a surface and may learn to pick up new types of objects.
[0012] Some objects have attributes that make them difficult to manipulate and transport using the grabber pad, grabber pad arms, and shovel. These difficulties can be overcome by following an algorithm that specifically configures the attributes that create the difficulties. For example, an object that is too large to fit completely within the shovel can be partially secured within the shovel by placing the grabber pad over the object's center of gravity, and the grabber pad arms can be lowered slightly, causing the grabber pad to exert a slight downward pressure to hold the object securely within the shovel or against the bottom or edge of the shovel. Small, lightweight, easily scattered objects such as plastic construction blocks or marbles can be scattered if swept too quickly with the grabber pad. Alternatively, the pad can contact such objects at a height point where direct and constant pressure from the pad acts on the objects to press them firmly against the floor, rather than sweeping them along the floor. In such cases, a reduced force may be applied initially and then increased as the object begins to move, or a series of gentle tapping motions may be employed by the grabber pad to provide a horizontal force that moves the object while avoiding a downward force that would increase friction with the floor and impede the object's movement. While many objects can simply be dropped from the shovel at their designated destination, such as a bin, a class of flat, stackable objects, such as books, CDs, DVDs, narrow boxes, etc., are easily arranged and neatly placed by being lifted above previously stacked objects and manipulated from the shovel by the grabber pad. Algorithms for handling objects such as these are disclosed herein.
[0013] 1A-1D show a robot 100 according to one embodiment. FIG. 1A shows a side view of the robot 100, and FIG. 1B shows a top view. The robot 100 may include a chassis 102, a mobility system 104, a sensing system 106, a capture and stow system 108, and a robot control system 1900. The capture and stow system 108 may further include a shovel 110, a shovel arm 112, a shovel arm pivot point 114, two grabber pads 116, two grabber pad arms 118, and two pad arm pivot points 122.
[0014] The chassis 102 may support and house other components of the robot 100. The mobility system 104 may include caterpillar tracks, conveyor belts, and the like in addition to the wheels shown, as is well understood in the art. The mobility system 104 may further include motors, servos, or other sources of rotational or kinetic energy to propel the robot 100 along its desired path. The components of the mobility system 104 may be mounted on the chassis 102 for the purpose of moving the entire robot without impeding or inhibiting the range of motion required by the capture and containment system 108. Elements of the sensing system 106, such as cameras, lidar sensors, or other components, may be mounted on the chassis 102 in positions that provide the robot 100 with a clear line of sight around the environment in at least some configurations of the chassis 102, shovel 110, grabber pad 116, and grabber pad arm 118 relative to one another.
[0015] The chassis 102 may house and protect all or a portion of the robotic control system 1900 (some of which may be accessed via a connection to a cloud server), including in some embodiments a processor, memory, and connections to the mobility system 104, the sensing system 106, and the capture and containment system 108. The chassis 102 may include other electronic components such as batteries, wireless communication devices, etc., as is well understood in the field of robotics. The robotic control system 1900 may function as described in more detail with respect to FIG. 19. The mobility system 104 and / or the robotic control system 1900 may incorporate motor controllers used to control the speed, direction, position, and smooth movement of the motors. Such controllers may also be used to detect force feedback and limit maximum current (provide overcurrent protection) to ensure safety and prevent damage.
[0016] The capture and containment system 108 may include a shovel 110, a shovel arm 112, a shovel arm pivot point 114, a grabber pad 116, a grabber pad arm 118, a pad pivot point 120, and a pad arm pivot point 122. In some embodiments, the capture and containment system 108 may include two grabber pad arms 118, a grabber pad 116, and their pivot points. In other embodiments, the grabber pad 116 may be attached directly to the shovel 110 without the grabber pad arm 118. Such embodiments are presented later in this disclosure.
[0017] The geometry of the shovel 110 and the placement of the grabber pad 116 and grabber pad arm 118 relative to the shovel 110 may describe a containment area in which objects are securely transported, as shown more clearly in Figures 2A-2E. As shown in Figures 2A-2C, servos, direct current (DC) motors or other actuators at the shovel arm pivot point 114, pad pivot point 120 and pad arm pivot point 122 may be used to adjust the placement of the shovel 110, grabber pad 116 and grabber pad arm 118 between a fully lowered shovel and a raised shovel and grabber position.
[0018] The connection points shown between the shovel arm and the grabber pad arm are exemplary locations and are not intended to limit the physical location of such connection points, which may be made in a variety of locations suitable for the chassis and arm structure and intended use application.
[0019] In some embodiments, gripping surfaces may be configured on the side of the grabber pad 116 that faces the object to be lifted. These gripping surfaces may provide cushioning, grit, elasticity, or some other feature that enhances friction between the grabber pad 116 and the object to be captured and contained. In some embodiments, the grabber pad 116 may include suction cups to better grip objects with smooth, flat surfaces. In some embodiments, the grabber pad 116 may be configured with sweeping bristles. These sweeping bristles may assist in moving small objects from the floor onto the shovel 110. In some embodiments, the sweeping bristles may be angled downward and inward from the grabber pad 116 so that as the grabber pad 116 sweeps an object toward the shovel 110, the sweeping bristles form a ramp, with the frontmost bristles sliding under the object and directing it upward toward the grabber pad 116, facilitating capture of the object within the shovel and reducing the tendency of the object to be pressed against the floor, increasing its friction, making it more difficult to move.
[0020] 1C and 1D show side and top views, respectively, of the chassis 102 along with general connectivity of the components of the mobility system 104, the sensing system 106, and the communication unit 134 in the context of the robotic control system 1900. In some embodiments, the communication unit 134 may include a network interface 1912, which is described in more detail with respect to the robotic control system 1900.
[0021] In one embodiment, the mobility system 104 may include a right front wheel 136, a left front wheel 138, a right rear wheel 140, and a left rear wheel 142. The robot 100 may have front wheel drive, in which the right front wheel 136 and the left front wheel 138 are actively driven by one or more actuators or motors, while the right rear wheel 140 and the left rear wheel 142 passively rotate on an axle while supporting the rear of the chassis 102. In another embodiment, the robot 100 may have rear wheel drive, in which the right rear wheel 140 and the left rear wheel 142 are actuated, and the front wheels passively rotate. In another embodiment, each wheel may be actively driven by a separate motor or actuator.
[0022] The sensing system 106 may further include cameras 124, such as a forward camera 126 and a rearward camera 128, a light detection and ranging (LIDAR) sensor, such as a LIDAR sensor 130, and an inertial measurement unit (IMU) sensor, such as an IMU sensor 132. In some embodiments, the forward camera 126 may include a front right camera 144 and a front left camera 146. In some embodiments, the rearward camera 128 may include a rearward left camera 148 and a rearward right camera 150.
[0023] Additional embodiments of robots that may be used to perform the disclosed algorithms are shown in Figures 2A-2E, 3A-3C, 4A-4C, 5, 6, 7, 8, 9, 10, 11, 12, and 13. Figures 15, 16, and 17, and Figures 18A and 18B show exemplary mechanical embodiments of robots that may be used to perform the disclosed algorithms.
[0024] Figure 2A shows the robot 100 as introduced with respect to Figure 1A positioned in a lowered shovel and grabber position 200a. In this configuration, the grabber pad 116 and grabber pad arm 118 are positioned in a lowered grabber position 204, and the shovel 110 and shovel arm 112 are positioned in a lowered shovel position 206 at the front 202 of the robot 100. In this position, the shovel 110 and grabber pad 116 roughly describe the storage area 210 shown.
[0025] 2B shows the robot 100 with a lowered shovel position and an elevated grabber position 200b. Through the action of servos or other actuators at the pad pivot point 120 and pad arm pivot point 122, the grabber pad 116 and grabber pad arm 118 are raised to an elevated grabber position 208 while the shovel 110 and shovel arm 112 maintain the lowered shovel position 206. In this configuration, the grabber pad 116 and shovel 110 may roughly describe the illustrated containment area 210 where objects taller than the height of the shovel 110 may remain within the shovel 110 and be held in place by the pressure exerted by the grabber pad 116.
[0026] The pad arm pivot point 122, pad pivot point 120, shovel arm pivot point 114, and shovel pivot point 502 (as shown in FIG. 5) may provide the robot 100 with a range of motion for these components that exceeds that shown in this application. The positions shown in the disclosed figures are exemplary only and are not meant to indicate limits to the range of motion of the robot's components.
[0027] 2C shows the robot 100 with an elevated shovel position and an elevated grabber position 200c. The grabber pad 116 and grabber pad arm 118 are in an elevated grabber position 208, while the shovel 110 and shovel arm 112 are in an elevated shovel position 212. In this position, the robot 100 may be able to drop objects from the shovel 110 and grabber pad arm 118 onto an area at the rear 214 of the robot 100.
[0028] As shown in Figures 27A-30C below, the transport positions may involve disposing the grabber pad 116, grabber pad arm 118, shovel 110 and shovel arm 112 in relative configurations between the extremes of a lowered shovel and grabber position 200a and an raised shovel and grabber position 200c.
[0029] 2D shows the robot 100 with an extended grabber pad 200d. Through the action of a servo or other actuator at the pad pivot point 120, the grabber pad 116 may be configured as an extended grabber pad 216 to allow the robot 100 to approach objects as wide as or wider than the robot chassis 102 and shovel 110. In some embodiments, the grabber pads 116 may be able to rotate approximately 360 degrees to rest parallel to and outboard of their associated grabber pad arms 118 when fully extended.
[0030] 2E shows the robot 100 with the grabber pads 200e retracted. The closed grabber pads 218 may loosely define the containment area 210 via their position relative to the shovel 110. In some embodiments, the grabber pads 116 may be able to rotate further than shown, through approximately 360 degrees, to rest parallel to and inside the sidewall of the shovel 110.
[0031] 3A-3C show a robot 100 as introduced in connection with FIGS. 1A-2E. In such an embodiment, the grabber pad arm 118 may be controlled by a servo or other actuator at the same connection point 302 as the shovel arm 112 to the chassis 102. The robot 100 may be shown as being disposed in a lower shovel and grabber position 300a, a lower shovel and grabber position 300b, and a raise shovel and grabber position 300c. The robot 100 may be configured to perform the algorithms disclosed herein.
[0032] The connection points shown between the shovel arm 112 / grabber pad arm 118 and the chassis 102 are exemplary locations and are not intended to limit the physical location of such connection points. Such connections may be made at a variety of locations depending on the configuration and intended use of the chassis 102 and arms.
[0033] 4A-4C show a robot 100 such as that introduced in connection with FIGS. 1A-2E. In such an embodiment, the grabber pad arm 118 may be controlled by a servo (or other actuator) at a connection point 402 with the chassis 102 that is different from the connection point that controls the shovel arm 112. The robot 100 may be shown as being disposed in a lower shovel and grabber position 400a, a lower shovel and grabber position 400b, and a raise shovel and grabber position 400c. The robot 100 may be configured to perform the algorithms disclosed herein.
[0034] The different connection points 402 between the shovel arm and the chassis and the grabber pad arm and the chassis shown are exemplary locations and are not intended to limit the physical locations of these connection points. Such connections may be made at various locations depending on the structure and intended use of the chassis and arms.
[0035] 5 shows a robot 100 such as that previously introduced at a front drop position 500. The arms of the robot 100 can be positioned to form a receiving area 210 as previously described.
[0036] The robot 100 may be configured with a shovel pivot point 502 where the shovel 110 is connected to the shovel arm 112. The shovel pivot point 502 allows the shovel 110 to tilt forward and downward while the shovel arm 112 is raised, allowing objects in the storage area 210 to slide out and be placed in an area on the front side 202 of the robot 100.
[0037] FIG. 6 illustrates a robot 600 according to one embodiment. The robot 600 may be configured to perform the algorithms disclosed herein. The robot 600 may include a number of features not described in detail here but introduced previously. In addition to these, the robot 600 may include a raised rear shovel arm pivot point 602, a split shovel arm 604, a right telescoping grabber pad arm 606, a left telescoping grabber pad arm 608, a linear actuator 610, a wrist actuator 612, a right front drive wheel 614, a left front drive wheel 616, and rear casters 618.
[0038] The raised rear shovel arm pivot point 602 may be connected to a split shovel arm 604 to raise and lower the shovel 110. This configuration may allow the front camera 126 to capture images without obstruction from the shovel arm. A groove or slot in the chassis 102 of the robot 600 may allow portions of the split shovel arm 604 to move without being hindered by the dimensions of the chassis 102.
[0039] The grabber pad arms may include a right telescoping grabber pad arm 606 and a left telescoping grabber pad arm 608. In this manner, the grabber pad arms may extend (increase in length) and retract (decrease in length). This movement may be generated by a linear actuator 610 configured as part of the grabber pad arms. A wrist actuator 612 is located at the pad pivot point 120 and allows the grabber pads to pivot and push objects into the shovel 110.
[0040] The mobility system in one embodiment may include a right front drive wheel 614, a left front drive wheel 616, and a rear caster 618. The front drive wheels provide the motive force that allows the robot 600 to navigate its environment, while the rear caster 618 may provide support to the rear of the robot 600 without limiting its range of motion. The right front drive wheel 614 and the left front drive wheel 616 may be independently actuated, allowing the robot 600 to turn in place and traverse floors.
[0041] 7 shows a robot 700 according to one embodiment. The robot 700 can be configured to perform the algorithms disclosed herein. The robot 700 can include many features not described in detail herein but introduced above. In addition, the robot 700 can include a single grabber pad 702, a single telescoping grabber pad arm 704, a linear actuator 706, a bearing 708, an opposing grabber pad arm pivot point 710, a pivot joint 712, and an opposing grabber pad arm 714.
[0042] The single grabber pad 702 may be raised and lowered by the grabber pad arm, plus extended and retracted by the action of a single telescoping grabber pad arm 704 driven by a linear actuator 706. A bearing 708 at the opposing pad arm pivot point 710 and the sliding joint 712 of the opposing grabber pad arm 714 may allow the force of the linear actuator 706 to be transferred through the single grabber pad 702, allowing symmetrical movement in both grabber pad arms while one arm is being actively moved. In another embodiment, the single grabber pad 702 may be positioned by the synchronous actuation of the right telescoping grabber pad arm 606 and the left telescoping grabber pad arm 608 as shown in FIG.
[0043] 8 shows a robot 800 according to one embodiment. The robot 800 may be configured to perform the algorithms disclosed herein. The robot 800 may include many features not described in detail herein but introduced above. In addition to these, the robot 800 may include a raised rear shovel arm pivot point 802 and a single shovel arm 804.
[0044] A raised rear shovel arm pivot point 802 may connect to a single shovel arm 804 to raise and lower the shovel 110. This configuration allows the forward camera 126 to capture images without interference from the shovel arm. A groove or slot in the chassis 102 of the robot 800 may allow the single shovel arm 804 to move without being hindered by the dimensions of the chassis 102.
[0045] 9 shows a robot 900 according to one embodiment. The robot 900 may be configured to perform the algorithms disclosed herein. The robot 900 may include many features not described in detail herein but introduced above. In addition to these, the robot 900 may include a single telescoping shovel arm 902 and a linear actuator 904.
[0046] A single telescoping shovel arm 902 may enable the shovel 110 to move away from and towards the chassis 102 by extension and retraction powered by a linear actuator 904 .
[0047] 10 illustrates a robot 1000 according to an embodiment. The robot 1000 may be configured to perform the algorithms disclosed herein. The robot 1000 may include many features not described in detail herein but introduced above. In addition to these, the robot 1000 may include a shovel mounted grabber pad arm 1002 and a shovel mounted pad arm pivot point 1004.
[0048] Rather than connecting to the chassis 102 as in other embodiments disclosed herein, the shovel mounted grabber pad arm 1002 may be connected to a shovel mounted pad arm pivot point 1004 located on the shovel 110. An actuator at the shovel mounted pad arm pivot point 1004 may allow the shovel mounted grabber pad arm 1002 to be raised and lowered relative to the shovel 110 in addition to being raised and lowered with the shovel 110.
[0049] 11 illustrates a robot 1100 according to one embodiment. The robot 1100 may be configured to perform the algorithms disclosed herein. The robot 1100 may include many features not described in detail herein but introduced above. In addition to these, the robot 1100 may include a shovel-mounted grabber pad 1102 and a shovel-mounted pad pivot point 1104.
[0050] Rather than connecting to a grabber pad arm, a shovel-mounted grabber pad 1102 may be connected to the shovel 110 at a shovel-mounted pad pivot point 1104. A wrist actuator at the shovel-mounted pad pivot point 1104 may allow the shovel-mounted grabber pad 1102 to pivot in and out of the shovel 110 to move objects into the shovel 110.
[0051] 12 shows a robot 1200 according to an embodiment. The robot 1200 may be configured to perform the algorithms disclosed herein. The robot 1200 may include many features not described in detail herein but introduced above. In addition to these, the robot 1200 may include a split shovel arm 1202, a raised rear shovel arm pivot point 1204, a shovel pivot point 1206, and a shovel pivot actuator 1208.
[0052] The robot 1200 may have a split shovel arm 1202 that connects to the chassis 102 at two raised rear shovel arm pivot points 1204. The actuators at each raised rear shovel arm pivot point 1204 may be actuated synchronously to raise and lower the shovel 110.
[0053] The shovel 110 may be connected to a split shovel arm 1202 at a shovel pivot point 1206. A shovel pivot actuator 1208 at the shovel pivot point 1206 may cause the shovel 110 to be raised by the split shovel arm 1202 and tilted forward and downward to a forward drop position 500 as shown in FIG.
[0054] 13 shows a robot 1300 according to one embodiment. The robot 1300 may be configured to execute the algorithms disclosed herein. The robot 1300 may include many features not described in detail herein but introduced above. In addition to these, the robot 1300 may include a mobility system 104 having a track 1302.
[0055] These tracks 1302 may improve the mobility and stability of the robot 1300 on some surfaces. Each of the left and right tracks 1302 may be independently actuated to allow the robot 1300 to turn while traversing or remain in place.
[0056] 14 illustrates a robot 1400 according to one embodiment. The robot 1400 may be configured to perform the algorithms disclosed herein. The robot 1400 may include many features not described in detail herein but introduced above. In addition, the robot 1400 may include a chassis 102, a shovel 110, a single grabber pad arm 1402, a single pad arm pivot point 1404, a single grabber pad 1406, and a single pad pivot point 1408.
[0057] The single grabber pad arm 1402 is connected to the chassis 102 at a single pad arm pivot point 1404, allowing the single grabber pad arm 1402 to move relative to the robot 1400. The single grabber pad arm 1402 is connected to the single grabber pad arm 1402 at a single pad pivot point 1408, allowing the single grabber pad 1406 to move relative to the single grabber pad arm 1402. Servos, DC motors, or other actuators at the single pad arm pivot point 1404 and single pad pivot point 1408 can drive the movement of the single grabber pad arm 1402 and single grabber pad 1406 to move objects within the shovel 110.
[0058] 15 illustrates a robot 1500 according to one embodiment. The robot 1500 may be configured to perform the algorithms disclosed herein. The robot 1500 may include, among other features, the chassis 102, the shovel 110, the shovel arm 112, the shovel arm pivot point 114, the grabber pad 116, the grabber pad arm 118, the pad pivot point 120, the pad arm pivot point 122, the front right camera 144, the front left camera 146, and the different connection points 402 for the shovel arm 112 and the grabber pad arm 118, as previously described.
[0059] 16 illustrates a robot 1600 according to one embodiment. The robot 1600 may be configured to perform the algorithms disclosed herein. The robot 1600 may include, among other features, grabber pads 116, right telescoping grabber pad arm 606, left telescoping grabber pad arm 608, the same connection points 302 between each shovel arm 112 and the grabber pad arms, and linear actuators 610 for extending the right telescoping grabber pad arm 606, the left telescoping grabber pad arm 608, and the grabber pads 116 synchronously or separately, as previously described.
[0060] 17 illustrates a robot 1700 according to one embodiment. The robot 1700 may be configured to perform the algorithms disclosed herein. The robot 1700 may include, among other features, a single grabber pad 702, a single telescoping grabber pad arm 704, a linear actuator 706, a bearing 708, a contralateral pad arm pivot point 710, and a contralateral grabber pad arm 714, as previously described.
[0061] 18A and 18B show a robot 1800 according to one embodiment. The robot 1800 may be configured to perform the algorithms disclosed herein. The robot 1800 may include, in addition to other features, the mobility system 104, the lidar sensor 130, and the tracks 1302, as previously described.
[0062] The robot features depicted in connection with Figures 6-18B may be present in various combinations in a particular embodiment, and as one of ordinary skill in the art will readily appreciate, these figures are not intended to limit the arrangement of the features described.
[0063] 19 illustrates an embodiment of a robotic control system 1900 for implementing the system components and process steps described herein. The robotic control system 1900 and some or all of its operational logic may be contained within the physical components of the robot and / or within a cloud server in communication with the robot. In one embodiment, aspects of the robotic control system 1900 on the cloud server control multiple robots at once, allowing multiple robots to work cooperatively within a workspace.
[0064] Input devices 1904 (e.g., of a robot or companion device such as a mobile phone or personal computer) include transducers that convert physical phenomena into machine-internal signals, typically electrical, optical or magnetic signals. The signals can be wireless in the form of electromagnetic radiation in the radio frequency (RF) range but potentially in the infrared or optical range. Examples of input devices 1904 are contact sensors that respond to touch or physical pressure from an object or proximity to a surface of an object, mice that respond to movement through space or across a plane, microphones that convert vibrations in a medium (typically air) into device signals, and scanners that convert optical patterns on a two-dimensional or three-dimensional object into device signals. Signals from input devices 1904 are provided to memory 1906 via various machine signal conductors (e.g., buses or network interfaces) and circuits.
[0065] The memory 1906 is typically known as a first or second level memory device and provides storage of signals received from the input device 1904, instructions and information for controlling the operation of the central processing unit or CPU 1902, and signals from the storage device 1910 (through configuration of things or state of things). The memory 1906 and / or storage device 1910 store computer executable instructions, which when applied and executed by the CPU 1902, form logic 1914 implementing embodiments of the processes disclosed herein. The logic 1914 may include portions of a computer program along with configuration data executed by the CPU 1902 or another processor. The logic 1914 may include one or more machine learning models 1916 used to perform the disclosed operations. In an embodiment, portions of the logic 1914 may also reside on a mobile or desktop computing device accessible by the user to facilitate direct user control of the robot.
[0066] The information stored in memory 1906 is typically directly accessible to the device's CPU 1902. Signals input to the device affect the operation of the robotic control system 1900 by causing a reconfiguration of the material / energy states within memory 1906, effectively creating a new mechanical configuration, and configuring the CPU 1902 with control signals (instructions) and data provided in conjunction with the control signals.
[0067] A second or third level storage device 1910 may provide slower but larger capacity mechanical memory functions. Examples of storage devices 1910 are hard disks, optical disks, large capacity flash memory or other non-volatile memory technologies, and magnetic memories.
[0068] In one embodiment, memory 1906 may include virtual storage accessible via a connection to a cloud server using network interface 1912, as described below. In such an embodiment, some or all of logic 1914 may be stored and processed remotely.
[0069] The CPU 1902 may change the configuration of the memory 1906 by signals in the storage device 1910. That is, the CPU 1902 may read data and instructions from the storage device 1910 in the memory 1906, which affect the operation of the CPU 1902 as instruction and data signals, and may also be provided to the output device 1908. The CPU 1902 may change the contents of the memory 1906 by sending signals to the machine interface of the memory 1906 to change its internal configuration, and then converting the signals to the storage device 1910 to change its material internal configuration. That is, the data and instructions may be backed up from the memory 1906, which is often volatile, to the storage device 1910, which is often non-volatile.
[0070] Output device(s) 1908 are transducers that convert signals received from memory 1906 into physical phenomena, such as vibrations in the air or patterns of light or vibrations on a mechanical display (i.e., tactile devices) or patterns of ink or other material (i.e., printers and 3D printers).
[0071] The network interface 1912 receives signals from the memory 1906 and converts them to electrical, optical or wireless signals for transmission to other machines, typically over a machine network. The network interface 1912 also receives signals from the machine network and converts them to electrical, optical or wireless signals for transmission to the memory 1906. The network interface 1912 may enable the robot to communicate with cloud servers, mobile devices, other robots, and other network-enabled devices.
[0072] 20 illustrates a sensor input analysis 2000 according to one embodiment. The sensor input analysis 2000 may inform the robot 100 of the dimensions of its immediate environment 2002 and the position of itself and other objects within that environment 2002.
[0073] As previously described, the robot 100 includes a sensing system 106. The sensing system 106 may include at least one of a camera 124, an IMU sensor 132, a lidar sensor 130, an odometry 2004, and an actuator force feedback sensor 2006. These sensors may capture data describing an environment 2002 around the robot 100.
[0074] Image data 2008 from the camera 124 may be used for object detection and classification 2010. Object detection and classification 2010 may be performed by algorithms and models configured in the robot control system 1900 of the robot 100. In this manner, the characteristics and types of objects in the environment 2002 may be identified.
[0075] Image data 2008, object detection and classification 2010 data, and other sensor data 2012 may be used for global / local map updates 2014. A global and / or local map may be stored by the robot 100 and represent its knowledge of the dimensions and objects within its cleaning environment 2002. This map may be used in navigation and strategy decisions related to the cleaning task.
[0076] The robot may use a combination of cameras 124, lidar sensors 130, and other sensors to maintain a global or local area map of the environment and position itself within it. In addition, the robot may perform object detection and object classification, and generate a visual recognition fingerprint for each object. The robot may utilize stereo cameras in conjunction with machine learning / neural network software architectures (e.g., semi-supervised or supervised convolutional neural networks) to efficiently classify the type, size, and location of different objects on the map of the environment.
[0077] The robot may determine the relative distance and angle to each object, which may then be used to locate the object on a global or local area map. The robot may utilize both forward and rear facing cameras to scan both in front of and behind the robot.
[0078] The image data 2008, object detection and classification 2010 data, other sensor data 2012, and global / local map update 2014 data may be stored as observations, current robot state, current object state, and sensor data 2016. The observations, current robot state, current object state, and sensor data 2016 may be used by the robot control system 1900 of the robot 100 in determining a navigation path and a task strategy.
[0079] 21 illustrates a main navigation, collection and placement process 2100 according to one embodiment. According to some examples, the method includes heading to a target object at block 2102. For example, a robot 100 as introduced in connection with FIG. 1A heads to the target object using a local or global map to navigate to a location near the target object depending on the observations, current robot state, current object state and sensor data 2016 identified as shown in FIG.
[0080] According to some examples, the method includes determining an object separation strategy in block 2104. For example, the robot control system 1900 shown in FIG. 1A may determine an object separation strategy to separate the target object from other objects in the environment based on the object's location in the environment. The object separation strategy may be determined using a machine learning model or a rule-based approach depending on the observations, the current robot state, the current object state, and the sensor data 2016 identified as shown in FIG. 20. In some cases, object separation may not be required and the related block may be skipped. For example, object separation may not be required in an area where there are several items to be picked up and moved or where furniture, walls, or other obstacles that may impede picking up the target object are not close to each other.
[0081] In some cases, there may be no effective separation strategy. For example, the robot control system 1900 shown in FIG. 1A may be unable to determine an effective separation strategy. If it is determined at decision block 2106 that there is no effective separation strategy, the target object may be marked as unsuccessfully picked at block 2120. The main navigation, collect and place process 2100 may proceed to block 2128 where the next target object is determined.
[0082] If a valid separation strategy is determined in decision block 2106, the robot 100 as introduced in relation to FIG. 1A may execute an object separation strategy to separate the target object from other objects in block 2108. The separation strategy may follow the strategy steps of separation, pick up and drop strategies 2200 shown in FIG. 22. The separation strategy may be a reinforcement learning based strategy that uses rewards and penalties in addition to the observations, current robot state, current object state and sensor data 2016 or a rule based strategy that depends on the observations, current robot state, current object state and sensor data 2016 as specified in FIG. 20. The reinforcement learning based strategy that depends on rewards and penalties is described in more detail with reference to FIG. 22.
[0083] The rule-based strategies may use conditional logic to determine the next step based on the observations, current robot state, current object state, and sensor data 2016 as developed in FIG. 20. Each rule-based strategy may have a list of available actions that it may consider. In one embodiment, a motion collision avoidance system may be used to determine the motion range associated with each action. The rule-based strategies for object separation may include: Navigating the robot to a position facing the target object to be separated but far enough away to open the grabber pad arm and grabber pad and lower the shovel; Open the grabber pad arm and grabber pad, lower the grabber pad arm and grabber pad, and lower the shovel; Rotate the robot slightly into place so that the target object is centered in the front view, - Open the grabber pad arms and grabber pads so that they are slightly wider than the target object; Slowly advance the grabber pad arm and the end of the grabber pad until they are positioned beyond the target object; -Closing the grabber pad arms and grabber pads slightly in a V-shape so that the grabber pad arms and grabber pads surround the target object; Move 100cm back and move the object into an open space; may include.
[0084] According to some examples, the method includes determining whether the separation was successful at decision block 2110. For example, the robot control system 1900 shown in FIG. 1A may determine whether the target object was successfully separated. If the separation strategy was not successful, the target object may be marked as unsuccessfully picked at block 2120. The main navigation, collect and place process 2100 proceeds to block 2128 where the next target object is determined. In some embodiments, rather than determining the next target object, a different strategy may be selected for the same target object. For example, if the target object cannot be separated by the current separation strategy, a different separation strategy may be selected and separation may be retried.
[0085] If the target object is successfully isolated, the method includes determining a pick-up strategy at block 2112. For example, the robot control system 1900 shown in FIG. 1A may determine the pick-up strategy. The pick-up strategy for a particular target object and location may be determined using a machine learning model or a rule-based approach depending on the observations, the current robot state, the current object state, and the sensor data 2016 identified as shown in FIG.
[0086] In some cases, there may be no valid pick-up strategy. For example, the robot control system 1900 shown in FIG. 1A may be unable to determine a valid pick-up strategy. If there is no valid pick-up strategy as determined in decision block 2114, the target object may be marked as unsuccessfully picked up in block 2120, as described above. A pick-up strategy may be: -Initial default positions for grabber pad arms and shovel before starting a pick up, Detecting hard surface vs. carpeted floor types, which can affect pick-up strategies; Final shovel and grabber pad arm position for transport, needs to be taken into consideration.
[0087] If there is a valid pick-up strategy determined in decision block 2114, the robot 100 as introduced in relation to FIG. 1A may execute the pick-up strategy in block 2116. The pick-up strategy may follow the strategy steps of separation strategy, pick-up strategy and drop strategy 2200 shown in FIG. 22. The pick-up strategy may be a reinforcement learning based strategy or a rule-based strategy that depends on the observations, the current robot state, the current object state and the sensor data 2016, as specified as shown in FIG. 20. The rule-based strategy for picking up an object may be: Navigating the robot to a position facing the target object but far enough away to open the grabber pad arms and grabber pads and lower the shovel; Open the grabber pad arm and grabber pad, lower the grabber pad arm and grabber pad, and lower the shovel; Rotating the robot slightly in place so that the target object is centered in the front view; - moving forward until the object is within the "pick-up zone" relative to the end of the shovel; - Identifying the center position of the object relative to the shovel on the right, left or center; -If it is on the right side, close the right grabber pad arm and grabber pad first, then close the left grabber pad arm and grabber pad later -If not, close the left grabber pad arm and grabber pad first, then close the right grabber pad arm and grabber pad afterwards. - Determining whether the target object is pushed into the shovel normally; - If yes, the pick up was successful - If no, lift the grabber pad arm and grabber pad and try again with the appropriate part of the strategy may include
[0088] According to some examples, the method includes determining whether the target object has been picked up at decision block 2118. For example, the robot control system 1900 shown in FIG. 1A may determine whether the target object has been picked up. A successful pick-up is determined by: Object detection within the area of the shovel and grabber pad arm (i.e., the containment area as previously described) to determine if an object is within the shovel / grabber pad arm / containment area; Force feedback from the actuator force feedback sensor indicating that the object is being held by the grabber pad arm; Tracking the movement of objects during their pick-up into the shovel's area and maintaining the state of those objects in memory (memory is often relied upon since objects may no longer be visible when the shovel is in its carrying position), Detection of an increase in the weight of the shovel during a lift, indicating that an object is in the shovel; Use of a classification model to determine whether an object is inside the shovel or not; Use of force feedback, weight augmentation and / or dedicated cameras to double-check that the object is within the shovel while the robot is moving; can be evaluated using:
[0089] If the pick-up strategy is unsuccessful, the target object may be marked as unsuccessfully picked at block 2120, as previously described. If the target object is successfully picked up, the method includes navigating to a drop location at block 2122. For example, the robot 100 as introduced in connection with FIG. 1A may be navigated to a predetermined drop location. The drop location may be a container or a designated area on the ground or floor. The navigation may be controlled by a machine learning model or a rule-based approach.
[0090] According to some examples, the method includes determining a dropping strategy at block 2124. For example, the robot control system 1900 shown in FIG. 1A may determine the dropping strategy. The dropping strategy may need to take into account the delivery location determined for the pick-up strategy. The dropping strategy may be determined using a machine learning model or a rule-based approach. A rule-based strategy for dropping an object may include: Navigate the robot to a position 100cm away from the side of the box; Rotating the robot in place to align it facing the box; - Moving towards the box while maintaining center alignment on the side of the box, - Stop 3cm from the side of the box, -Ensure that the robot is correctly positioned against the sides of the box; - If yes, raise the shovel upwards and backwards to drop the target object into the box -If no, leave the box and restart the process may include.
[0091] An object drop strategy may involve navigating with the rear camera when attempting a backward drop and with the front camera when attempting a forward drop.
[0092] According to some examples, the method includes executing a drop strategy at block 2126. For example, the robot 100 as introduced in connection with FIG. 1A may execute a drop strategy. The drop strategy may follow the strategy steps of separation strategy, pick-up strategy, and drop strategy 2200 shown in FIG. 22. The drop strategy may be a reinforcement learning based strategy or a rule-based strategy. Once the drop strategy is executed at block 2126, the method may proceed to determining a next target object at block 2128. For example, the robot control system 1900 shown in FIG. 1A may determine the next target object. Once a new target object is determined, the process is repeated for the new target object.
[0093] Strategies such as the separation, pick up and drop strategies mentioned above may be simple strategies or may incorporate reward and collision avoidance elements. These strategies may follow a general approach such as the strategy steps of the separation, pick up and drop strategies 2200 shown in FIG.
[0094] In some embodiments, the object separation strategy comprises: - Using V-shaped grabber pad arms and grabber pads on the floor to encircle and push objects; - Accurately grasp and push objects using V-shaped grabber pad arms and grabber pads; Loosely roll large objects with the grabber pad arm and grabber pad raised Dispersing dense clutter by loosely gripping and pushing the pile; A single grabber pad arm / grabber pad is placed on the floor between the object and the clutter and then rotated; Put small toys into the shovel and let it drop to separate them; Using a single grabber pad arm / grabber pad to move the object away from the wall; may include.
[0095] In some embodiments, the pick-up strategy comprises: -Closing the grabber pad arm / grabber pad on the floor to pick up simple objects; Pick up piles of small objects, such as small plastic blocks, by closing the grabber pad arms / grabber pad on the ground; Picking up small, rolling objects such as balls by tapping them on the top with the grabber pad arm / grabber pad and rolling them into the shovel; Picking up deformable objects such as clothing by repeatedly compressing the object into the shovel using the grabber pad arm / grabber pad; Grabbing large, soft objects such as large stuffed toys by grabbing and compressing them with the grabber pad arms / grabber pads; Grabbing a large ball by rolling it on the elevated grabber pad arm / grabber pad and holding it against the shovel; Passing the grabber pad over a flat object, such as a puzzle piece, and then unsteadying it sideways to pick it up; -grasping books and other large flat objects, Grabber pad arms / grabber pads to lift clothes and lift them above the shovel and drop them into the shovel; - rolling the ball by initiating the movement of a first grabber pad arm and immediately initiating the movement of a second grabber pad arm; may include.
[0096] In some embodiments, the drop strategy comprises: - Dropping backwards into a box, - Dropping forward into the box, - Forward release to floor, - Forward release against a wall, Stacking books or other flat objects; - Direct dropping of large objects using grabber pad arms / grabber pads instead of relying on shovels; may include.
[0097] 22 illustrates strategy steps for a separation, pick up, and drop strategy 2200 according to one embodiment. According to some examples, the method includes determining an action from a policy at block 2202. For example, the robot control system 1900 shown in FIG. 1A may determine the action from the policy. The next action may be based on the policy along with the observations, the current robot state, the current object state, and the sensor data 2016. This determination may be made through a process for determining an action from a policy 2300 shown in FIG.
[0098] In one embodiment, the strategy may incorporate rewards or penalties 2212 in determining an action from the policy in block 2202. These rewards or penalties 2212 are primarily used to train the reinforcement learning model and in some embodiments are not applied to the ongoing movement of the robot. Training the reinforcement learning model may be done using simulation or by recording the model inputs / outputs / rewards / penalties during the robot's movements. The recorded data may be used to train the reinforcement learning model to select the action that maximizes the reward and minimizes the penalty. In some embodiments, the rewards or penalties 2212 for object picking using reinforcement learning are: - A small penalty added every second, A reward when the target object first touches the edge of the shovel, - Reward when the target object is pushed completely into the shovel, - Penalty when the target object is lost from the shovel, - Penalty for collision with obstacles or walls (exceeding maximum force feedback) Penalty for lifting non-target objects, Penalties if the robot gets stuck or runs over an object, may include.
[0099] In some embodiments, the reward or penalty 2212 for object separation using reinforcement learning (e.g., moving the target object away from the wall to the right) is: - A small penalty added every second, Reward when the right grabber pad arm is between the target object and the wall, Reward if the distance of the target object from the wall is more than 10 cm, - Penalty for accidentally colliding with the target object, - Penalty for collision with an obstacle or wall (exceeding the maximum force feedback value), Penalties if the robot gets stuck or runs over an object, may include.
[0100] In some embodiments, the reward or penalty 2212 for dropping an object using reinforcement learning is: - A small penalty added every second, -Reward for the robot docking correctly with the box, - Reward if the target object is successfully dropped into the box, - Penalty in case of collision when moving boxes, Penalty for collisions with obstacles or walls (exceeding the maximum force feedback), Penalties if the robot gets stuck or runs over an object, may include.
[0101] In at least one embodiment, the techniques described herein may use a reinforcement learning approach in which the problem is modeled as a Markov Decision Process (MDP) represented as a tuple (S, O, A, P, r, γ), where S is the set of states in the environment, O is the set of observations, A is the set of actions, and P: S×A×S (outside 1) TIFF2024540846000002.tif2279 is the state transition probability function Yes, r:S×A (outside 2) TIFF2024540846000003.tif2279 is the reward function and γ is the discount factor.
[0102] In at least one embodiment, the goal of the training is to perform an action a at time t. t =π(o t ) maximizes the sum of discounted future rewards from state st. (Outside 3) The goal is to learn TIFF2024540846000004.tif2279.
[0103]
number
[0104] In at least one embodiment, action a t After this, the environment is sampled from P to state s t From state s t+1 In at least one embodiment, state s t Actions in a t The quality of doing something is known as the Q function. (outside 4) Measured by TIFF2024540846000006.tif2279.
[0105] In one embodiment, data from the mobile collision avoidance system 2214 may be used in determining an action from the policy in block 2202. Each strategy may have an associated list of available actions that it considers. A strategy may use the mobile collision avoidance system to determine the range of motion for each action involved in executing the strategy. For example, the mobile collision avoidance system may be used to see if the shovel can be lowered to the ground without hitting the grabber pad arm or grabber pad (if they are closed under the shovel), a nearby obstacle such as a wall, or an object (such as a ball) that may have rolled under the shovel.
[0106] According to some examples, the method includes performing an action at block 2204. For example, a robot 100 as introduced in relation to FIG. 1A may perform the action determined from block 2202. The action may be based on observations, a current robot state, a current object state, and sensor data 2016. The action may be performed via operation of the robot motors and other actuators 2210 of the robot 100. The real-world environment 2002 may be affected by the operation of the robot 100. Changes in the environment 2002 may be detected as described in relation to FIG.
[0107] According to some examples, the method includes checking progress toward the goal at block 2206. For example, the robot control system 1900 shown in FIG. 1A may check the progress of the robot 100 toward the goal. If the check determines at decision block 2208 that the goal of the strategy has been achieved or that a fatal error has occurred, execution of the strategy is halted. If the goal has not been achieved and no fatal error has occurred, the strategy may return to block 2202.
[0108] 23 illustrates a process for determining an action from a policy 2300 according to one embodiment. The process of determining an action from a policy 2300 may take into account a type of strategy 2302 and determine available actions to be used based on the type of strategy 2302 in block 2304. The reinforcement learning algorithm or rule-based algorithm may utilize both simple actions and pre-defined compound actions. Examples of simple actions to control individual actuators are: - Moving the left grabber pad arm to a new position (rotating it up or down), - moving the left grabber pad list to a new position (rotating it left or right), - Moving the right grabber pad arm to a new position (rotating it up or down), - moving the right grabber pad list to a new position (rotating left or right), Lifting the excavator to a new position (rotating it up or down), Changing the angle of the shovel (with a second motor or actuator for forward drop), Driving the left wheel Right wheel driving, may include.
[0109] Examples of predefined compound actions are: Driving the robot according to a path to a position / waypoint; Rotating the robot left or right in place; Centering the robot on an object; Aligning the grabber pad arms with the top / bottom / center of the object; - moving the object forward until it touches the edge of the shovel; Close both grabber pad arms and push the object in one smooth motion. Lifting the shovel and grabber pad arms together while gripping the object; -Close both grabber pad arms and push the object with a quick tap and slight release; Place the shovel lightly on the floor / carpet, Pushing the shovel down onto the floor / carpet, -Close the grabber pad arms and hold them in position until resistance is met / pressure is applied; - Vibration and side-to-side rotation closes the grabber pad arms, creating instability and slight bouncing of flat objects over the shovel edge; may include.
[0110] In block 2308, the process of determining actions from policy 2300 may take the list of available actions 2306 determined in block 2304 and determine an operating range 2312 for each action. The operating range 2312 may be determined based on the observations, the current robot state, the current object state, and the sensor data 2016 available to the robot control system. The action type 2310 may also be indicated to the mobile collision avoidance system 2214, which may determine the operating range 2312.
[0111] The process of determining an action from the policy 2300, block 2308, may determine an observation list 2314 based on the determined operating range 2312. An example observation list 2314 may include: Detected and classified objects in the environment; Global or local environment maps, Condition 1: The left arm is positioned 20 degrees inward. Condition 2: The right arm is positioned 150 degrees inward. Condition 3: The target object is 15 cm from the end of the shovel. Condition 4: The target object is 5 degrees to the right of the center. Maximum range of action 1: Drives forward up to 1cm Maximum range of action 2: Drive backwards up to 10cm Action 3 maximum range: Left arm opens up to 70 degrees Action 4 maximum range: Open the right arm up to 90 degrees Action 5 maximum range: Close the left arm up to 45 degrees Action 6 maximum range: Close the right arm up to 0 degrees Action 7 Max Range: Turn left up to 45 degrees Action 8 Max Range: Turn up to 45 degrees right may include.
[0112] At block 2316, a reinforcement learning model may be executed based on the observation list 2314. The reinforcement learning model may return actions 2318 appropriate for the strategy that the robot 100 attempts to complete based on an associated policy.
[0113] Figure 24 illustrates a placement process 2400 according to one embodiment. The placement process 2400 may be performed by a robot 100 as introduced in connection with Figure 1A as part of the algorithms disclosed herein. The robot may include the sensing system, control system, mobility system, grabber pads, grabber pad arms and shovels shown in Figures 1A-1D or similar systems and features performing equivalent functions as are well understood in the art.
[0114] At block 2402, the robot may detect a destination where an object carried by the robot is intended to be placed. At block 2404, the robot may determine a destination approach path to the destination. The path may be determined to avoid obstacles near the destination. In some embodiments, the robot may perform additional navigation steps to push or move the object away from the destination approach path. The robot may also determine an object placement pattern, the object placement pattern being at least one of a placing pattern and a dropping pattern. Some objects that can be neatly stacked, such as books, other media, narrow boxes, etc., may be best put away by carefully stacking them. Other objects may not be neatly stacked, but may be easily placed by dropping them into a box. Based on the attributes of the object, the robot may determine which object placement pattern is best suited for the object.
[0115] At block 2406, the robot may approach the destination via a destination approach path. The manner in which the robot navigates the destination approach path may be determined based on the object placement pattern. If the object being carried is to be dropped onto the back of the robot's chassis, the robot may reverse traverse the destination approach path and stop with the back of the chassis closest to the destination. Alternatively, for an object to be stacked or placed in front of the shovel, i.e., in the area of the shovel opposite the chassis, the robot may move forward along the destination approach path such that the shovel is closest to the destination.
[0116] At decision block 2408, the robot may proceed in one of at least two ways depending on whether the object is to be placed or dropped. If the object placement pattern is intended to be a placing pattern, the robot may proceed to block 2410. If the object placement pattern is intended to be a dropping pattern, the robot may proceed to block 2416.
[0117] For objects being placed via a placement pattern, the robot may stop with the destination in front of the shovel and grabber pad at block 2410. At block 2412, the robot may lower the shovel and grabber pad to a placement height. For example, when placing a book on top of an existing stack of books, the placement height may be slightly higher than the top of the tallest book in the stack so that the book can be placed without getting in the way of the stack or dropping the book from a height that may have enough momentum to collapse or destabilize the stack. Finally, at block 2414, the robot may use its grabber pad to push the object out of the containment area to the destination. In one embodiment, the shovel may be tilted forward to allow the object to be dropped, with or without the assistance of a grabber pad to push the object off the shovel.
[0118] If, at decision block 2408, the robot determines to proceed with an object placement pattern that is a drop pattern, the robot may proceed to block 2416. At block 2416, the robot may stop with the destination behind the shovel and grabber pad, and thus behind the robot chassis as shown in FIG. 1A. At block 2418, the robot may raise the shovel and grabber pad to a placement height. In one embodiment, the object may be positioned to be dropped from the containment area into the destination area by raising the shovel and grabber pad arms from a transport position to a placement height. Otherwise, at block 2420, the robot may extend the grabber pad and drop the object from the containment area such that the object comes to rest at or within the destination area. In one embodiment, the shovel may be tilted forward to drop the object, with or without the assistance of the grabber pad to push the object off the shovel.
[0119] The disclosed algorithm may include a capture process 2500 as shown in Figure 25. The capture process 2500 may be performed by a robot 100 as introduced in connection with Figure 1A. The robot may have the sensing system, control system, mobility system, grabber pads, grabber pad arms and shovels shown in Figures 1A-1D or similar systems and features performing equivalent functions as are well understood in the art.
[0120] The capture process 2500 may begin at block 2502 where the robot detects a starting location and attributes of an object to be picked up. The starting location may be determined against a learned map of landmarks in a room the robot is programmed to clear. Such a map may be stored in memory in the robot's electrical system. These systems are described in more detail with reference to FIG. 19. The object attributes may be detected based on input from a sensing system, which may include cameras, LIDAR, or other sensors. In some embodiments, data detected by such sensors may be compared to a database of common objects to identify attributes such as deformability and dimensions. In some embodiments, the robot may use known landmark attributes to calculate object attributes such as dimensions. In some embodiments, machine learning may be used to improve the detection and analysis of attributes.
[0121] In block 2504, the robot may determine an approach path to the starting location. The approach path may take into account the shape of the surrounding space, obstacles detected around the object, and components the robot may be configured with as the robot approaches the object. The robot may further determine a gripping height for initial contact with the object. This gripping height may take into account the estimated center of gravity of the object so that the grabber pads move the object with the least chance of the object slipping under or around the object or deflect the object in some direction other than into the shovel. The robot may determine a gripping pattern for the movement of the grabber pads during capture of the object such that the object is contacted from a direction with forces applied at intervals optimized to propel the object into the shovel. Finally, the robot may determine a transfer position of the grabber pads and the shovel that will secure the object in a storage area for transfer after the object is captured. This position may take into account attributes such as the object's dimensions, its weight, and its center of gravity.
[0122] At block 2506, the robot may extend its grabber pads outward and forward relative to the grabber pad arms, raising the grabber pads to a gripping height. This may allow the robot to get as close as possible to the object without leaving the room for this extension after the approach. Alternatively, the robot may perform part of the approach with the arms folded near the chassis and shovel to prevent colliding with obstacles along the approach path. In some embodiments, the robot may first navigate the approach path and deploy the arms and shovel to clear the object from the approach path. At block 2508, the robot may finally approach the object via the approach path and stop when the object is placed between the grabber pads.
[0123] At block 2510, the robot may execute the grasping pattern determined at block 2502 to grab an object within the containment area. The containment area is an area roughly described by the dimensions of the shovel and the placement of the grabber pad arm relative to the shovel. It will be understood that this is an area in which the object to be transported has a minimal chance of slipping, dislodging, or falling off the shovel and grabber pad arm during movement. At decision block 2512, the robot may verify that the object is within the containment area. If the object is within the containment area, the robot may proceed to block 2514.
[0124] At block 2514, the robot may apply light pressure to the object with the grabber pad to hold the object still within the containment area. In some embodiments, this pressure may be downward to hold the object that extends above the top of the shovel against the sides and surface of the shovel. In other embodiments, this pressure may be applied horizontally to hold the object within the shovel against the back of the shovel. In some embodiments, the pressure may be against the bottom of the shovel to prevent the formation of gaps that would allow the object to slip off the front of the shovel.
[0125] At block 2516, the robot may raise the shovel and grabber pad to the transfer position determined at block 2502. The robot may then transport the object to the destination at block 2518. The robot may follow a transition path between the start location and the destination where the object is to be placed. To place the object at the destination, the robot may follow the placement process 2400 shown in FIG.
[0126] If, at decision block 2512, it is determined that the object is not detected within the containment area or is partially or precariously located within the containment area, the robot may withdraw the grabber pad from the shovel 2946 and extend it forward relative to the grabber pad arm to return the grabber pad to the gripping height at block 2520. The robot may then return to block 2510. In some embodiments, the robot may back away from the object if it is not feasible to simply release and try again to capture the object at block 2522. This may occur if the object has been repositioned or moved by the first attempt to capture it. At block 2524, the robot may re-determine an approach path to the object. The robot may then return to block 2508.
[0127] 26 shows an initial process diagram 2600 according to one embodiment of the placement process 2400 and capture process 2500 described above. In step 2602, the robot may drive towards a target object or group of objects. The robot may navigate to approach the target object or group of objects using a local map or a global map.
[0128] In step 2604, the robot may adjust its approach angle and move the obstacle. An optimal approach angle for a particular object, such as approaching a book from its spine, may be determined. A grabber pad arm may be used to push the obstacle out of the way, and the robot may be actuated to adjust its approach angle.
[0129] In step 2606, the robot may adjust the height of its arms based on the type of object or group of objects. The strategy for picking up a target object or group of objects, including the height of the arms, may vary depending on the object to be picked up. For example, a basketball may be pushed from above so that it rolls. A stuffed animal may be pushed from the center to prevent it from slipping and tipping on its side making it difficult to push or onto a grabber pad arm. A book may be pushed from the side very close to the floor. A Lego may be pushed against the floor with an arm.
[0130] In step 2608, the robot may drive the arms to align over an object or group of objects that may be in contact with the shovel or scoop and within the inner area of the two grabber pad arms.
[0131] In step 2610, the robot may use its arms to push an object or group of objects against the shovel or scoop. The arms may be used intelligently according to the grasping pattern as previously described. In some cases, both arms may push together gradually, but adjustments may be made as the object rolls and moves.
[0132] At step 2612, the robot may determine if the object has been picked up. Using a camera sensor or other sensors, it may be confirmed or detected if the object or group of objects has been successfully pushed onto the shovel or scoop. If the object has not been picked up, at step 2614, the robot may release and open the arms. The arms are first released slightly to prevent the object from being pinched, and then moved over the object or group of objects to prevent the object from being pushed further out of the shovel or scoop than it is currently. This allows the robot to make incremental progress in the pick-up if the initial action is not sufficient. From here, the robot may return to step 2604. If at step 2612, the object is detected to be within the shovel, the robot may proceed to step 2616.
[0133] In step 2616, the robot may apply light pressure to the top of the object or against the shovel or scoop. Based on the type of object, the robot may apply pressure to the object to hold the object or group of objects within the shovel or scoop. For example, the robot may hold the top of a basketball tightly, grab a stuffed animal, or push down on a book or against the shovel or scoop to hold a group of small objects such as marbles or plastic blocks.
[0134] In step 2618, the robot may lift the object or group of objects while continuing to hold it with the grabber pad arm. The shovel or scoop and arm may be raised together to the desired angle, such as 45 degrees, for the object or group of objects to be carried without rolling off or becoming dislodged from the shovel or scoop. Once the shovel and arm are raised to the desired angle, the arm may continue to apply pressure to keep the object secured within the shovel.
[0135] In step 2620, the robot moves to a destination to place the object. The robot may navigate to the destination using a local or global map to place the object or group of objects. For example, this may be a container intended to hold an object, a stack of books, or a designated portion of the floor where the object or group of objects is out of the way.
[0136] In step 2622, the robot may move a shovel or scoop holding an object or group of objects above the destination. The height or position of the shovel may be adjusted to match the destination. For example, the shovel or scoop may be elevated above a container or aligned with the top area of an existing stack of books.
[0137] In step 2624, the robot may place an object or group of objects at a destination using its grabber pad arm. The arm may be opened to drop the object or group of objects into a container or may be used to push an object forward from a shovel or scoop. For example, a basketball may be dropped over the robot's back into a container, a book may be carefully pushed forward onto an existing stack of books. Finally, the process ends with the object being successfully dropped or placed at the destination in step 2626.
[0138] Figures 27A-D show a process for stackable objects 2700 according to one embodiment. Figure 27A shows a side view of the robot performing steps 2702-2710, while Figure 27B shows a top view of the performance of these same steps. Figure 27C shows a side view of steps 2712-2720, and Figure 27D shows a top view of these steps. The stackable objects can be books, cases for holding compact discs (CDs), digital video discs (DVDs) or other media, narrow boxes such as puzzle boxes, or any other objects that can be easily and neatly stacked.
[0139] As shown in FIG. 27A and FIG. 27B, the robot first moves to a stackable object 2722 located at a start position 2724, as shown in step 2702. The robot may move to the stackable object 2722 by following an approach path 2726. As shown in step 2704 and step 2712, the robot may adjust its grabber pad arm to a gripping height 2728 based on the type of object. For a stackable object such as a book, this may be just above the top of the book. The robot may drive its arm to align over the object 2730, as shown in step 2706. The robot may adopt a gripping pattern 2732, as shown in step 2708, and use its arm to press the book against the shovel or scoop. Using the grabber pad arm, as shown in step 2710, the robot may apply light pressure 2734 to the top of the book to hold the book firmly in or on top of the shovel.
[0140] As shown in FIG. 27C and FIG. 27D, the robot may continue to hold the book with its grabber pad arm and lift the book while maintaining the book in a transfer position 2736 in the shovel in step 2712. In step 2714, the robot may move to a destination 2738 where the book is intended to be placed, following a destination approach path 2740. The robot may adjust the shovel and grabber pad arm to position the book at a placement height 2742 in step 2716. In the case of a stackable object such as a book, this may position the book horizontally with an area above the top of the existing stacked books. In step 2718, the robot may push the book with its arm and place it at the destination (i.e., on top of the stacked books) using a placement pattern 2744. The book may thus be dropped or placed at its destination in step 2720.
[0141] This process for stackable object 2700 may be performed by any of the robots disclosed herein, such as those shown in Figures 1A-1D, 2A-2E, 3A-3C, 4A-4C, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 and 17, and 18A and 18B.
[0142] Figures 28A-D show a process for a large, slightly deformable object 2800 according to one embodiment. Figure 28A shows a side view of the robot performing steps 2802-2810, and Figure 28B shows a top view of the performance of these same steps. Figure 28C shows a side view of steps 2812-2820, and Figure 28D shows a top view of these steps. The large, slightly deformable object can be an object such as a basketball that extends outside the dimensions of the shovel and can respond to pressure with little deformation or change in shape.
[0143] As shown in FIG. 28A and FIG. 28B, the robot may first move to a large, slightly deformable object 2822, such as a basketball, located at a starting position 2824, following an approach path 2826 in step 2802. The robot may adjust the grabber pad arm to a gripping height 2828 based on the type of object in step 2804. For a large, slightly deformable object 2822, such as a basketball, this may be near or on the top of the basketball. The robot may drive its arm to align over the object 2830 in step 2806. The robot may adopt a gripping pattern 2832 in step 2808 and use its arm to push or roll the basketball into the shovel or scoop. Using the grabber pad arm in step 2810, the robot may apply light pressure 2834 to the top of the basketball to hold it firmly in or on the shovel.
[0144] As shown in FIG. 28C and FIG. 28D, the robot may continue to hold the basketball with its grabber pad arms and lift the basketball while maintaining the ball at a transfer position 2836 in the shovel, step 2812. The robot may then move to a destination 2838 where the basketball is intended to be placed, following a destination approach path 2840, step 2814. In step 2816, the robot may adjust the shovel and grabber pad arms to position the basketball at a placement height 2842. For an object such as a basketball, this may position the shovel and ball in an area above the robot that is angled or pointed toward the container. In step 2818, the robot may open its arms to release the object into the destination container using a drop pattern 2844. The basketball is dropped from the shovel 2846 and placed in the destination container, step 2820.
[0145] The process for the large, slightly deformable object 2800 may be performed by any of the robots disclosed herein, such as those shown in Figures 1A-1D, 2A-2E, 3A-3C, 4A-4C, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 and 17, and 18A and 18B.
[0146] Figures 29A-D show a process for a large, highly deformable object 2900 according to one embodiment. Figure 28A shows a side view of the robot performing steps 2902-2910 and Figure 28B shows a top view of the performance of these same steps. Figure 29C shows a side view of steps 2912-2920 and Figure 29D shows a top view of these steps. A large, highly deformable object can be an object such as a stuffed animal, bean bag toy, empty backpack, etc. that extends outside the dimensions of the shovel and can respond to pressure with significant deformation or change in shape.
[0147] As shown in FIG. 29A and FIG. 29B, the robot first moves to a large, highly deformable object 2922, such as a stuffed toy, located at a starting position 2924, in step 2902. The robot may adjust its grabber pad arm to a gripping height 2928 based on the type of object, in step 2904. For large, highly deformable objects, such as a stuffed toy, this may be near the vertical center of the object or may be the predicted center of gravity. The robot may drive its arm to align over the object 2930, in step 2906. The robot adopts the gripping pattern 2832, in step 2908, and uses its arm to press the stuffed toy against the shovel or scoop. Using the grabber pad arm, in step 2910, the robot may apply light pressure 2934 to the top of the stuffed toy to hold it firmly in or on the shovel.
[0148] As shown in FIG. 29C and FIG. 29D, the robot may continue to hold the stuffed animal with its grabber pad arms and lift the stuffed animal while maintaining the stuffed animal in a transfer position 2936 in the shovel, step 2912. The robot may then move to a destination 2938 where the stuffed animal is intended to be placed, following a destination approach path 2940, step 2914. In step 2916, the robot may adjust the shovel and grabber pad arms to position the stuffed animal at a placement height 2942. For an object such as a stuffed animal, this may position the shovel and stuffed animal in an area above the robot that is inclined or pointed toward the container. In step 2918, the robot may open its arms to release the object into the destination container using a drop pattern 2944. The stuffed animal rolls, slides, or falls off the shovel 2846 and is placed in the destination container, step 2920.
[0149] The process for the large, slightly deformable object 2800 may be performed by any of the robots disclosed herein, such as those shown in Figures 1A-1D, 2A-2E, 3A-3C, 4A-4C, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 and 17, and 18A and 18B.
[0150] Figures 30A-30D show a process for small, easily scattered objects 3000 according to one embodiment. Figure 30A shows a side view of the robot performing steps 3002-3010 and Figure 30B shows a top view of the performance of these same steps. Figure 30C shows a side view of steps 3012-3020 and Figure 30D shows a top view of these steps. Small, easily scattered objects can be small, lightweight objects such as small plastic blocks, marbles, cereal, etc. that can easily scatter when in contact with the grabber pad arms of the robot or can slide off the shovel during transport if proper care is not taken.
[0151] As shown in FIG. 30A and FIG. 30B, the robot first moves to a small, easily scattered object 3022, such as a group of plastic blocks, located at a start position 3024, in step 3002. The robot may adjust its grabber pad arms to a gripping height 3028 based on the type of object, in step 3004. For small, easily scattered objects, this is a height close to or touching the floor. The robot may drive its arms to align over the object 3030, in step 3006. The robot employs a gripping pattern 2832, in step 3008, using its arms to press the object against a shovel or scoop. The gripping pattern 3032 for such objects may use small forces or small arcing movements rather than continuous pressure. In step 3010, the robot may close its arms over the front of the shovel and apply small pressure against the shovel to prevent the object from rolling or sliding out.
[0152] As shown in FIG. 30C and FIG. 30D, the robot may continue to block the shovel from opening with its grabber pad arm and lift the block while maintaining the object in a transfer position 3036 in the shovel, step 3012. The robot may then move to a destination 3038 where the object is intended to be placed, following a destination approach path 3040, step 3014. In step 3016, the robot may adjust the shovel and grabber pad arm to position the object at a placement height 3042. For objects such as small plastic blocks, this may position the shovel in an area above the robot that is inclined or pointed toward the container. In step 3018, the robot may open its arms to release the object captured by the arms into the destination container, using a drop pattern 3044. The block rolls, slides, or falls off the shovel 3046 and is placed in the destination container, step 3020.
[0153] The process for small, easily scattered objects 3000 may be performed by any of the robots disclosed herein, such as those shown in Figures 1A-1D, 2A-2E, 3A-3C, 4A-4C, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 and 17, and 18A and 18B.
[0154] 31 illustrates a robotic system 3100 in one embodiment. The robotic system 3100 receives inputs from one or more sensors 3102 and one or more cameras 3104 and provides these inputs for processing by localization logic 3106, mapping logic 3108, and perception logic 3110. Output of the processing logic is provided to a path planner 3112, a pick-up planner 3114, and a motion controller 3116 of the robotic system 3100, which in turn drive the system's motor and servo controllers 3118.
[0155] The cameras may be arranged in a front stereo configuration and may also include a rear camera, or a single front camera may be used, or a single front camera along with a single rear camera. In some cases, other camera configurations (e.g., one or more side cameras or cameras facing at an angle) may also be used.
[0156] One or more of the localization logic 3106, mapping logic 3108, and perception logic 3110 may be located and / or executed on the mobile robot or may be executed in a computing device that communicates wirelessly with the robot, such as a mobile phone, laptop computer, tablet computer, or desktop computer. In some embodiments, one or more of the localization logic 3106, mapping logic 3108, and perception logic 3110 may be located and / or executed on "the cloud," i.e., a computer system coupled to the robot via the Internet or other network.
[0157] The perception logic 3110 is coupled by an image segmentation activation 3144 signal to detect objects within the field of view of the camera 3104 utilizing any one or more of known image segmentation and object recognition algorithms. The perception logic 3110 may provide a calibration and objects 3120 signal for mapping purposes. The localization logic 3106 uses any one or more of known algorithms to localize the mobile robot within its environment. The localization logic 3106 outputs a local to global transformation 3122 reference frame transformation, which the mapping logic 3108 combines with the calibration and objects 3120 signal to generate an environment map 3124 for the pick up planner 3114 and an object tracker 3126 signal for the path planner 3112.
[0158] In addition to the object tracking 3126 signal from the mapping logic 3108, the path planner 3112 also utilizes the current state of the system 3128 from the system state settings 3130, a synchronization signal 3132 from the pick up planner 3114, and motion feedback 3134 from the motion controller 3116. The path planner 3112 converts these inputs into navigation waypoints 3136 that drive the motion controller 3116. The pick up planner 3114 uses the image segmentation 3138 input from the perception logic 3110, 3124 from the mapping logic 3108, and the synchronization signal 3132 from the path planner 3112 to convert local perception into manipulation actions 3140 (e.g., of a robotic gripper, shovel) to the motion controller 3116. Embodiments of the algorithms utilized by the path planner 3112 and the pick up planner 3114 are described in more detail below.
[0159] In one embodiment, a simultaneous localization and mapping (SLAM) algorithm may be used to generate a global map and simultaneously localize the robot on the map. Many SLAM algorithms are known in the art and commercially available.
[0160] The motion controller 3116 converts the local perception from navigation waypoints 3136, manipulation actions 3140, and image segmentation 3138 signals into target motion 3142 signals to the motor and servo controller 3118.
[0161] FIG. 32 illustrates a robotics process 3200 in one embodiment. At block 3202, the robotics process 3200 wakes up the robot sleeping at the base station. At block 3204, the robotics process 3200 navigates the robot around its environment using cameras to map the type, size, and location of toys, clothes, obstacles, and other objects. At block 3206, the robotics process 3200 operates a neural network to identify the type, size, and location of objects based on images from the left and right stereo cameras. At open loop block 3208, the robotics process 3200 runs a corresponding bin for each category of objects. At block 3210, the robotics process 3200 selects a specific object to pick up within the category. At block 3212, the robotics process 3200 performs path planning. At block 3214, the robotics process 3200 navigates adjacent to and facing the target object. At block 3216, the robotic process 3200 actuates the arm to move other objects out of the way and push the target object onto the front shovel. At block 3218, the robotic process 3200 tilts the front shovel upward to keep them on the shovel (creating a "bowl" shape for the shovel). At block 3220, the robotic process 3200 actuates the arm to close at the front to keep the object from getting under the wheels while the robot navigates to the next location. At block 3222, the robotic process 3200 does path planning and navigates close to the container for the current object category for collection. At block 3224, the robotic process 3200 aligns the robot to the side of the container. At block 3226, the robotic process 3200 lifts the shovel upward and backward to lift the target object over the side of the container. At block 3228, the robotic process 3200 returns the robot to the base station.
[0162] In a less sophisticated mode of operation, the robot may opportunistically pick up objects in its field of view and drop them into a container without first building a global map of the environment. For example, the robot may simply explore until it finds an object to pick up, then explore again until it finds a matching container. This approach works effectively in single-room environments where the area to explore is limited.
[0163] FIG. 33 also illustrates a robotic process 3300 in one embodiment, where the robotic system sequences through an embodiment of a state space map 3400 as shown in FIG.
[0164] The sequence starts with the robot sleeping (sleep state 3402) and charging at the base station (block 3302). The robot is woken up, for example according to a schedule, and enters a search mode (environment search state 3404, wake action 3406 and schedule start time 3408). In the environment search state 3404, the robot scans the environment using the camera (and other sensors), updates its environment map, and locates itself on the map (block 3304, search set interval 3410). The robot may transition from the environment search state 3404 to the sleep state 3402 if there are no more objects to be picked up 3412 or if the battery is low 3414.
[0165] From the environment exploration state 3404, the robot may transition to an object organization state 3416 where it operates to move items on the floor and organize them by category 3418. This transition may be triggered by the robot determining that objects on the floor are too close to each other 3420 or that a path to one or more objects is blocked 3422. If neither of these trigger conditions are met, the robot may transition from the environment exploration state 3404 directly to the object pickup state 3424 if the environment map includes at least one container for drop of the object category 3426 and there are unblocked objects in the container category for pick-up 3428. Similarly, the robot may transition from the object organization state 3416 to the object pickup state 3424 under these latter conditions. The robot may return from the object organization state 3416 to the environment exploration state 3404 if there are no objects ready for pick-up 3430.
[0166] In the environment exploration state 3404 and / or object organization state 3416, image data from the camera is processed to identify different objects (block 3306). The robot selects a particular object type / category to pick up, determines the next waypoint to navigate to, and identifies the type of target object and the location to perform the pick-up based on the environment map (blocks 3308, 3310, and 3312).
[0167] In the pick up object state 3424, the robot selects a goal location adjacent to the target object (block 3314). The robot uses a path planning algorithm to navigate to its new location while avoiding obstacles. The robot actuates the left and right pusher arms to create an opening large enough for the target object to pass through, but not so large that other unwanted objects are collected as the robot moves forward (block 3316). The robot moves forward so that the target object is between the left and right pusher arms, which work together to push the target object onto the collection shovel (block 3318).
[0168] The robot may continue in the pick up state 3424 to identify other target objects of the selected type to be picked up based on the environment map. If other such objects are detected, the robot selects a new target location adjacent to the target object. Using a path planning algorithm, the robot navigates to its new location while avoiding obstacles, while still carrying the previously collected target object. The robot actuates the left and right pusher arms to create an opening large enough for the target object to pass through, but not so large that other unwanted objects are collected as the robot moves forward. The robot moves forward so that the target object is between the left and right pusher arms. Again, the left and right pusher arms work together to push the target object onto the collection shovel.
[0169] When all identified objects in a category have been picked up (3432) or the shovel has reached capacity (3434), the robot transitions to the Drop Object state (3436) where it uses a map of the environment to select a goal location adjacent to the bin for the type of object collected and navigates to that new location while avoiding obstacles using a path planning algorithm (block 3320). The robot backs up towards the bin and enters a docking position where the back of the robot is aligned with the back of the bin (block 3322). The robot lifts the shovel and rotates backwards on the rigid arm at the back of the robot (block 3324). This lifts the target object above the top of the bin and dumps it into the bin.
[0170] If there are more items to be picked up (3438) or if the map of the environment is incomplete (3440), the robot may return from the object dropping state 3436 to the environment exploring state 3404. The robot may resume exploration, and the process may be repeated for each other type of object in the environment that has an associated collection bin (block 3326). .
[0171] The robot may transition from the dropped object state 3436 to the sleep state 3402 if there are no more objects to pick up (3412) or if the battery is low (3414). Once the battery is sufficiently charged or at the next activation or scheduled pick-up interval, the robot may resume searching and the process may be repeated for each other type of object in the environment that has an associated collection bin (block 3326).
[0172] 35 illustrates a robot control algorithm 3500 for a robotic system in one embodiment. The robot control algorithm 3500 begins by selecting one or more categories of objects to organize (block 3502). Within the selected category, a grouping is identified that determines a target category and a starting location for a path, block 3504. Any number of well-known clustering algorithms may be utilized to identify groupings of objects within a category.
[0173] A path to the starting goal location is formed, the path including zero or more waypoints (block 3506). Motion feedback is provided to the path planning algorithm. Waypoints may be selected to avoid static and / or dynamic (moving) obstacles (objects not in the target group and / or category). The robot's motion controller works to retrace the waypoints to the target group (block 3508). If the goal location is reached, the target group is evaluated including additional criteria to determine if it is safe to organize (block 3510).
[0174] The robot's perception system operates (block 3512) to provide image segmentation to determine a sequence of actions to be generated for the robot's manipulators (e.g., arms) and positioning system (e.g., wheels) to organize the group (block 3514). The sequence of actions is repeated until the target group is organized or until organization fails (return to block 3510). Engagement of the perception system may be triggered by proximity to the target group. Once the target group is organized, these actions are repeated (block 3516), provided there is sufficient battery life remaining in the robot and there are groups in the category to organize.
[0175] In response to low battery life, the robot returns to the docking station to charge (block 3518). However, provided that battery life is sufficient and the categories are organized, the robot enters object pick-up mode (block 3520) and picks up one of the organized groups and returns it to the drop bin. Entering pick-up mode may be conditioned on the presence of an environment map that includes at least one drop bin of target objects and an unobstructed object in the target group for pick-up. Provided that no groups of objects are ready for pick-up, the robot continues to explore the environment (block 3522).
[0176] 36 illustrates a robot control algorithm 3600 for a robotic system in one embodiment. A target object within a selected object category is identified (item 3602), and a goal position for the robot is determined as an adjacent position of the target object (item 3604). A path to the target object is determined as a series of waypoints (item 3606), and the robot is navigated along the path while avoiding obstacles (item 3608).
[0177] Upon reaching the adjacent position, the target object is evaluated to determine whether it is safe to manipulate (item 3610). Provided that the target object is safe to manipulate, the robot operates to lift the object using the robot's manipulator arm, e.g., a shovel (item 3612). At this point, the target object and nearby objects may be analyzed using the robot's perception module to better control the manipulation (item 3614). .
[0178] Once the target object is placed on the shovel or other manipulator arm, it is secured (item 3616). Provided that the robot does not have capacity for additional objects or it is the last object in the selected category, an object drop mode is initiated (item 3618). Otherwise, the robot may start the process again (item 3602).
[0179] 37 illustrates a robot control algorithm 3700 according to one embodiment. In block 3702, a left and right camera of the robot as disclosed herein or some other configuration of robot cameras may provide input that may be used to generate scale-invariant keypoints in the robot's workspace.
[0180] "Scale invariant keypoints" or "visual keypoints" in this disclosure refer to unique visual features that can be maintained across different viewpoints, such as photographs taken from different regions, that are an aspect in an image taken of a robot's workspace that can be used to identify a feature of a region or an object within a region when that feature or object is photographed in other images taken from different angles, at different scales, or with different resolutions from the original photography.
[0181] The scale-invariant keypoints can be detected by an augmented reality robot interface installed on a robot or a mobile device based on images captured by the robot's camera or the mobile device's camera. The scale-invariant keypoints can help the robot or the augmented reality robot interface on the mobile device to determine the geometric transformation between camera frames displaying matching content. This helps to confirm or fine-tune the estimate of the robot's or mobile device's position within the robot's workspace.
[0182] Scale-invariant keypoints can be detected, transformed and matched for use via algorithms well understood in the art, such as (but not limited to) Scale Invariant Feature Transform (SIFT), Speed Up Robust Features (SURF), Oriented Robust Binary Features (ORB) and SuperPoints.
[0183] An object located within the robot's workspace is detected at block 3704 based on input from the left and right cameras, thereby defining a starting location for the object and classifying the object into a category. At block 3706, a re-identification fingerprint is generated for the object, and the re-identification fingerprint is used to determine the visual similarity of the object to future detected objects. The future detected object may be the same object that is re-detected as part of an update or transformation of the global area map or may be a similar object similarly located at a future time, and the re-identification fingerprint may be used to aid in faster classification of the object.
[0184] At block 3708, the robot may be located within the robot's workspace. The robot may be located using input from at least one of a left camera, a right camera, a light detection and ranging (LIDAR) sensor, and an inertial measurement unit (IMU) sensor. The robot's workspace may be mapped to create a global area map including scale-invariant keypoints, objects, and starting positions of the objects. At block 3710, objects within the robot's workspace may be re-identified based on at least one of a starting position, a category, and a re-identification fingerprint. At block 3712, each object may be assigned a persistent unique identifier.
[0185] In block 3714, the robot may receive camera frames from an augmented reality robot interface installed as an application on a mobile device operated by a user, and update the global area map with the starting position and scale-invariant keypoints using a camera frame-to-global area map transformation based on the camera frames. In the camera frame-to-global area map transformation, the global area map may be searched to find a set of scale-invariant keypoints that match the keypoints detected in the mobile camera frames using a specific geometric transformation. This transformation may maximize the number of matching keypoints and minimize the number of non-matching keypoints while maintaining geometric consistency.
[0186] User indicators are generated for the object at block 3716, which may include next target, target order, danger, too big, fragile, clutter, and blocking travel path. The global area map and object details are sent to the mobile device at block 3718, which may include at least one of a visual snapshot, a category, a starting location, a persistent unique identifier, and a user indicator for the object. This information may be sent using wireless signaling such as BlueTooth or Wifi, as supported by the communications 134 module introduced in FIG. 1C and the network interface 1912 introduced in FIG. 19.
[0187] The updated global area map, object, starting position, scale invariant key points and object details may be displayed on the mobile device using the augmented reality robot interface. The augmented reality robot interface accepts user input to the augmented reality robot interface, which indicates a change in object type, followed by object property overrides including put away, not put away and a change in user indicator at block 3720. The object property overrides may be sent from the mobile device to the robot and used to update the global area map, user indicator and object details at block 3722. Returning to block 3718, the robot may resend the updated global area map to the mobile device to resynchronize this information.
[0188] 38 illustrates a robotic system 3800 according to one embodiment. The robotic system 3800 may include a robot 100, a charging station 3802, a plurality of destination bins 3804 in which objects 3806 may be placed and each of which is associated with at least one object category, such as object category 3808, object category 3810, and object category 3812, and logic 3814 that enables the robot 100 to perform the disclosed methods. The robot 100, the charging station 3802, the objects 3806, and the destination bins 3804 may be located in an area that may be considered a workspace 3816 of the robot.
[0189] The robot 100 may detect features of the robot's workspace 3816 using the sensors and cameras shown in Figures 1C and 1D. These features may include scale-invariant keypoints 3818 of walls, corners, furniture, etc. The robot 100 may also detect objects 3806 on the floor of the robot's workspace 3816, destination boxes 3804 where the objects 3806 may be placed based on a category that the robot 100 may determine based on user input, recognition of similarity to previously processed objects, machine learning, or some combination thereof. The robot 100 may use the sensors and cameras to localize itself within the robot's workspace 3816. The robot 100 may combine all of this data into a global area map 3820 as described with respect to Figure 37.
[0190] In one embodiment, the robotic system 3800 may also include a mobile device 3822 with an augmented reality robot interface application 3824 installed and capable of providing a camera frame 3826. The robotic system 3800 may include a user possessing a mobile device 3822, such as a tablet or smartphone. The mobile device 3822 may have an augmented reality robot interface application 3824 installed that functions according to the present disclosure. The augmented reality robot interface application 3824 may provide a camera frame 3826 using a camera configured as part of the mobile device 3822. The camera frame 3826 may include a ground plane 3806 that may be identified and used to locate the mobile device 3822 within the robot system 3800 such that information regarding the robot's workspace 3816 detected by the robot 100 is transformed according to a camera frame to global area map transformation 3830, allowing the robot 100 and the mobile device 3822 to maintain synchronization with respect to objects 3806 within the robot's workspace 3816 and user indicators and object property overrides that may be added to those objects 3828.
[0191] The global area map 3820, in one embodiment, may be a top-down two-dimensional representation of the robot's workspace 3816. The global area map 3820 may undergo a camera frame to global area map transformation 3830 so that information detected by the robot 100 may be represented in the augmented reality robot interface application 3824 from the user's perspective. The global area map 3820 may be updated to include a mobile device position 3832, a robot position 3834, an object start position 3836, and an object drop position 3838. In one embodiment, the global area map 3820 may identify furniture or other objects 3806 as obstacles 3840. Objects 3806 other than the target object currently being considered by 100 may be considered obstacles 3840 during that phase of picking. In one embodiment, the augmented reality robot interface application 3824 may also show the mobile device position 3832 and the robot position 3834, which are not shown.
[0192] Various functional operations described herein may be implemented in logic, which may be referenced using a noun or noun phrase that reflects the operation or function. For example, an association operation may be performed by an "associator" or a "correlator." Similarly, switching may be performed by selection by a "switch," a "selector," etc. "Logic" refers to circuitry (hardware) that includes control and / or procedural signals and / or settings and values (resistance, impedance, capacitance, inductance, current / voltage ratings, etc.) that may be applied to affect the operation of a device by non-transitory machine-readable media, including machine memory circuits and machine-executable instructions (software and firmware), and / or material and / or material-energy configurations thereof. Magnetic media, electronic circuits, electrical and optical memories (both volatile and non-volatile), and firmware are examples of logic. Logic specifically excludes pure signals or software per se (however, machine memory that includes software and thereby forms a configuration of matter is not excluded).
[0193] In this disclosure, different entities (which may be variously referred to as "units," "circuits," other components, etc.) may be described or claimed as being "configured" to perform one or more tasks or operations. In this application, the formula "an entity" configured to "perform one or more tasks" refers to a structure (i.e., a physical thing, such as an electronic circuit). More specifically, this formula is used to indicate that this structure is configured to perform one or more tasks during operation. A structure may be said to be "configured" to perform a task even if the structure is not currently operating. A "credit distribution circuit configured to distribute credits to multiple processor cores" is intended to cover, for example, an integrated circuit having a circuit that performs this function during operation, even if the integrated circuit is not currently in use (e.g., the power supply is not connected). Thus, an entity described or described as being configured to perform a task refers to a physical thing, such as a device, a circuit, a memory that stores executable program instructions to perform a task. In this application, the phrase is not used to refer to an intangible thing.
[0194] The term "configured" is not intended to mean "configurable." For example, an unprogrammed Field Programmable Gate Array (FPGA) is not considered to be "configured" to perform a particular function, even if it is "configurable" to perform that function after programming.
[0195] A statement in the appended claims that a structure is "configured to" perform one or more tasks is expressly intended not to invoke 35 U.S.C. 112(f) with respect to that claim element. Thus, claims in this application that do not include "means" structure for "performing a function" should not be construed under 35 U.S.C. 112(f).
[0196] As used herein, the term "based on" is used to describe one or more factors that influence a decision. This term does not exclude that additional factors may influence the decision. That is, the decision may be based only on the specified factor or may be based on the specified factor and other non-specified factors. Consider the phrase "determining A based on B." This phrase specifies that B is a factor used to determine A or that influences the decision of A. This phrase does not exclude that the decision of A may also be based on some other factor, such as C. This phrase is also intended to cover embodiments in which A is determined only based on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on."
[0197] As used herein, the phrase "in response to" describes one or more factors that trigger an effect. This phrase does not exclude the possibility that additional factors may affect or trigger the effect. That is, the effect may be responsive only to these factors or to the specified factors and other unspecified factors. Consider the phrase "performing A in response to B." This phrase specifies that B is the factor that triggers the execution of A. This phrase does not exclude that performing A may be responsive to other factors, such as C. This phrase is also intended to cover embodiments in which A is performed only in response to B.
[0198] As used herein, the terms "first," "second," etc. are used as labels for the nouns they precede and do not imply any kind of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise specified. For example, in a register file having eight registers, the terms "first register" and "second register" are used to refer to any two of the eight registers and not, for example, simply to logical registers 0 and 1.
[0199] As used in the claims, the term "or" is used as an inclusive or and not an exclusive or. For example, "at least one of x, y, or z" means any one of x, y, and z, as well as any combination thereof.
[0200] As used herein, the term "and / or" in reference to two or more elements should be construed to mean only one element or a combination of elements. For example, "element A, element B, and / or element C" may include element A only, element B only, element C only, elements A and element B, elements A and element C, elements B and element C, or elements A, B, and C. In addition, "at least one of element A or element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Also, "at least one of element A and element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.
[0201] The subject matter of the present disclosure has been specifically described herein to satisfy legal requirements. However, the description itself is not intended to limit the scope of the present disclosure. Rather, the inventors acknowledge that the subject matter of the claims may be embodied in other ways to include different steps or combinations of steps similar to those described herein in connection with other current or future technologies. Furthermore, the terms "step" and / or "block" may be used to refer to different elements of the methods used herein, but should not be construed as implying a particular order between the various steps disclosed herein, unless the order of the individual steps is explicitly described.
[0202] While exemplary embodiments have been described in detail, it should be apparent that modifications and variations are possible without departing from the scope of the claimed disclosure. The scope of the disclosed subject matter is not limited to the described embodiments, but rather is set forth in the following claims.
Claims
1. Receiving a starting position and attributes of a target object to be lifted by a robot, the robot including a robot control system, a shovel, a grabber pad arm with a grabber pad, and at least one wheel or one track for movement of the robot; determining an object separation strategy, including using at least one of a reinforcement learning based strategy including rewards and penalties, a rule based strategy and a dependency on observations, current object state and sensor data; executing the object separation strategy to separate the target object from other objects; determining a pick-up strategy, the pick-up strategy comprising: an approach path of the robot to the target object; a grip height for initial contact with the target object; a gripping pattern for movement of the grabber pad during capture of the target object; a holding position of the grabber pad and the shovel for securing the target object in a storage area on the robot for transport, the storage area including at least two of the grabber pad arm, the grabber pad, and the shovel; and executing the pick-up strategy, the pick-up strategy comprising: extending the grabber pad outwardly and forwardly relative to the grabber pad arm to raise the grabber pad to the gripping height; approaching the target object via the approach path and stopping when the target object is located between the grabber pads; executing the grasping pattern to capture the target object within the containment area; determining whether the target object is within the containment area; executing the pick-up strategy, Provided that the target object is within the containment area, applying pressure to the target object with the grabber pad to immobilize and hold the target object within the containment area; Raising at least one of the shovel and the grabber pad to the holding position for holding the target object; and provided that the target object is not within the containment area; Varying the picking strategy using at least one of a different reinforcement learning based strategy, a different rule based strategy and depending on different observations, current object states and sensor data; Executing the modified pick-up strategy; and and The method includes:
2. Navigating to a drop location at the destination; determining a drop strategy using a machine learning model or a rule-based approach; Executing the drop strategy, the execution of the drop strategy comprising: determining a destination approach path and an object placement pattern, the object placement pattern being one of a drop pattern or a place pattern; approaching the destination via the destination approach route; Provided that the object arrangement pattern is the installation pattern, stopping at the destination with the shovel and grabber pad forward; Lowering the shovel and the grabber pad to a deployment height; using the grabber pad to at least one of push the target object out of the storage area to the drop location and tilting the shovel forward to drop the target object out of the storage area to the drop location; and On the condition that the object arrangement pattern is the drop pattern, stopping at the destination with the shovel and the grabber pad rearward; Raising the shovel and the grabber pad to the deployment height; extending the grabber pad to allow the target object to drop from the containment area to the drop location; and The method of claim 1 further comprising:
3. The rule based strategy for the drop strategy comprises: navigating the robot to a position adjacent a side of a box; rotating said robot in place to align it facing said box; driving the robot towards the box while maintaining alignment about a side of the box; stopping a short distance from the side of the box; Using the rear camera to navigate when attempting a backward fall; Using the forward camera to navigate when attempting a forward fall; Ensuring that the robot is correctly positioned relative to the sides of the box; Provided that the robot is accurately positioned, At least one of lifting the shovel upward and backward to drop the target object into the box and lifting the shovel and tilting the shovel forward to drop the target object into the box; and if the robot is not correctly positioned, then leave the box and re-execute the drop strategy. The method of claim 2 , comprising at least one of:
4. The rewards and penalties for executing the drop strategy are: A penalty is added for each second over the maximum time, a reward for the robot docking correctly with the storage bin; A reward when the target object is successfully dropped into the storage box; a penalty for a collision that moves the bin; A penalty for collisions with obstacles or walls that exceed the maximum force feedback, and a penalty if the robot gets stuck or runs over the target object; The method of claim 2 , comprising at least one of:
5. The reward and penalty for implementing the object separation strategy are: A penalty is added for each second over the maximum time, a reward if the correct grabber pad arm is between the target object and the wall; a reward if the distance of the target object from the wall exceeds a predetermined distance; A penalty for accidentally colliding with the target object; A penalty for collisions with obstacles or walls that exceed the maximum force feedback, and a penalty if the robot gets stuck or runs over the target object; The method of claim 1 , comprising at least one of:
6. The rewards and penalties for implementing the pick-up strategy are: A penalty is added for each second over the maximum time, a reward for the first contact of the target object with the edge of the shovel; A reward when the target object is pushed completely into the shovel; a penalty if the target object is lost from the shovel; Penalty for collisions with obstacles or walls that exceed the maximum force feedback, Penalty for picking up a non-target object, and a penalty if the robot gets stuck or runs over the target object; The method of claim 1 , comprising at least one of:
7. A rule-based strategy for separating objects is navigating the robot to a position facing the target object to be separated but far enough away to open the grabber pad arm and the grabber pad and lower the shovel; opening the grabber pad arm and the grabber pad, lowering the grabber pad arm and the grabber pad, and lowering the shovel; rotating the robot slightly in place so that the target object is in the center of the front view; opening the grabber pad arms and the grabber pad slightly wider than the target object; slowly driving the grabber pad arm and the end of the grabber pad forward until they are positioned beyond the target object; Slightly closing the grabber pad arms and the grabber pad in a V-shape so that the grabber pad arms and the grabber pad surround the target object; and driving backwards a short distance, thereby moving the target object into an open space; The method of claim 1 , comprising at least one of:
8. and evaluating the success of picking up the target object, the evaluating the success of picking up the target object comprising: detecting the target object within the containment area of the shovel and grabber pad arm to determine whether the target object is within the containment area; receiving force feedback from an actuator force feedback sensor indicating that the target object is being held by the grabber pad arm; tracking the movement of the target object during pick-up within the shovel area and maintaining a state of the target object in a memory; detecting an increase in weight of the shovel during lifting of the target object, indicating that the target object is within the shovel; utilizing a classification model to determine whether the target object is within the shovel; and reconfirming that the target object is within the shovel during the robot's movement using at least one of the force feedback, the weight gain, and a dedicated camera; The method of claim 1 , comprising at least one of:
9. The reinforcement learning strategy and the rule-based strategy include actions to control individual actuators, the actions to control individual actuators being: Move the left grabber pad arm to a new position by rotating it up or down; Move the left grabber pad list to a new position by rotating it left or right; Move the right grabber pad arm to a new position by rotating it up or down; Move the right grabber pad list to a new position by rotating it left or right; Lifting the shovel to a new position by rotating it upwards or downwards; Using a second motor or a second actuator, varying the shovel angle to effect a forward drop of the target object; driving the left wheels or left track of the robot; and driving the right wheels or right track of the robot; The method of claim 1 , comprising at least one of:
10. The reinforcement learning strategy and the rule-based strategy include a composite action that controls an actuator, the composite action controlling the actuator comprising: driving the robot following a path to a location or waypoint; rotating the robot left or right in place; Centering the robot relative to the target object; aligning the grabber pad arm with a top, bottom or central section of the target object; driving the object forward until it contacts the edge of the shovel; closing both grabber pad arms and pushing the target object in with one smooth motion; Lifting the shovel and grabber pad arm together while gripping the target object; closing both grabber pad arms and pushing the target object with a quick tap and slight release; placing the shovel lightly on the floor; pushing the shovel down onto the floor; closing said grabber pad arms and holding the position until resistance is encountered; vibrating the grabber pad arms closed and rotating left or right to create instability and slightly bounce flat objects over the edge of the shovel; The method of claim 1 , comprising at least one of:
11. A robot comprising: Shovel and a grabber pad arm having a grabber pad; At least one wheel or one track for the movement of the robot; A processor; a memory storing instructions that, when executed by the processor, enable operation and control of the robot; A robot including: A base station; A plurality of boxes for containing objects; a robot control system in at least one of the robot and a cloud server; Logic and A robot system comprising: The logic comprises: receiving a starting position and attributes of a target object to be lifted; determining an object separation strategy, including using at least one of a reinforcement learning based strategy including rewards and penalties, a rule based strategy and a dependency on observations, current object state and sensor data; executing the object separation strategy to separate the target object from other objects; determining a pick-up strategy, the pick-up strategy comprising: an approach path of the robot to the target object; a grip height for initial contact with the target object; a gripping pattern for movement of the grabber pad during capture of the target object; a holding position of the grabber pad and the shovel for securing the target object in a storage area on the robot for transport, the storage area including at least two of the grabber pad arm, the grabber pad, and the shovel; and Executing the pick-up strategy, the executing the pick-up strategy comprising: extending the grabber pad outwardly and forwardly relative to the grabber pad arm to raise the grabber pad to the gripping height; approaching the target object via the approach path and stopping when the target object is located between the grabber pads; executing the grasping pattern to capture the target object within the containment area; determining whether the target object is within the containment area; executing the pick-up strategy, Provided that the target object is within the containment area, applying pressure to the target object with the grabber pad to immobilize and hold the target object within the containment area; Raising at least one of the shovel and the grabber pad to the holding position for holding the target object; and provided that the target object is not within the containment area; Varying the picking strategy using at least one of a different reinforcement learning based strategy, a different rule based strategy and depending on different observations, current object states and sensor data; Executing the modified pick-up strategy; and A robot system that performs the above operations.
12. Navigating to a drop location at the destination; determining a drop strategy using a machine learning model or a rule-based approach; Executing the drop strategy, the execution of the drop strategy comprising: determining a destination approach path and an object placement pattern, the object placement pattern being one of a drop pattern or a place pattern; approaching the destination via the destination approach route; Provided that the object arrangement pattern is the installation pattern, stopping at the destination with the shovel and grabber pad forward; Lowering the shovel and the grabber pad to a deployment height; using the grabber pad to at least one of push the target object out of the storage area to the drop location and tilting the shovel forward to drop the target object out of the storage area to the drop location; and On the condition that the object arrangement pattern is the drop pattern, stopping at the destination with the shovel and the grabber pad rearward; Raising the shovel and the grabber pad to the deployment height; extending the grabber pad to allow the target object to drop from the containment area to the drop location; and The robotic system of claim 11 further comprising logic.
13. The rule based strategy for the drop strategy comprises: navigating the robot to a position adjacent a side of a box; rotating said robot in place to align it facing said box; driving the robot towards the box while maintaining alignment about a side of the box; stopping a short distance from the side of the box; Using the rear camera to navigate when attempting a backward fall; Using the forward camera to navigate when attempting a forward fall; Ensuring that the robot is correctly positioned relative to the sides of the box; Provided that the robot is accurately positioned, At least one of lifting the shovel upward and backward to drop the target object into the box and lifting the shovel and tilting the shovel forward to drop the target object into the box; and if the robot is not correctly positioned, then leave the box and re-execute the drop strategy. The robotic system of claim 12 , comprising at least one of:
14. The rewards and penalties for executing the drop strategy are: A penalty is added for each second over the maximum time, a reward for the robot docking correctly with the storage bin; A reward when the target object is successfully dropped into the storage box; a penalty for a collision that moves the bin; A penalty for collisions with obstacles or walls that exceed the maximum force feedback, and a penalty if the robot gets stuck or runs over the target object; The robotic system of claim 12 , comprising at least one of:
15. The reward and penalty for implementing the object separation strategy are: A penalty is added for each second over the maximum time, a reward if the correct grabber pad arm is between the target object and the wall; a reward if the distance of the target object from the wall exceeds a predetermined distance; A penalty for accidentally colliding with the target object; A penalty for collisions with obstacles or walls that exceed the maximum force feedback, and a penalty if the robot gets stuck or runs over the target object; The robotic system of claim 11 , comprising at least one of:
16. The rewards and penalties for implementing the pick-up strategy are: A penalty is added for each second over the maximum time, a reward for the first contact of the target object with the edge of the shovel; A reward when the target object is pushed completely into the shovel; a penalty if the target object is lost from the shovel; Penalty for collisions with obstacles or walls that exceed the maximum force feedback, Penalty for picking up a non-target object, and a penalty if the robot gets stuck or runs over the target object; The robotic system of claim 11 , comprising at least one of:
17. A rule-based strategy for separating objects is navigating the robot to a position facing the target object to be separated but far enough away to open the grabber pad arm and the grabber pad and lower the shovel; opening the grabber pad arm and the grabber pad, lowering the grabber pad arm and the grabber pad, and lowering the shovel; rotating the robot slightly in place so that the target object is in the center of the front view; opening the grabber pad arms and the grabber pad slightly wider than the target object; slowly driving the grabber pad arm and the end of the grabber pad forward until they are positioned beyond the target object; Slightly closing the grabber pad arms and the grabber pad in a V-shape so that the grabber pad arms and the grabber pad surround the target object; and driving backwards a short distance, thereby moving the target object into an open space; The robotic system of claim 11 , comprising at least one of:
18. The method further includes logic for evaluating success of picking up the target object, the logic for evaluating success of picking up the target object comprising: detecting the target object within the containment area of the shovel and grabber pad arm to determine whether the target object is within the containment area; receiving force feedback from an actuator force feedback sensor indicating that the target object is being held by the grabber pad arm; tracking the movement of the target object during pick-up within the shovel area and maintaining a state of the target object in a memory; detecting an increase in weight of the shovel during lifting of the target object, indicating that the target object is within the shovel; utilizing a classification model to determine whether the target object is within the shovel; and reconfirming that the target object is within the shovel during the robot's movement using at least one of the force feedback, the weight gain, and a dedicated camera; The robotic system of claim 11 , comprising at least one of:
19. The reinforcement learning strategy and the rule-based strategy include actions to control individual actuators, the actions to control individual actuators being: Move the left grabber pad arm to a new position by rotating it up or down; Move the left grabber pad list to a new position by rotating it left or right; Move the right grabber pad arm to a new position by rotating it up or down; Move the right grabber pad list to a new position by rotating it left or right; Lifting the shovel to a new position by rotating it upwards or downwards; Using a second motor or a second actuator, varying the shovel angle to effect a forward drop of the target object; driving the left wheels or left track of the robot; and driving the right wheels or right track of the robot; The robotic system of claim 11 , comprising at least one of:
20. The reinforcement learning strategy and the rule-based strategy include a composite action that controls an actuator, the composite action controlling the actuator comprising: driving the robot following a path to a location or waypoint; rotating the robot left or right in place; Centering the robot relative to the target object; aligning the grabber pad arm with a top, bottom or central section of the target object; driving the object forward until it contacts the edge of the shovel; closing both grabber pad arms and pushing the target object in with one smooth motion; Lifting the shovel and grabber pad arm together while gripping the target object; closing both grabber pad arms and pushing the target object with a quick tap and slight release; placing the shovel lightly on the floor; pushing the shovel down onto the floor; closing said grabber pad arms and holding the position until resistance is encountered; vibrating the grabber pad arms closed and rotating left or right to create instability and slightly bounce flat objects over the edge of the shovel; The robotic system of claim 11 , comprising at least one of:
