Robot for stacking elements

Abstract

An example autonomous vehicle includes a body configured to move along a surface and an end effector holding a first element having first and second features that mate with complementary third and fourth features on a second element. At least one of the end effector or the body is controllable to move in at least four degrees of freedom. The autonomous device includes one or more sensors to detect the second element and obtain information for positioning the third and fourth features of the second element or other compatible device. The autonomous device also includes a control system to control at least one of the end effector or the body to move in the at least four degrees of freedom to stack the first element on top of the second element.

Description

TECHNICAL FIELD

[0001] This specification generally relates to examples of mobile robots or components thereof configured to move in multiple degrees of freedom to stack one element on top of another element. BACKGROUND

[0002] Forklifts or other drivable machines can be used to stack elements in a space, such as a warehouse or manufacturing plant. Examples of elements include pallets and containers. An example pallet includes a flat transport structure that supports cargo during lifting. An example container includes a transportable structure with one or more vertical walls and has interlocking mechanisms designed to prevent relative motion between elements. SUMMARY

[0003] An example autonomous vehicle includes a body configured to move along a surface and an end effector holding a first element having a first feature and a second feature that match a complementary third feature and a fourth feature on a second element. At least one of the end effector or the body is controllable to move in at least four degrees of freedom. The at least four degrees of freedom include forward / backward, up / down, left / right, and at least one of yaw, pitch, or roll. The autonomous device includes one or more sensors to detect the second element and obtain information for positioning the third feature and the fourth feature of the second element (or another compatible device). The autonomous device also includes a control system that controls at least one of the end effector or the body to move in the at least four degrees of freedom to stack the first element on top of the second element by performing operations including moving the first element to align the first feature with the third feature, moving the first element to align the second feature with the fourth feature, and moving the first element into contact with the second element such that the first feature matches the third feature and the second feature matches the fourth feature. The autonomous device can include one or more of the following features, alone or in combination.

[0004] The control system can be programmed to determine, based on the information, a first vector from the autonomous vehicle to the second element and a second vector from a sensor of the one or more sensors to the first element, and control at least one of the end effector or the body to move based on a difference between the first vector and the second vector. The control system can be configured (e.g., programmed) to move the first element in at least one of a forward / backward, up / down, or left / right motion to align the first feature with the third feature. The control system can be configured to move the first element in at least one of a pitch, yaw, or roll motion to align the second feature with the fourth feature. The control system can be configured to control at least one of the end effector or the body to move in at least five degrees of freedom, where the at least five degrees of freedom include: forward / backward, up / down, left / right, and at least two of yaw, pitch, or roll. The control system can be configured to control at least one of the end effector or the body to move in six degrees of freedom, where the six degrees of freedom include: forward / backward, up / down, left / right, pitch, yaw, and roll.

[0005] The one or more sensors can be configured to locate the third feature and the fourth feature in a three-dimensional (3D) space. The one or more sensors can be configured to detect the second element in the three-dimensional (3D) space and obtain 3D coordinates of the second element. The one or more sensors can include one or more of a light detection and ranging (LIDAR) sensor, a time-of-flight (TOF) sensor, a radar sensor, a sonar sensor, a two-dimensional camera sensor, or a three-dimensional camera sensor.

[0006] The control system can include one or more processing devices that reside on the autonomous vehicle and are configured to execute software to perform operations independent of data input from an external source. The one or more processing devices that reside on the autonomous vehicle can be configured to execute software to perform operations based at least in part on data input from a fleet system external to the autonomous vehicle. The one or more processing devices on the autonomous vehicle can be programmed to receive information from the fleet system, and the one or more processing devices can be programmed to control movement of the body toward the second element based on the information. The one or more processing devices on the autonomous vehicle can be programmed to identify a type of the second element based on the information.

[0007] An autonomous vehicle can include a computer memory storing a database containing data identifying different types of elements and attributes of different types of elements. A control system can include one or more processing devices residing on the autonomous vehicle and programmed to obtain data from the database to identify a type of a second element and attributes of the second element or a compatible location. The attributes can relate to third and fourth features of the second element. The one or more processing devices can be programmed to receive information from a fleet system external to the autonomous vehicle. The one or more processing devices can be programmed to move to a location proximate the second element based on the information and, after reaching the location, control at least one of an end effector or a body to move in at least four degrees of freedom to stack the first element on top of the second element independent of input from the fleet system. The received information can include a map of an area in which the second element is located.

[0008] The first and second elements can each be or include compatible interlocking devices. The end effector can be configured to move in at least four degrees of freedom. The control system can be configured to control the end effector to move in the at least four degrees of freedom to stack the first element on top of the second element. The end effector can be configured to move in at least five degrees of freedom, where the at least five degrees of freedom include: forward / backward, up / down, left / right, and at least one of pitch, yaw, or roll. The control system can be configured to control the end effector to move in the at least five degrees of freedom to stack the first element on top of the second element. The end effector can be configured to move in six degrees of freedom, where the six degrees of freedom include: forward / backward, up / down, left / right, pitch, yaw, and roll. The control system can be configured to control the end effector to move in the six degrees of freedom to stack the first element on top of the second element.

[0009] The end effector can include tines holding the first element. The tines can include first and second tines. The first tine can be configured to move independent of the second tine in at least one degree of freedom, and the second tine can be configured to move independent of the first tine in at least one degree of freedom. At least one of the third or fourth features on the second element is or includes in a corner or a side or any other feature of the second element.

[0010] An example method performed by an autonomous vehicle includes the following operations: picking up, using an end effector connected to a body of the autonomous vehicle, a first element having a first feature and a second feature that match a complementary third feature and a fourth feature on a second element; moving the autonomous vehicle holding the first element along a surface toward a location of the second element; detecting the second element, wherein detecting includes locating the third feature and the fourth feature of the second element; and controlling at least one of the end effector or the body to move in at least four degrees of freedom to stack the first element on top of the second element by performing the following operations, which include: moving the first element to align the first feature with the third feature, moving the first element to align the second feature with the fourth feature, and moving the first element into contact with the second element such that the first feature matches the third feature and the second feature matches the fourth feature. The at least four degrees of freedom include forward / backward, up / down, left / right, and at least one of yaw, pitch, or roll.

[0011] Any two or more of the features described in this specification, including in this summary section, can be combined to form specific embodiments of the application not specifically described in this document.

[0012] Example robots and techniques described herein, or portions thereof, can be implemented using a computer program product, which includes instructions stored on one or more non-transitory machine-readable storage media and which are executable on one or more processing devices to control (e.g., coordinate) operations described herein. Example robots and techniques described herein, or portions thereof, can be implemented as a device or electronic system, which can include one or more processing devices and memory storing executable instructions for implementing various operations.

[0013] The details of one or more specific embodiments are set forth in the accompanying drawings and the detailed description below. Other features, objects, and advantages will be apparent from the detailed description and drawings, and from the claims. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a photo-realistic drawing showing a perspective view of an example autonomous vehicle.

[0015] Figure 2 is a schematic drawing illustrating six degrees of freedom.

[0016] Figure 3 is a block diagram showing a perspective view of an autonomous vehicle of Figure 1

[0017] Figure 4 is a block diagram showing a perspective view of an example element.

[0018] ​Figure 5 is a block diagram showing a perspective view of an example element.

[0019] Figure 6 is a photorealistic drawing showing a close-up perspective view of an example fit between two elements.

[0020] Figure 7 is a flowchart showing an example operation of a stacked element that can be performed at least in part by an autonomous vehicle of Figure 1

[0021] Figure 8 is a photorealistic drawing showing a perspective view of an example pallet stack.

[0022] Figure 9 is a drawing showing a perspective view of an example bracket that can be used to stack elements.

[0023] Figures 10 to 15 is a block diagram containing element perspective views and illustrating an example element stacking process of Figure 7

[0024] Like reference numbers in different drawings indicate like elements. DETAILED DESCRIPTION

[0025] Examples of autonomous vehicles configured to stack elements are described herein. Stacking can include placing one element on top of another element of the same or different type. Autonomous vehicles used as examples herein include mobile robots (or simply “robots”); however, any appropriate type of autonomous vehicle can be used, including but not limited to an automated driving machine or a stationary robot. Elements used as examples herein include pallets and containers; however, any appropriate type of element can be used, including but not limited to boxes, brackets, crates, or bins. As described above, an example pallet includes a flat transport structure that supports a load during lifting. The pallet also includes a mechanism that an end effector of a robot can enter and use to connect and lift the pallet. An example container includes a transportable structure having one or more vertical walls and having an interlocking mechanism designed to prevent relative motion.

[0026] ​​An example autonomous vehicle, such as a robot, includes a body configured to move along a surface, such as a warehouse floor. An end effector, such as a fork containing one or more tines, is configured to hold a first element having features that match complementary or compatible features on a second element. The second element includes any type of device that is compatible with the first element. For example, the first or second element can include any appropriate type of compatible interlocking element. In an example, the end effector is configured to hold a first element having first and second features that match complementary third and fourth features on a second element located on the floor or in a rack. The end effector, the body, or a combination of the end effector and the body is controllable to move in at least four degrees of freedom to move the first element relative to the second element. In an example, the at least four degrees of freedom include forward / backward, up / down, left / right, and at least one of yaw, pitch, or roll. One or more sensors, such as two-dimensional (2D) sensors and / or three-dimensional (3D) sensors, are configured to detect the second element and obtain information for positioning features (e.g., corners or alignment mechanisms) of the first and second elements.

[0027] A control system, which can include one or more processing devices, is configured (e.g., programmed) to control the end effector, the robot body, or both the end effector and the robot body to move in at least four degrees of freedom to stack the first element on top of the second element, where examples of the processing devices are described herein. The stacking can include aligning the first, second, third, and fourth features of the elements. Operations performed to perform the stacking can include, but are not limited to, the following: moving the first element to align the first feature with the third feature, moving the first element to align the second feature with the fourth feature, and after these alignments, moving the first element into contact with the second element such that the first feature matches the third feature and the second feature matches the fourth feature. In an example where the elements are boxes and the features are corners of the boxes, the operations include moving the first box to align a first corner of the first box with a third corner of the first box, moving the first box to align a second corner of the first box with a fourth corner of the first box, and after these alignments, moving the first box into contact with the second box such that the first corner matches the third corner and the second corner matches the fourth corner. In some examples, each corner can include a part, such as a pin or a notch, to which each alignment is made.

[0028] Figure 1An example of a robot 10 is shown that is configured to move in multiple degrees of freedom to stack one element on top of another element. In this example, the robot 10 is autonomously controllable, even though it includes mechanisms 14 for manual control. In examples, autonomous controllability includes the robot moving automatically based on sensor input, and in some cases, based on input from a remote system, such as a fleet control system. The robot 10 includes a body 12 with wheels (not shown) to enable the robot 10 to travel across a surface, such as a warehouse, factory floor, or other terrain. The robot 10 also includes a support area 15 that is configured to support the weight of an element, such as a pallet, container, or any other device to be stacked, using an end effector 16. In this example, the robot 10 can be controlled to transport an element from one location to another.

[0029] As shown in Figure 1 In this example, the end effector 16 includes a fork made up of two tines 20, 21, as shown in FIG. 1. Other types of end effectors can also be used, such as a plate or a clamp. The tines can be configured to move vertically in the direction of arrow 22. This enables the tines to pick up an element and move the element to the appropriate vertical height for stacking. The tines can also be configured to move horizontally in the direction of arrow 23. In some examples, the tines are connected to each other and thus move together. In some examples, each tine can be configured to move independently and separately horizontally in the direction of arrow 23. That is, each tine can move relative to the other tine to adjust the distance (or spacing) between the two. This adjustment can be necessary to accommodate elements with different socket positions. In some examples, each tine can be configured to move independently and separately vertically in the direction of arrow 23. In some implementations, one of the tines can be moved away to allow a single tine to interact with an element or other elements. For example, tine 20 can be rotated 90° in the direction of arc 24, leaving tine 21 in a position to interact with an element or other elements located in front of the robot 10. The other tine 21 can be similarly operated.

[0030] The end effector, the robot body, or the combination of the end effector and the robot body can move in four, five, or six degrees of freedom in order to move and manipulate one element so that the element is stacked on top of another element. That is, the end effector, the robot body, or the combination of the end effector and the robot body can move in four, five, or six degrees of freedom in order to move and manipulate one element to be placed on top of another element of a similar or different type. Figure 2Movements relative to the six degrees of freedom of the Cartesian X-axis, Y-axis, and Z-axis 27 are illustrated. The six degrees of freedom include forward / backward (surge) 28, up / down (heave) 29, left / right (sway) 30, yaw 33, pitch 34, and roll 35.

[0031] In some implementations, the end effector is controllable to move independently of the robot body in at least four degrees of freedom, including forward / backward, up / down, left / right, and at least one of yaw, pitch, or roll. In some implementations, the end effector is controllable to move independently of the robot body in at least five degrees of freedom, including forward / backward, up / down, left / right, and at least two of yaw, pitch, or roll. In some implementations, the end effector is controllable to move independently of the robot body in six degrees of freedom, including forward / backward, up / down, left / right, yaw, pitch, and roll. These movements enable the element to move in four, five, or six degrees of freedom.

[0032] In some implementations, the robot body is controllable to move in at least four degrees of freedom, including forward / backward, up / down, left / right, and at least one of yaw, pitch, or roll. In some implementations, the robot body is controllable to move in at least five degrees of freedom, including forward / backward, up / down, left / right, and at least two of yaw, pitch, or roll. In some implementations, the robot body is controllable to move in six degrees of freedom, including forward / backward, up / down, left / right, yaw, pitch, and roll. These movements enable the element to move in four, five, or six degrees of freedom.

[0033] Because the end effector is connected to the robot body, the end effector can move with the body. The end effector can also be configured to move independently of the robot body in the movements described above. That is, the body can move in a number of degrees of freedom, and the end effector can move separately from the robot body in the same number of degrees of freedom, a fewer number of degrees of freedom, or a greater number of degrees of freedom. For example, if the body moves forward, the end effector can move forward with the body, but the end effector can also move further forward or left / right independently of the movement of the body.

[0034] In some implementations, the end effector and the body can be controlled together to move the element in at least four degrees of freedom, including forward / backward, up / down, left / right, and at least one of yaw, pitch, or roll. In some implementations, the end effector and the body can be controlled together to move the element in at least five degrees of freedom, including forward / backward, up / down, left / right, and at least one of yaw, pitch, or roll. In some implementations, the end effector and the body can be controlled together to move the element in six degrees of freedom, including forward / backward, up / down, left / right, yaw, pitch, and roll. For example, the body can be configured to move forward / backward and left / right, and the end effector can be configured to move up / down and at least one of yaw, pitch, or roll (four degrees), at least two of yaw, pitch, or roll (five degrees), or all of yaw, pitch, or roll (six degrees). Different combinations of movement can be achieved than those described here.

[0035] Also refer to Figure 3 The block diagram shows one or more sensors 36a, 36b, 36c, 36d, and 36e located on robot 10 for detecting the position of the robot itself, for detecting an element to be picked up, and / or for detecting the position of placed (e.g., stacked) elements. The sensors can also be used to detect the position of alignment features on the elements and to track their positions as the robot and / or elements move relative to the stack. The sensors are configured to acquire 3D data from at least a position in front of the end effector. Examples of sensors include 2D and 3D sensors. For example, robot 10 may include a 3D camera, a light detection and ranging (LIDAR) scanner, an optical sensor, a sonar sensor, a time-of-flight (TOF) sensor, a radar sensor, a 2D camera sensor, or any suitable combination thereof. Multiple 2D sensors can be used to acquire 3D data using 2D sensors. It is worth noting that the example robot is not limited to these types of sensors.

[0036] In the example, robot 10 includes a 3D camera located 40 degrees in front of the robot. The 3D camera can capture red, green, blue, and depth (RGBD) data. In this example, the front of the robot faces the direction of travel. Figure 3In the example, the front of the robot may include an arc spanning 180° or less from one side of the robot to the opposite side. In the example, robot 10 may include multiple LiDAR scanners located at the front of the robot. Each LiDAR scanner is configured to detect objects within a sensing plane. Two or more LiDAR scanners may be configured and arranged to acquire 2D data in orthogonal sensing planes. When appropriately correlated and / or combined, this 2D data constitutes 3D information obtained from the front of the robot. Combinations of these and / or other sensors may be used to obtain 3D data representing the space in front of the robot. The 3D data may include 3D (e.g., Cartesian XYZ) coordinates representing the space.

[0037] In implementations that include multiple sensors on the front of the robot, the sensors can be located at different positions, such as different heights. Furthermore, one or more sensors can be movable on the robot. In some examples, sensors 36a and 36c can be located on the end effector and can move with it. Sensors located here enable the robot to detect and image components, such as pallets or containers, located directly in front of the end effector. Data from such sensors allows the end effector to identify sockets in the component, and thus facilitates the end effector entering the sockets in the component. In some examples, one or more sensors can be located on the body of the robot. For example, one or more sensors 36d can be located at the midpoint of the robot, one or more sensors 36b can be located at the bottom of the robot, and / or one or more sensors 36e can be located at the top of the robot. Strategically placing sensors in these or other locations allows the robot to capture images of components in front of it even when the robot is holding a component that is blocking the sensor. Example components include, but are not limited to, supports, pallets, fixed or movable devices including compatible interlocking mechanisms, stacks of one or more components, or supports containing stacks of one or more components. For example, if correctly positioned, if one sensor is blocked, such as by one or more components on the end effector, another sensor will not be blocked by those components, enabling continuous sensing. The data captured by the sensors is sent to the robot's control system for processing and use.

[0038] In some implementations, robot 10 may include additional sensors at locations other than the front of the robot. For example, sensors of the type described herein may be included on one or both sides of the robot and / or the rear of the robot. In this example, the rear of the robot is the opposite side of the front of the robot. Figure 3 In the example, the rear 41 of the robot includes an arc spanning 180° or less from one side of the robot to the opposite side. Figure 3In the example, the robot's sides 42, 43 may include an arc spanning 180° or less from the robot's direction of travel to the opposite direction. These sensors can assist in operations such as object detection and localization.

[0039] A lidar scanner, 3D camera, and / or any sensor constitute the robot's vision system. Visual data acquired by the vision system can be used to determine the robot's position within the space it is traversing. In some implementations, the control system 46 stores a mapping map of the space to be traversed in a computer memory. The components of the control system 46 are... Figure 3 The mapping is shown in dashed lines because at least a portion of the control system can be inside the robot. The mapping map can be located on the robot or anywhere accessible to the control system. The mapping map can include the positions of landmarks (such as posts, corners, windows, poles, and other distinguishable features of the space serving as a reference for the robot). The mapping map can include the size and distinguishing features of the landmarks, such as color, shape, texture, etc., such as posts, corners, windows, poles, and other distinguishable features of the space serving as a reference for the robot. The mapping map can also include measurements indicating the size of the space, measurements indicating the size and position of the landmarks, measurements indicating the distances between the landmarks, and coordinate information identifying the positions of the landmarks in the space. The control system uses information from the mapping map to move throughout the space and uses visual data from the vision system and data from the mapping map to determine the robot's position in the space. For example, the robot can identify the positions of three landmarks in the space. By knowing the robot's position relative to these landmarks, the positions of the landmarks on the mapping map and therefore in the space, and the distances between the landmarks, the control system can determine the robot's position in the space. This information can be used to locate the component to be picked up, or to locate another component to be placed, such as a component that a stacking robot is holding or will hold. Examples of other components include similar types of components, compatible interlocking devices, another component that is similar or different types, or supports.

[0040] In some implementations, the on-board control system on the robot can use a pre-planned route through the map to identify where to position an element to be picked up, or to position one or more of the other elements on which the element the robot is holding is to be placed: an element stack, a compatible interlocking device, or a holder. In some implementations, the on-board control system on the robot can not use a pre-planned route through the map to identify where to position an element to be picked up, or to position one of the other elements on which the element the robot is holding is to be placed. In the latter example, the robot can move through the space or around the space and, upon detecting an object, attempt to identify the element on which the robot is controlled to move. The identification can be performed by reference to a database containing element attributes, as explained herein. After the robot identifies the element of interest, the robot can move through the space and around the space to identify one of the other elements on which the element the robot is holding is to be placed. The location of the other element can be pre-programmed, or the robot can search the entire space using its sensor system to identify the other element. In this regard, in some implementations, the element the robot is holding or the other element on which that element is to be placed can contain a marker such as a barcode, QR code, or serial number. The robot can identify such markers on the element and the other element, compare the identified markers, and when a match is detected, determine that the element is to be placed on the other element. In some implementations, these operations can be performed based on the dimensions of the element and the other element, or other distinguishing features on the element and the other element such as color or markings.

[0041] The control system 46 can include circuitry or an on-board computing system that controls operation of the robot. The circuitry or on-board computing system is “on-board” in the sense that it is located on the robot itself. The control system can include, for example, one or more microcontrollers, one or more microprocessors, programmable logic such as a field programmable gate array (FPGA), one or more application specific integrated circuits (ASICs), solid state circuitry, or any appropriate combination of two or more of these types of processing devices. In some implementations, the on-board components of the control system can be in communication with a remote computing system. The computing system is remote in the sense that it is not located on the robot itself. For example, the control system can also include computing resources that are distributed to at least a portion of a remote service, e.g., a central service or a cloud service, that is not on the robot. Commands provided by the remote computing system can be communicated for execution by the on-board computing system. In some implementations, the control system includes only on-board components. In some implementations, the control system includes a combination of on-board components and a remote computing system. In some implementations, the control system can be configured (e.g., programmed) to implement control functions and robot movements without local or remote input from a user. In some implementations, the control system can be configured to implement control functions, including positioning, based at least in part on input from a user.

[0042] In some implementations, the remote control system can include a fleet control system. The fleet control system can include one or more computing devices that operate together to control, influence, or instruct a plurality of robots of the type described herein. For example, the fleet control system can be configured to coordinate operation of a plurality of robots, including instructing the robots to move to a location where an element is located and a location where the element is to be stacked (e.g., placed). In some implementations, the fleet control system can store, maintain, and update a map of a space in which one or more robots are to operate. The map can be accessible by each robot through the fleet control system, or the map can be downloaded to all or some of the robots operating in the space periodically, intermittently, or aperiodically. The map can then be used to identify where an element is located and where the element is to be stacked. For example, a robot can use the map to position itself proximate to an element in order to pick up the element; the robot can use the map to navigate to another element at which the robot is to place the element that the robot has picked up; and the robot can use the map to position itself proximate to the other element at which the robot is to place the element that the robot has picked up. In this example, positioning can include moving the robot (which can include moving the body, the end effector, or both) so that its end effector is aligned with a socket in the element to be picked up.

[0043] In some implementations, the control system, including its remote portions, can be distributed among multiple robots operating in the space. For example, where one of the robots can receive the map from the fleet controller and distribute the map to robots operating locally within the space. Similarly, one or more robots within the space can send commands and control signals to other robots.

[0044] Whether on the robot, off the robot, or a combination of both, the control system can include a computer memory that stores a database containing data identifying different types of elements and attributes of different types of elements. For example, the database can include attributes identifying the configuration of the element, the model of the element, the number of sockets in the element, the dimensions of the element including length, width, and depth (XYZ), the material from which the element is made, the weight of the element, the element interface (or stacking) features, markings or indicia on the element such as color, serial number, bar code, QR code, etc., and any other appropriate information that can be needed by the robot to pick up the element, move the element, and stack the element on a support or compatible interlocking device such as another element. The robot can use this information to identify the element and control its body and / or its end effector to pick up and move the element. For example, an on-board control system on the robot can obtain information from this type of local or remote database and can use this information to identify the element and as the stacking element as described herein.

[0045] In this regard, different types of elements can have different types of interface features. In some examples, the interface features include components, protrusions, and / or notches that enable one element to be stably stacked on another element. In some elements, such as Figure 4 Element 50 (a pallet in this example) has interface features on the top and bottom of each of the four corners of the element. Those on the bottom are shown in dashed lines. In this case, the interface features include a pin 51 on the top of each corner of element 50 and a complementary socket 52, where the pin from another element fits on the bottom of each corner. Similar pins and sockets can be located elsewhere on the element. Thus, to stack multiple such elements, the pin on the top of one element fits into the socket on the bottom of another element. In Figure 5In another example element 54, the interface features include posts 57 at the top four corners of element 54 and complementary notches 60 at the bottom four corners of the element. Those on the bottom are shown in dashed lines. Similar notches and posts can be located elsewhere on the element. Thus, to stack multiple such elements, the posts on the top of one element fit into the complementary notches of the bottom of another element. In another example, the top of an element can lock into the bottom of a corresponding corner of the same type of element. These are referred to as interlocking corners. Thus, to stack multiple such elements, the interlocking features of the elements are identified and aligned so that the interlocking features of the top element are in the same vertical position as the interlocking features of the element below on which the top element is stacked. In Figure 6 In a close-up view of two elements 63, 64, the interlocking features of the two elements include complementary structures 65, 66 that fit together. In yet another example, the interface features include the corners of the element itself. For example, the top and bottom of the element can be substantially flat. To provide the most stable stack, in this example, the edges of the element should also be aligned.

[0046] Figure 7 is a flowchart showing example operations that can be performed by a control system and robot to stack one element on another. According to the process 70, the robot 10 is controlled 71 to move to the location of an element to be picked up. As explained previously, the robot can use a map to navigate the space containing the element and can use a database to identify the element to be picked up based on its location and / or identifying features, dimensions, markings, etc. Once at the location of the element, the robot positions 72 itself so that its end effector can be inserted into the element. In the case of a pallet, the end effector is inserted into the socket of the pallet. Using the end effector, the robot picks up 73 the element and navigates 74 to be near the location where the element is to be placed, e.g., a stack. As explained, the element includes features that match complementary or compatible features on another element, such as its corners or parts. The robot moves autonomously - e.g., based on information in the map, sensors, and signals from the control system - to a location that is proximate to the stack, the compatible interlocking device, or the rack where the element is to be placed. No human drives the robot through the space.

[0047] After moving, the robot 10 detects 75 the element on which the element that the robot is holding is to be stacked. For example, the detected element can be a compatible interlocking device or a rack. As in the case above, the robot can use a map to navigate to a location that is proximate to the detected element and can use a database to identify the location on which the element that the robot is holding is to be stacked based on identifying features, dimensions, markings, etc. The robot can also use identifying features, dimensions, markings, etc. to identify the target destination as well. Figure 8 An example stack of elements 80 is shown, and Figure 9Example bracket 81 is shown. The component can be placed on a component stack on a rack, or on a component stack on the floor, that is, not on a rack, but directly on a rack, or placed on a device with compatible interlocking features, which may not involve stacking in itself.

[0048] At this point, the onboard control system performs a coordinate system transformation between the robot and the environment to determine how to manipulate the element. More specifically, sensors on the robot sense the detected element relative to a robot-based coordinate system. This information is converted into an environmental coordinate system that identifies the position where the element will be placed. To this end, the control system determines a first vector from the robot to the target position and a second vector from the sensors on the robot to the element that the robot is holding. The control system determines the difference between the first and second vectors and uses this difference (reflecting the difference in the coordinate system) to control the movement of the element relative to the target position.

[0049] Also refer to Figure 1 and Figures 10 to 15 In order to stack one component (76) on top of another component (such as...) Figure 8 (On pallets stacked on 81), robot 10 controls at least one of its end effector 16 or its body 12 in four, five, or six degrees of freedom to perform the following operations: moving element 83 to align (77) a first corner feature of element 83 (also referred to as an “alignment feature”) with a first corner feature of element 84; moving element 83 to align (78) a second corner feature of element 83 with a second corner feature of element 84; and moving element 83 to contact element 84 (79) such that the first corner features of the two elements match, and such that the second corner features of the two elements match. As described herein, corner features may include pins and sockets, posts and notches, corners themselves, or other mechanical features to engage a top element with a bottom element. As part of the stacking process, sensors on the robot identify the positions of the corner features and continue to monitor their positions.

[0050] in this regard, Figures 10 to 15 Examples of stacking operations are shown. Each figure includes a diagram (85) illustrating six degrees of freedom of movement. However, the robot is omitted from these figures. Figure 10 In this configuration, the robot body, end effector, or both are controlled to tilt the element forward in the direction of arrow 87, such that the front 90 of element 83 is lower than the rear 91 of element 83. Figure 11In one implementation, the robot body, end effector, or both are controlled to roll the elements 83 left / right in the direction of arrow 88 so that the corner 94 of element 83 is at the lowest point of that element in the environmental coordinate space. Sensors on the robot 10 locate the alignment posts 95 and sockets 96 on the elements 83 and 84. Examples of sensors on the robot are described herein. In this example, the posts and sockets are located on the left and right sides of the elements, e.g., at the linear arrangement or corners of the elements.

[0051] In one implementation, the robot body, end effector, or both are controlled to roll the elements 83 left / right in the direction of arrow 88 so that the corner 94 of element 83 is at the lowest point of that element in the environmental coordinate space. Sensors on the robot 10 locate the alignment posts 95 and sockets 96 on the elements 83 and 84. Examples of sensors on the robot are described herein. In this example, the posts and sockets are located on the left and right sides of the elements, e.g., at the linear arrangement or corners of the elements. Figure 12 In one implementation, the robot body, end effector, or both are controlled to align the Cartesian XY coordinates of the socket 100 at the corner 94 of element 83 to the post 101 at the corner 97 of element 84 (this alignment is represented by dashed line 98), and to perform a coarse yaw alignment of element 83 to element 84 during this XY coordinate alignment. At this time, the robot body, end effector, or both can also be controlled to begin lowering the element down - in the negative Cartesian Z direction 99 - to move the socket 100 toward the post 101. Figure 13 In one implementation, the robot body, end effector, or both are controlled to perform yaw 107 and roll 108 movements to align the Cartesian XY coordinates of the socket 103 at the corner 104 of element 83 to the post 106 at the corner 107 of element 84. This alignment is represented by dashed line 93. In one implementation, the robot body, end effector, or both are controlled to perform a pitch movement 110 to vertically align the elements. The alignment is represented by dashed lines (not labeled) on the four corners of the elements. In one implementation, the robot body, end effector, or both are controlled to move element 83 down so that its socket mates with the post of element 84. Figure 14 In one implementation, the robot body, end effector, or both are controlled to perform yaw 107 and roll 108 movements to align the Cartesian XY coordinates of the socket 103 at the corner 104 of element 83 to the post 106 at the corner 107 of element 84. This alignment is represented by dashed line 93. In one implementation, the robot body, end effector, or both are controlled to perform a pitch movement 110 to vertically align the elements. The alignment is represented by dashed lines (not labeled) on the four corners of the elements. In one implementation, the robot body, end effector, or both are controlled to move element 83 down so that its socket mates with the post of element 84. Figure 15 In one implementation, the robot body, end effector, or both are controlled to perform yaw 107 and roll 108 movements to align the Cartesian XY coordinates of the socket 103 at the corner 104 of element 83 to the post 106 at the corner 107 of element 84. This alignment is represented by dashed line 93. In one implementation, the robot body, end effector, or both are controlled to perform a pitch movement 110 to vertically align the elements. The alignment is represented by dashed lines (not labeled) on the four corners of the elements. In one implementation, the robot body, end effector, or both are controlled to move element 83 down so that its socket mates with the post of element 84.

[0052] In the preceding example, elements 83 and 84 are aligned using corners 94 and 104 and corresponding alignment features 100 and 103. These corners can be initially selected using an optimization process, e.g., based on their visibility and availability to sensors. In some implementations, it is not necessarily required to do so. In some implementations, three or more interlocking features can be aligned prior to mating as described herein. In some implementations, the features used to align the elements can not be located on the corners of the elements. For example, features on opposite sides of the elements or adjacent sides of the elements, rather than corners, can be used to align the elements. These additional features can also be used to confirm or enhance the alignment performed based on the corner features, corners, or combinations thereof.

[0053] The example robots described herein can be controlled, at least in part, using one or more computer program products, e.g., one or more computer programs tangibly embodied in one or more information carriers, such as one or more non-transitory machine-readable media, for execution by, or to control the operation of, one or more data processing apparatus, e.g., a programmable processor, a computer, multiple computers, and / or a programmable logic component.

[0054] A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a network.

[0055] Actions associated with implementing all or part of the testing can be performed by one or more programmable processors executing one or more computer programs to perform some of the functions described herein. All or part of the testing can be implemented using special purpose logic circuitry, e.g., an FPGA (field programmable gate array) and / or an ASIC (application-specific integrated circuit).

[0056] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The elements of a computer (including a server) include one or more processors for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more machine-readable storage media, such as a mass storage device (e.g., a magnetic resistant, optical, or magnetic disk) for storing data. Machine-readable storage media suitable for embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0057] Any mechanical or electrical connection herein can include a direct physical connection or an indirect connection including intermediate components.

[0058] The elements of different implementations described herein can be combined to form other implementations not specifically described herein. A plurality of elements of a structure described herein can be combined into a single element to perform the function of the plurality of elements.

Claims

1. An autonomous vehicle, comprising: a body configured to move along a surface; an end effector to hold a first element having a first feature and a second feature that match a complementary third feature and a fourth feature on a second element, at least one of the end effector or the body being controllable to move in at least four degrees of freedom, the at least four degrees of freedom including forward / backward, up / down, left / right, and at least one of yaw, pitch, or roll; one or more sensors to detect the second element and obtain information for locating the third feature and the fourth feature of the second element; and a control system to control at least one of the end effector or the body to move in the at least four degrees of freedom to stack the first element on top of the second element by performing operations including moving the first element to align the first feature with the third feature, moving the first element to align the second feature with the fourth feature, and moving the first element into contact with the second element such that the first feature matches the third feature and the second feature matches the fourth feature, wherein the control system is configured to move the first element in at least one of forward / backward, up / down, or left / right motion to align the first feature with the third feature; and wherein the control system is configured to move the first element in at least one of pitch, yaw, or roll motion to align the second feature with the fourth feature.

2. The autonomous vehicle of claim 1, wherein the control system is programmed to determine a first vector from the autonomous vehicle to the second element and a second vector from a sensor of the one or more sensors to the first element based on the information, and to control the at least one of the end effector or the body to move based on a difference between the first vector and the second vector. at least two of forward / backward, up / down, left / right, and yaw, pitch, or roll.

3. The autonomous vehicle of Claim 1, wherein the control system is configured to control the at least one of the end effector or the body to move in at least five degrees of freedom, including: forward / backward, up / down, left / right, pitch, yaw, and roll.

4. The autonomous vehicle of claim 1, wherein the control system is configured to control the at least one of the end effector or the body to move in six degrees of freedom, including:

5. The autonomous vehicle of claim 1, wherein the one or more sensors are configured to locate the third feature and the fourth feature in three-dimensional (3D) space.

6. The autonomous vehicle of claim 1, wherein the one or more sensors are configured to detect the second element in three-dimensional (3D) space and obtain 3D coordinates of the second element.

7. The autonomous vehicle of claim 1, wherein the one or more sensors include one or more of a light detection and ranging (LIDAR) sensor, a time-of-flight (TOF) sensor, a radar sensor, a sonar sensor, a two-dimensional camera sensor, or a three-dimensional camera sensor. ​ 8. The autonomous vehicle of claim 1, wherein the control system comprises one or more processing devices resident on the autonomous vehicle and configured to execute software to perform the operations independent of data input from an external source.

9. The autonomous vehicle of claim 1, wherein the control system comprises one or more processing devices resident on the autonomous vehicle and configured to execute software to perform the operations based at least in part on data input from a fleet system external to the autonomous vehicle.

10. The autonomous vehicle of claim 1, wherein the control system comprises one or more processing devices resident on the autonomous vehicle programmed to receive information from a fleet system external to the autonomous vehicle, the one or more processing devices programmed to control movement of the body toward the second element based on the information.

11. The autonomous vehicle of claim 1, wherein the control system comprises one or more processing devices resident on the autonomous vehicle programmed to receive information from a fleet system external to the autonomous vehicle, the one or more processing devices programmed to identify a type of the second element based on the information.

12. The autonomous vehicle of claim 1, further comprising: a computer memory storing a database containing data identifying different types of elements and attributes of the different types of elements; wherein the control system comprises one or more processing devices resident on the autonomous vehicle programmed to obtain data from the database to identify a type of the second element and attributes of the second element, the attributes related to the third and fourth features of the second element.

13. The autonomous vehicle of claim 1, wherein the control system comprises one or more processing devices resident on the autonomous vehicle programmed to receive information from a fleet system external to the autonomous vehicle, the one or more processing devices programmed to move to a position in proximity to the second element based on the information and, after reaching the position, control at least one of the end effector or the body to move in the at least four degrees of freedom such that the first element is stacked on top of the second element independent of input from the fleet system.

14. The autonomous vehicle of claim 13, wherein the information comprises a map of an area in which the second element is located.

15. The autonomous vehicle of claim 1, wherein the first and second elements each comprise a pallet.

16. The autonomous vehicle of claim 1, wherein the end effector is configured to move in the at least four degrees of freedom; and wherein the control system is configured to control the end effector to move in the at least four degrees of freedom to stack the first element on top of the second element.

17. The autonomous vehicle of Claim 1, wherein the end effector is configured to move in at least five degrees of freedom, the at least five degrees of freedom comprising: at least one of forward / backward, up / down, left / right, and pitch, yaw, or roll; and wherein the control system is configured to control the end effector to move in the at least five degrees of freedom to stack the first element on top of the second element.

18. The autonomous vehicle of Claim 1, wherein the end effector is configured to move in six degrees of freedom, including: forward / backward, up / down, left / right, pitch, yaw, and roll; and wherein the control system is configured to control the end effector to move in the six degrees of freedom to stack the first element on top of the second element.

19. The autonomous vehicle of claim 1, wherein the end effector comprises tines for holding the first element, the tines comprising a first tine and a second tine, the first tine configured to move independently of the second tine in at least one degree of freedom, and the second tine configured to move independently of the first tine in the at least one degree of freedom.

20. The autonomous vehicle of claim 1, wherein at least one of the third feature or the fourth feature on the second element is located at a corner of the second element.

21. A method performed by an autonomous vehicle, the method comprising: picking up, using an end effector connected to a body of the autonomous vehicle, a first element having a first feature and a second feature, the first feature and the second feature matching a complementary third feature and a fourth feature on a second element; moving the autonomous vehicle holding the second element along a surface toward a location of the second element; detecting the second element, wherein detecting comprises locating the third feature and the fourth feature of the second element; and controlling at least one of the end effector or the body to move in at least four degrees of freedom to stack the first element on top of the second element by performing operations comprising moving the first element to align the first feature with the third feature, moving the first element to align the second feature with the fourth feature, and moving the first element into contact with the second element such that the first feature matches the third feature and the second feature matches the fourth feature; wherein the at least four degrees of freedom comprise at least one of forward / backward, up / down, left / right, and at least one of yaw, pitch, or roll, wherein the operations comprise moving the first element in at least one of forward / backward, up / down, or left / right motion to align the first feature with the third feature; and moving the first element in at least one of pitch, yaw, or roll motion to align the second feature with the fourth feature.

Images