Systems, devices, and methods for robotic learning and execution of skills

Abstract

Systems, devices, and methods for robotic learning and execution of skills are described. The robotic device may include a memory, a processor, sensors, and one or more moving components (e.g., a manipulating element and / or a transport element). The processor may be operatively coupled to the memory, the moving elements, and the sensors and configured to acquire information about the environment (including one or more objects located within the environment). In some embodiments, the processor may be configured to learn the skill by demonstration, exploration, user input, etc. In some embodiments, the processor may be configured to execute the skill and / or arbitrate between different behaviors and / or actions. In some embodiments, the processor may be configured to learn environmental constraints. In some embodiments, the processor may be configured to learn using a generic model of the skill.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 62 / 723,694, filed August 28, 2018, and entitled "SYSTEMS, APPARATUS, AND METHODS FOR ROBOTIC LEARNING AND EXECUTION OF SKILLS."

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 723,694, entitled "METHOD AND SYSTEM FOR ROBOTIC LEARNING OF EXECUTION PROCESSES RELATED TO PERCEPTUALLY CONSTRAINED MANIPULATION SKILLS," filed February 25, 2017, and U.S. Provisional Patent Application No. 62 / 463,628, entitled "METHOD AND SYSTEM FOR ROBOTIC EXECUTION OF A PERCEPTUALLY CONSTRAINED MANIPULATION SKILL LEARNED VIA HUMAN INTERACTION," filed February 23, 2018, which claims priority to U.S. Provisional Patent Application No. 62 / 463,630, entitled "METHOD AND SYSTEM FOR ROBOTIC EXECUTION OF A PERCEPTUALLY CONSTRAINED MANIPULATION SKILL LEARNED VIA HUMAN INTERACTION," filed February 25, 2017. This application also claims priority to U.S. patent application Ser. No. 16 / 456,919, entitled "SYSTEMS, APPARATUS, AND METHODS FOR ROBOTIC LEARNING AND EXECUTION OF SKILLS," filed June 28, 2019, which is a continuation-in-part of International PCT Application No. PCT / US2018 / 019520, entitled "SYSTEMS, APPARATUS, AND METHODS FOR ROBOTIC LEARNING AND EXECUTION OF SKILLS."

[0003]

[0003] This application is also a continuation-in-part of International PCT Application No. PCT / US2018 / 019520.

[0004]

[0004] The disclosures of each of the above-referenced applications are incorporated herein by reference in their entirety.

[0005] government support This invention was made with United States government support under Grant Nos. 1621651 and 1738375 awarded by the National Science Foundation under Phase I of the Small Business Innovation Program. The United States government has certain rights in this invention.

[0006] Technical Field

[0006] The present disclosure relates generally to systems, devices, and methods for robotic learning and performance of skills. More particularly, the present disclosure relates to robotic devices capable of learning and performing skills in unstructured environments. [Background technology]

[0007] background

[0007] Robots can be used to perform and automate a variety of tasks. Robots can perform tasks by moving through an environment, such as an office building or a hospital. Robots may be equipped with wheels, tracks, or other moving components that enable the robot to move autonomously around the environment. However, robots that do not have arms or other manipulators cannot manipulate objects in the environment. These robots are therefore limited in their ability to perform tasks; for example, such robots may not be able to pick up and deliver objects without a human being present at the pick-up or delivery point to manipulate the object.

[0008] Robots that include arms or other manipulators may be able to pick up objects and deliver them to a location without human assistance. For example, a robot with an arm equipped with an end effector, such as a gripper, can use the gripper to pick up one or more objects from multiple different locations and deliver the objects to a new location, all without human assistance. These robots can be used to automate certain tasks, thereby allowing human operators to focus on other tasks. However, most commercial robots do not include manipulators due to the challenges and complexity of programming the manipulator's movements.

[0009]

[0009] Additionally, most commercial robots are designed to operate in structured environments (e.g., factories, warehouses, etc.). Unstructured environments (e.g., human-involved environments such as hospitals and homes) can pose additional challenges for programming robots. In unstructured environments, a robot cannot rely on complete knowledge of its surroundings but must be able to recognize changes in its surroundings and adapt based on those changes. Thus, in unstructured environments, a robot must continuously acquire information about the environment to be able to make autonomous decisions and perform tasks. Often, a robot's motion (e.g., the movement of an arm or end effector within the environment) is also constrained by objects and other obstacles in the environment, further increasing the challenges of robot perception and manipulation. Given the uncertain and dynamic nature of unstructured environments, a robot generally cannot be pre-programmed to perform a task.

[0010]

[0010] Therefore, there is a need for a robotic system that can recognize and adapt to dynamic, unstructured environments and perform tasks within those environments without relying on pre-programmed operational skills. Summary of the Invention [Means for solving the problem]

[0011] overview Systems, devices, and methods for robotic learning and execution of skills are described. In some embodiments, the device includes a memory, a processor, a manipulative element, and a set of sensors. The processor is operably coupled to the memory, the manipulative element, and the set of sensors and may be configured to: obtain a representation of an environment using a subset of sensors from the set of sensors; identify a plurality of markers within the representation of the environment, where each marker from the plurality of markers is associated with a physical object from a plurality of physical objects located in the environment; present information indicative of a position of each marker from the plurality of markers within the representation of the environment; receive a selection of a set of markers from the plurality of markers associated with a set of physical objects from the plurality of physical objects; obtain, for each position from a plurality of positions associated with movement of the manipulative element within the environment, sensory information associated with the manipulative element, where movement of the manipulative element is associated with a physical interaction between the manipulative element and the set of physical objects; and generate, based on the sensory information, a model configured to define a behavior of the manipulative element to perform the physical interaction between the manipulative element and the set of physical objects.

[0012] In some embodiments, the manipulation element may include a plurality of joints and an end effector. In some embodiments, the manipulation element may include a base or a carrying element.

[0013] The set of physical objects may be stationary or moving. In some embodiments, the set of physical objects may include fixtures, storage containers, structures (e.g., doors, handles, furniture, walls), etc. In some embodiments, the set of physical objects may include people (e.g., patients, doctors, nurses, etc.).

[0014]

[0014] In some embodiments, the plurality of markers are fiducial markers and the representation of the environment is a visual representation of the environment.

[0015] In some embodiments, two or more markers from the plurality of markers may be associated with one physical object from the set of physical objects. Alternatively or additionally, in some embodiments, one marker from the plurality of markers may be associated with two or more physical objects from the set of physical objects.

[0016]

[0016] In some embodiments, a method includes obtaining a representation of the environment using a set of sensors; identifying a plurality of markers within the representation of the environment, each marker from the plurality of markers associated with a physical object from a plurality of physical objects located in the environment; presenting information indicating a position of each marker from the plurality of markers within the representation of the environment; after the presenting, receiving a selection of a set of markers from the plurality of markers associated with a set of physical objects from the plurality of physical objects; obtaining, for each position from a plurality of positions associated with movement of the manipulating element in the environment, sensory information associated with the manipulating element, the movement of the manipulating element being associated with a physical interaction between the manipulating element and the set of physical objects; and generating, based on the sensory information, a model configured to define a behavior of the manipulating element to perform the physical interaction between the manipulating element and the set of physical objects.

[0017]

[0017] In some embodiments, the method further includes receiving a selection of a first subset of features from the set of features, wherein the model is generated based on sensor data associated with the first subset of features and not based on sensor data associated with a second subset of features from the set of features that is not included in the first set of features.

[0018]

[0018] In some embodiments, a method includes obtaining a representation of an environment using a set of sensors; identifying a plurality of markers within the representation of the environment, each marker from the plurality of markers associated with a physical object from a plurality of physical objects located in the environment; presenting information indicative of a position of each marker from the plurality of markers within the representation of the environment; in response to receiving a selection of a set of markers from the plurality of markers associated with the set of physical objects from the plurality of physical objects, identifying a model associated with performing a physical interaction between a manipulating element and the set of physical objects, the manipulating element including a plurality of joints and an end effector; and generating a trajectory for the manipulating element using the model, the trajectory defining movement of the plurality of joints and the end effector associated with performing the physical interaction.

[0019]

[0019] In some embodiments, the method further includes displaying to a user a trajectory of the manipulative element within a representation of the environment, and after displaying, receiving input from the user, and in response to the input indicating approval of the trajectory of the manipulative element, performing a movement of the manipulative element (e.g., multiple joints, an end effector, a transport element) to perform a physical interaction.

[0020] In some embodiments, a model is associated with (i) a set of stored markers and (ii) sensory information indicative of at least one of a position, orientation, or configuration of a manipulating element (e.g., a plurality of joints, an end effector, a transport element) at points along a stored trajectory of the manipulating element associated with the set of stored markers. A method for generating a trajectory of the manipulating element may include, for example, calculating a transformation function between the set of markers and the set of stored markers, and, for each point, transforming at least one of a position or orientation of the manipulating element using the transformation function, determining, for each point, a planned configuration of one or more components of the manipulating element (e.g., a plurality of joints, an end effector, a transport element) based on the configuration of said components along the stored trajectory, and, for each point, determining a portion of the trajectory between the point and successive points based on the planned configuration of said components with respect to the point.

[0021] In some embodiments, the robotic device is configured to learn and perform a skill associated with a transport element. The transport element may be, for example, a set of wheels, a set of tracks, a set of crawlers, or any other suitable device that enables movement of the robotic device within an environment, such as from a first room to a second room or between multiple floors of a building. When the robotic device is moved from a first location to a second location, for example, through a doorway or by an object such as a human, the robotic device may be configured to learn the skill associated with the transport element by acquiring sensory information associated with the transport element, the environment around the robotic device, and / or another component of the robotic device. The sensory information may be recorded at specific points (e.g., at key frames) during operation of the robotic device. Based on the sensory information, the robotic device may be configured to generate a model configured to prescribe the behavior of the transport element (or other component of the robotic device (e.g., a manipulation element)) to execute movement of the robotic device from the first location to the second location.

[0022] In some embodiments, a robotic device is configured to learn a specialized skill using a generic version of the skill. The robotic device can begin performing the generic skill in a particular environment and pause the performance when the robotic device reaches a portion of the performance that requires specialization of the skill in the particular environment. The robotic device can then prompt the user to perform that portion of the skill. The robotic device can continue performing the skill and / or prompt the user to perform specific portions of the skill until the skill is complete. The robotic device then adapts the generic skill based on the performance of the specific portions of the skill to generate a specialized model for performing the skill.

[0023] In some embodiments, the robotic device is configured to learn environmental constraints. The robotic device can record information about its surroundings or objects in that environment to obtain general knowledge about the environment. The robotic device can apply this general knowledge to a set of models related to different skills to be performed in the environment.

[0024] In some embodiments, a robotic device can use behavior arbitration to continuously and autonomously act within a dynamic environment, such as a human social environment. The robotic device may have various resources or components (e.g., a manipulating element, a carrying element, a head, a camera, etc.) and can apply an arbitration algorithm to determine when and how to use these resources. The arbitration algorithm can define a set of rules that allows different actions or behaviors to be prioritized based on information captured by the robotic device. The robotic device may be configured to continuously decide among performing different actions and behaviors (and thus use different resources or components) as new information is captured by the robotic device. As part of behavior arbitration, the robotic device may be configured to handle the interruption of an action and switch to a new action. By continuously monitoring itself and its surrounding environment and performing continuous arbitration, the robotic device can continually engage in socially appropriate behaviors over time.

[0025] In some embodiments, the robotic device is configured to receive social context information related to an environment and combine that information with a navigation map of the environment (e.g., overlay the social context information on the navigation map). The robotic device may be provided with the social context information by a user (e.g., a human operator), or the robotic device may capture the social context information through interactions in the environment. As the robotic device engages in interactions with humans in the environment over time, the robotic device may refine the social context information. In some embodiments, the robotic device may be configured to associate certain types of behaviors or actions with specific locations in the environment based on the social context information related to those locations.

[0026] In some embodiments, the robotic device is configured to interact with a human operator, either on-site or via a network connection to a remote device operated by the human operator. The human operator, referred to herein as a “robot supervisor,” can use various software-based tools to inform the robotic device about its surrounding environment and / or instruct the robotic device to perform specific actions. These actions may include, for example, navigation behaviors, manipulation behaviors, head behaviors, sounds, lights, etc. The robot supervisor can control the robotic device to collect information while learning or performing an action and / or tag certain behaviors as good or bad so that the robotic device can use that information to improve its ability to arbitrate among different actions or to perform a particular action in the future.

[0027]

[0027] Other systems, processes, and features will become apparent to one with skill in the art upon examination of the following figures and detailed description. All such additional systems, processes, and features are intended to be included within the scope of this specification, be within the scope of the invention, and be protected by the accompanying claims.

[0028] BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Those skilled in the art will appreciate that the drawings are primarily for illustrative purposes and are not intended to limit the scope of the inventive subject matter described herein. The drawings are not necessarily to scale, and in some cases, various aspects of the inventive subject matter disclosed herein may be shown exaggerated, or enlarged, in the drawings to facilitate an understanding of different features. In the drawings, like reference numerals generally refer to like features (e.g., functionally similar and / or structurally similar elements). [Brief explanation of the drawings]

[0029] [Figure 1]

[0029] A block diagram illustrating a configuration of a system including a robotic device according to some embodiments. [Figure 2]

[0030] FIG. 1 is a block diagram illustrating a configuration of a robotic device, according to some embodiments. [Figure 3]

[0031] FIG. 1 is a block diagram illustrating the configuration of a control unit associated with a robotic device, according to some embodiments. [Figure 4]

[0032] FIG. 1 is a schematic diagram of a manipulative element of a robotic device, according to some embodiments. [Figure 5]

[0033] FIG. 1 is a schematic diagram of a robotic device, according to some embodiments. [Figure 6A]

[0034] FIG. 1 is a schematic diagram of objects in an environment as seen by a robotic device, according to some embodiments. [Figure 6B]

[0034] FIG. 1 is a schematic diagram of objects in an environment as seen by a robotic device, according to some embodiments. [Figure 7A]

[0035] FIG. 1 is a schematic diagram of objects in an environment as seen by a robotic device, according to some embodiments. [Figure 7B]

[0035] FIG. 1 is a schematic diagram of objects in an environment as seen by a robotic device, according to some embodiments. [Figure 8]

[0036] FIG. 1 is a flow diagram illustrating a method for sensing or scanning an environment performed by a robotic device, according to some embodiments. [Figure 9]

[0037] FIG. 1 is a flow diagram illustrating a method for learning and performing a skill performed by a robotic device, according to some embodiments. [Figure 10]

[0038] FIG. 1 is a flow diagram illustrating a method for learning a skill to be performed by a robotic device, according to some embodiments. [Figure 11]

[0039] FIG. 1 is a flow diagram illustrating a method for executing a skill performed by a robotic device, according to some embodiments. [Figure 12]

[0040] FIG. 1 is a block diagram illustrating a system architecture for robot learning and execution, including user actions, according to some embodiments. [Figure 13]

[0041] FIG. 1 is a flow diagram illustrating the operation of a robotic device within an environment, according to some embodiments. [Figure 14]

[0042] FIG. 1 is a flow diagram illustrating a method for requesting and receiving input from a user, such as a robot supervisor, according to some embodiments. [Figure 15]

[0043] FIG. 1 is a flow diagram illustrating a method for learning skills and environmental constraints performed by a robotic device, according to some embodiments. [Figure 16]

[0044] FIG. 1 is a flow diagram illustrating a method for learning a skill from a generic skill model performed by a robotic device, according to some embodiments. [Figure 17]

[0045] FIG. 1 is a flow diagram illustrating a method for learning environmental constraints performed by a robotic device, according to some embodiments. [Figure 18]

[0046] FIG. 1 is a block diagram illustrating an example of components of a robotic device that perform behavior arbitration, according to some embodiments. [Figure 19]

[0047] FIG. 1 is a schematic diagram of a layer of a map of an environment generated by a robotic device, according to some embodiments. [Figure 20]

[0048] FIG. 1 is a block diagram illustrating the configuration of a control unit associated with a robotic device, according to some embodiments. [Figure 21]

[0049] 1 illustrates the flow of information provided by and received from a robotic device to a map maintained by the robotic device, according to some embodiments. [Figure 22]

[0050] FIG. 1 illustrates a flow diagram illustrating different learning modes of a robotic device, according to some embodiments. [Figure 23]

[0051] FIG. 1 illustrates a flow diagram illustrating an example learning behavior of a robotic device, according to some embodiments. [Figure 24]

[0051] A flow diagram illustrating an example learning behavior of a robotic device, according to some embodiments. [Figure 25]

[0051] A flow diagram illustrating an example learning behavior of a robotic device, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0030] Detailed Description

[0052] Described herein are systems, devices, and methods for robotic learning and performance of skills. In some embodiments, the systems, devices, and methods described herein relate to robotic devices that can learn skills through human demonstration and interaction and perform the learned skills in unstructured environments.

[0031] overview

[0053] In some embodiments, the systems, devices, and methods described herein relate to robots that can learn skills (e.g., manipulation skills) through a Learning from Demonstration ("LfD") process, in which a human demonstrates an action to a system through kinesthetic teaching (e.g., a human physically and / or remotely directing the robot to perform the action) and / or by performing the action themselves. Such systems, devices, and methods are designed to avoid the need to pre-program the robot with manipulation skills, and to enable the robot to be adaptive and learn skills through observation. For example, a robot can acquire and perform a manipulation skill using machine learning techniques. After learning a skill, the robot can perform the skill in multiple different environments. A robot can learn and / or perform a skill based on visual data (e.g., sensed visual information). Alternatively or additionally, a robot can learn and / or perform a skill using tactile data (e.g., torque, force, and other non-visual information). Robot training can occur in a factory before the robot is deployed or in the field (e.g., in a hospital) after the robot is deployed. In some embodiments, a robot can be taught skills and / or adapted to operate in an environment by a user untrained in robotics and / or programming. For example, a robot can have learning algorithms that utilize natural human behavior and include tools that can guide a user through a demonstration process.

[0032]

[0054] In some embodiments, robots can be designed to interact with humans and collaborate with them to perform tasks. In some embodiments, robots can utilize common social behaviors to behave in a socially predictable and acceptable manner around humans. Mobile robots can also be designed to navigate within an environment while interacting with humans within that environment. For example, a robot can be programmed to vocalize specific phrases to navigate around humans, step aside to allow humans to pass, and use gaze to intentionally communicate while navigating. In some embodiments, a robot may be equipped with sensors that enable the robot to sense and track humans within its surrounding environment and use that information to trigger gaze and other social behaviors.

[0033]

[0055] In some embodiments, a robot can be designed to suggest options for achieving a goal or performing an action during an LfD process. For example, the robot can suggest several different options for achieving a goal (e.g., picking up an object) and indicate which of these options is most likely to be effective and / or efficient for achieving the goal. In some embodiments, the robot can adapt a skill based on user input (e.g., the user indicates relevant features to include in the skill model).

[0034]

[0056] In some embodiments, a robotic device may be capable of learning and / or performing skills in an unstructured environment (e.g., a dynamic and / or human environment) where the robotic device does not have complete information about the environment in advance. An unstructured environment may include, for example, indoor and outdoor situations and may include one or more humans or other objects that are movable within the environment. Because most natural or real-world environments are unstructured, robotic devices that can adapt and operate in unstructured environments, such as the robotic devices and / or systems described herein, can provide a significant improvement over existing robotic devices that cannot adapt to unstructured environments. Unstructured environments may include indoor situations (e.g., buildings, offices, homes, rooms, etc.) and / or other types of enclosed spaces (e.g., airplanes, trains, and / or other types of movable compartments), as well as outdoor situations (e.g., parks, beaches, outdoor yards, fields). In some embodiments, the robotic devices described herein can operate in an unstructured hospital environment.

[0035]

[0057] FIG. 1 is a high-level block diagram illustrating a system 100 according to some embodiments. The system 100 can be configured to learn and perform skills, such as manipulation skills, in an unstructured environment. The system 100 may be implemented as a single device or across multiple devices connected to a network 105. For example, as shown in FIG. 1, the system 100 may include one or more computing devices, such as one or more robotic devices 102 and 110, a server 120, and one or more additional computing devices 150. While four devices are shown, it is understood that the system 100 may include any number of computing devices, including computing devices not specifically shown in FIG. 1.

[0036]

[0058] Network 105 may be implemented as a wired and / or wireless network and may be any type of network (e.g., a local area network (LAN), a wide area network (WAN), a virtual network, a telecommunications network) used to operatively couple computing devices, including robotic devices 102 and 110, server 120, and one or more computing devices 150. As described in further detail herein, in some embodiments, for example, the computing devices are computers connected to each other by an Internet Service Provider (ISP) and the Internet (e.g., network 105). In some embodiments, a connection between any two computing devices may be defined by network 105. As shown in FIG. 1, for example, a connection may be defined between robotic device 102 and any one of robotic device 110, server 120, or one or more computing devices 150. In some embodiments, these computing devices may communicate with each other (e.g., send and / or receive data) and with network 105 through intermediate and / or alternative networks (not shown in FIG. 1). Such intermediate and / or alternative networks may be networks of the same type and / or different types than network 105. Each computing device may be any type of device configured to transmit data over network 105 for transmission and / or receive data from one or more of the other computing devices.

[0037]

[0059] In some embodiments, the system 100 includes a single robotic device (e.g., the robotic device 102). The robotic device 102 can be configured to sense information about the environment, learn skills through human demonstration and interaction, interact with the environment, and / or learn environmental constraints through human demonstration and input, and / or perform those skills in the environment. In some embodiments, the robotic device 102 can self-explore and / or request user input to learn additional information about the skill and / or the environment. A more detailed diagram of an example robotic device is shown in FIG. 2.

[0038]

[0060] In another embodiment, the system 100 includes multiple robotic devices (e.g., robotic devices 102 and 110). The robotic device 102 can transmit data to and / or receive data from the robotic device 110 via the network 105. For example, the robotic device 102 can transmit information sensed by the robotic device 102 about its environment (e.g., the location of an object) to the robotic device 110 and can receive information about the environment from the robotic device 110. The robotic devices 102 and 110 can also transmit information to and / or receive information from each other to learn and / or perform a skill. For example, the robotic device 102 can learn a skill in an environment and transmit a model representing the learned skill to the robotic device 110, which can then use the model to perform the skill in the same or a different environment. The robotic device 102 may be located in the same location as the robotic device 110 or in a different location. For example, robotic devices 102 and 110 may be located in the same room in a building (e.g., a hospital building) so that they can learn and / or perform a skill together (e.g., moving a heavy or large object). Alternatively, robotic device 102 may be located on the first floor of a building (e.g., a hospital building) and robotic device 110 may be located on the second floor of the building, and the two robotic devices may communicate with each other to convey information about the different floors (e.g., where objects are located on these floors, where resources may be located, etc.).

[0039]

[0061] In some embodiments, system 100 includes one or more robotic devices (e.g., robotic devices 102 and / or 110) and server 120. Server 120 may be a dedicated server that manages robotic devices 102 and / or 110. Server 120 may be located in the same or a different location as robotic devices 102 and / or 110. For example, server 120 may be located in the same building (e.g., a hospital building) as one or more robotic devices and may be managed by a local administrator (e.g., a hospital administrator). Alternatively, server 120 may be located in a remote location (e.g., a location associated with a manufacturer or supplier of the robotic devices).

[0040]

[0062] In some embodiments, system 100 includes one or more robotic devices (e.g., robotic devices 102 and / or 110) and an additional computing device 150. Computing device 150 may be any suitable processing device configured to initiate and / or perform a particular function. For example, in a hospital setting, computing device 150 may be a diagnostic and / or therapeutic device that can connect to network 105 and communicate with other computing devices, including robotic devices 102 and / or 110.

[0041]

[0063] In some embodiments, one or more robotic devices (e.g., robotic devices 102 and / or 110) may be configured to communicate with server 120 and / or computing device 150 via network 105. Server 120 may include one or more components located remotely from the robotic devices and / or near the robotic devices on a premises. Computing device 150 may include one or more components located remotely from the robotic devices, near the robotic devices on a premises, and / or integrated into one of the robotic devices. Server 120 and / or computing device 150 may include a user interface that allows a user (e.g., a local user or a robot supervisor) to control the operation of the robotic device. For example, a user can interrupt and / or modify the execution of one or more actions performed by the robotic device. These actions may include, for example, navigation behavior, manipulation behavior, head behavior, sound / lights, and / or other components of the robotic device. In some embodiments, a robot supervisor can remotely monitor the robotic device and control its operation for safety purposes. For example, a robot supervisor can command the robotic device to stop or modify the performance of an action to avoid endangering a human or damaging the robotic device or another object in the environment. In some embodiments, a nearby user can instruct or demonstrate an action to the robotic device. In some embodiments, the robotic device may be configured to request user intervention at certain points during the performance of an action.For example, the robotic device may request user intervention when it is unable to ascertain certain information about itself and / or the environment around it, when it is unable to determine a trajectory for completing an action or navigating to a particular location, when the robotic device has been previously programmed to do so (e.g., during learning using an interactive learning template, as described further below), or at other times. In some embodiments, the robotic device may request user feedback at certain points during learning and / or execution. For example, the robotic device may prompt the user to specify what information to collect (e.g., information related to a manipulation element or a transport element, information related to the surrounding environment) and / or when to collect information (e.g., timing of key frames during a demonstration). Alternatively or additionally, the robotic device may request that the user tag past or current behaviors of the robotic device as good or bad examples of an action in a particular context. The robotic device may be configured to use this information to improve future performance of that action in that particular context.

[0042] Systems and Devices

[0064] 2 schematically illustrates a robotic device 200 according to some embodiments. The robotic device 200 includes a control unit 202, a user interface 240, at least one operating element 250, and at least one sensor 270. Additionally, in some embodiments, the robotic device 200 optionally includes at least one transport element 260. The control unit 202 includes a memory 220, storage 230, a processor 204, a graphics processor 205, a system bus 206, and at least one input / output interface (“I / O interface”) 208. The memory 220 may be, for example, random access memory (RAM), a memory buffer, a hard drive, a database, erasable programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), read-only memory (ROM), and / or the like. In some embodiments, memory 220 stores instructions that cause processor 204 to perform modules, processes, and / or functions related to sensing or scanning an environment, learning a skill, and / or performing a skill. Storage 230 may be, for example, a hard drive, a database, cloud storage, a network-attached storage device, or other data storage device. In some embodiments, storage 230 may store sensor data including, for example, state information about one or more components of robotic device 200 (e.g., manipulative element 250), learned models, marker position information, etc.

[0043]

[0065] The processor 204 of the control unit 202 may be any suitable processing device configured to initiate and / or execute functions related to inspecting an environment, learning a skill, and / or performing a skill. For example, the processor 204 may be configured to execute a skill by generating a model for the skill based on sensor information or using the model to generate a trajectory for performing the skill, as described further herein. More specifically, the processor 204 may be configured to execute modules, functions, and / or processes. In some embodiments, the processor 204 may be a general-purpose processor, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), and / or the like.

[0044]

[0066] Graphics processor 205 may be any suitable processing device configured to drive and / or perform one or more display functions (e.g., functions associated with display device 242). In some embodiments, graphics processor 205 may be a low-power graphics processing unit, such as, for example, a dedicated graphics card or an integrated graphics processing unit.

[0045]

[0067] The system bus 206 may be any suitable component that enables the processor 204, memory 220, storage 230, and / or other components of the control unit 202 to communicate with each other. The one or more I / O interfaces 208 connected to the system bus 206 may be any suitable component that enables communication between the internal components of the control unit 202 (e.g., the processor 204, memory 220, storage 230) and external input / output devices, such as a user interface 240, one or more operating elements 250, one or more transport elements 260, and one or more sensors 270.

[0046]

[0068] User interface 240 may include one or more components configured to receive input and provide output to other devices and / or to a user operating a device (e.g., a user operating robotic device 200). For example, user interface 240 may include a display device 242 (e.g., a display, a touchscreen, etc.), an audio device 244 (e.g., a microphone, a speaker), and, optionally, one or more additional input / output devices ("I / O devices") 246 configured to receive input and / or generate output for a user.

[0047]

[0069] The one or more manipulating elements 250 may be any suitable component capable of manipulating and / or interacting with stationary and / or moving objects, including, for example, a human. In some embodiments, the one or more manipulating elements 250 may include multiple segments coupled to one another by joints capable of providing translation along one or more axes and / or rotation about one or more axes. The one or more manipulating elements 250 may optionally include an end effector capable of engaging and / or otherwise interacting with objects in the environment. For example, the manipulating element may include a gripping mechanism capable of releasably engaging (e.g., grasping) an object in the environment for picking up and / or transporting the object. Other examples of end effectors include, for example, one or more vacuum engagement mechanisms, one or more magnetic engagement mechanisms, one or more suction mechanisms, and / or combinations thereof. In some embodiments, one or more manipulating elements 250 may be retractable within a housing of the robotic device 200 when not in use to reduce one or more dimensions of the robotic device. In some embodiments, one or more manipulating elements 250 may include a head or other humanoid component configured to interact with the environment and / or one or more objects in the environment, including humans. In some embodiments, one or more manipulating elements 250 may include a carrying element or base (e.g., one or more carrying elements 260). A detailed view of an example manipulating element is shown in FIG. 4.

[0048]

[0070] The one or more transport elements 260 may be any suitable component configured for movement, such as, for example, wheels or tracks. The one or more transport elements 260 may be provided at the base of the robotic device 200 to enable the robotic device 200 to move around an environment. For example, the robotic device 200 may include multiple wheels that enable the robotic device 200 to navigate through a building, such as a hospital. The one or more transport elements 260 may be designed and / or dimensioned to facilitate movement through narrow and / or restricted spaces (e.g., small passageways or hallways, small rooms such as storage rooms, etc.). In some embodiments, the one or more transport elements 260 may be rotatable about an axis and / or movable relative to each other (e.g., along tracks). In some embodiments, the one or more transport elements 260 may be retractable within the base of the robotic device 200 when not in use to reduce one or more dimensions of the robotic device. In some embodiments, one or more conveying elements 260 may be part of or form part of an operating element (eg, one or more operating elements 250).

[0049]

[0071] The one or more sensors 270 may be any suitable component that enables the robotic device 200 to capture information about the environment and / or objects in the environment surrounding the robotic device 200. The one or more sensors 270 may include, for example, an imaging device (e.g., a camera such as an RGB-D (red-green-blue-depth) camera or a webcam), an audio device (e.g., a microphone), an optical sensor (e.g., a light detection and ranging (i.e., lidar) sensor, a color detection sensor), a proprioceptive sensor, a position sensor, a tactile sensor, a force or torque sensor, a temperature sensor, a pressure sensor, a motion sensor, a sound detector, etc. For example, the one or more sensors 270 may include at least one imaging device, such as a camera, for capturing visual information about objects and the environment surrounding the robotic device 200. In some embodiments, the one or more sensors 270 may include a tactile sensor (e.g., a sensor that can communicate force, vibration, touch, and other non-visual information to the robotic device 200).

[0050]

[0072] In some embodiments, the robotic device 200 may have humanoid features (e.g., a head, torso, arms, legs, and / or a base). For example, the robotic device 200 may include a face with eyes, a nose, a mouth, and other humanoid features. These humanoid features can form part of and / or may be part of one or more operational elements. Although not schematically shown, the robotic device 200 may also include actuators, motors, couplers, connectors, power sources (e.g., an internal battery), and / or other components that connect, actuate, and / or drive different portions of the robotic device 200.

[0051]

[0073] 3 is a block diagram that schematically illustrates a control unit 302 according to some embodiments. Control unit 302 may include similar components as control unit 202 and may be structurally and / or functionally similar to control unit 202. For example, control unit 302 includes processor 304, graphics processor 305, memory 320, one or more I / O interfaces 308, system bus 306, and storage 330, which may be structurally and / or functionally similar to processor 204, memory 220, one or more I / O interfaces 308, system bus 206, and storage 230, respectively.

[0052]

[0074] Memory 320 stores instructions that can cause processor 304 to execute modules, processes, and / or functions shown as active sensing 322, marker identification 324, learning and model generation 326, trajectory generation and execution 328, and success monitoring 329. Active sensing 322, marker identification 324, learning and model generation 326, trajectory generation and execution 328, and success monitoring 329 may be implemented as one or more programs and / or applications coupled to hardware components (e.g., sensors, operating elements, I / O devices, processors, etc.). Active sensing 322, marker identification 324, learning and model generation 326, trajectory generation and execution 328, and success monitoring 329 may be implemented by a single robotic device or multiple robotic devices. For example, a robotic device may be configured to implement active sensing 322, marker identification 324, and trajectory generation and execution 328. As another example, the robotic device may be configured to implement active sensing 322, marker identification 324, optionally learning and model generation 326, and trajectory generation and execution 328. As another example, the robotic device may be configured to implement active sensing 322, marker identification 324, trajectory generation and execution 328, and optionally success monitoring 329. Although not shown, memory 320 may also store programs and / or applications related to an operating system and general robotic operation (e.g., power management, memory allocation, etc.).

[0053]

[0075] Storage 330 stores information related to learning and / or performing skills. Storage 330 stores, for example, state information 331, one or more models 334, object information 340, and machine learning library 342. State information 331 may include information about the state of a robotic device (e.g., robotic device 200) and / or the environment in which the robotic device is operating (e.g., a building, such as a hospital). In some embodiments, state information 331 may indicate the location of the robotic device within the environment, such as a room, a floor, or an enclosed space. For example, state information 331 may include a map 332 of the environment and may indicate the location of the robotic device within the map 332. State information 331 may also include one or more locations of one or more objects (or markers representing and / or associated with the objects) within the environment (e.g., within map 332). Thus, state information 331 may identify the location of the robotic device relative to one or more objects. Objects may include any type of physical object located in the environment, including objects that define a space or opening (e.g., a surface or wall that defines a doorway). Objects may be stationary or movable. Examples of objects in an environment such as a hospital include equipment, fixtures, devices, tools, furniture, and / or people (e.g., nurses, doctors, patients, etc.).

[0054]

[0076] In some embodiments, the state information 331 may include a representation or map of the environment, such as that shown in FIG. 19. The map of the environment may include, for example, a navigational layer, a static semantic layer, a social layer, and a dynamic layer. Information learned by the robotic device, for example, from demonstrations, user input, sensed sensor information, etc., may be fed into different layers of the map and organized for later reference by this robotic device (and / or other robotic devices). For example, the robotic device may rely on information learned about different objects (e.g., doors) in the environment to determine how to arbitrate between different behaviors (e.g., waiting for a closed doorway to be opened before passing through it, or requesting help to open the door before passing through it), as described further herein.

[0055]

[0077] The object information 340 may include information related to one or more physical objects in the environment. For example, the object information may include information that identifies or quantifies different characteristics of an object, such as, for example, location, color, shape, and surface features. The object information may also identify codes, symbols, and other markers associated with the physical object (e.g., quick response or "QR" codes, bar codes, tags, etc.). In some embodiments, the object information may include information that characterizes an object in the environment (e.g., a doorway or passageway is closed, a door handle is a type of door handle, etc.). The object information may enable the control unit 302 to identify one or more physical objects in the environment.

[0056]

[0078] The state information 331 and / or object information 340 may be examples of environmental constraints, as used and described herein. Environmental constraints may include information about an environment that may be presented or viewed in various dimensions. For example, environmental constraints may vary by location, time, social interaction, specific context, etc. The robotic device may learn environmental constraints from user input, skill demonstrations, interaction with the environment, self-exploration, skill execution, etc.

[0057]

[0079] The machine learning library 342 may include modules, processes, and / or functions related to different algorithms for machine learning and / or model generation for different skills. In some embodiments, the machine learning library may include methods such as hidden Markov models (i.e., "HMMs"). An example of an existing machine learning library for Python is scikit-learn. The storage 330 may also include additional software libraries related to, for example, robot simulation, motion planning and control, kinematics teaching and perception, etc.

[0058]

[0080] One or more skill models 334 are models generated to perform different actions and represent skills learned by the robotic device. In some embodiments, each model 334 is associated with a set of markers attached to different physical objects in an environment. Marker information 335 can indicate which markers are associated with a particular model 334. Each model 334 can also be associated with sensory information 336 collected, for example, by one or more sensors of the robotic device, during kinesthetic teaching and / or other demonstration of a skill. The sensory information 336 can optionally include manipulative element information 337 associated with manipulative elements of the robotic device when the robotic device performs an action during the demonstration. The manipulative element information 337 can include, for example, the position and configuration of a joint, the position and configuration of an end effector, the position and configuration of a base or a transport element, and / or the forces and torques acting on the joints, end effector, transport element, etc. Manipulative element information 337 may be recorded at specific points (e.g., key frames) during the performance and / or execution of a skill, or throughout the performance and / or execution of a skill. Sensory information 336 may also include information associated with the environment in which the skill is being performed and / or executed (e.g., the location of markers within the environment). In some embodiments, each model 334 may be associated with success criteria 339. The success criteria 339 can be used to monitor the performance of a skill. In some embodiments, the success criteria 339 may include information associated with visual and tactile data sensed using one or more sensors (e.g., cameras, force / torque sensors, etc.). The success criteria 339 may be tied, for example, to visually detecting the movement of an object, detecting a force acting on a component of the robotic device (e.g., the weight of the object), detecting an engagement of a component of the robotic device with an object (e.g., a change in pressure or force acting on a surface), etc.An example of the use of haptic data in robotic learning of manipulation skills is described in a paper entitled "Learning Haptic Affordances from Demonstration and Human-Guided Exploration" by Chu et al., published at the 2016 IEEE Haptics Symposium (HAPTICS) (Philadelphia, PA, 2016, pp. 119-125), which is incorporated herein by reference and available at http: / / ieeexplore.ieee.org / document / 7463165 / . An example of using visual data for robotic learning of manipulation skills is described in a paper titled "Simultaneously Learning Actions and Goals from Demonstration" by Akgun et al., published in Autonomous Robots (Volume 40, Issue 2, February 2016, pp. 211-227), available at https: / / doi.org / 10.1007 / s10514-015-9448-x (the "Akgun paper"), which is incorporated herein by reference.

[0059]

[0081] Optionally, sensory information 336 may include transport element information 338. Transport element information 338 may be associated with the movement (e.g., navigating through a doorway, transporting an object, etc.) of a transport element (e.g., one or more transport elements 260) of the robotic device as the robotic device undergoes a skill demonstration. Transport element information 338 may be recorded at specific points during the performance and / or execution of a skill (e.g., key frames associated with the skill) or throughout the performance and / or execution of the skill.

[0060]

[0082] In some embodiments, an initial set of environmental constraints (e.g., state information 331, object information 340, etc.) and / or skills (e.g., one or more models 334) may be provided to the robotic device, for example, by a remote manager or supervisor. The robotic device may adapt and / or augment its knowledge of environmental constraints and / or skills based on its own interactions with the environment or humans in the environment (e.g., patients, nurses, doctors, etc.), demonstrations, etc., and / or by additional user input. Alternatively or additionally, the robot supervisor may update the robotic device's knowledge of environmental constraints and / or skills based on new information collected by the robotic device or one or more other robotic devices (e.g., one or more other robotic devices in a similar or same environment (e.g., a hospital)) and / or provided to the robot supervisor by an external party (e.g., a supplier, manager, manufacturer, etc.). Such updates may be provided periodically and / or continuously as new information about the environment or skills is provided to the robotic device and / or the robot supervisor.

[0061]

[0083] FIG. 4 schematically illustrates a manipulative element 350 according to some embodiments. The manipulative element 350 may form part of a robotic device, such as, for example, robotic device 102 and / or 200. The manipulative element 350 may be implemented as an arm including two or more segments 352 joined together by joints 354. The joints 354 may allow one or more degrees of freedom. For example, the joints 354 may provide translation along one or more axes and / or rotation about one or more axes. In some embodiments, the manipulative element 350 may have seven degrees of freedom provided by the joints 354. While FIG. 4 illustrates four segments 352 and four joints 354, one skilled in the art will understand that the manipulative element may include a different number of segments and / or joints.

[0062]

[0084] The manipulation element 350 includes an end effector 356 that can be used to interact with objects in an environment. For example, the end effector 356 can be used to engage with and / or manipulate different objects. Alternatively or additionally, the end effector 356 can be used to interact with movable or dynamic objects, including, for example, a human. In some embodiments, the end effector 356 can be a gripper that can releasably engage or grasp one or more objects. For example, the end effector 356 implemented as a gripper can pick up and move an object from a first location (e.g., a storage unit) to a second location (e.g., an office, a room, etc.).

[0063]

[0085] Sensors 353, 355, 357, and 358 may be disposed on different components of manipulative element 350 (e.g., segment 352, joint 354, and / or end effector 356). Sensors 353, 355, 357, and 358 may be configured to measure sensory information, including environmental information and / or manipulative element information. Examples of sensors include position encoders, torque and / or force sensors, contact and / or tactile sensors, imaging devices such as cameras, temperature sensors, pressure sensors, light sensors, etc. In some embodiments, sensor 353 disposed on segment 352 may be a camera configured to capture visual information about the environment. In some embodiments, sensor 353 disposed on segment 352 may be an accelerometer configured to measure the acceleration of segment 352 and / or enable calculation of the movement speed and / or position of segment 352. In some embodiments, sensor 355 disposed on joint 354 may be a position encoder configured to measure the position and / or configuration of joint 354. In some embodiments, sensor 355 disposed on joint 354 may be a force or torque sensor configured to measure a force or torque applied to joint 354. In some embodiments, sensor 358 disposed on end effector 356 may be a position encoder and / or a force or torque sensor. In some embodiments, sensor 357 disposed on end effector 356 may be a contact or tactile sensor configured to measure engagement of end effector 356 with an object in the environment. Alternatively or additionally, one or more of sensors 353, 355, 357, and 358 may be configured to record information about one or more objects and / or markers in the environment. For example, sensor 358 disposed on end effector 356 may be configured to track the location of an object in the environment and / or the position of the object relative to end effector 356. In some embodiments, one or more of sensors 353, 355, 357, and 358 may also track whether an object, such as a human, is moving in the environment.Sensors 353, 355, 357, and 358 can transmit the sensory information they record to a computing device located on the robotic device (e.g., an on-board control unit (e.g., control unit 202 and / or 302)), or sensors 353, 355, 357, and 358 can transmit the sensory information to a remote computing device (e.g., a server (e.g., server 120)).

[0064]

[0086] Manipulation element 350 may optionally include coupling elements 359 that allow manipulating element 350 to be removably coupled to a robotic device (such as any of the robotic devices described herein). In some embodiments, manipulating element 350 may be coupled to a robotic device in a fixed position and / or may be coupleable to multiple locations on the robotic device (e.g., to the right or left side of the robotic device's torso as shown in FIG. 5). Coupling elements 359 may include any type of mechanism that allows manipulating element 350 to be coupled to a robotic device, such as, for example, a mechanical mechanism (e.g., a fastener, a latch, a mount), a magnetic mechanism, a friction fit, etc.

[0065]

[0087] 20 is a block diagram that schematically illustrates a control unit 1702 of a robotic system, according to some embodiments. The control unit 1702 may include components similar to other control units described herein (e.g., control units 202 and / or 302). For example, the control unit 1702 includes a processor 1704, a graphics processor 1705, memory 1720, one or more I / O interfaces 1708, a system bus 1706, and storage 1730, which may be structurally and / or functionally similar to the processor, memory, one or more I / O interfaces, a system bus, and storage of the control units 202 and / or 302, respectively. The control unit 1702 may be located on the robotic device and / or on a remote server connected to one or more robotic devices.

[0066]

[0088] Memory 1720 stored instructions that can cause processor 1704 to execute modules, processes, and / or functions including active sensing 1722, learning skills and behaviors 1724, and executing actions 1726, and optionally including resource arbitration 1728 and data tracking and analysis 1758. Active sensing 1722, learning skills and behaviors 1724, executing actions 1726, resource arbitration 1728, and data tracking and analysis 1758 may be implemented as one or more programs and / or applications coupled to hardware components (e.g., sensors, operating elements, I / O devices, processors, etc.). Active sensing 1722, learning skills and behaviors 1724, executing actions 1726, resource arbitration 1728, and data tracking and analysis 1758 may be implemented by a single robotic device or by multiple robotic devices. In some embodiments, active sensing 1722 may include active sensing or scanning of an environment, as described herein. In other embodiments, active sensing 1722 may include active scanning of an environment and / or detecting or sensing information related to the environment, one or more objects in the environment (including, for example, a human in the environment), and / or one or more conditions related to the robotic device or system.

[0067]

[0089] Similar to storage 330, storage 1730 stores information related to an environment and / or objects within the environment, and skill learning and / or performance (e.g., tasks and / or social behaviors). Storage 1730 stores, for example, state information 1732, one or more skill models 1734, object information 1740, machine learning libraries 1742, and / or one or more environmental constraints 1754. Optionally, storage 1730 may also store tracking information 1756 and / or one or more arbitration algorithms 1758.

[0068]

[0090] State information 1732 includes information about the state of the robotic device (such as any of the robotic devices described herein) and / or the environment in which the robotic device is operating. In some embodiments, state information 1732 may include a map of the environment along with additional static and / or dynamic information related to objects in the environment. For example, state information 1732 may include a navigation map of a building along with static and / or dynamic information about objects within the building (e.g., fixtures, equipment, etc.) and social context information related to people and / or social situations within the building. FIG. 19 provides a schematic diagram of an example map or representation 1600 of a building. Representation 1600 includes a navigation layer 1610, a static semantic layer 1620, a social layer 1630, and a dynamic layer 1640. The navigation layer 1610 provides a general layout or map of the building, which may identify one or more floors 1612, with one or more walls 1614, one or more staircases 1616, and other elements (e.g., hallways, openings, boundaries) incorporated into the building. The static semantic layer 1620 identifies objects and / or spaces within the building, such as one or more rooms 1622, one or more objects 1624, and one or more doors 1626. The static semantic layer 1620 may identify which one or more rooms 1622 or other spaces are or are not accessible to a robotic device. In some embodiments, the static semantic layer 1620 may provide a three-dimensional map of objects located within a building. The social layer 1630 provides social context information 1632. The social context information 1632 includes information related to people within the building, such as past interactions between one or more robotic devices and one or more people. The social context information 1632 can be used to track interactions between one or more robotic devices and one or more humans, and these interactions can be used to generate and / or adapt existing models of skills involved in one or more interactions between a robotic device and a human.For example, social context information 1632 may indicate that a human is typically at a particular location, thereby allowing a robotic device with knowledge of that information to adapt the execution of a skill that requires the robotic device to move near the human's location. Dynamic layer 1640 provides information about one or more objects and other elements within a building that may move and / or change over time. For example, dynamic layer 1640 may track one or more movements 1644 and / or one or more changes 1646 associated with one or more objects 1642. In one embodiment, dynamic layer 1640 may monitor the expiration date of an object 1642 and identify when the object 1642 has expired.

[0069]

[0091] Representation 1600 may be available to and / or managed by, for example, control unit 1602 of a robotic device. Control unit 1602 may include components similar to other control units described herein (e.g., control units 202, 302, and / or 1702). Control unit 1602 may include storage (e.g., similar to other storage elements described herein, such as storage 330 and / or 1730) that stores representation 1600 and state information 1604 including information related to one or more robotic devices (e.g., the configuration of elements of a robotic device, the location of the robotic device within a building, etc.). Control unit 1602 may be located on the robotic device and / or on a remote server connected to one or more robotic devices. As one or more robotic devices collect information about their surrounding environment, one or more robotic devices may be configured to update and maintain state information 1604 including information related to building representation 1600.

[0070]

[0092] In some embodiments, the control unit 1602 may also optionally include a data tracking and analysis element 1606. The data tracking and analysis element 1606 may be, for example, a computational element (e.g., a processor) configured to perform data tracking and / or analysis of information collected by one or more robotic devices (e.g., information included in representation 1600 and / or other state information 1604). For example, the data tracking and analysis element 1606 may be configured to manage inventory (e.g., tracking expiration dates, monitoring and recording inventory usage, ordering new inventory, analyzing and recommending new inventory, etc.). In a hospital context, the data tracking and analysis element 1606 may manage the use and / or maintenance of medical supplies and / or equipment. In some embodiments, the data tracking and analysis element 1606 may generate aggregate data displays (e.g., reports, charts, etc.), which may be used to comply with official laws, regulations, and / or standards. In some embodiments, the data tracking and analysis element 1606 may be configured to analyze information related to humans, such as patients in a hospital. For example, a robotic device may be configured to collect information about one or more patients in a hospital and pass that information to the data tracking and analysis component 1606 for analysis and / or summarization for use in various functions, including, for example, diagnostic testing and / or screening. In some embodiments, the data tracking and analysis component 1606 may be configured to perform data tracking and / or analysis functions by accessing and / or obtaining information from third-party systems (e.g., hospital electronic medical records, security system data, insurance data, vendor data, etc.) and using and / or analyzing that data with or without data collected by one or more robotic devices.

[0071]

[0093] As shown in FIG. 20 , the one or more skill models 1734 are models that can be used to learn and / or perform various actions or skills, including tasks and / or behaviors. The one or more models 1734 may be similar to the one or more models 334 as described herein with reference to FIG. 3 . For example, the one or more models 1734 may include information related to one or more objects involved in the performance of a skill (e.g., objects manipulated by the robotic device, objects with which the robotic device interacts while performing a skill, objects considered by the robotic device while performing a skill). As described above, examples of objects include, for example, stationary and / or movable objects, such as furniture, equipment, people, and / or surfaces defining an opening (e.g., a doorway). The information related to the one or more objects may include, for example, markers that identify the objects and / or features of the objects. Additionally or alternatively, the one or more models 1734 may include sensory information collected by the robotic device, for example, during skill learning and / or performance or active sensing. As described above, the sensory information may include information related to one or more components of the robotic device when learning and / or performing the skill (e.g., a manipulation element, a transport element), and / or information related to the environment in which the skill is learned and / or performed (e.g., the location of one or more objects in the environment, social context information, etc.). In some embodiments, the model 1734 may be associated with success criteria, such as, for example, visual and / or tactile data sensed using one or more sensors of the robotic device, that indicate successful performance of the skill.

[0072]

[0094] Object information 1740 may include information related to one or more physical objects (e.g., location, color, shape, surface features, and / or identification code). Machine learning library 1742 may include modules, processes, and / or functions related to generating models of different machine learning algorithms and / or different skills.

[0073]

[0095] Environmental constraints 1754 include information related to objects and / or conditions in the environment that may limit the operation of the robotic device within the environment. For example, environmental constraints 1754 may include information related to the size, configuration, and / or location of objects within the environment (e.g., a supply bin, a room, a doorway, etc.) and / or information indicating that access to a particular area (e.g., a room, a hallway, etc.) is restricted. Environmental constraints 1754 may affect the learning and / or execution of one or more actions within the environment. As such, environmental constraints 1754 may be part of each model for a skill that is performed within a context that includes the environmental constraint.

[0074]

[0096] Tracking information 1756 includes information related to the representation of the environment (e.g., representation 1600) and / or information obtained from third party systems (e.g., hospital electronic medical records, security system data, insurance data, vendor data, etc.) that is tracked and / or analyzed by a data tracking and analysis component such as data tracking and analysis component 1606 or processor 1704 running data tracking and analysis 1758. Examples of tracking information 1756 include inventory data, supply chain data, point-of-use data, and / or patient data, as well as aggregate data compiled from such data.

[0075]

[0097] The one or more arbitration algorithms 1758 include algorithms for arbitrating or selecting between different actions to perform (e.g., how to use different resources or components of the robotic device). The one or more arbitration algorithms 1758 may be rules and, in some embodiments, may be learned as further described with reference to FIG. 24 . These algorithms may be used by a robotic device to select among different actions when the robotic device has multiple resources and / or objectives to manage. For example, a robotic device operating in an unstructured and / or dynamic environment (e.g., an environment that includes humans) may be exposed to several conditions at any one time that may require different behaviors or actions from the robotic device. In such cases, the robotic device may be configured to select among different actions based on one or more arbitration algorithms 1758, which may assign different priorities to various actions based on predefined rules (e.g., emotional state, environmental constraints, socially defined constraints, etc.). In some embodiments, the arbitration algorithm 1758 can assign different scores or values ​​to multiple actions based on information gathered by the robotic device regarding the surrounding environment, current conditions, and / or other factors.

[0076]

[0098] Similar to the one or more I / O interfaces 208 and / or 308, the one or more I / O interfaces 1708 may be any suitable one or more components that enable communication between the internal components of the control unit 1702 and external devices, such as a user interface, an operating element, a transport element, and / or a computing device. The one or more I / O interfaces 1708 may include a network interface 1760 that can connect the control unit 1702 to a network (e.g., network 105 as shown in FIG. 1). The network interface 1760 enables communication between the control unit 1702 (which may be located on the robotic device or on another network device that communicates with one or more robotic devices) and a remote device, such as a computing device that may be used by a robot supervisor to monitor and / or control one or more robotic devices. The network interface 1760 may be configured to provide wireless and / or wired connections to a network.

[0077]

[0099] 5 schematically illustrates a robotic device 400 according to some embodiments. The robotic device 400 includes a head 480, a torso 488, and a base 486. The head 480 may be connected to the torso 488 by segments 482 and one or more joints (not shown). The segments 482 may be movable and / or flexible to allow the head 480 to move relative to the torso 488. The head 480, segments 482, etc. may be examples of one or more operational elements and may include similar functionality and / or structure as one or more other operational elements described herein.

[0078]

[0100] Head 480 includes one or more imaging devices 472 and / or other sensors 470. The imaging devices 472 and / or other sensors 470 (e.g., lidar sensors, motion sensors, etc.) can enable robotic device 400 to scan an environment and obtain a representation (e.g., a visual or other semantic representation) of the environment. In some embodiments, imaging device 472 may be a camera. In some embodiments, imaging device 472 may be movable so that it can be used to focus on different areas of the environment around robotic device 400. Imaging device 472 and / or other sensors 470 can collect sensory information and send the sensory information to a computing device or processor (e.g., control unit 202 or 302, etc.) onboard robotic device 400. In some embodiments, head 480 of robotic device 400 has a human form and may include one or more human features (e.g., eyes, nose, mouth, ears, etc.). In such an embodiment, imaging device 472 and / or other sensors 470 may be implemented as one or more human features. For example, imaging device 472 may be implemented as an eye on head 480.

[0079]

[0101] In some embodiments, the robotic device 400 can scan an environment for information about objects in the environment (e.g., physical structures, devices, items, people, etc.) using the imaging device 472 and / or other sensors 470. The robotic device 400 can engage in active sensing, or the robotic device 400 can begin sensing or scanning in response to a trigger (e.g., input from a user, a detected event or change in the environment).

[0080]

[0102] In some embodiments, the robotic device 400 can engage in adaptive sensing, which can perform sensing based on stored knowledge and / or user input. For example, the robotic device 400 can identify areas in an environment to scan for objects based on prior information the robotic device 400 has about the objects. With reference to FIG. 6A , the robotic device 400 can scan a scene (e.g., an area of ​​a room) and obtain a representation 500 of the scene. In representation 500, the robotic device 400 identifies that a first object 550 is located in area 510 and a second object 560 is located in areas 510 and 530. The robotic device 400 can store the locations of objects 550 and 560 in its internally stored map of the environment so that the robotic device 400 can use that information to locate objects 550 and 560 when performing future scans. For example, when the robotic device 400 returns to the scene and scans it a second time, the robotic device 400 may obtain a different view of the scene, as shown in FIG. 6B . When performing this second scan, the robotic device 400 can obtain a representation 502 of the scene. To locate objects 550 and 560 in representation 502, the robotic device 400 can refer to information that it previously stored regarding the location of these objects when it obtained representation 500 of the scene. The robotic device 400 can take into account that its location in the environment may have changed and can recognize that objects 550 and 560 may be located in different areas of representation 502. Based on this information, the robotic device 400 can know to look at area 510 for object 550, but to look at areas 520 and 540 for object 560. By using the previously stored information regarding the locations of objects 550 and 560, the robotic device 400 can automatically identify areas to carefully scan (e.g., by zooming in, slowly moving the camera within the area) for objects 550 and 560.

[0081]

[0103] In some embodiments, the robotic device 400 may also know to more carefully sense or scan different areas of a scene based on human input. For example, a human may indicate to the robotic device 400 that certain areas of a scene contain one or more objects of interest, and the robotic device 400 may scan those areas more carefully to identify those objects. In such embodiments, the robotic device 400 may include an input / output device 440, such as a display with a keyboard or other input device and / or a touchscreen, as shown schematically in FIG. 5.

[0082]

[0104] In some embodiments, the robotic device 400 can scan the environment and identify that an object, such as a human, is moving within the environment. For example, as shown in FIGS. 7A and 7B, an object 660 may be moving within the environment while an object 650 remains stationary. FIG. 7A shows a representation 600 of a scene showing the object 660 within areas 610 and 630, and FIG. 7B shows a representation 602 of a scene showing the object 660 within areas 620 and 640. In both representations 600 and 602, the object 650 may remain in the same location in area 610. The robotic device 400 can identify that the object 660 has moved within the scene and adjust its actions accordingly. For example, if the robotic device 400 was to interact with the object 660, the robotic device 400 could change its trajectory (e.g., move closer to the object 660) and / or change the trajectory of a manipulative element or other component configured to interact with the object 660. Alternatively or additionally, if the robotic device 400 was to interact with the object 650 (and / or another object in the scene), the robotic device 400 could take the movement of the object 660 into account while planning a course for interacting with the object 650. In some embodiments, the robotic device 400 can engage in active sensing such that the actions of the robotic device 400 can be adjusted in near real time.

[0083]

[0105] 5, base 486 may optionally include one or more transport elements implemented as wheels 460. Wheels 460 may enable robotic device 400 to move about within an environment (e.g., a hospital). Robotic device 400 also includes at least one manipulating element implemented as arm 450. Arm 450 may be structurally and / or functionally similar to other manipulating elements described herein (e.g., manipulating element 350). Arm 450 may be fixedly attached to torso 488 of robotic device 400, or optionally, manipulating element 450 may be removably coupled to torso 488 by a coupling element (e.g., coupling element 359) that may attach to coupling portion 484 of robotic device 400. Coupling portion 484 may be configured to engage coupling element 359 and provide an electrical connection between arm 450 and an on-board computing device (e.g., control unit 202 or 302) so that the on-board computing device can provide power to and / or control components of arm 450 and receive information collected by sensors (e.g., sensors 353, 355, 357, and 358) located on operating element 450.

[0084]

[0106] Optionally, the robotic device 400 may also include one or more additional sensors 470 located on the segments 482, the torso 488, the base 486, and / or other portions of the robotic device 400. The one or more sensors 470 may be, for example, an imaging device, a force or torque sensor, a motion sensor, a light sensor, a pressure sensor, and / or a temperature sensor. The sensors 470 may enable the robotic device 400 to capture visual and non-visual information about the environment.

[0085] method

[0107] 8-11 are flow diagrams illustrating a method 700 that may be performed by a robotic system (e.g., robotic system 100) including one or more robotic devices, according to some embodiments. For example, all or a portion of method 700 may be performed by a single robotic device (such as any of the robotic devices described herein). Alternatively, all of method 700 may be performed sequentially by multiple robotic devices, each performing a portion of method 700 in turn. Alternatively, all or a portion of method 700 may be performed simultaneously by multiple robotic devices.

[0086]

[0108] As shown in FIG. 8 , a robotic device can scan an environment and obtain a representation of the environment at 702. The robotic device can scan the environment using one or more sensors (e.g., one or more sensors 270 or 470 and / or one or more imaging devices 472). In some embodiments, the robotic device can scan the environment using a movable camera, and the position and / or focus of the camera can be adjusted to capture multiple areas within a scene of the environment. At 704, based on information collected during sensing, the robotic device can identify one or more markers within the captured representation of the environment by analyzing the data. The markers may be associated with one or more objects within the scene marked with visual markers or fiducial markers (e.g., visible markers such as QR codes, barcodes, tags, etc.). Alternatively or additionally, the robotic device can identify markers associated with one or more objects within the environment through object recognition using object information (e.g., object information 340) stored in memory (e.g., storage 330) on the robotic device. The object information may include information that describes different features of the object, such as location, color, shape, surface features, etc. In some embodiments, the object information may be organized as numerical values ​​that represent different features of the object, sometimes referred to as a feature space.

[0087]

[0109] After identifying the one or more markers, at 706, the robotic device may optionally present the one or more markers within a representation of the environment. In some embodiments, the representation of the environment may be a visual representation, such as, for example, an augmented view of the environment. In such embodiments, the robotic device may display the visual representation of the environment, for example, on a display screen, and display the locations of the one or more markers within the visual representation of the environment. Alternatively or additionally, the representation of the environment may be a semantic representation of the environment, with the locations of the one or more markers indicated by semantic markers within the environment.

[0088]

[0110] In some embodiments, the robotic device may present a representation of the environment with one or more markers to the user and, optionally, at 708, may prompt the user, for example, using a user interface or other type of I / O device, to accept or reject one or more markers in the representation of the environment. If the user does not accept one or more markers (708: NO), method 700 may return to 702, where the robotic device may obtain a second representation of the environment by rescanning the environment. If the user accepts one or more markers (708: YES), method 700 may proceed to 708, where the robotic device may store information associated with the markers (e.g., location, feature, etc.) in memory (e.g., storage 330). For example, the robotic device may store the location of one or more markers in an internal map of the environment (e.g., map 332).

[0089]

[0111] In some embodiments, the robotic device may identify one or more markers at 704 and immediately begin saving the locations of the one or more markers and / or other information associated with the markers at 710 without prompting the user to approve the one or more markers. In such embodiments, the robotic device may analyze the locations of the one or more markers before saving them. For example, the robotic device may have previously stored information regarding the locations of one or more markers (e.g., obtained during a previous scan of the environment and / or input into the robotic device by a user or computing device) and may compare the locations of the one or more markers to the previously stored information to check accuracy and / or identify changes in the marker locations. Specifically, if the previously stored information indicates that a particular marker should be located at a location different from the location identified by the robotic device, the robotic device may verify the location by initiating an additional scan of the environment before saving the marker location. Alternatively or additionally, the robotic device may send a notification to the user indicating that the marker location has changed. In such cases, the robotic device may save the new location of the marker but may also save a message indicating that the marker location has changed. The user or computing device can then later review the messages and reconcile any changes in the location of the markers.

[0090]

[0112] As shown in FIG. 9 , method 700 optionally proceeds to 712, where the robotic device can prompt a user, e.g., using a user interface or other type of I / O device, to select a set of markers from one or more markers identified in the representation of the environment. At 714, the user can make a selection, and the robotic device can receive the selection from the user. Alternatively, in some embodiments, instead of prompting the user to make a selection, the robotic device can automatically select a set of markers. The robotic device may be programmed to select markers based on certain predefined or learned rules and / or conditions. For example, the robotic device may be instructed to select markers associated with a particular type of object (e.g., supplies) during a particular time of day or when foot traffic in the building is low. In the latter case, the robotic device can determine when foot traffic in the building is low by actively moving through the building (e.g., patrolling and monitoring hallways and rooms) and sensing or scanning the environment. In this way, the robotic device can know to select markers associated with particular objects when foot traffic in the building is lower than most other times.

[0091]

[0113] After the robotic device receives a selection of a set of markers from a user and / or automatically selects a set of markers, method 700 may proceed to learning 716 a skill or performing 718 a skill.

[0092]

[0114] For any particular skill, before the robotic device can perform or implement the skill, the robotic device can be taught the skill. For example, to acquire a manipulation skill, the robotic device can be taught by LfD (e.g., kinesthetic teaching), in which a user (e.g., a nearby user or a remotely located robot supervisor) or another robotic device can demonstrate the skill to the robotic device. For example, a manipulation element such as an arm of the robotic device can move through a series of waypoints to interact with an object. As another example, a mobile base of a robotic device (e.g., a base with transport elements such as wheels, tracks, crawlers, etc.) can be navigated around objects in an environment by a user (e.g., by using a joystick, user interface, or other type of physical or virtual control device, or by physically directing or moving (e.g., pulling, pushing) the robotic device).

[0093]

[0115] With kinesthetic teaching, a user can physically demonstrate a skill to a robotic device. Training or teaching can occur in a mass-production setting (e.g., a manufacturing environment) where a robotic device can be taught using an aggregate model that represents the generic behavior of a skill. Alternatively or additionally, teaching can occur in the field after the robotic device is deployed (e.g., in a hospital) so that the robotic device can learn to perform a skill in a specific field environment. In some embodiments, a robotic device can be taught in a field setting and then transmit information related to the learned skill to one or more other robotic devices so that these other robotic devices also have knowledge of the taught skill when operating in the same field setting. Such embodiments can be useful when multiple robotic devices are deployed at a single field setting. In this case, each robotic device can send and receive information to other robotic devices so that the other robotic devices can collectively learn a set of skills related to the field environment.

[0094]

[0116] 10, method 700 proceeds to 720-724, where a user can teach a skill to a robotic device using an LfD teaching process. In one embodiment, a skill may be defined as grasping an object at a particular location, picking up the object, moving the object to another location, and setting the object down at the other location. In another embodiment, a skill may involve interacting with the robotic device's surrounding environment (e.g., a door).

[0095]

[0117] At 720, a user (or another robotic device) can teach a robotic device, including a manipulation element (e.g., manipulation element 250, 350, or 450) and / or a transport element (e.g., one or more transport elements 260, 460), actions. For example, a user can teach a robotic device's manipulation element (and / or other components of the robotic device (e.g., a transport element)) with a demonstration related to performing a particular skill (e.g., interacting with a human, engaging with and / or manipulating an object, and / or other interaction with one or more humans and / or the surrounding environment). In some embodiments, a user can demonstrate to a robotic device how to interact with and / or navigate through a door. A user can demonstrate how the robotic device, for example, interacts with a door handle using one or more of its manipulation elements. For example, a robotic device's manipulation element, such as an arm, can be taught a series of movements relative to a door handle (e.g., entered by visual input or identified by a fiducial marker). For example, while or after a door handle is turned using a manipulating element, the robotic device may be demonstrated to push the door open, for example, by moving one or more transport elements. As described further below, a combination of the movements of one or more manipulating elements and one or more transport elements may be used to build a model for performing a skill or behavior for future interactions with that door and / or similar doors.

[0096]

[0118] In some embodiments, the robotic device can be supervised by a locally present user (e.g., by a user physically moving the robotic device and / or providing input to the robotic device). In some embodiments, the robotic device can be supervised by a user (e.g., a robot supervisor) located remotely from the robotic device, for example, using a remote or cloud interface.

[0097]

[0119] While directing a manipulative element (and / or other components of the robotic device) to move, the user can indicate to the robotic device when to capture information about the state of the manipulative element (e.g., joint configuration, joint forces and / or joint torques, end-effector configuration, end-effector position), another component of the robotic device, and / or the environment (e.g., location of an object associated with a selected marker and / or other objects in the environment). For example, the robotic device can receive a signal from the user at 722 at a waypoint or keyframe during movement of the manipulative element to capture information about the manipulative element and / or the environment. In response to receiving the signal, the robotic device can capture information about the manipulative element, other components of the robotic device, and / or the environment at that keyframe at 724. Manipulative element information can include, for example, joint configuration, joint torque, end-effector position, and / or end-effector torque. Environmental information can include, for example, the position of a selected marker relative to the end-effector, indicating to the robotic device when objects in the environment may have moved. If the motion is still continuing (728: NO), the robotic device may wait to capture information about the operational element and / or the environment at additional key frames. In some embodiments, the robotic device may be programmed to capture key frame information without receiving a signal from the user. For example, while the operational element is being moved by the user, the robotic device may monitor changes in the segments and joints of the operational element. When these changes exceed a threshold or when there is a directional change in the trajectory of a segment or joint, the robotic device may autonomously select that point in time as a key frame and record information about the operational element and / or the environment at that key frame.

[0098]

[0120] During operation of the manipulative element (and / or other components of the robotic device (e.g., the transport element)), the robotic device may also continuously or periodically record sensory information (e.g., information about the manipulative element, one or more other elements of the robotic device (e.g., the transport element), and / or the environment) at 730 without receiving a signal from the user. For example, as the user moves the manipulative element throughout a demonstration, the robotic device may record information about the trajectories of the segments and joints and their configurations. During the demonstration, the robotic device may also record information about one or more environmental constraints (e.g., static information about one or more objects in the environment (e.g., location or size of doorways, supply bins, etc.) and / or dynamic information about one or more objects in the environment (e.g., traffic levels in a room, user movements around the robotic device, etc.)). In some embodiments, sensory information recorded by the robotic device during a demonstration can add to and / or modify one or more layers of a map of the environment, such as that shown in FIG. 19 .

[0099]

[0121] In some embodiments, the robotic device may include an audio device (e.g., 244), such as a microphone, and the definition of key frames may be controlled by voice commands. For example, a user can indicate to the robotic device that they are going to demonstrate by saying, "I will guide you." A demonstration may begin when the user indicates the first key frame by saying, "start here." An intermediate key frame may be indicated by saying, "go here." And the final key frame, representing the end of the demonstration, may be indicated by saying, "end here." Suitable examples of demonstration teaching are provided in Akgun's paper.

[0100]

[0122] In some embodiments, while demonstrating a skill to a robotic device, a user can indicate, e.g., through one or more inputs to the robotic device, which one or more portions of the skill are generic and which one or more portions of the skill are more specific to a particular environment or situation. For example, while demonstrating to a robotic device how to move equipment from a first location to a second location (e.g., a room) and then drop the equipment off at the second location, a user can indicate to the robotic device that the action of navigating from the first location to the second location is generic, but the action of dropping the equipment off at the second location is specific and requires additional information specific to the environment to perform (e.g., a specific tag or marker for dropping the equipment off). When the robotic device later uses a model for the skill to move equipment between different locations, the robotic device can know to request and / or scan specific information regarding the unloading before executing the skill (e.g., request and / or scan information about a specific tag to determine the location before unloading).

[0101]

[0123] In some embodiments, after demonstrating a skill to the robotic device, the user can indicate portions of the skill that are generic or specific and / or change portions of the skill that the user previously indicated as generic or specific. In some embodiments, the robotic device can determine whether portions of a set of skills are generic or specific after learning a set of skills. In some embodiments, the robotic device can recommend to the user that portions of the set of skills are generic or specific and request confirmation from the user. Based on the user's confirmation, the robotic device can store this information for reference when learning and / or performing future skills related to this set of skills. Alternatively, the robotic device can automatically determine and classify different portions of a skill as generic or specific without user input.

[0102]

[0124] Upon completion of the action or demonstration (728: YES), the robotic device can generate a model of the demonstrated skill based on a subset of all recorded sensory information (e.g., manipulation element information, transport element information, environmental information). For example, at 732, the robotic device can optionally prompt the user, e.g., using a user interface or other type of I / O device, to select features relevant to learning the skill, and at 734, the robotic device can receive the feature selection from the user. Alternatively or additionally, the robotic device can know to select particular features to use in generating the model based on previous instructions from the user. For example, the robotic device can recognize, e.g., based on the sensory information, that object picking is being demonstrated, and can automatically select one or more features of the sensory information (e.g., joint configuration, joint torque, end effector torque) to include as relevant features for generating the skill model, e.g., based on past demonstrations of picking up the same or different objects.

[0103]

[0125] At 736, the robotic device can generate a model of the skill using the selected features. The model may be generated using a stored machine learning library or algorithm (e.g., machine learning library 342). In some embodiments, the model may be represented as an HMM algorithm including multiple parameters, such as a number of hidden states, a feature space (e.g., features included in a feature vector), and an emission distribution for each state modeled as a Gaussian distribution. In some embodiments, the model may be represented as a support vector machine, or “SVM,” model, which may include parameters, such as a kernel type (e.g., linear, radial, polynomial, sigmoidal), a cost parameter or function, weights (e.g., equal, class balanced), a loss type or function (e.g., hinge, rectangular hinge), and an answer type or problem type (e.g., dual, primal). The model may be associated with relevant sensory information and / or other sensory information recorded by the robotic device during the performance of the skill. The model may also be associated with marker information indicating a set of markers manipulated during the performance of the skill and / or features associated with one or more physical objects bound to those markers. The robotic device may store the model in memory (e.g., storage 230 or 330) at 738.

[0104]

[0126] In some embodiments, information related to the performance (e.g., sensory information, models, etc.) can be used to add to and / or modify layers of a map or representation of the environment, such as the representation 1600 shown in Figure 19. For example, Figure 21 shows

[0105]

[0127] Optionally, at 740, the robotic device may determine whether the user will perform another demonstration of the skill. If another demonstration is to be performed (740: YES), method 700 may return to 720, where the user (or another robotic device) may direct the robotic device for additional demonstrations. If the demonstration is complete (740: NO), method 700 may optionally return to the beginning and perform a new scan of the environment. Alternatively, in some embodiments, method 700 may end.

[0106]

[0128] In another embodiment, the skill may be a navigation behavior, such as navigation between two locations or navigation around and / or through objects in an environment. Similar to learning a skill using a manipulative element as described herein, at 720, a user (or other robotic device) can teach a navigation behavior to a robotic device including a set of transport elements (e.g., one or more transport elements 260, 460). For example, the user can use a joystick or other control device to control the movement of the set of transport elements (and / or other components of the robotic device (e.g., manipulative elements)) and / or physically guide such elements (e.g., push or pull the robotic device) to enable the robotic device to perform a navigation behavior. While controlling the movement of the set of transport elements (and / or other components of the robotic device), the user can signal the robotic device when to capture sensory information, such as information about the state of the set of transport elements (e.g., the angle of each transport element, the configuration of each transport element, the spacing between transport elements if movable relative to one another, etc.), other components of the robotic device, and / or the environment (e.g., the location of the robotic device in a map, the location and / or boundaries of objects in the environment). The user can signal the robotic device at key frames during the movement of the set of transport elements (and / or other components of the robotic device), e.g., at the beginning, end, and / or transition points between moving the set of transport elements in a first direction and moving the set of transport elements in a second direction. In response to receiving a signal from the user at 722, the robotic device can capture snapshots of the movement (including information about the set of transport elements, other components of the robotic device, and / or the environment) at the key frames. Alternatively or additionally, the robotic device may be configured to autonomously select points during the movement of the set of transport elements (and / or other components of the robotic device) at which to capture snapshots of the movement.For example, the robotic device can monitor the angle and / or configuration of the set of conveying elements (e.g., when a user takes over control of the robotic device (e.g., by controlling the robotic device suing a controlling device, by pushing the robotic device), etc.), and can automatically capture a snapshot of that information each time the robotic device detects a change in angle and / or configuration. In some embodiments, the robotic device may be configured to continuously collect information about the set of conveying elements, other components of the robotic device, and / or the environment while a user controls the movement of the robotic device.

[0107]

[0129] Similar to learning a skill using a manipulative element as described herein, the robotic device can continue capturing sensory information related to the operation of the transport element (and / or other components of the robotic device) until the operation is completed (728: YES). The robotic device can optionally receive a selection of features relevant to learning navigation behavior and / or autonomously identify relevant features in the collected sensory information (e.g., by recognizing that a particular navigation skill is being demonstrated and identifying the features relevant to learning that skill) at 732-734. The robotic device can then generate a model for navigation behavior based on the sensory information related to the relevant features at 736 and store the model and sensory information at 738 for later use in generating a trajectory for the robotic device to execute.

[0108]

[0130] An example of a navigation behavior is navigating through a doorway. While a robotic device can be configured to navigate through standard doorways in a building, the robotic device may not be able to navigate through non-standard doorways (e.g., small or oddly shaped doorways). Thus, the robotic device may prompt a user to demonstrate to the robotic device how it can safely navigate through a doorway. The robotic device may navigate to one side of a doorway and then have the user demonstrate to the robotic device how to pass through the doorway to the other side. During the demonstration, the robotic device may passively record information such as the location of the robotic device in a map and / or sensor data (e.g., door boundaries, configurations of conveying and / or operating elements, and other sensory information as described herein). In some embodiments, the robotic device may learn how to navigate through a doorway using an interactive learning template, as further described herein with reference to FIG. 16 .

[0109]

[0131] In some embodiments, the user may be on-site near the robotic device, while in other embodiments, the user may be located remotely and control the robotic device from a computing device (e.g., server 120, computing device 150) via a network connection to the robotic device (e.g., as shown in FIG. 1 and described above).

[0110]

[0132] In some embodiments, as described herein, a robotic device may actively scan its surrounding environment to monitor changes in the environment. Thus, during learning and / or execution, the robotic device may engage in continuous sensing or scanning of the environment and accordingly update its representation of the environment and the environmental information stored by the robotic device.

[0111]

[0133] In some embodiments, a robotic device can be configured to learn socially appropriate behaviors, i.e., actions that take into account interactions with humans. For example, a robotic device can be configured to learn manipulation or navigation skills to be performed around humans. The robotic device can learn socially appropriate behaviors through demonstration by a human operator (e.g., a nearby user or a remotely located robot supervisor). In some embodiments, a robotic device may be configured to learn behaviors in an interactive context (e.g., by a human operator intervening in the robotic device's autonomous performance of a skill and demonstrating to the robotic device how to perform the skill). When a robotic device detects human operator intervention (e.g., a nurse pushing the robotic device to one side), the robotic device may be configured to switch to a learning mode in which the robotic device is controlled by the human operator and passively records information related to the demonstration and the perceptual context in which the demonstration takes place. The robotic device can use this information to generate a revised model of the skill and associate the model with the appropriate social context in which the robotic device should later perform the skill. As an example, the robotic device can detect when a human operator intervenes in the autonomous execution of the navigation plan and, in response to detecting the intervention, switch to a learning model controlled by the human operator. In the presence of a human, the robotic device can record information about its surrounding environment and its own behavior as the human operator demonstrates how the navigation plan should be modified (e.g., that instead of waiting for a clear aisle, the robotic device should move to the side of the aisle to let the human through). Further details regarding interactive learning are described herein with reference to FIG. 16.

[0112]

[0134] In the execution mode shown in FIG. 11 , the robotic device can optionally prompt the user at 750, e.g., using a user interface or other type of I / O device, to select a model, such as a model previously generated by the robotic device in learn mode. The robotic device can receive the model selection at 752. In some embodiments, the robotic device can receive the model selection from the user, or alternatively, the robotic device can automatically select a model based on certain rules and / or conditions. For example, the robotic device may be programmed to select a model when it is in a certain area of ​​a building (e.g., when it is in a certain room or floor), during a certain time or day of the week, etc. Alternatively or additionally, the robotic device can know to select a certain model based on a selected set of markers. At 754, the robotic device can determine whether to move near the selected markers before generating a trajectory and performing a skill on the selected markers. For example, the robotic device can determine whether to move to be more desirable for performing the skill (e.g., closer or closer to the markers, facing the markers from a certain angle) based on the selected set of markers and the selected model. The robotic device may make this determination based on sensory information recorded during the performance of the skill. For example, the robotic device may recognize that it was positioned closer to a marker when the skill was performed and adjust its position accordingly.

[0113]

[0135] If the robotic device determines to move relative to the selected marker (754: YES), the robotic device may move its position (e.g., adjust its location and / or orientation) at 756, and method 700 may return to 702, where the robotic device again scans the environment to obtain a representation of the environment. Method 700 may proceed through various steps and return to 754. If the robotic device determines not to move relative to the selected marker (754: NO), the robotic device may generate a trajectory of action, for example, for a manipulative element of the robotic device.

[0114]

[0136] Specifically, at 758, the robotic device can calculate a function that transforms (e.g., translates) between the selected set of markers and the set of markers associated with the selected model (e.g., one or more markers selected when the robotic device learned the skill, i.e., generated the selected model), referred to herein as the “saved markers” or “saved set of markers.” For example, a robotic device can be taught a skill using a first set of markers that were in one or more particular locations and / or one or more orientations relative to a portion of the robotic device, such as an end effector, and the robotic device can later perform the skill using a second set of markers that are in one or more different locations and / or one or more different orientations relative to the manipulative element. In such a case, the robotic device can calculate a transformation function that transforms between one or more positions and / or one or more orientations of the first set of markers and one or more positions and / or one or more orientations of the second set of markers.

[0115]

[0137] At 760, the robotic device may use the calculated transformation function at each keyframe recorded when the skill was taught to transform the position and orientation of a portion of the manipulative element (e.g., an end effector of the manipulative element). Optionally, at 762, the robotic device may take into account any environmental constraints, e.g., characteristics of objects and / or areas in the environment, such as size, configuration, and / or location. The robotic device may limit the movement of the manipulative element based on the environmental constraints. For example, if the robotic device recognizes that it is attempting to perform a skill in a supply room, the robotic device may take into account the size of the supply room when transforming the position and orientation of the manipulative element to avoid the portion of the manipulative element contacting walls or other physical structures in the supply closet. The robotic device may also take into account other environmental constraints related to the supply room, such as the size of bins in the supply room, the location of shelves in the supply room, etc. The robotic device can be provided with information about and / or taught one or more environmental constraints prior to performing a skill in a situation having the one or more environmental constraints, as further described herein with reference to FIG. 17 .

[0116]

[0138] Optionally, at 762, the robotic device can use inverse kinematics equations or algorithms to determine 762 the joint configurations of the manipulative element for each keyframe. The positions and orientations of the end effector and set of markers may be provided in task space (e.g., the Cartesian space in which the robotic device is operating), and the joint orientations can be provided in joint or configuration space (e.g., an n-dimensional space in which the robotic device is represented by a point and associated with the configuration of the manipulative element, where n is the number of degrees of freedom of the manipulative element). In some embodiments, the inverse kinematics calculations can be guided by joint configuration information recorded when the robotic device was taught a skill (e.g., joint configurations recorded during a teaching demonstration using the manipulative element). For example, the inverse kinematics calculations can be seeded (e.g., provided as an initial guess for the calculations or provided with a bias) using the joint configurations recorded at each keyframe. Additional conditions can also be imposed on the inverse kinematics calculations, such as requiring that the calculated joint configurations do not deviate from the joint configurations of adjacent keyframes by more than a predefined amount. At 764, the robotic device can plan a trajectory between joint configurations, e.g., in joint space, from one key frame to the next, to generate a complete trajectory for the manipulative element to perform the skill. Optionally, the robotic device can take into account environmental constraints as described herein.

[0117]

[0139] In some embodiments, after the robotic device transforms the position and orientation of a portion of the manipulative element (e.g., an end effector), the robotic device can plan a trajectory for the manipulative element in the task space. In such embodiments, method 700 can proceed directly from 760 to 764.

[0118]

[0140] At 766 and 768, the robotic device can optionally present the trajectory to the user and prompt the user to approve or reject the trajectory, for example, using a user interface or other I / O device. Alternatively, the robotic device may approve or reject the trajectory based on internal rules and / or conditions and by analyzing associated sensory information. If the trajectory is rejected (768: NO), the robotic device can optionally modify one or more parameters of the selected model at 770 and generate a second trajectory at 758-764. The model parameters can be modified, for example, by selecting different features (e.g., different sensory information) to include in model generation. In some embodiments where the model is an HMM model, the robotic device can modify the model parameters based on the determined success or failure, where the robotic device tracks the log-likelihood of different models with different parameters and selects the model with a higher log-likelihood than the other models. In some embodiments where the model is an SVM model, the robotic device can modify the parameters by changing the feature space or configuration parameters (e.g., kernel type, cost parameter or cost function, weights) as described herein.

[0119]

[0141] If the trajectory is approved (768: YES), the robotic device may move the manipulative element to execute the generated trajectory at 772. While the manipulative element executes the planned trajectory, the robotic device may record and / or store sensory information, e.g., information about the manipulative element and / or the environment, at 774, e.g., using one or more sensors on the manipulative element and other components of the robotic device.

[0120]

[0142] Optionally, at 774, the robotic device can determine whether the execution of the skill was successful (e.g., whether the interaction with the object meets predefined success criteria). For example, the robotic device can scan the environment and determine the current state of the environment and the robotic device, including, for example, the location of one or more objects and / or the position or orientation of those objects relative to a manipulative element or another component of the robotic device, and determine whether the current state matches the predefined success criteria. The predefined and / or learned success criteria may be provided by a user or, in some embodiments, may be provided by a different robotic device and / or computing device. The predefined and / or learned success criteria may indicate information about different characteristics of the environment and / or robotic device that are associated with success. In some embodiments, the user can also provide input to the robotic device that indicates the execution was successful.

[0121]

[0143] In one particular example where a skill is defined as grasping an object at a particular location and picking up the object, success of the skill may be taught and / or defined as detecting one or more markers associated with the object in a particular relationship relative to each other and / or to the robotic device, or detecting that sufficient force or torque is (or was) experienced by an end effector or joint (e.g., a wrist joint) of a manipulative element (meaning that the manipulative element is supporting the weight of the object and thus has picked up the object). If the execution was unsuccessful (776: NO), the robotic device may optionally modify model parameters at 770 and / or generate a new trajectory at 758-764. If the execution was successful (776: YES), data related to the successful interaction (e.g., data indicating that the execution was successful and how it was successful) may be recorded, and method 700 may optionally return to the beginning and perform a new scan of the environment. Alternatively, in some embodiments, method 700 may end.

[0122]

[0144] In some embodiments, the robotic device is configured to perform a skill involving one or more movements of a manipulative element, a transport element, and / or another component of the robotic device (e.g., head, eyes, sensors, etc.). The robotic device may be configured to plan a trajectory for the skill, similar to that described above with respect to the manipulative element. For example, at 750-752, the robotic device may prompt and receive a user selection of a model for the skill, or alternatively, may autonomously select a model for the skill. At 758, the robotic device may calculate a function that transforms between a set of markers currently identified in the environment and a set of stored markers associated with the selected model (e.g., one or more markers identified when the robotic device learned the skill and stored with the model of the skill). Such transformations may include transforming keyframes associated with the transport element or other components of the robotic device. At 760, the robotic device may use the calculated function to transform the configuration of one or more components of the robotic device at each keyframe. At 764, the robotic device may plan a trajectory for one or more components of the robotic device between each transformed keyframe. While transforming the key frames and / or planning the trajectory, the robotic device can optionally take into account any environmental constraints associated with the context in which the skill is being performed. At 772, the robotic device can perform movement of one or more components of the robotic device according to the planned trajectory. In some embodiments, the robotic device may determine the joint configuration at 762, present the planned trajectory to the user at 766, and / or perform other optional steps as shown in FIG. 11 . An example of a skill involving movement of a transport element may be opening a door. A user (e.g., a nearby user or a remotely located robot supervisor) can instruct the robotic device to open the door using the robot's base (e.g., by pushing the door open).A user can direct the robotic device by, for example, using a control device such as a joystick or by using a base to physically direct the robotic device to perform an action to open a door (e.g., pushing or pulling the robotic device). The robotic device can sense and record information while the user is directing the action and use that information to generate a skill model. The robotic device can later use the skill model to perform an action to open that door or other doors in the environment, for example, by transforming keyframes associated with transport elements during the user's skill demonstration.

[0123]

[0145] FIG. 12 is a block diagram illustrating a system architecture for robot learning and execution, including actions performed by a user, according to some embodiments. System 800 can be configured for robot learning and execution. System 800 may include one or more robotic devices, such as any of the robotic devices described herein, that can perform modules, processes, and / or functions illustrated in FIG. 12 as active sensing 822, marker identification 824, learning and model generation 826, trajectory generation and execution 828, and success monitoring 829. Active sensing 822, marker identification 824, learning and model generation 826, trajectory generation and execution 828, and success monitoring 829 may correspond to one or more steps performed by the robotic device, as described with reference to method 700 shown in FIGS. 8-11. For example, active sensing 822 may include step 702 of method 700, marker identification 824 may include one or more of steps 704-710 of method 700, learning and model generation 826 may include one or more of steps 712-738, trajectory generation and execution 828 may include one or more of steps 712, 714, 718, and 750-774, and success monitoring 829 may include one or more of steps 774 and 776.

[0124]

[0146] System 800 can be connected to (e.g., in communication with) one or more devices including, for example, one or more cameras 872, an arm 850 (including gripper 856 and one or more sensors 870), a display device 842, and a microphone 844. System 800 can receive input from a user using display device 842, microphone 844, and / or other I / O devices (not shown) related to one or more user actions 880. User actions 880 can include, for example, a user approving a marker or requesting a rescan of the environment 882, a user selecting one or more markers 884, a user selecting relevant information and generating a model 886, a user selecting a model for performing a skill 888, a user approving a trajectory for performing a skill 890, a user confirming the success of an performed skill 892, and a user teaching a skill through kinesthetic learning 894.

[0125]

[0147] For active sensing 822, system 800 scans the environment using one or more cameras 872 and records sensory information about the environment, including information associated with one or more markers in the environment. For marker identification 824, system 800 can analyze the sensory information to identify one or more markers in the environment and receive one or more inputs from the user, for example, using display device 842 (showing 822 that the user approves the marker or requests a rescan of the environment). For learning and model generation 826, system 800 can receive sensory information collected by one or more cameras 872 and / or one or more sensors 870 on arm 850 and use that information to generate a model of the skill. As part of learning and model generation 826, system 800 can receive one or more inputs from the user, for example, using display device 842 and / or microphone 844 (indicating user selection of one or more sets of markers for teaching the skill 884, user selection of particular features of recorded sensory information for use in generating a model 886, and / or user demonstrating a skill 894). For trajectory generation and execution 828, system 800 can generate a planned trajectory and control the movement of arm 850 to execute the trajectory. For trajectory generation and execution 828, system 800 can receive one or more inputs from the user, for example, using display device 842 (indicating user selection of a model for generating a trajectory 888, and / or user approval or rejection of the generated trajectory 890). For success monitoring 829, system 800 can determine whether the skill was successfully performed by analyzing sensory information recorded by one or more sensors 870 during the skill's performance. As part of success monitoring 829, system 800 may receive one or more inputs from the user (shown 892 indicating the user confirmed that the execution was successful), for example, using display device 842 and / or microphone 844.

[0126]

[0148] While particular one or more devices and / or connections between system 800 and the one or more devices are shown in FIG. 12 , it is understood that any of the embodiments described herein may enable one or more additional devices (not shown) to communicate with system 800 to receive information from and / or transmit information to system 800.

[0127]

[0149] 13-17 are flow diagrams illustrating methods 1300 and 1400 that may be performed by a robotic system including one or more robotic devices (e.g., robotic system 100) according to embodiments described herein. For example, methods 1300 and / or 1400 may be performed by a single robotic device and / or multiple robotic devices.

[0128]

[0150] As shown in FIG. 13 , the robotic device is configured to operate in an execution mode at 1301. In the execution mode, the robotic device can autonomously plan and execute actions within an environment. To determine which actions to perform and / or plan how to perform the actions, the robotic device can scan the environment and collect information about the environment and / or objects within the environment at 1304 and use this information to build and / or modify a representation or map of the environment (e.g., representation 1600) at 1305. The robotic device can repeatedly (e.g., at predefined times and / or time intervals) or continuously scan the environment and update the representation of the environment based on the information it collects about the environment. Similar to the methods described above, the robotic device can collect information about the environment using one or more sensors (e.g., one or more sensors 270 or 470 and / or one or more imaging devices 472).

[0129]

[0151] In some embodiments, the robotic device can repeatedly and / or continuously scan an environment at 1304 for data tracking and / or analysis at 1307. For example, the robotic device can move through an environment (e.g., a building such as a hospital) autonomously or remotely driven (e.g., by a robot supervisor) and collect data about the environment, objects within the environment, etc. This information can be used by a data tracking and analysis component (e.g., data tracking and analysis component 1606), for example, managing hospital supplies and / or equipment. In some embodiments, the robotic device can collect information about various people (e.g., patients) to track such patients' behavior (e.g., medication and / or treatment compliance), perform diagnostic tests, and / or conduct screenings, etc.

[0130]

[0152] In some embodiments, the robotic device can repeatedly and / or continuously scan its environment at 1304 for modeling and learning purposes at 1309. For example, the robotic device can collect data about its environment and / or objects within the environment (e.g., including humans and their behavior in response to robot actions) and use that information to develop new behaviors and / or actions. In some embodiments, the robotic device can collect a large amount of information about an environment, which can be used by the robotic device to further refine and / or generate a model specific to that environment. In some embodiments, the robotic device can provide this information to a robot supervisor (e.g., a remote user), who can use this information to further adapt the robotic device to a particular environment (e.g., by generating and / or modifying skill models and / or behaviors within that environment). The robot supervisor can repeatedly (e.g., at specific time intervals) and / or continuously (e.g., in real time) fine-tune the information collected by the robotic device and / or other robotic devices and / or the parameters of the models the robot is using. In some embodiments, this active exchange of information with the robotic device allows the robotic supervisor to repeatedly and / or continuously adapt the robotic device to a particular environment. For example, the robotic supervisor can modify a planned path that the robotic device may be using to navigate through an environment, which in turn can change the information and / or one or more models used by the robotic device to generate the movements of its transport elements. These changes, both provided by the robotic supervisor and / or implemented by the robotic device, can be fed into one or more layers (e.g., the social layer or semantic layer of the map) of the environment (e.g., map 1600).

[0131]

[0153] At 1306, the robotic device may determine which one or more actions to perform in the environment, i.e., arbitrate among a set of resources or components on which a particular action can be performed. The robotic device may select among different actions based on one or more arbitration algorithms (e.g., one or more arbitration algorithms 1758). The robotic device may arbitrate autonomously and / or with user input. For example, the robotic device may be configured to request user input when it is unable to determine the current state of one or more of its components and / or objects in the environment, or when it is unable to determine which action to perform and / or plan how to perform a selected skill. In some embodiments, the robotic device may be configured to autonomously select among a set of actions when it is familiar with a particular situation (e.g., has previously learned and / or performed actions in that situation) and to request user input when it encounters a new situation. When the robotic device encounters a new situation, the robotic device may request that the user select an appropriate action for the robotic device to perform, or alternatively, the robotic device may prompt the user to select an action and confirm the selection of the action.

[0132]

[0154] In some embodiments, a human operator (e.g., a robot supervisor) may monitor the robotic device and send a signal to the robotic device when the human operator wants to intervene in the performance of an action by the robotic device. The human operator may be located near the robotic device and / or on a remote computing device connected via a network to the robotic device or a nearby device that may be used to monitor the robotic device. For example, the human operator may decide to intervene in the performance of an action by the robotic device when the human operator wants to teach the robotic device a new skill or behavior for safety reasons (e.g., to prevent harm to the environment surrounding the human or the robotic device) and / or to prevent damage to the robotic device.

[0133]

[0155] In response to determining that user input is required (e.g., due to an unfamiliar situation or in response to a signal from the user) (1312: YES), the robotic device can optionally prompt the user to provide user input at 1314. For example, the robotic device can display a prompt to the user requesting user input on an on-board display or a display located on a remote device (e.g., using a remote or cloud interface). The robotic device can receive the user input at 1315 and perform arbitration based on the user input. If user input is not required (1312: NO), the robotic device continues to perform arbitration autonomously.

[0134]

[0156] After the robotic device selects an action to perform, the robotic device can plan and execute the action at 1308. As described above, the action can be associated with a task, such as a manipulation action (e.g., involving a manipulative element, such as a manipulative element described herein) or a movement (e.g., involving a transport element, such as a transport element described herein) and / or a social behavior. While performing the action, the robotic device can continue to scan its surrounding environment at 1304. If the robotic device detects a change in its current state and / or the state of the environment (e.g., the location of an object in the environment), the robotic device can evaluate the change at 1310 to determine whether the robotic device should abort the execution of the action. For example, the robotic device may decide to abort the execution of the action in response to detecting physical engagement of one or more of its components with a human or other object in the environment (e.g., when a human operator contacts a manipulative element, transport element, etc.) or when it detects the presence of a human or other object in the immediate vicinity. Additionally or alternatively, the robotic device may decide to suspend execution of an action in response to receiving a signal from a user (e.g., a robot supervisor). In some embodiments, the robotic device may be pre-configured to suspend execution of an action at a particular point during execution of the action, for example, when the robotic device may require the user to demonstrate part of the action as defined in the interactive learning template. Further details regarding learning using interactive learning templates are described herein with reference to FIG. 15.

[0135]

[0157] If the robotic device determines to abort the execution of the action (1310: YES), the robotic device may determine whether user input is required at 1312. As described above, if the robotic device determines that user input is required (1312: YES), the robotic device may optionally prompt and / or receive user input at 1314-1315. The robotic device may then return to sensing or scanning the surrounding environment at 1304, arbitrating for a set of resources at 1306, and / or performing an action at 1308. Optionally, as shown in FIGS. 14 and 15 , when the robotic device determines that user input is required, the robotic device may switch to a learning mode at 1402 and then proceed to learning a skill at 1404 or learning environmental constraints at 1406. If the robotic device determines that user input is not required (1312: NO), the robotic device can return to sensing or scanning the surrounding environment at 1304, arbitrating for a set of resources at 1306, and / or performing an action at 1308. If the robotic device determines that the action does not need to be interrupted (1310: NO), the robotic device can continue performing the action at 1308.

[0136]

[0158] In some embodiments, the robotic device may be configured to determine when it may need to ask for user input, for example, to help it perform a particular task or behavior. Such assistance may be requested both during the performance of a particular action and / or while the robotic device is navigating an environment. The robotic device may be configured to use its learned model and information about the environment and objects in that environment to determine 1312 when it needs to ask for user input and how to ask for that user input. For example, the robotic device may use knowledge of the types of doors and different users in its environment to determine when and how to ask for help (e.g., asking different users to help open different types of doors (e.g., doors with keypads that require specific permissions or knowledge), and knowing to ask certain types of users who may be more likely to provide help than others (e.g., nurses vs. doctors)).

[0137]

[0159] In some embodiments, for example, the user input received by the robotic device at 1315 may include feedback from the user regarding whether the selection and / or execution of a skill was appropriate and / or successful. For example, a user may tag an action (e.g., a task or behavior) being performed by (or previously performed by) the robotic device as a good or bad example of the action. The robotic device may store this feedback (e.g., as success criteria associated with this and / or other actions) and use it to adjust the selection and / or execution of that or other actions in the future.

[0138]

[0160] In some embodiments, the robotic device is configured to engage in socially appropriate behaviors. The robotic device may be designed to operate in an environment with humans. When operating around humans, the robotic device may be configured to continuously plan 1306-1308 how its actions may be perceived by humans. For example, when the robotic device is moving, the robotic device may monitor its surroundings in search of humans in 1304 and engage in social interactions with humans it encounters in 1308. When the robotic device is not performing any task (e.g., stationary), the robotic device may continue to monitor its surroundings in search of humans in 1304 and determine 1306 whether the robotic device may need to perform one or more socially appropriate behaviors based on how humans may perceive the robotic device's presence. In such embodiments, the robotic device may be configured with an underlying framework (e.g., one or more arbitration algorithms) to plan and execute socially appropriate behaviors given a particular context or situation. The robotic device may be configured to manage multiple resources (e.g., manipulation elements, carrying elements, humanoid components such as heads or eyes, sound generators, etc.) and generate appropriate behaviors based on one or more of these resources.

[0139]

[0161] FIG. 18 provides an example of a component of a robotic device that arbitrates for an attention mechanism (e.g., an eye element). For example, the robotic device may have a camera or other element that can be perceived by a human as having a line of sight (e.g., an eye element). A human near the robotic device may recognize the direction the eye element is pointed as the robotic device is looking at. Thus, the robotic device may be configured to determine where the eye element is pointed when the robotic device is operating and / or near a human. The robotic device may continually arbitrate the use of the eye element so that the robotic device maintains socially appropriate behavior when operating around and / or in the presence of humans.

[0140]

[0162] As shown in FIG. 18 , different components of the robotic device may request a gaze target (e.g., pointing the eye element in a particular direction). A first component associated with active sensing (e.g., a camera, laser, or other sensor) may determine 1510 that an object is within the field of view of the eye element. The first component may further determine 1514 that the object is a human. In response to determining that the object is a human, the first component may send a request 1516 to the central resource manager 1508 (e.g., a control unit (e.g., one or more control units 202, 302, and / or 1702) and / or components of the control unit) to point the eye element toward the human's face. A second component associated with navigation actions (e.g., a camera, laser, or other sensor) may determine 1520 that a particular location (e.g., an intermediate or final navigation destination) is within the field of view of the eye element. In response to determining that the location is within line of sight, the second component can send a request 1522 to the central resource manager 1508 to direct an eye element to the location. A third component associated with the manipulation action (e.g., a sensor on the manipulation element, a camera, or other sensor) can determine 1530 that the robotic device is engaged with an object (e.g., the robotic device is carrying an object, a human has touched the robotic device). In response to determining that the robotic device is engaged with an object, the third component can send a request 1532 to the central resource manager 1508 to direct an eye element to the object.

[0141]

[0163] The central resource manager 1508 receives requests 1516, 1522, and 1532 from the components of the robotic device and can arbitrate 1540 to determine in which direction the eye element should point. When determining in which direction the eye element should point, the central resource manager 1508 can take into account specified rules or constraints (e.g., socially appropriate constraints). These specified rules can include, for example, not moving the eye element twice for a predefined period (e.g., about 5 seconds), having the eye element point in a direction for at least a predefined minimum period (e.g., about 5 seconds), moving the eye element after a predefined maximum period, which can differ for non-human objects and humans, etc. The specified rules can be encoded into an arbitration algorithm for managing gaze resources.

[0142]

[0164] In some embodiments, the central resource manager 1508 can arbitrate gaze resources based on social context and other information collected about the environment. For example, a robotic device may be configured to associate certain behaviors with particular locations (e.g., waiting for a person to pass through a crowded doorway or hallway before attempting to navigate through the doorway or hallway, operating quieter in quiet areas (e.g., refraining from using sound and / or speech functionality), etc.). The robotic device may be configured to capture social context information associated with such locations and add it to a representation of the environment (such as described herein with reference to FIG. 19) used for navigation. In some embodiments, a human operator can also provide social context information to the robotic device, and the robotic device can adapt this information over time as it learns more about the environment in which it operates.

[0143]

[0165] 15-17 show flow diagrams of a robotic device operating in learn mode. As described above, the robotic device may operate in run mode and switch to operate in learn mode, for example, at 1312, when the robotic device determines that it requires user input. Alternatively or additionally, the robotic device may be configured to operate in learn mode, for example, when the robotic device is initially deployed to a new environment (e.g., a new area, building, etc.). The robotic device may operate in learn mode until a user indicates to the robotic device that the robotic device is ready to switch to operate in run mode and / or until the robotic device determines that it is ready to switch to operate in run mode.

[0144]

[0166] When operating in the learning mode, the robotic device may learn a skill at 1404 and / or learn environmental constraints at 1406. The robotic device may be configured to learn a skill with or without an existing model of the skill (e.g., a generic model of the skill). When learning a skill without an existing model (e.g., learning a skill without relying on prior knowledge), the robotic device may generate a model of the skill after being taught performance of the skill, as described herein with reference to FIG. 10. When learning a skill using an existing model (e.g., a generic model of the skill), the robotic device may begin performing the skill using the existing model and request user input when the robotic device needs the user to demonstrate a portion of the skill. The existing model of the skill may be part of, or form part of, an interactive learning template (i.e., a template for teaching the robotic device to learn a skill with input from a user at specified points during performance of the skill).

[0145]

[0167] FIG. 16 illustrates a process for learning a skill using an interactive learning template. A robotic device may be deployed in a particular context (e.g., a building). The robotic device may select an existing model of the skill at 1410. The existing model of the skill may be a generic model of the skill that is not specific to the environment or context in which the robotic device is operating. At 1412, the robotic device may generate a plan to perform the skill using the skill model, following a process similar to that described herein with reference to FIG. 11. At 1414, the robotic device may begin performing the skill by performing one or more steps or portions of the skill that do not require user input and / or specialization. Upon reaching a portion of the skill (e.g., a portion of the skill that the robotic device cannot determine how to perform or that requires specialization to the context in which the robotic device is performing the skill) (1416: YES), the robotic device may prompt the user to provide a demonstration of that portion of the skill at 1418. In some embodiments, the interactive learning template may indicate to the robotic device when the robotic device should prompt the user for a demonstration. Alternatively or additionally, the robotic device may autonomously determine that user input is required for the robotic device to be able to perform a portion of a skill, for example, if the robotic device is unable to determine the state of objects in the environment and / or is unable to generate a plan to perform a particular portion of a skill given constraints imposed by the environment. In some embodiments, a user (e.g., a robot supervisor) may also monitor the robotic device's performance of a skill and signal the robotic device when the user wants the robotic device to demonstrate a portion of a skill. Upon receiving a signal from the user, the robotic device may decide to proceed to 1418.

[0146]

[0168] At 1420, a user can teach the robotic device an action. For example, the user can demonstrate a portion of a skill by moving one or more components (e.g., a manipulation element, a transport element, etc.) of the robotic device. While teaching the robotic device an action, the user can optionally indicate to the robotic device at 1422 when to capture information about the state of one or more of the components of the robotic device and / or the environment. For example, the robotic device can receive a signal from the user to capture sensory information, including information about the manipulation element, the transport element, and / or the environment, at key frames during the robotic device's movement. Alternatively, the robotic device can autonomously determine when to capture sensory information. For example, while the robotic device is being moved by the user, the robotic device can monitor changes in one or more of its components, and when those changes exceed a threshold or a directional change occurs in the trajectory of the component, the robotic device can autonomously select that time as a key frame and record information about the robotic device and / or the environment at that key frame. In response to receiving a signal from a user or determining to capture sensory information autonomously, the robotic device can capture sensory information using one or more of its sensors at 1424. During operation of the robotic device, the robotic device can also record sensory information at 1430, continuously or periodically, without receiving a signal from a user.

[0147]

[0169] Upon completion of the action or performance (1426: YES), the robotic device may optionally receive, at 1432, a selection of features relevant to learning the portion of the skill that was demonstrated. In some embodiments, the robotic device may autonomously perform the feature selection at 1432 and / or prompt the user to confirm the selection made by the robotic device. At 1436, the robotic device may save information related to the performance. If skill performance is not complete (1417: NO), the robotic device may continue performing the skill at 1414 and, if necessary, prompt the user for further performance of portions of the skill at 1418. The robotic device may continue to repeat the interactive learning process until action performance is complete (1417: YES), at which point the robotic device may generate, at 1438, a model of the skill having portions specific to the context in which the robotic device performed the skill. The robotic device may then learn another skill and / or environmental constraints. Alternatively, if the robotic device does not need to learn additional skills and / or environmental constraints, the robotic device may switch to execution mode at 1301 and begin sensing or scanning the environment, performing arbitration, and / or performing actions.

[0148]

[0170] Using the interactive learning templates, the robotic device is taught and / or provided with an initial set of models for skills. The initial set of models may be developed before the robotic device is deployed in the field in a particular environment (e.g., a hospital). For example, this initial set of models may be developed and made available to the robotic device in a factory setting or at a training site. Once deployed in the field, the robotic device can adapt or specialize the initial set of models to the environment, for example, through interactive learning sessions as described herein. Additionally, as new models are developed (e.g., off-site at a factory or training site), the new models may be made available to the robotic device, for example, via a network connection. Thus, the systems and methods described herein enable users and / or entities to continue developing new models of skills and providing them to robotic devices even after those robotic devices are deployed in the field.

[0149]

[0171] An example of an interactive learning session may involve adapting a generic model for moving an item into a room. A robotic device may be equipped with the generic model and deployed on-site at a hospital. Once deployed at the hospital, the robotic device may autonomously begin performing a skill to move an item into a patient's room, at 1412-1414. For example, the robotic device may autonomously navigate to the patient's room using a hospital map. Once the robotic device has navigated to the patient's room, the robotic device may determine that the robotic device needs a user to demonstrate where in the patient's room the item should be dropped off (1416: YES). The robotic device may prompt the user to move it to a specific drop-off location, at 1418. The user may control the operation of the robotic device, at 1420, using, for example, a joystick or other type of control device. As described above, the user may be on-site near the robotic device or may be located remotely. While the user moves the robotic device to the unloading location, the robotic device can capture sensory information about its current state and / or its surrounding environment at 1424 and / or 1430. Upon arriving at the unloading location, the robotic device can switch back to autonomous execution at 1414. For example, the robotic device can perform known arm movements (e.g., locating an item in a container loaded on the robotic device, grasping the item with a manipulative element, and positioning the manipulative element in a generic position to release the item). The robotic device can make a second determination (1416: YES) that the robotic device needs the user to demonstrate a portion of a skill (e.g., lowering an item onto a shelf). The robotic device can prompt the user for another demonstration at 1418, and the user can move the manipulative element to place the item in place on the shelf. The robotic device can again capture sensory information during the operation of the manipulative element at 1424 and / or 1430.The robotic device can regain control at 1414 and autonomously open the gripper of the manipulative element to place the item on the shelf, then retract the manipulative element back to its resting position. The robotic device can then determine (1417: YES) that execution is complete and generate (1438) a specialization model of the skill based on information captured by the robotic device during the two user demonstrations.

[0150]

[0172] Other examples of interactive learning sessions may include adapting navigation actions to navigate through certain passageways and / or doorways, for example, as described above.

[0151]

[0173] In some embodiments, a user (or robotic device) can modify a model of a skill after the skill is demonstrated and / or after the model of the skill is generated. For example, a user can iterate through keyframes captured during a skill performance and decide whether to retain, modify, and / or delete these keyframes. By modifying and / or deleting one or more keyframes, the user can modify the model of the skill generated based on these keyframes. An example of using iterative and adaptive versions of keyframe-based performances is described in a paper titled "Trajectories and keyframes for kinesthetic teaching: a human-robot interaction perspective" by Akgun et al., published in Proceedings of the 7th Annual ACM / IEEE International Conference on Human-Robot Interaction (2012), pp. 391-98, which is incorporated herein by reference. As another example, an interactive graphical user interface ("GUI") can be used to display keyframes captured during a skill demonstration to a user so that the user can indicate how much variance is acceptable associated with the position, orientation, etc. of the robotic device's components during the planned execution. An example of using a GUI with keyframes is described in a paper titled "An Evaluation of GUI and Kinesthetic Teaching Methods for Constrained-Keyframe Skills," authored by Kurenkov et al., published at the IEEE / RSJ International Conference on Intelligent Robots and Systems (2015), available at http: / / sim.ece.utexas.edu / static / papers / kurenkov_iros2015.pdf, which is incorporated herein by reference.These examples allow for modification of the model of a skill after the skill has been learned. The systems and methods described herein further provide learning templates that enable a robotic device to plan and execute certain portions of a skill while leaving other portions of the skill to user performance during ongoing execution of the skill. The systems and methods described herein also provide a robotic device that can determine, for example, autonomously or based on information provided to the robotic device prior to the learning process, when to collect data and / or when to request performance of portions of a skill.

[0152]

[0174] FIG. 17 illustrates a process for learning environmental constraints. As described above, a robotic device can be configured to learn environmental constraints, which can be applicable to a set of skills to be learned and / or performed in a situation that includes the environmental constraints. At 1450, the robotic device can optionally prompt the user to select a type of constraint and / or an object associated with the constraint. The type of constraint may be, for example, a barrier (e.g., a wall, a surface, a boundary), a size and / or dimension, a restricted area, an object location, a time constraint, etc. At 1452, the robotic device can receive the selection of the type of constraint and / or object. Optionally, at 1454, a user (or another robotic device) can direct the robotic device in an action to demonstrate the environmental constraint. For example, a user can demonstrate the size of an equipment container to the robotic device by moving a manipulative element of the robotic device along one or more edges of the container. As another example, a user can demonstrate a shelf location by moving a manipulative element of the robotic device along the surface of the shelf. During operation of the robotic device, the robotic device may capture information related to the constraints at 1456 using one or more sensors (e.g., cameras, lasers, haptics, etc.). At 1458, the robotic device may store the captured information related to the environmental constraints for use with models of skills to be performed in the situation that include the environmental constraints. In some embodiments, the robotic device may include information related to the environmental constraints in each model of a skill to be performed in the situation. Alternatively, the robotic device may be configured to reference information related to the environmental constraints when planning and / or performing a skill in the situation. In some embodiments, the environmental constraints may be added to a representation of the environment (e.g., representation 1600).

[0153]

[0175] In some embodiments, the robotic device can learn the environmental constraints without requiring user demonstration or robotic device movement. For example, after receiving a constraint type and / or object selection at 1452, the robotic device may be configured to scan the environment for relevant information related to the environmental constraints at 1456. In some embodiments, the robotic device can present the information it has captured and / or determined to be relevant to the environmental constraints to the user so that the user can review and / or modify which information is relevant to the environmental constraints. The robotic device can then store the relevant information at 1458.

[0154]

[0176] In some embodiments, a robotic device can use transport elements (e.g., wheels or tracks) to navigate an environment and learn one or more environmental constraints. For example, a robotic device can navigate a hallway or corridor, e.g., while receiving a demonstration and / or performing a skill, and learn that the hallway is crowded during a particular time frame. In some embodiments, a robotic device can learn various environmental constraints and / or behaviors and associate them with different conditions (e.g., time, location, etc.). For example, a robotic device can recognize that a hallway is crowded based on information collected by one or more of its sensors, user input, and / or information derived from sensed information (e.g., during the demonstration and / or performance of a skill). As another example, a robotic device can determine whether the robotic device should respond "hello" or "good night" when a user enters a particular room at various times of day.

[0155]

[0177] In some embodiments, the robotic device can learn environmental constraints autonomously or interactively. For example, the robotic device can acquire an initial set of environmental constraints at 1458 by moving through an environment (e.g., navigating a hallway or corridor). A user (e.g., a robot supervisor or local user) can review the constraints and decide at 1459 whether to adjust the constraints by directly modifying them and / or by demonstration. For example, some constraints may be provided through interaction with the robotic device (e.g., demonstration), while other constraints (e.g., object locations (e.g., the distance a shelf extends from a wall)) may be provided by input (e.g., to a user interface (e.g., user interface 240)).

[0156]

[0178] 21 illustrates an example of a map 2120 (e.g., a map of a building (e.g., a hospital, etc.)) containing information about entering and exiting the building. The map 2120 may be stored and / or maintained by one or more robotic devices, such as any of the robotic devices described herein. The map 2120 may be similar to the map 1600 shown in FIG. 19. For example, the map 2120 may include one or more layers 2122, such as a navigational layer, a static layer, a dynamic layer, and a social layer.

[0157]

[0179] The map 2120 can provide information that the robotic device can use while operating in the learn mode 2102 or the run mode 2112. For example, a robotic device operating in the learn mode 2102 can obtain information about one or more environmental constraints, one or more objects, and / or one or more social contexts (e.g., similar to the state information 331, 1732, the object information 340, 1740, the one or more environmental constraints 1754, the social context 1632, etc.) by accessing the map 2120. Such information can enable the robotic device to determine the location of one or more objects, identify one or more characteristics of one or more objects, analyze one or more environmental constraints, select one or more skill models to use, prompt one or more users for one or more inputs, etc., as described herein. Additionally or alternatively, a robotic device operating in execution mode 2112 can access map 2120 to obtain information about one or more environmental constraints, one or more objects, and / or one or more social contexts and use that information to assess the environment, arbitrate between different behaviors or skills, determine which skills to perform, and / or adapt a behavior or skill to suit a particular environment.

[0158]

[0180] A robotic device operating in learn mode 2102 can also provide information that adds to and / or modifies information in map 2120. For example, the robotic device can incorporate sensed information 2104 (e.g., information collected by one or more sensors of the robotic device) and / or derived information 2106 (e.g., information derived by the robotic device based on, for example, an analysis of sensed information 2104) into map 2120. The robotic device can incorporate such information by adding information to map 2120 and / or adapting existing information in map 2120. Additionally, a robotic device operating in execute mode 2112 can provide information (e.g., sensed information 2114, derived information 2116) that adds to and / or modifies information in map 2120.

[0159]

[0181] As an example, a robotic device undergoing a demonstration of how to navigate through doorways (e.g., such as the demonstrations shown in FIGS. 10 and 16 ) can collect information during the demonstration that is provided to one or more layers 2122 of map 2120. The doorways may be specific doorways that exist in various locations throughout a building, as shown in map 2122, for example. The characteristics and / or properties of the doorways can be derived by the robotic device based on information recorded and / or sensed by various sensors on the robotic device. For example, the robotic device can sense the size of the doorway and determine that it is a narrow doorway. The robotic device can add this information about the doorway to map 2120, such as, for example, as raw or processed sensor information, as derived rules, using one or more semantic labels, etc. The robotic device can also use this information to adapt the context layer of the map (e.g., the social or behavioral layer of the map (e.g., the social layer 1630 shown in FIG. 19 )). For example, based on the doorway being a narrow doorway, the robotic device may determine to engage in particular behavior(s) and / or action(s), including, for example, passing through the doorway when opened by the user (or one or more other robotic devices) instead of following and / or parallelly passing through the doorway with the user (or one or more other robotic devices), helping the robotic device navigate through the doorway (e.g., by holding the doorway open and / or directing the robotic device to perform various one or more actions), seeking out a particular user familiar with the doorway, etc.

[0160]

[0182] In some embodiments, the map 2120 may be maintained centrally for one or more robotic devices described herein. For example, the map 2120 may be stored on a remote computing device (e.g., a server) and centrally hosted for a group of robotic devices operating together, for example, in a hospital. The map 2120 may be updated as information (e.g., sensed information 2104, 2114 and derived information 2106, 2116) is received from the group of robotic devices as they operate in the learning mode 2102 and / or the performing mode 2112, and provided to each robotic device as needed to perform actions, behaviors, etc. As individual robotic devices in the group learn new information, this information may be shared with other robotic devices as they encounter similar environments and / or perform similar skills (e.g., actions, behaviors), so that one or more layers 2122 of the map 2120 may be adapted. In some embodiments, a local copy of the map 2120 may be stored on each robotic device, and this local copy may be updated or synchronized (e.g., at predetermined intervals, during rest or downtime) with a centrally maintained copy of the map 2120. By periodically updating the map (e.g., with newly collected information such as sensed information 2104, 2114 and derived information 2106, 2116) and / or sharing the map among multiple robotic devices, each robotic device may have access to a more comprehensive map that provides the robotic device with more accurate information for interacting with and / or performing skills within its surrounding environment.

[0161]

[0183] In some embodiments, the map 2120 may be a mixed-initiative map in which information (e.g., one or more environmental constraints, one or more rules, one or more skills, etc.) may be learned (e.g., during learning 2102 and / or performing 2112) and / or provided by a user (e.g., with user input 2130). For example, the robotic device may construct the map 2120 by incorporating information given directly by a user (e.g., a robot supervisor or local user), information learned through demonstration and / or performance, or information provided interactively by a combination of demonstration and / or performance and user input (e.g., requested by the robotic device in real time or retrospectively in connection with an interaction). For example, a user may indicate to the robotic device that the robotic device should proceed more slowly through a certain area on the map 2120 (e.g., a narrow or crowded passage). In some embodiments, the robotic device presents the map to the user, and the user may draw an area on the map to indicate that the robot should proceed more slowly in that area. The robotic device may attempt to navigate that area on the map 2120 multiple times and learn that the area should be avoided at certain times of day (e.g., due to overcrowding). Alternatively or additionally, the robotic device may encounter a new situation (e.g., a partitioned passage) that the robotic device is not accustomed to dealing with autonomously. In such cases, the robotic device may communicate with a user (e.g., a robot supervisor and / or a local user) to obtain more information about the area and / or determine how to navigate around the area. When requesting user input, the robotic device may broadly inquire about the situation and / or provide a list of examples that the user labels (e.g., to enable the robotic device to autonomously derive appropriate behavior).In some embodiments, the robotic device may detect and / or derive information about an area without user input and propose one or more rules associated with the area to the user for confirmation. In some embodiments, the user, upon viewing the proposed rules and / or other information from the robotic device, may modify the rules and / or other information before accepting them. Figure 24, described below, provides a more detailed description of such a process.

[0162]

[0184] 22-25 are flow diagrams illustrating different means by which any of the robotic devices described herein, including, for example, robotic devices 102, 110, 200, 400, etc., may learn one or more skills, one or more environmental constraints, etc. In some embodiments, a robotic device operating according to a method described herein may be constantly learning (e.g., operating in a learning mode). For example, the robotic device may continuously collect, store, analyze, and / or update information as it navigates through an environment and / or engages in particular behaviors or actions, adapting and / or adding to its library of learned information (e.g., skills, behaviors, environmental constraints, maps, etc.). In other embodiments, the robotic device may switch between operating in a learning mode and an execution mode. While operating in a learning mode 2202, the robotic devices described herein may learn by demonstration 2204, by execution 2206, by exploration 2208, by derivation / user input 2210, and / or any combination thereof.

[0163]

[0185] In some embodiments, the robotic device can learn by demonstration 2204, for example, as shown and described above with reference to FIGS. 8-10. For example, the robotic device can learn new skills or environmental constraints by demonstrating a skill using the LfD teaching process. The robotic device can analyze and extract information from past skill demonstrations in a human environment. As an example, a user can demonstrate to the robotic device how to move a control element (e.g., an arm) of the robotic device in an equipment room with a human and the robotic device. From one or more demonstrations, the robotic device can define one or more environmental constraints for the movements of one or more existing demonstrations that constrain the movements of the robotic device when planning and / or executing future skills. The robotic device can generate movements and build a graph (in which environmental constraints are encoded) using existing demonstrations (e.g., a sequence of one or more keyframes), each of which provides a path through an n-dimensional space of nodes and movements. Subsequently, when the robotic device is presented with a new environment and needs to adapt its skills to the new environment, the robotic device can use the constructed graph to efficiently sample from its existing set of demonstrated moves to plan moves for the new environment. By incorporating such environmental constraint learning into the initial skill demonstration, the robotic device can quickly adapt to the new environment without requiring new environmental constraints to be defined. In the new environment, the robotic device can scan the environment to obtain additional information about the environment and / or one or more objects in the environment (e.g., people, furniture, doors, etc.), and then use the existing skill model to plan a motion trajectory through the new environment without significantly deviating from its existing set of demonstrated moves.

[0164]

[0186] In some embodiments, the robotic device can learn through demonstration 2204 and user input 2210. For example, the robotic device can engage in interactive learning with a user, for example, using an interactive learning template. As described above, with interactive learning, an initial set of robotic device and / or skill models is provided. The initial set of models may be developed by other robotic devices in a factory setting and / or by the robotic device itself in one or more settings. While operating in a particular environment, the robotic device can adapt or specialize this initial set of skills through interactive learning sessions in which the robotic device can autonomously perform certain parts of the skills while leaving others to user demonstration. Further details of such an interactive learning process are described with reference to FIG. 16 .

[0165]

[0187] In some embodiments, the robotic device can learn skills as it executes 2206. For example, the robotic device can collect, store, analyze, and / or update information as the robotic device executes behaviors, actions, etc. (e.g., at 774 in FIG. 11 and 1304, 1305, and 1307 in FIG. 13). In one embodiment, the robotic device can execute a skill that requires the robotic device to navigate a hallway several times a day. The robotic device can learn that a hallway has higher traffic during certain times of day. The robotic device can then adapt its behavior, for example, to avoid passing through that hallway during those high traffic times (e.g., by taking an alternate route and / or by waiting to pass through that hallway at a time with less traffic).

[0166]

[0188] In some embodiments, the robotic device can learn by exploration 2208. FIG. 23 is a flow diagram illustrating an exemplary method for learning by exploration. The robotic device can scan an environment and collect information about the environment and / or objects in the environment, at 2302. Optionally, based on the information collected by the robotic device, the robotic device can determine to explore the environment by executing a skill. To execute the skill, the robotic device can select an existing skill model, at 2304, plan the execution of the skill using the skill model, at 2306, and execute the skill, at 2308. For example, the robotic device can scan the environment and identify a doorway. The robotic device can determine the execution of a skill that includes steps of opening the door and moving through the doorway. At 2304, the robotic device can select an existing skill model previously provided and / or learned to navigate through the doorway. At 2306, the robotic device can generate a plan to open the door and move through the doorway using the existing skill (e.g., according to a process similar to the process described with reference to FIG. 11 ). The robotic device can then execute the skill according to the generated plan at 2308. While executing the skill, the robotic device can collect information about the environment and / or the execution of the skill and compare that information with information collected during previous interactions with the doorway at 2310. Based on this comparison, the robotic device can evaluate whether its interaction with the doorway was successful or unsuccessful (e.g., based on success criteria such as those described above). The robotic device can optionally generate a skill model specific to the doorway at 2312 based on its execution of the skill.

[0167]

[0189] In some embodiments, a robotic device can begin performing a skill and determine that user input is required to further complete the skill. For example, as described above with reference to FIG. 16, when a robotic device begins performing a skill and reaches a portion of the skill that the robotic device cannot determine how to perform, it can solicit user input to proceed with the performance of the skill.

[0168]

[0190] In some embodiments, the robotic device can execute a skill and determine that execution failed. For example, the robotic device may plan and / or execute a movement through a doorway, but detect that it was unable to move through the doorway (e.g., due to a failed attempt to open the door or navigate through a narrow doorway). The robotic device may decide to scan the environment again (2314: YES) and re-plan and re-execute the skill at 2306-2308. In some embodiments, the robotic device may select a different skill at 2304 (e.g., a skill more specific to the narrow doorway) and then re-plan and re-execute based on the different skill. The robotic device can continue learning by exploration by resensing, re-planning, and re-executing the skill until the robotic device determines that it has met a certain objective (e.g., successfully executed the skill a predetermined number of times or in a predetermined number of ways, learned sufficient information about the environment, etc.). In 2316, the robotic device may store information it has collected about the environment and / or objects in the environment, and optionally, a model it has adapted or generated based on the performance of the skill in the environment.

[0169]

[0191] In some embodiments, the robotic device can engage in learning by exploration 2208 using user input 2210. For example, the robotic device can perform one or more skills in a particular environment to explore and / or learn more information about the environment. While the robotic device is exploring, a user (e.g., a remote supervisor or a local user) can provide input to the robotic device based on information the user perceives and / or learns about the environment (e.g., directly by being in the environment and / or through the robotic device). The user can then provide one or more inputs to the robotic device that further guide its exploration of the environment. For example, a robotic device that encounters a doorway can plan one or more actions to navigate through the doorway. Upon executing those actions, the robotic device may fail to navigate through the doorway. The user, upon observing a failed attempt by the robotic device, can provide one or more inputs to the robotic device to guide the robotic device in a second attempt to navigate through the doorway. For example, a user may determine that a doorway is a narrow doorway and indicate to the robotic device that the robotic device should first ask a nearby user or robotic device to open the door before navigating through the doorway. Upon receiving the user input, the robotic device may ask the user to open the door before navigating through the doorway.

[0170]

[0192] In some embodiments, the robotic device can learn through user input 2210 by receiving information about the environment and / or objects in the environment from the user, or by receiving an initial set of models, rules, etc. from the user. In some embodiments, the robotic device may initially learn a skill by demonstration (e.g., as described above with reference to FIG. 10 ) and learn to adapt the skill based on user input. For example, the robotic device can learn the skill of moving equipment from a first location to a second location and then dropping off the equipment at the second location. During user demonstration of the skill, the user can indicate to the robotic device that dropping off the equipment at the second location requires specific information unique to the environment or situation. The user can later provide input to the robotic device specifying specific information for the robotic device to observe while dropping off the equipment for different pieces of equipment and / or different locations. For example, the user can provide information about a tag or marker that identifies where the robotic device should drop off the equipment.

[0171]

[0193] In some embodiments, the robotic device can learn behaviors or rules (e.g., one or more arbitration algorithms 1858, as described above) based on the stored information and / or user input 2210. FIG. 24 is a flowchart of an exemplary method for robot learning of behaviors and / or rules. The robotic device can analyze the stored information (e.g., one or more maps, one or more models, one or more user inputs) at 2402. The robotic device can derive the behaviors and / or rules based on the analysis of the stored information at 2404. For example, a user (e.g., a robot supervisor or local user) can define a rule for the robotic device to be more interactive when more humans are near the robotic device. The robotic device may determine, based on the analysis of the stored information, that more humans are near it during certain times of the day (e.g., around lunchtime). Therefore, the robotic device may further derive that it should be more interactive during that time of day. In some embodiments, the robotic device can automatically implement this behavior as a new rule. Alternatively, the robotic device may suggest a new rule to the user to be more interactive at that time of day, at 2406. The robotic device may receive user input in response to the rule suggestion, at 2408. For example, the user may view the rule on a local and / or remote display of the robotic device and choose to accept or reject it. Based on the user input, the robotic device may modify its behavior and / or the rule, at 2410. For example, if the user indicates that the rule is acceptable, the robotic device may save the new rule and enforce it in the future.Alternatively, if the user indicates that the rule is acceptable but with further tweaking or modification (e.g., be more interactive only in the cafeteria, around lunchtime), the robotic device can adapt the rule and save the adapted rule for future implementation.

[0172]

[0194] In some embodiments, a robotic device can learn skills and / or environmental constraints by adapting existing skills and / or environmental constraints based on user input. Figure 25 is a flow diagram of an example method for adapting skills and / or environmental constraints based on user input. The robotic device can receive user input at 2502. The robotic device can associate the user input with one or more skills and / or environmental constraints at 2504. Such association may be based, for example, on further input by the user (e.g., the user specifying the skill and / or environmental constraint to which the input pertains) and / or derivation by the robotic device (e.g., the robotic device determining that such input is relevant to a particular skill or environmental constraint based on the input and / or one or more rules, attributes, etc. associated with the input and / or the particular skill or environmental constraint). For example, the user may provide the robotic device with new information to identify an object, such as a QR code, barcode, tag, etc. The user may specify that the new information pertains to a particular object, and the robotic device may associate the new information with the object. The robotic device may further associate the new information about the object with one or more skills (e.g., a grasping skill or a movement skill) involved in interacting with the object. The robotic device may optionally modify and / or create a skill or environmental constraint based on the input at 2506-2508.

[0173]

[0195] In some embodiments, the robotic device can allow a user to retroactively label information associated with skills and / or environmental constraints. For example, the robot can present skills or environmental constraints to a user, for example, using a user interface (e.g., user interface 240) and / or a remote interface. The user can choose to add, modify, and / or remove labels associated with skills or environmental constraints, and such input can be received at the robotic device at 2502. The robotic device can then associate the input with the skills or environmental constraints at 2504 and can modify and / or create new skills or environmental constraints based on the input at 2506-2508. [Example]

[0174] Example

[0196] It will be appreciated that the present disclosure may include any one up to all of the following examples.

[0175]

[0197] Example 1: An apparatus having a memory, a processor, a control element, and a set of sensors, the processor operatively coupled to the memory, the control element, and the set of sensors and configured to: obtain a representation of an environment using a subset of sensors from the set of sensors; identify a plurality of markers within the representation of the environment, where each marker from the plurality of markers is associated with a physical object from a plurality of physical objects located in the environment; present information indicative of a position of each marker from the plurality of markers within the representation of the environment; receive a selection of a set of markers from the plurality of markers associated with a set of physical objects from the plurality of physical objects; obtain, for each position from a plurality of positions associated with movement of the control element within the environment, sensory information associated with the control element, where the movement of the control element is associated with a physical interaction between the control element and the set of physical objects; and generate, based on the sensory information, a model configured to specify a behavior of the control element to perform the physical interaction between the control element and the set of physical objects.

[0176]

[0198] Example 2: The apparatus of example 1, wherein the set of physical objects includes a person.

[0177]

[0199] Example 3: The apparatus of example 1 or 2, wherein the manipulating element comprises an end effector configured to engage a subset of the physical objects from the set of physical objects.

[0178]

[0200] Example 4: The device of any one of Examples 1 to 3, wherein the subset of sensors is a subset of a first sensor, and the processor is configured to acquire sensory information using a second subset of sensors from the set of sensors, the second subset of sensors being different from the first subset of sensors.

[0179]

[0201] Example 5: An apparatus described in any one of Examples 1 to 4, wherein the operating element includes a plurality of movable components connected by a plurality of joints, and the set of sensors includes at least one of a sensor configured to measure a force acting on a joint from the plurality of joints, or a sensor configured to detect engagement of a movable component from the plurality of movable components with a physical object from the set of physical objects.

[0180]

[0202] Example 6: The device of example 5, wherein the set of sensors further comprises a sensor configured to measure a position of a joint or moving component relative to a portion of the device.

[0181]

[0203] Example 7: The apparatus of example 5, wherein the set of sensors further comprises at least one of a light sensor, a temperature sensor, an audio capture device, and a camera.

[0182]

[0204] Example 8: The apparatus of Example 1, wherein the manipulating element includes (i) a plurality of joints and (ii) an end effector configured to move a physical object from the set of physical objects, and the set of sensors includes a sensor configured to measure a force applied to the end effector or at least one of a joint from the plurality of joints coupled to the end effector when the end effector moves the physical object.

[0183]

[0205] Example 9: The apparatus of any one of Examples 1 to 8, wherein the sensory information includes sensor data associated with a set of features, and the processor is further configured to receive a selection of a first subset of features from the set of features, and the processor is configured to generate the model based on the sensor data associated with the first subset of features and not based on sensor data associated with a second subset of features from the set of features that is not included in the first set of features.

[0184]

[0206] Example 10: The apparatus of Example 9, wherein the processor is further configured to prompt the user to select at least one feature from the set of features in response to a selection made by the user, such that the processor receives a selection of the subset of the first features.

[0185]

[0207] Example 11: The apparatus of any one of Examples 1 to 10, wherein the plurality of markers are fiducial markers and the representation of the environment is a visual representation of the environment.

[0186]

[0208] Example 12: The apparatus of any one of Examples 1 to 11, wherein the processor is further configured to store a model and information associated with the model in the memory, and the information associated with the model includes (i) a set of markers and (ii) sensory information.

[0187]

[0209] Example 13: The device described in Example 1, wherein the operating element includes a plurality of joints, and the sensory information includes, for each position from a plurality of positions associated with movement of the operating element, information indicating a current state of each joint from the plurality of joints.

[0188]

[0210] Example 14: The device of any one of Examples 1 to 13, wherein the processor is further configured to prompt the user to select at least one marker from the plurality of markers after the presentation, such that in response to a selection made by the user, the processor receives a selection of the set of markers.

[0189]

[0211] Example 15: An apparatus described in any one of Examples 1 to 14, wherein the processor is configured to obtain a representation of the environment by sensing or scanning an area of ​​interest within the environment using a set of sensors.

[0190]

[0212] Example 16: A method comprising: acquiring a representation of an environment using a set of sensors; identifying a plurality of markers within the representation of the environment, each marker from the plurality of markers associated with a physical object from a plurality of physical objects located in the environment; presenting information indicative of a position of each marker from the plurality of markers within the representation of the environment; and after the presenting, receiving a selection of a set of markers from the plurality of markers associated with a set of physical objects from the plurality of physical objects; acquiring, for each position from a plurality of positions associated with movement of a manipulative element within the environment, sensory information associated with the manipulative element, the movement of the manipulative element being associated with a physical interaction between the manipulative element and the set of physical objects; and generating, based on the sensory information, a model configured to specify a behavior of the manipulative element to perform the physical interaction between the manipulative element and the set of physical objects.

[0191]

[0213] Example 17: The method of Example 16, wherein the sensory information includes sensor data associated with a set of features, the method further comprising receiving a selection of a first subset of features from the set of features, and wherein generating includes generating the model based on sensor data associated with the first subset of features and not based on sensor data associated with a second subset of features from the set of features that is not included in the first set of features.

[0192]

[0214] Example 18: The method of Example 17, further comprising prompting the user to select at least one feature from the set of features in response to a selection made by the user, such that a selection of the first subset of features is received.

[0193]

[0215] Example 19: The method of any one of Examples 16-18, wherein the plurality of markers are fiducial markers and the representation of the environment is a visual representation of the environment.

[0194]

[0216] Example 20: A method as described in any one of Examples 16 to 19, wherein the operating element includes a plurality of joints, and the sensory information includes, for each position from a plurality of positions associated with the movement of the operating element, information indicating the current state of each joint from the plurality of joints.

[0195]

[0217] Example 21: The method described in any one of Examples 16 to 20, further comprising prompting the user to select at least one marker from the plurality of markers after the presentation, such that a selection of a set of markers is received in response to a selection made by the user.

[0196]

[0218] Example 22: A method described in any one of Examples 16 to 21, wherein obtaining a representation of the environment includes sensing or scanning an area of ​​interest within the environment using a set of sensors.

[0197]

[0219] Example 23: A non-transitory processor-readable medium storing code representing instructions executed by a processor, the code causing the processor to: obtain a representation of an environment using a set of sensors; identify a plurality of markers within the representation of the environment, where each marker from the plurality of markers is associated with a physical object from a plurality of physical objects located in the environment; present information indicative of a position of each marker from the plurality of markers within the representation of the environment; identify a model associated with performing a physical interaction between a manipulative element and the set of physical objects in response to receiving a selection of a set of markers from the plurality of markers associated with the set of physical objects from the plurality of physical objects, where the manipulative element includes a plurality of joints and an end effector; and generate a trajectory for the manipulative element defining movement of the plurality of joints and the end effector associated with performing the physical interaction using the model.

[0198]

[0220] Example 24: A non-transitory processor-readable medium as described in Example 23, wherein the code that causes the processor to identify a model associated with performing a physical interaction includes code that causes the processor to prompt a user to identify the model.

[0199]

[0221] Example 25: A non-transitory processor-readable medium described in Example 23 or 24, wherein the code that causes the processor to identify a model associated with performing a physical interaction includes code that causes the processor to identify the model based on a selection of a set of markers.

[0200]

[0222] Example 26: A non-transitory processor-readable medium described in any one of Examples 23 to 25, further having code that causes a processor to display to a user a trajectory of a manipulative element within a representation of the environment, and after the display, receive input from the user, and in response to the input indicating approval of the trajectory of the manipulative element, perform movements of a plurality of joints and an end effector to perform a physical interaction.

[0201]

[0223] Example 27: A non-transitory processor-readable medium described in any one of Examples 23 to 26, wherein the trajectory is a first trajectory, and the non-transitory processor-readable medium further has code that causes a processor to generate a modified model by modifying a set of parameters associated with the model in response to an input that does not indicate approval of the trajectory of the operating element, and generate a second trajectory of the operating element using the modified model.

[0202]

[0224] Example 28: A non-transitory processor-readable medium described in any one of Examples 23 to 26, wherein the trajectory is a first trajectory, the model is a first model, and the non-transitory processor-readable medium further has code that causes a processor to, in response to an input that does not indicate approval of the trajectory of the operating element, generate a second model based on sensor data associated with a set of features different from the set of features used to generate the first model, and generate a second trajectory of the operating element using the second model.

[0203]

[0225] Example 29: A non-transitory processor-readable medium described in any one of Examples 23 to 28, further having code that causes a processor to perform the following: perform movements of multiple joints and end effectors to perform a physical interaction; acquire sensory information related to the performance of the physical interaction; and determine, based on the sensory information, whether the performance of the physical interaction meets pre-defined and / or learned success criteria.

[0204]

[0226] Example 30: A non-transitory processor-readable medium as described in Example 29, further having code that causes a processor to: generate a signal indicating that the physical interaction was successful in response to a determination that the execution of the physical interaction meets pre-defined and / or learned success criteria; generate a modified model by modifying the model based on sensory information in response to a determination that the execution of the physical interaction does not meet the pre-defined and / or learned success criteria; and generate a second trajectory of the operating element using the modified model.

[0205]

[0227] Example 31: A non-transitory processor-readable medium described in any one of Examples 23 to 30, wherein the model is associated with (i) a set of stored markers, (ii) sensory information indicating at least one of a position or orientation of the manipulative element at points along a stored trajectory of the manipulative element associated with the set of stored markers, and (iii) sensory information indicating a configuration of a plurality of joints at the points along the stored trajectory, and the code for causing the processor to generate a trajectory for the manipulative element includes code for causing the processor to: calculate a transformation function between the set of markers and the set of stored markers; for each point, transform at least one of the position or orientation of the manipulative element using the transformation function; for each point, determine a planned configuration of the plurality of joints based on the configuration of the plurality of joints at the point along the stored trajectory; and for each point, determine a portion of the trajectory between the point and successive points based on the planned configuration of the plurality of joints relative to the point.

[0206]

[0228] Example 32: A non-transitory processor-readable medium described in any one of Examples 23 to 30, wherein the model is associated with (i) a set of stored markers and (ii) sensory information indicating at least one of a position or orientation of the manipulative element at points along a stored trajectory of the manipulative element associated with the set of stored markers, and the code for causing the processor to generate a trajectory of the manipulative element includes code for causing the processor to: calculate a transformation function between the set of markers and the set of stored markers; for each point, transform at least one of the position or orientation of the manipulative element using the transformation function; for each point, determine a planned configuration of a plurality of joints; and for each point, determine a portion of the trajectory between the point and successive points based on the planned configuration of the plurality of joints relative to the point.

[0207]

[0229] Example 33: A non-transitory processor-readable medium as described in any one of claims 23 to 32, further having code that causes a processor to: determine whether to change a location of the manipulative element based on a distance between a first location of the manipulative element and a location of a physical object from a set of physical objects; and, in response to a determination to change the location of the manipulative element, move the manipulative element from the first location to a second location that is closer to the location of the physical object; and the code that causes the processor to generate a trajectory of the manipulative element includes code that causes the processor to generate a trajectory based on the location of the physical object relative to the second location of the manipulative element after the movement.

[0208]

[0230] While various inventive embodiments have been described and illustrated herein, those skilled in the art will readily envision numerous other means and / or structures for performing the functions described herein and / or obtaining one or more of the results and / or advantages described herein, and each such variation and / or modification is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and that the actual parameters, dimensions, materials, and / or configurations will depend on the specific application or applications for which the teachings of the present invention are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. Accordingly, it is to be understood that the above-described embodiments are presented by way of example only, and that, within the scope of the appended claims and their equivalents, inventive embodiments may be practiced otherwise than as specifically described and claimed. The inventive embodiments of the present disclosure are directed to each individual feature, system, product, material, kit, and / or method described herein. Furthermore, any combination of two or more such features, systems, products, materials, kits, and / or methods is within the inventive scope of the present disclosure, if those features, systems, products, materials, kits, and / or methods are not mutually inconsistent.

[0209]

[0231] Also, various inventive concepts may be embodied as one or more methods, examples of which have been provided. Acts performed as part of a method may be ordered in any suitable manner. Thus, embodiments may be constructed in which acts are performed in an order different from that illustrated, and these embodiments may include performing some acts simultaneously even though in the exemplary embodiment they are shown as sequential acts.

[0210]

[0232] Some embodiments and / or methods described herein may be performed by other software (executed on hardware), hardware, or a combination thereof. Hardware modules may include, for example, general-purpose processors, field-programmable gate arrays (FPGAs), and / or application-specific integrated circuits (ASICs). Software modules (executed on hardware) may be expressed in various software languages ​​(e.g., computer code), including C, C++, Java™, Ruby, Visual Basic™, and / or other object-oriented, procedural, or other programming languages ​​and development tools. Examples of computer code include, but are not limited to, microcode or microinstructions, machine instructions such as those generated by a compiler, code used to create web services, and files containing high-level instructions executed by a computer using an interpreter. For example, embodiments may be implemented using imperative programming languages ​​(e.g., C, Fortran, etc.), functional programming languages ​​(e.g., Haskell, Erlang, etc.), logic programming languages ​​(e.g., Prolog), object-oriented programming languages ​​(e.g., Java, C++, etc.), or other suitable programming languages ​​and / or development tools. Additional examples of computer code include, but are not limited to, control signals, encryption code, and compression code.

[0211]

[0233] Although terms such as "first," "second," and "third" may be used herein to describe various elements, it will be understood that these elements are not limited by these terms. These terms are used only to distinguish one element from another. Thus, a first element described herein could be termed a second element without departing from the teachings of the present disclosure.

[0212]

[0234] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "having" ("comprises" and / or "comprising") or "including" ("includes" and / or "including") as used herein specify the presence of stated features, regions, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, regions, integers, steps, operations, elements, components, and / or groups thereof.

Claims

1. An operating element including a plurality of joints; a conveying element configured to move along a surface; A set of sensors and Memory and a processor operatively coupled to the memory, the operating element, the transport element, and the set of sensors, acquiring, using the set of sensors, information about an environment, which is an unstructured environment, and a current state of the robotic device, wherein the current state of the robotic device is a current state of the transport element and the manipulating element, and the information acquired using the set of sensors includes information about positions of physical objects in the environment and information about configurations of the plurality of joints of the manipulating element; determining, based on the information of the environment, that user input is required; and In response to determining that the user input is required, requesting the user input from a user; selecting, using an arbitration algorithm, a skill to execute from a plurality of skills based on the information of the environment, the current state of the robotic device, and the user input, wherein the execution of the skill includes a physical interaction between the operational element and the physical object in the environment; generating a plan for executing at least a portion of the skill based on at least one of the information of the environment, the current state of the robotic device, and the user input using a model for executing at least a portion of the skill, the plan including one or more movements of at least one of the manipulation element or the transport element; moving the at least one of the operating element or the conveying element based on the plan; a processor configured to: A robotic device comprising:

2. the portion of the skill is a first portion, and the processor: determining that a second portion of the skills requires specialization for the environment; In response to determining that the second portion of the skill requires specialization, requesting second user input from a user; In response to receiving the second user input, generating a plan to perform the second portion of the skill based on the second user input; moving the at least one of the operating element and the transport element based on the plan to perform the second portion of the skill; The robotic device of claim 1 , further configured to:

3. the portion of the skill is a first portion, and the processor: determining that a second portion of the skills requires specialization for the environment; In response to determining that the second portion of the skill requires specialization, requesting a performance from the user; acquiring, for each location from a plurality of locations associated with the demonstration, information collected by the set of sensors and relating to at least one of the environment, the operating element, or the transport element; generating a specialized model for performing the skill in the environment using the model and the information collected by the set of sensors; and The robotic device of claim 1 , further configured to:

4. the processor: acquiring information of at least one of the environment, the operation element, or the transport element using the set of sensors when the at least one of the operation element or the transport element is moving based on the plan; determining, based on the information of the at least one of the environment, the operating element, or the transport element, that the operation of the at least one of the operating element or the transport element failed to meet one or more success criteria; modifying the plan to perform at least a portion of the skill in response to determining that the action failed to meet the one or more success criteria; moving the at least one of the operating element or the conveying element based on the correction plan; The robotic device of claim 1 , further configured to:

5. the processor: acquiring information of at least one of the environment, the operation element, or the transport element using the set of sensors when the at least one of the operation element or the transport element is moving based on the plan; updating a map of the environment based on the information of the at least one of the environment, the operating element, or the transport element; The robotic device of claim 1 , further configured to:

6. the map of the environment includes at least a semantic layer and a context layer, and the processor: storing information related to one or more objects in the environment in the semantic layer; determining one or more behaviors to perform near the one or more objects based on the information related to the one or more objects; storing the one or more behaviors in the context layer; 6. The robotic device of claim 5, configured to update the map of the environment by:

7. the processor: displaying the plan to execute the at least some of the skills to a user; and receiving, after the display, a third user input indicating a confirmation to initiate performance of the at least part of the skill; and further configured to: The robotic device of claim 1 , wherein the processor is configured to move the at least one of the manipulation element or the transport element in response to receiving the third user input.

8. the processor: displaying the plan to execute the at least some of the skills to a user; and receiving a fourth user input after said display indicating a change to said plan; modifying the plan based on the fourth user input; and and further configured to: The robotic device of claim 1 , wherein the processor is configured to move the at least one of the manipulation element or the transport element based on the plan after modifying the plan.

9. the operating element includes a plurality of movable components connected by a plurality of joints; the set of sensors a sensor configured to measure torque acting on a joint from said plurality of joints; a sensor configured to detect engagement of a moving component from the plurality of moving components with a physical object from the set of physical objects; The robotic device of claim 1 , comprising at least one of:

10. The robotic device of claim 1 , wherein the manipulative element includes an end effector configured to engage one or more objects in the environment.

11. the processor: presenting the information of the environment to a user; receiving a fifth user input, the fifth user input including an instruction to perform a social behavior, the portion of the skill involving an interaction with at least one human; identifying the at least one human in the environment; performing the social behavior in proximity to the at least one human; and The robotic device of claim 1 , further configured to:

12. the processor: acquiring information about another portion of the environment that is different from the portion of the environment using the set of sensors; In response to determining that user input is required to perform another portion of the skill in the other portion of the environment, prompting a user to provide a sixth user input related to performing the other portion of the skill; generating another plan to execute the other portion of the skill based on the sixth user input; and The robotic device of claim 1 , further configured to:

13. the processor: moving the manipulation element and the transport element based on the plan to perform the portion of the skill while acquiring additional information about the environment; suspending the execution of the skill in response to identifying a change in the environment based on the additional information of the environment; prompting the user to provide feedback based on the changes in the environment; and generating another plan to execute another portion of the skill based on the feedback; and The robotic device of claim 1 , further configured to:

14. acquiring, using a set of sensors of a robotic device, information of an environment, the environment being an unstructured environment, and a current state of the robotic device, the current state of the robotic device being a current state of a transport element and a manipulation element of the robotic device, the information acquired using the set of sensors including information of positions of physical objects in the environment and information of configurations of the plurality of joints of the manipulation element; determining, based on the information of the environment, that user input is required; and In response to determining that the user input is required, requesting the user input from a user; selecting, using an arbitration algorithm, a skill to execute from a plurality of skills based on the information of the environment, the current state of the robotic device, and the user input, wherein the execution of the skill includes a physical interaction between the operational element and the physical object in the environment; generating a plan for executing the portion of the skill based on at least one of the information of the environment, the current state of the robotic device, and the user input using a model for executing the portion of the skill, the plan including one or more movements of at least one of a manipulation element or a transport element of the robotic device; moving the at least one of the operating element or the conveying element based on the plan; A method comprising:

15. acquiring information of at least one of the environment, the operating element, or the transport element using the set of sensors of the robotic device when the at least one of the operating element or the transport element is moving based on the plan; updating a map of the environment stored in the memory based on the information of the at least one of the environment, the operating element, or the transport element; sending information indicative of the update to the map of the environment to a computing device; receiving information from the computing device indicative of updates to the map of the environment received by the computing device from a set of robotic devices other than the robotic device; The method of claim 14 further comprising:

16. Using the set of sensors to acquire information about another portion of the environment that is different from the portion of the environment; In response to determining that user input is required to perform another portion of the skill in the other portion of the environment, prompting a user to provide a second user input related to performing the other portion of the skill; generating another plan to execute the other portion of the skill based on the second user input; The method of claim 14 further comprising:

17. moving the manipulation element and the transport element based on the plan to perform the portion of the skill while acquiring additional information about the environment; suspending the execution of the skill in response to identifying a change in the environment based on the additional information of the environment; prompting the user to provide feedback based on the changes in the environment; and generating another plan to execute another portion of the skill based on the feedback; and The method of claim 14 further comprising:

18. acquiring information of at least one of the environment, the operation element, or the transport element using the set of sensors when the at least one of the operation element or the transport element is moving based on the plan; determining, based on the information of the at least one of the environment, the operating element, or the transport element, that the operation of the at least one of the operating element or the transport element failed to meet one or more success criteria; modifying the plan to perform at least a portion of the skill in response to determining that the action failed to meet the one or more success criteria; moving the at least one of the operating element or the conveying element based on the correction plan; 16. The method of claim 15, further comprising:

Images