Humanoid robot interface and workflow

Abstract

A humanoid robot system includes at least one humanoid robot and a control system configured to be communicatively coupled to the at least one humanoid robot and to perform actions. Actions include running a mission creator to create one or more work tasks for at least one humanoid robot and running a mission assigner to assign one or more work tasks to at least one humanoid robot. In one embodiment, actions further include running a diagnostic tool on at least one humanoid robot.

Description

Technical Field

[0005] , , , ,

[0003] , , , ,

[0004]

[0001] (Technical Field) The present disclosure describes systems and methods associated with a humanoid robot user interface.

Background Art

[0006] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, at least one humanoid robot includes one or more eyes, a mouth, and a chest display.

[0007] In another aspect, which may be combined with one, some, or all of the aforementioned aspects, an action providing communication from at least one humanoid robot to a human operator includes activating one or more eyes, mouths, or chest displays to provide visual communication.

[0008] In another aspect that can be combined with one, some, or all of the aforementioned aspects, the communications include at least one of the following: startup sequences, startup greetings, processing and acknowledgment communications, work communications, greeting communications between work tasks, maintenance mode communications, error communications, or charging communications.

[0009] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, is that visual communication includes optical communication using one or more LEDs.

[0010] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, the control system includes a handheld controller.

[0011] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, the controller communicates wirelessly with at least one humanoid robot.

[0012] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, the operation further includes controlling at least one humanoid robot to perform the movement of at least one humanoid robot using a handheld controller.

[0013] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, movement includes at least one of walking, squatting, rotation of at least one humanoid robot upper body assembly, rotation of at least one humanoid robot, loading an object, or bending forward.

[0014] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, the operation further includes presenting images captured by at least one humanoid robot on a handheld controller.

[0015] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, the operation further includes presenting the identification of at least one humanoid robot from a team of humanoid robots on a handheld controller.

[0016] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, is that at least one humanoid robot includes multiple humanoid robots.

[0017] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, multiple humanoid robots are divided into at least two teams of humanoid robots.

[0018] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, is that the action of running the Mission Creator to create one or more work tasks for at least one humanoid robot includes running the Mission Creator to create a mission for at least one team of humanoid robots.

[0019] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, is that a mission involves multiple tasks.

[0020] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, is that the action of a mission assigner to assign one or more work tasks to at least one humanoid robot includes assigning multiple tasks individually to humanoid robots in at least one team of humanoid robots.

[0021] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, the operation further includes presenting a visual view of at least one team of humanoid robots to a human operator on a display device.

[0022] In another exemplary implementation, a method for operating a humanoid robot system includes using a control system to initialize at least one humanoid robot within the humanoid robot system; using the control system to run a mission creator to create one or more work tasks for at least one humanoid robot; and using the control system to run a mission assigner to assign one or more work tasks to at least one humanoid robot.

[0023] One aspect that can be combined with the exemplary implementation is to use a control system to run a diagnostic tool on at least one humanoid robot.

[0024] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, further includes providing communication from at least one humanoid robot to a human operator using a control system.

[0025] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, at least one humanoid robot includes one or more eyes, a mouth, and a chest display, and providing communication from at least one humanoid robot to a human operator includes activating one or more of the one or more eyes, mouth, or chest display to provide visual communication.

[0026] In another aspect that can be combined with one, several, or all of the foregoing aspects, the communication includes at least one of startup sequence, startup greeting, processing and confirmation communication, work communication, greeting communication between work tasks, maintenance mode communication, error communication, or charging communication.

[0027] In another aspect that can be combined with one, several, or all of the foregoing aspects, the visual communication includes optical communication using one or more LEDs.

[0028] In another aspect that can be combined with one, several, or all of the foregoing aspects, the control system includes a handheld controller.

[0029] Another aspect that can be combined with one, several, or all of the foregoing aspects further includes wireless communication between the handheld controller and at least one humanoid robot.

[0030] Another aspect that can be combined with one, several, or all of the foregoing aspects further includes controlling at least one humanoid robot to perform the movement of at least one humanoid robot using the handheld controller.

[0031] In another aspect that can be combined with one, several, or all of the foregoing aspects, the movement includes at least one of walking, squatting, rotation of the upper body assembly of at least one humanoid robot, rotation of at least one humanoid robot, loading of an object, or forward flexion.

[0032] Another aspect that can be combined with one, several, or all of the foregoing aspects further includes presenting an image captured by at least one humanoid robot on the handheld controller.

[0033] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, further includes presenting the identification of at least one humanoid robot from a team of humanoid robots on a handheld controller.

[0034] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, is that at least one humanoid robot includes multiple humanoid robots.

[0035] In another aspect, which can be combined with one, some, or all of the aforementioned aspects, multiple humanoid robots are divided into at least two teams of humanoid robots.

[0036] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, is that running the Mission Creator to create one or more work tasks for at least one humanoid robot includes running the Mission Creator to create a mission for at least one team of humanoid robots.

[0037] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, is that a mission involves multiple tasks.

[0038] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, is that performing a mission assigner to assign one or more work tasks to at least one humanoid robot, which includes assigning multiple tasks individually to humanoid robots within at least one team of humanoid robots.

[0039] Another aspect, which can be combined with one, some, or all of the aforementioned aspects, further includes presenting a visual view of at least one team of humanoid robots to a human operator on a display device.

[0040] Implementations of the systems and methods described herein may include one, some, or all of the following features. For example, an implementation of the disclosure may provide optimized and efficient control, diagnostics, and communication with one or more humanoid robots, including a team of humanoid robots assigned one or more tasks within a robotic system workflow.

[0041] Details of one or more implementations of the subject matter described herein are provided in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will be evident from the description, drawings, and claims. [Brief explanation of the drawing]

[0042] [Figure 1] Figure 1 is a schematic diagram illustrating an exemplary implementation of a humanoid robot according to this disclosure.

[0043] [Figure 2A] Figures 2A-2D illustrate exemplary implementations of a controller for a humanoid robot within a robot workflow system according to this disclosure. [Figure 2B] Figures 2A-2D illustrate exemplary implementations of a controller for a humanoid robot within a robot workflow system according to this disclosure. [Figure 2C] Figures 2A-2D illustrate exemplary implementations of a controller for a humanoid robot within a robot workflow system according to this disclosure. [Figure 2D] Figures 2A-2D illustrate exemplary implementations of a controller for a humanoid robot within a robot workflow system according to this disclosure.

[0044] [Figure 3A] Figures 3A and 3B illustrate exemplary implementations of a mission assigner for one or more humanoid robots in a robotic workflow system as described herein. [Figure 3B]Figures 3A and 3B illustrate exemplary implementations of a mission assigner for one or more humanoid robots in a robotic workflow system as described herein.

[0045] [Figure 4A] Figures 4A and 4B illustrate exemplary implementations of a mission creator for one or more humanoid robots in a robot workflow system as described herein. [Figure 4B] Figures 4A and 4B illustrate exemplary implementations of a mission creator for one or more humanoid robots in a robot workflow system as described herein.

[0046] [Figure 5A] Figures 5A and 5B illustrate exemplary implementations of diagnostic tools for one or more humanoid robots in a robotic workflow system as described herein. [Figure 5B] Figures 5A and 5B illustrate exemplary implementations of diagnostic tools for one or more humanoid robots in a robotic workflow system as described herein.

[0047] [Figure 6A] Figures 6A-6C illustrate exemplary implementations of human-interaction display components for a humanoid robot within a robot workflow system as described herein. [Figure 6B] Figures 6A-6C illustrate exemplary implementations of human-interaction display components for a humanoid robot within a robot workflow system as described herein. [Figure 6C] Figures 6A-6C illustrate exemplary implementations of human-interaction display components for a humanoid robot within a robot workflow system as described herein.

[0048] [Figure 6D] Figures 6D and 6E are schematic diagrams illustrating exemplary implementations of the humanoid robot deployment system according to this disclosure. [Figure 6E]Figures 6D and 6E are schematic diagrams illustrating exemplary implementations of the humanoid robot deployment system according to this disclosure.

[0049] [Figure 6F] Figure 6F is a schematic diagram illustrating an exemplary implementation of an operator panel for a humanoid robot deployment system according to the present disclosure.

[0050] [Figure 6G] Figure 6G is a schematic diagram illustrating an exemplary implementation of one or more humanoid robot deployment systems integrated into a case-picking process as described herein.

[0051] [Figure 7] Figure 7 shows an exemplary implementation of the robot workflow control architecture described herein.

[0052] [Figure 8] Figure 8 shows an exemplary implementation of the robot workflow control scheme according to this disclosure.

[0053] [Figure 9] Figure 9 shows an exemplary implementation of the robot workflow control suite described herein.

[0054] [Figure 10] Figure 10 shows a schematic diagram of a control system that may be used in the workflow of Figure 1 according to this disclosure. [Modes for carrying out the invention]

[0055] (Detailed explanation) Figure 1 is a schematic diagram illustrating an exemplary implementation of the humanoid robot 100 as described herein. Generally, many different types and forms of humanoid robots can be implemented using the UI and user experience (UX) systems and methods described herein, and the humanoid robot 100 provides only one embodiment of the specific features of different humanoid robots envisioned herein.

[0056] In this disclosure, the term “humanoid robot” may generally refer to a robot whose shape is human, for example, having a torso, trunk, two torso appendages (i.e., arms / hands), two trunk appendages (i.e., legs / feet), and a head or skull appendage. However, the term “humanoid robot” may also refer to a robot that resembles only a part of a human, such as only a torso with torso appendages, or only a trunk with trunk appendages. In addition, this disclosure describes aspects of humanoid robots that may apply to quadruped robots or other non-humanoid robots (e.g., pairs of linear actuators that form part of a joint assembly or appendage, operate in combination, and adjust the joint assembly or appendage in two degrees of freedom through differential linear action).

[0057] Humanoid robot 100 includes, in accordance with this disclosure, a head 102, an upper body assembly 104, and a lower body assembly 106. Generally, humanoid robot 100 comprises a general-purpose robot product that performs useful tasks in the real world (without the use of emotions), such as tasks that are hazardous, harmful, or even routine, which are impossible (or possible) to perform by humans. Exemplary tasks may include handling hazardous or harmful substances (e.g., ammunition, radioactive materials, chemicals), loading and unloading items or objects (e.g., items or objects that are impossible or otherwise immovable by one or more humans), or tasks performed in a hazardous or dangerous environment.

[0058] The humanoid robot 100 can be autonomously controlled (not tethered to any external control system) or controlled by a human (e.g., tethered or wirelessly) and perform tasks (such as those described in more detail here). For example, the humanoid robot 100 can perform useful tasks in a space occupied or not occupied by a human, with mobility and kinematic movement that at least partially mimics that of a human. In some respects, the humanoid robot 100 is designed for practical portability and mobility and mass production.

[0059] The humanoid robot 100 can implement various levels of autonomy. For example, an exemplary implementation of the humanoid robot 100 can be made possible for untethered walking motion testing with some limited operational capabilities. In some aspects, the exemplary implementation of the humanoid robot 100 can be configured for full operation and walking motion.

[0060] The upper body assembly 104 includes, for example, a torso assembly 120, a shoulder assembly 108, an upper arm assembly 110, a forearm assembly 112, and a neck assembly 118. The lower body assembly 106 includes a hip joint assembly 122 (which connects to the torso assembly 120), an upper leg assembly 114, a lower leg assembly 116, and a foot assembly 124 (which is part of the lower leg assembly 116 in some respects).

[0061] The shoulder assembly 108 can provide flexion and extension of the arm of the humanoid robot 100 (e.g., raising the arm forward and backward). The shoulder assembly 108 can provide abduction and adduction (AA) of the arm of the humanoid robot 100. The upper arm assembly 110 can provide internal / external rotation (IE) of the arm of the humanoid robot 100. The combination of the upper and forearm assemblies 110 and 112 (e.g., in some aspects, in combination with radial actuators) can provide flexion-extension (FE) of the forearm of the humanoid robot 100.

[0062] The appendages of the humanoid robot 100 may have at least two degrees of freedom of movement. For example, the two degrees of freedom of the shoulder, roll and yaw, can be provided through differential linear action of the linear actuators of the shoulder assembly 108. The two degrees of freedom of the torso, roll and pitch, can be provided through differential linear action of the linear actuators of the torso assembly 120. The two degrees of freedom of the ankle, roll and pitch, can be provided through differential linear action of the linear actuators of the lower leg assembly 116. The two degrees of freedom of the hip joint, roll and pitch, can be provided through differential linear action of a pair of (smaller) linear actuators and (larger) thigh linear actuators of the upper leg assembly 114.

[0063] Although not shown in Figure 1, some components, such as motor controllers for linear and radial actuators and other control components, include and / or are connected by wiring or cables. One problem that causes some robots to not look human-like and conform to a humanoid envelope is excessive cabling that is visible outside the robot's outer shell. This excessive cabling can also present a risk of snagging. The exemplary implementation of humanoid robot 100 minimizes external cabling by keeping the cables internal or minimizing external cabling. The exemplary implementation of humanoid robot 100 also helps to keep the cables within the humanoid envelope without putting excessive stress on them. More specifically, the exemplary implementation of humanoid robot 100 can define wire paths across joints and minimize stress on both sides of the joints, which allows for little to no strain on the board connectors to which the cables connect.

[0064] Cables and board joints can be subjected to excessive stress when the ratio of cable path length change to total cable length is too high. Minimizing the change in cable path length through the range of motion of the joint relative to the total cable length ensures that the cable does not stretch and does not place unnecessary stress on the cable, connector, or board. Furthermore, bending the cable at an excessively sharp radius can induce localized stress within the cable, which can propagate and apply stress to the connector or board. Exemplary implementations of the humanoid robot 100 can implement features to minimize cable path length change and maximize bending radius.

[0065] Figures 2A-2D show exemplary implementations of a controller 200 for a humanoid robot 100 in a robotic workflow system according to this disclosure. The controller 200 is an exemplary implementation of a human-operated handheld controller 200 that can communicate with the humanoid robot 100 (wired or wirelessly). In some aspects, the humanoid robot 100 is manually controlled using the controller 200 (by programmable buttons and / or thumbsticks, etc.). Alternatively, the humanoid robot 100 can be controlled by the controller 200 in combination with other control elements in a robotic workflow system.

[0066] In some aspects, the display unit 201 of the controller 200 can display images (still images or videos) captured by one or more image acquisition devices on the humanoid robot 100 (for example, in real time). Thus, the operator of the controller 200 can view real-time images (still images or videos) within the humanoid robot 100's viewing path. The display unit 201 can be used, among other things, to assign tasks to the humanoid robot 100, such as moving to an intermediate point or loading an object. In some aspects, once commanded to perform a task (through the controller 200), the humanoid robot 100 performs the task autonomously (for example, through control software on the humanoid robot 100).

[0067] In some respects, the programmable buttons 203 may be, or include, buttons and triggers on the rear or front of the controller 200 that can be programmed, for example, to perform specific movements. Thumbstick movement 205 can be performed using the controller 200, as is customary in games (and also exemplified by a stick with four directional arrows that represents the movement or rotation of the humanoid robot 100).

[0068] Figure 2C shows other components of an exemplary implementation of the controller 200. For example, the D-pad 207 can be operated to move the humanoid robot 100 directionally and / or switch modes or focus areas. The L-thumbstick 209 can be operated to move the humanoid robot 100 directionally. The R-thumbstick 213 can be operated to rotate / turn the humanoid robot 100. Buttons 203 may be programmable for specific movements of the humanoid robot 100. Menus 211 can be configured and accessed for advanced functionality of the humanoid robot 100.

[0069] Figure 2D shows the features of the display portion of an exemplary implementation of the controller 200. The map 215 (or minimap) can display the point of focus, orientation, and neighbors relative to the humanoid robot 100. The display provides a visual view 217 of the control area available in first-person, third-person, and controller POV relative to the humanoid robot 100. The data layer 219 is located within the view area and may be visible. Context actions 221 can be used to assign simple tasks to the humanoid robot 100 in the environment, such as moving to an intermediate point or an object action. Feature navigation 223 can provide access to additional functionality of the humanoid robot 100. Identifiers 225 can provide data and information related to the identification of a particular humanoid robot 100 (for example, among many humanoid robots 100 being controlled).

[0070] Figures 3A and 3B illustrate exemplary implementations of a mission assigner 300 for one or more humanoid robots 100 in a robot workflow system according to this disclosure. For example, a mission assigner (or mission assigner) can provide tasks and missions (a group of tasks linked together) that can be assigned to individual humanoid robots 100 or to a group of humanoid robots 100 working together as a team in the mission assigner. With respect to a workflow involving multiple humanoid robots 100, the mission assigner can also act as a fleet manager, providing robot status, task status and progress, and rapid assignment capabilities. As shown in Figure 3A, the assigner 300 displays a graphic view (which may be an image view) of the humanoid robots 100 in the workflow setting, utilization, diagnostics, uptime, and location of the humanoid robots 100 on the "team", a description of the assigned tasks, a description of the activities, and specific task progress, location, and assignment for each humanoid robot 100 on the team.

[0071] As shown in Figure 3B, the Mission Assigner 300 can display the functionality, features, and UI details associated with the Mission Assigner. For example, Navigation 301 can provide access to features and functionality. Spatial View 303 can display the location area of ​​the selected humanoid robot 100. Team Status 305 can provide a dashboard view of data about the selected humanoid robot 100. Rapid Assignment 307 can be used to quickly assign a task to one humanoid robot 100 or its team. Activity Timeline 309 displays recent activities logged by the selected humanoid robot 100. Fleet View 311 is an overview of fleet details viewed by a team or all humanoid robots 100.

[0072] Figures 4A and 4B illustrate exemplary implementations of a mission creator (or control device) 400 for one or more humanoid robots 100 within a robot workflow system, as disclosed herein. For example, a mission control device is a component of a robot workflow system that can be used to create tasks, missions, and behaviors available in a mission assigner 300 and a controller 200. Missions for a humanoid robot 100 or a team of humanoid robots 100 can be created and customized to meet operational needs and adapt to specific environmental factors. Operators of the robot workflow can configure missions, interactions, and behaviors according to the work environment, brand values, and worker and customer expectations regarding interaction with the humanoid robots 100.

[0073] As shown in Figure 4B, the Mission Creator 400 may include one or more features. For example, Navigation 401 may provide access to features and functionalities. The Action Library 403 may provide a set of predetermined actions that can be used to create missions for one or more humanoid robots 100. The Spatial View 405 may be used to help build and simulate missions for one or more humanoid robots 100. The Behavior Details Panel 407 may provide detailed adjustments to variables in the mission. The Action Bar 409 may display actions associated with mission building, such as save and simulate. The Behavior Tree 411 is an area that may be used to build missions.

[0074] Figures 5A and 5B show exemplary implementations of a diagnostic tool 500 for one or more humanoid robots 100 in a robotic workflow system, as disclosed herein. The diagnostic tool 500 can be used for troubleshooting with error codes, diagnostic software, and customer support to diagnose problems relating to one or more humanoid robots 100. Viewing the component and system status, functionality, and performance of the humanoid robots 100 can be critical for both external and internal teams. When something goes wrong with the humanoid robots 100, the diagnostic tool 500 may be the first place a user would consult. Therefore, insights into the details of what is happening, as well as details about the next steps and contact with customer support, can be crucial.

[0075] The diagnostic tool 500 can provide features as shown in Figure 5B. For example, the navigation 501 can provide access to features and functionality. The component health overview 503 provides an overview view of the overall health status of the components and systems of the humanoid robot 100. The alert popup 505 can provide details about the problem, next steps, and how to contact customer support when a health anomaly is detected in the humanoid robot 100. The health overview 507 is a dashboard view of health-related data for the humanoid robot 100. The system details 509 are high-level statuses of various systems, with the ability to examine each in more detail.

[0076] Figures 6A-6C show exemplary implementations of human interaction display components for a humanoid robot 100 in a robot workflow system according to this disclosure. For example, a human-interactive display component can communicate information to a human operator. In some aspects, it is crucial for smooth operation to always understand what the humanoid robot 100 is doing. There are several ways in which this information can be physically conveyed through the humanoid robot 100. Apart from body posture, communication occurs in the upper body and head, and in some aspects through the eyes 150, mouth 155, and chest display 160 (each implemented as LEDs, for example) in the head. The eyes 150 and mouth 155 can work together to form facial expressions. These expressions include human-based patterns such as greetings, smiles, and introductions. They also reflect technical modes such as charging and error states. The chest panel 160 may be a bidirectional touch-enabled display. The chest panel 160 communicates the identification, mode, task status, and battery status of the humanoid robot 100. Additional details and functionalities can be accessed directly from panel 160 or remotely through a control device (such as controller 200).

[0077] As shown in Figures 6B and 6C, the facial expressions (on face 151, including eyes 150 and mouth 155) and the chest display (on chest panel 160) are designed to work together to avoid redundancy and distraction. The status of the humanoid robot 100 can be communicated through the facial expressions on face 151 and the chest display on chest panel 160. For example, when the humanoid robot 100 is powered on and in the process of starting up and initiating the system, a startup sequence can be communicated. When the humanoid robot 100 reaches the end of the startup sequence, a startup greeting can be communicated. When the humanoid robot 100 is processing an assigned task and confirming that the task has been understood, processing and confirmation communications can be displayed. When the humanoid robot 100 is performing an assigned task, work communications can be displayed. When the humanoid robot 100 is in the middle of a task and wants to greet someone, inter-task greeting communications can be displayed. When the humanoid robot 100 is put into maintenance mode for repair, modification, or inspection, maintenance mode communication may be displayed. When the humanoid robot 100 has detected errors, error communication is displayed. When the humanoid robot 100 is charging, charging communication is displayed, and if the humanoid robot 100 is charging in the middle of a task, task communication is also displayed on the chest panel 160.

[0078] Figures 6D and 6E are schematic diagrams illustrating exemplary implementations of the humanoid robot deployment system according to the present disclosure. For example, Figure 6D shows an exemplary implementation of the humanoid robot deployment system 600 for a single humanoid robot 604 (e.g., the humanoid robot 100 or any other humanoid robot according to the present disclosure). In this exemplary implementation, the humanoid robot deployment system 600 includes a base station 602, which includes, for example, at least one spare battery 610 and a charging location for the humanoid robot 604 and the spare battery 610. Thus, in this embodiment, the humanoid robot 604 can return to the base station 602, dock with it, and have its onboard battery charged, or be fitted with a different fully charged spare battery 610. The exemplary implementation of the humanoid robot deployment system 600 may also include an operator panel 606 (more fully described with reference to Figure 6F) which is communicatively coupled to the base station 602. A controller 608 (such as a wireless controller in the form of a tablet or other mobile device) may also be included in the humanoid robot deployment system 600. Controller 608 may be similar to, or identical to, controller 200, for example.

[0079] Figure 6E shows an exemplary implementation of a humanoid robot deployment system 650 for multiple humanoid robots 604 (e.g., humanoid robot 100 or any other humanoid robot as disclosed herein). In this exemplary implementation, the humanoid robot deployment system 650 includes multiple base stations 602 (e.g., one per humanoid robot 604), each including, for example, at least one spare battery 610 and a charging location for a particular humanoid robot 604 assigned to a base station 602. The exemplary implementation of the humanoid robot deployment system 650 may also include, in this example, a single operator panel 606 that is communicatively coupled to the base stations 602 within the humanoid robot deployment system 650. In this embodiment, multiple controllers 608 (e.g., one per humanoid robot 604) may also be included within the humanoid robot deployment system 650.

[0080] Exemplary implementations of the humanoid robot deployment system 600 and the humanoid robot deployment system 650 can provide functionality and features that facilitate the operation of one or more humanoid robots 604 associated with the system. For example, the base station 602 provides a location where the humanoid robot 604 can dock, as well as charging capacity for the humanoid robot 604 (i.e., with ground power) and a separate battery 610. While the humanoid robot 604 is working, the spare battery 610 is being charged. When the humanoid robot 604 runs out of its battery, the robot 604 can either automatically dock and automatically recharge (e.g., autonomously without any human intervention) or have a human quickly replace the battery, thereby achieving, for example, a 22-hour / 7-day operational time.

[0081] In some respects, the charging dock on base station 602 can charge the humanoid robot 604 simultaneously with the spare battery 610. In some respects, the charging dock on base station 602 can charge the humanoid robot 604 or the spare battery 610 at any given time. During battery replacement, the humanoid robot 604 may remain in a charged state while the battery replacement occurs.

[0082] In some respects, the operator panel 606 and the base station 602 can be connected (e.g., wired or wirelessly) to communicate within a closed private network (i.e., optionally encrypted). Thus, the humanoid robot deployment system 600 (or 650) can form a secure boundary protecting the humanoid robot 604 from external access. Optionally, the humanoid robot deployment system 600 or 650 can be connected to an external network (e.g., through the operator panel 606) via an Ethernet® jack, WiFi, 4G / 5G, or other protocols.

[0083] Figure 6F is a schematic diagram showing an exemplary implementation of an operator panel 606 for a humanoid robot deployment system according to the present disclosure. In the exemplary implementation, the operator panel 606 may include multiple lights / switches that provide multiple indicators / functions. In this exemplary implementation, the operator panel 606 includes four lights / switches of different colors. A first light-switch 601 may be, for example, blue and may indicate that the humanoid robot 604 is docked to the base station 602. A blinking first light-switch 601 may indicate that the humanoid robot 604 is moving to the base station 602. A first light-switch 601 may be operated to pause the current job and call the humanoid robot 604 back to the base station 602.

[0084] The second light-switch 603 may be, for example, green, and can indicate that the humanoid robot 604 is actively performing a mission within the work area. The second light-switch 603 can be operated to instruct the humanoid robot 604 to start or resume work that is initiated from the front of the job queue.

[0085] A third light-switch 605 may be, for example, yellow, and can indicate that the humanoid robot 604 is temporarily suspended in a sustainable position within the work area. The third light-switch 605 may be a soft-stop switch and can be operated to instruct the humanoid robot 604 to temporarily suspend its current behavior in the next available sustainable safe position. Other actions can also trigger a soft stop. For example, in the exemplary case-picking workflow in Figure 6G, opening the safety gate can also trigger a soft stop.

[0086] The fourth light-switch 607 may be red, for example, to indicate that the humanoid robot 604 has been emergency stopped and that any power to the joints of the humanoid robot 604 has been removed. The fourth light-switch 607 can be operated to immediately emergency stop the humanoid robot 604.

[0087] Figure 6G is a schematic diagram showing an exemplary implementation of one or more humanoid robot deployment systems integrated into a case-picking process 670 according to the present disclosure. In this exemplary process 670 (or workflow 670), multiple humanoid robot deployment systems 600 (or one or more humanoid robot deployment systems 650) may be used to operate multiple humanoid robots 604 from a base station 602 in the workflow 670. As shown in this embodiment, the base station 602 may be jointly installed within a work area 672 (bounded, for example, by a safety fence 674) together with one or more pallets holding products 676. In this embodiment, one or more operator panels 606 may also be jointly installed with the work area 672 to provide motion control of the humanoid robots 604 (for example, as described with reference to Figure 6F).

[0088] In this exemplary workflow 670, the humanoid robot 604 works autonomously, for example, loading products 676 from pallets in the work area 672 onto a conveyor 678, where such products 676 are transported out of the work area 672. The humanoid robot 604 can return to the base station 602 to recharge when needed (e.g., autonomously) or when commanded by the operator panel 606. In this embodiment, the humanoid robot 604 can also be stopped in an emergency, for example, if a gate or door of the work area 672 (e.g., within the fence 674) is opened.

[0089] Figure 7 shows an exemplary implementation of the robot workflow control architecture 700 according to this disclosure. The robot workflow control architecture 700 provides an exemplary architecture that can be implemented using a humanoid robot 100 or a team of humanoid robots 100. The core functional areas and exemplary features / workflows in architecture 700 can be modified, for example, depending on the capabilities of a particular humanoid robot 100 and the workflow needs of the user of the humanoid robot 100 (or multiple humanoid robots 100).

[0090] Figure 8 shows an exemplary implementation of the robot workflow control scheme 800 as disclosed herein. The robot workflow control scheme 800 provides an exemplary scheme that can be implemented using a humanoid robot 100 or a team of humanoid robots 100. As shown, the control functionality progresses from technical, internal, and complex to simple control abstractions used by the customer or user of the humanoid robot 100 (or multiple humanoid robots 100). The functionality of scheme 800 can be modified, for example, depending on the capabilities of a particular humanoid robot 100 and the workflow needs of the user of the humanoid robot 100 (or multiple humanoid robots 100).

[0091] Figure 9 shows an exemplary implementation of the robot workflow control suite 900 as disclosed herein. For example, the described workflow functionalities, such as mission assigner, mission control, task assignment, and diagnostics, can be accessed or otherwise used from several different control components, including backend devices such as the controller 200 and servers, or frontend user devices such as tablets, phones, laptops, and desktop workstations.

[0092] Figure 10 shows a schematic diagram of a control system that may be used in a robot workflow according to this disclosure. For example, all or part of control system 1000 (or multiple control systems 1000) may be used for the operations described above, for example, as one or more components of a robot workflow control suite 900, or as part thereof. Controller 1000 is intended to include various forms of digital computers, such as printed circuit boards (PCBs), processors, digital networks, or others. In addition, the system may include portable storage media such as a Universal Serial Bus (USB) flash drive. For example, a USB flash drive may store an operating system and other applications. The USB flash drive may include input / output components such as a wireless transmitter or USB connector that can be inserted into a USB port of another computing device.

[0093] The controller 1000 includes a processor 1010, memory 1020, storage device 1030, and input / output device 1040. Components 1010, 1020, 1030, and 1040 are interconnected using a system bus 1050. The processor 1010 is capable of processing instructions for execution within the controller 1000. The processor may be designed using one of several architectures. For example, the processor 1010 may be a CISC (Complex Instruction Set Computer) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimum Instruction Set Computer) processor.

[0094] In one implementation, processor 1010 is a single-threaded processor. In another implementation, processor 1010 is a multi-threaded processor. Processor 1010 is capable of processing instructions stored in memory 1020 or on storage device 1030 in order to display graphic information for a user interface on input / output device 1040.

[0095] Memory 1020 stores information within the control system 1000. In one implementation, memory 1020 is a computer-readable medium. In another implementation, memory 1020 is a volatile memory unit. In yet another implementation, memory 1020 is a non-volatile memory unit.

[0096] The storage device 1030 is capable of providing large-capacity storage for the controller 1000. In one implementation, the storage device 1030 is a computer-readable medium. In various different implementations, the storage device 1030 may be a floppy disk device, a hard disk device, an optical disk device, a tape device, flash memory, a solid-state device (SSD), or a combination thereof.

[0097] The input / output device 1040 provides input / output operations for the controller 1000. In one implementation, the input / output device 1040 includes a keyboard and / or a pointing device. In another implementation, the input / output device 1040 includes a display unit for displaying a graphical user interface.

[0098] A feature described may be implemented within a digital electronic network, or within computer hardware, firmware, software, or a combination thereof. The device may be implemented within a computer program product, which may be tangibly embodied within an information carrier, for example, a machine-readable storage device for execution by a programmable processor, and the method steps may be carried out by a programmable processor that performs the function of the described implementation by executing a program of instructions, acting on input data, and generating an output. Advantageously, the feature described may be implemented within one or more computer programs executable on a programmable system, which includes at least one programmable processor, at least one input device, and at least one output device, which receive data and instructions from a data storage system and are coupled to transmit data and instructions thereto. A computer program is a set of instructions that are used directly or indirectly within a computer to perform an activity or produce a result. Computer programs can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use within a computing environment.

[0099] Processors suitable for executing instruction programs include, in examples, one of both general-purpose and special-purpose microprocessors and one or more processors of any type of computer. Generally, a processor will receive instructions and data from read-only memory or random-access memory or both. Essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer also includes, or is operationally coupled to, one or more mass storage devices for storing data files, and such devices include magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include, in examples, semiconductor memory devices such as EPROMs, EEPROMs, solid-state drives (SSDs), and flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and all forms of non-volatile memory, including CD-ROMs and DVD-ROMs. Processors and memory can be complemented by or incorporated into ASICs (Application-Specific Integrated Circuits).

[0100] To provide user interaction, the feature can be implemented on a computer having a display device such as a CRT (cathode ray tube), LCD (liquid crystal display), or LED (light-emitting diode) monitor for displaying information to the user, and a pointing device such as a keyboard and mouse or trackball, thereby allowing the user to provide input to the computer. In addition, such activities can be implemented via a touchscreen flat panel display and other suitable mechanisms.

[0101] The system can be implemented in a control system that includes backend components such as data servers, middleware components such as application servers or internet servers, or frontend components such as client computers with a graphical user interface or internet browser, or any combination thereof. The components of this system can be connected by digital data communication in any form or medium, such as a communication network. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), peer-to-peer networks (with ad-hoc or static components), grid computing infrastructure, and the internet.

[0102] This specification contains many specific implementation details, which should be interpreted not as limitations on the scope of any invention or claim, but rather as descriptions of features specific to a particular implementation of a particular invention. Certain features described herein in the context of a separate implementation may also be implemented in a single implementation or in combination. Conversely, various features described in the context of a single implementation may also be implemented in multiple implementations, individually or in any preferred secondary combination. Furthermore, while features described above as acting in a combination, and even initially claimed as such, one or more features from a claimed combination may, in some cases, be removed from the combination, and the claimed combination may be subject to secondary combinations or variations of secondary combinations.

[0103] Similarly, while actions are depicted in a specific order within the diagrams, this should not be understood as requiring that such actions be performed in a specific or sequential order shown, or that all illustrated actions be performed, in order to achieve the desired result. In some situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0104] Several implementations are described. Nevertheless, it should be understood that various modifications can be made without departing from the spirit and scope of this disclosure. For example, the exemplary operations, methods, or processes described herein may include more or fewer steps than those described. Furthermore, the steps in such exemplary operations, methods, or processes may be performed in a different order than those illustrated in the description or figures. Thus, other implementations are also within the scope of the following claims.

Claims

1. It is a humanoid robot system, At least one humanoid robot, A control system configured to be communicatively coupled to and perform actions on at least one of the humanoid robots, Equipped with, The aforementioned operation is, The mission creator is executed to create one or more work tasks for the aforementioned at least one humanoid robot, The mission assigner executes to assign one or more work tasks to at least one of the humanoid robots. A humanoid robot system, including [a specific robot].

2. The humanoid robot system according to claim 1, further comprising the operation of a diagnostic tool on the at least one humanoid robot.

3. The humanoid robot system according to claim 1, further comprising providing communication from the at least one humanoid robot to a human operator.

4. The humanoid robot system according to claim 3, wherein the at least one humanoid robot comprises one or more eyes, a mouth, and a chest display, and providing the communication from the at least one humanoid robot to the human operator includes activating one or more of the one or more eyes, the mouth, or the chest display to provide visual communication.

5. The aforementioned communication is Startup sequence, Startup greetings, Processing and confirmation communication, Work communication, Greetings and communication between work tasks, Maintenance mode communication, Error communication, or Charging and communication A humanoid robot system according to claim 3, comprising at least one of the following.

6. The humanoid robot system according to claim 3, wherein the visual communication comprises optical communication using one or more LEDs.

7. The control system comprises a handheld controller, as described in claim 1, for the humanoid robot system.

8. The humanoid robot system according to claim 7, wherein the controller communicates wirelessly with the at least one humanoid robot.

9. The humanoid robot system according to claim 7, wherein the operation further includes controlling the at least one humanoid robot to perform movement of the at least one humanoid robot using the handheld controller.

10. The humanoid robot system according to claim 9, wherein the movement comprises at least one of walking, squatting, rotating the upper body assembly of the at least one humanoid robot, rotating the at least one humanoid robot, loading an object, or bending forward.

11. The humanoid robot system according to claim 7, further comprising the operation of presenting images captured by the at least one humanoid robot on the handheld controller.

12. The humanoid robot system according to claim 7, further comprising the operation of presenting the identification of at least one humanoid robot from the team of humanoid robots on the handheld controller.

13. The humanoid robot system according to claim 1, wherein the at least one humanoid robot comprises a plurality of humanoid robots.

14. The humanoid robot system according to claim 13, wherein the plurality of humanoid robots are divided into at least two teams of humanoid robots.

15. The operation of running the mission creator to create one or more work tasks relating to the at least one humanoid robot comprises running the mission creator to create a mission relating to at least one team of humanoid robots, wherein the mission comprises a plurality of tasks, according to claim 13.

16. The humanoid robot system according to claim 13, wherein the operation of the mission assigner to assign one or more work tasks to the at least one humanoid robot includes assigning the multiple tasks individually to the humanoid robots in the at least one team of humanoid robots.

17. The humanoid robot system according to claim 13, further comprising the operation of presenting a visual view of the at least one team of humanoid robots to a human operator on a display device.

18. A method for operating a humanoid robot system, Using a control system, initialize at least one humanoid robot within the humanoid robot system, Using the control system, execute the mission creator to create one or more work tasks for the at least one humanoid robot, Using the control system, the mission assigner executes a mission assigner to assign one or more work tasks to the at least one humanoid robot. Methods that include...

19. The method according to claim 18, comprising using the control system to run a diagnostic tool on the at least one humanoid robot.

20. The method according to claim 18, comprising providing communication from the at least one humanoid robot to a human operator using the control system.

21. The method according to claim 20, wherein the at least one humanoid robot comprises one or more eyes, a mouth, and a chest display, and providing the communication from the at least one humanoid robot to the human operator includes activating one or more of the one or more eyes, the mouth, or the chest display to provide visual communication.

22. The aforementioned communication is Startup sequence, Startup greetings, Processing and confirmation communication, Work communication, Greetings and communication between work tasks, Maintenance mode communication, Error communication, or Charging and communication The method according to claim 20, comprising at least one of the following.

23. The method according to claim 20, wherein the visual communication comprises optical communication using one or more LEDs.

24. The method according to claim 18, wherein the control system comprises a handheld controller.

25. The method according to claim 24, comprising wireless communication between the handheld controller and the at least one humanoid robot.

26. The method according to claim 24, comprising controlling the at least one humanoid robot to perform movement of the at least one humanoid robot using the handheld controller.

27. The method according to claim 26, wherein the movement comprises at least one of walking, squatting, rotating the upper body assembly of the at least one humanoid robot, rotating the at least one humanoid robot, loading an object, or bending forward.

28. The method according to claim 24, comprising displaying images taken by the at least one humanoid robot on the handheld controller.

29. The method according to claim 24, comprising presenting the identification of at least one humanoid robot from the team of humanoid robots on the handheld controller.

30. The method according to claim 18, wherein the at least one humanoid robot comprises a plurality of humanoid robots.

31. The method according to claim 30, wherein the plurality of humanoid robots are divided into at least two teams of humanoid robots.

32. The method according to claim 30, wherein running the mission creator to create one or more work tasks relating to at least one humanoid robot includes running the mission creator to create a mission relating to at least one team of humanoid robots, the mission comprising a plurality of tasks.

33. The method according to claim 30, wherein performing the mission assigner to assign one or more work tasks to the at least one humanoid robot includes assigning the multiple tasks individually to the humanoid robots in the at least one team of humanoid robots.

34. The method according to claim 30, comprising presenting a visual view of at least one team of humanoid robots to a human operator on a display device.

Images