Autonomous robots

Abstract

An autonomous robot guidance system and related methods are disclosed. An exemplary autonomous robot includes a processor circuit that detects a first reference based on image data collected while the robot is in a first driving mode, wherein the first driving mode corresponds to an autonomous driving mode, and in response to the detection of the first reference, switches the robot from the first driving mode to a second driving mode, moving the robot in the second driving mode in a direction corresponding to the position of the first reference relative to the robot; generates navigation data for the robot, wherein the navigation data indicates the position of the robot in the environment, switches the robot from the second driving mode to the first driving mode, and after switching from the second driving mode, drives the robot in the first driving mode based on the navigation data.

Description

Technical Field

[0001] This application claims the benefit of U.S. Patent Application No. 18 / 190,593, filed on March 27, 2023, the entire contents of which are incorporated herein by reference.

[0002] This disclosure generally relates to autonomous robots, and more specifically to autonomous robot guidance systems and related methods.

Background Art

[0003] During the operation of an autonomous robot in an environment, the position of the robot relative to the environment is monitored over time. Such position data is used, for example, to instruct the robot regarding travel to a specific location within the environment.

[0004] During the operation of an autonomous robot in an environment, the position of the robot relative to the environment is monitored over time. For example, an estimated position of the robot at a given time can be estimated using a dead reckoning algorithm based on a known starting location and estimated values of the robot's speed and direction of travel over time. However, in some instances, tracking of the robot's position relative to the environment may be interrupted, which can result in inaccurate position data being used to direct the robot's travel. A fixed visual reference system, such as a coded marker (e.g., a QR code (registered trademark)) placed on a wall of a warehouse, can be used to realign the robot's orientation relative to the environment. For example, when the marker is within the field of view of the robot's camera, the fixed visual reference can be decoded by the robot to obtain a value corresponding to the location data of the fixed visual reference. The location data of the fixed visual reference can be used to adjust, update, or recalibrate the robot's navigation data (e.g., the coordinates of the robot within the environment).

[0005] In some cases, despite the existence of techniques such as dead reckoning and / or fixed fiducials, a robot may be unable to reorient itself due to factors such as prolonged periods of not seeing the fixed fiducials or loss of wireless communication by the robot. In some such cases, a user can put the robot into free rotation mode (e.g., via inputs provided to the robot), in which mode the robot exits autonomous driving mode and can be manually operated by the user (e.g., pushed, pulled, rotated). When the robot is in free rotation mode, the user can move the robot to a specific location, such as a location containing a fixed fiducial, to allow the robot to perform the reorientation process.

[0006] However, some robots may have different shapes, sizes, weights, etc., which can affect whether a user can manually move the robot. For example, a robot with a low profile or a robot that carries loads on a forklift may not be ergonomically safe and / or ergonomic for a user to push and / or pull by hand. Therefore, while a free rotation mode allows a user to manually move the robot, some robots are not designed or intended to be moved manually. Furthermore, factors such as the weight of the load carried by the robot may also affect whether a user can manually move the robot. [Overview of the project]

[0007] According to a first aspect of the present disclosure, an autonomous robot is provided comprising an image sensor, a memory, machine-readable instructions, and a processor circuit, wherein the processor circuit is configured to execute machine-readable instructions such that the autonomous robot detects a first reference based on image data corresponding to the output of the image sensor collected while the autonomous robot is in a first operating mode, where the first operating mode corresponds to an autonomous operating mode, and in response to the detection of the first reference, switches the autonomous robot from the first operating mode to a second operating mode, moves the autonomous robot in the second operating mode in a direction corresponding to the position of the first reference relative to the autonomous robot, generates navigation data for the autonomous robot, where the navigation data indicates the position of the autonomous robot in the environment, switches the autonomous robot from the second operating mode to the first operating mode, and after switching from the second operating mode, drives the autonomous robot in the first operating mode based on the navigation data.

[0008] The processor circuit may be configured to detect a second criterion based on image data corresponding to the output of image sensors collected while the autonomous robot is in a second operating mode, and to generate navigation data for the autonomous robot based on the second criterion. The first criterion may be movable relative to the environment in which the autonomous robot is positioned, while the second criterion is fixed relative to the environment. The first criterion may be a virtual criterion.

[0009] The processor circuit may be configured to switch the autonomous robot from a first operating mode to a second operating mode in response to the detection of a first reference over a threshold time period. The processor circuit may be configured to switch the autonomous robot from a first operating mode to a second operating mode in response to the detection of a first reference within a portion of the image sensor's field of view. While the autonomous robot is in the second operating mode, the processor circuit may be configured to move the autonomous robot to maintain the position of the first reference relative to the image sensor's field of view. While the autonomous robot is in the second operating mode, the processor circuit is configured to detect a gesture based on image data, where the gesture is associated with the movement of the first reference, to determine a maneuver based on the gesture, and to move the autonomous robot based on the maneuver.

[0010] This specification discloses exemplary visual reference systems for facilitating the orientation (reorientation) of an autonomous robot's navigation system to an environment using non-fixed visual references (e.g., virtual markers). Examples disclosed herein provide visual references displayed via a user device carried by an individual in an environment (e.g., a warehouse). Non-fixed visual references complement the guidance provided by fixed references and are therefore referred to herein as auxiliary references. When an auxiliary reference is detected within the field of view of the robot's image sensor(s), a navigation awareness control circuit causes the robot to exit autonomous driving mode and enter guided driving mode. In guided driving mode, the robot travels based on the position of the auxiliary reference (e.g., toward the auxiliary reference). In some examples, a user can use the auxiliary reference displayed on a user device to orient (e.g., guide) the robot to a fixed reference in the environment. The robot can then use the fixed reference to update its navigation data (e.g., its position in the environment) and re-enter autonomous driving mode.

[0011] In the examples disclosed herein, an autonomous robot may output an indication that its navigation data is outdated, inaccurate, or otherwise incorrect. In other words, the robot may be unable to orient itself in the environment due to, for example, an obstruction of a fixed reference point or an interruption of wireless communication. Alternatively, the robot may be identified as missing by a dispatcher or warehouse management system. In the examples disclosed herein, a user may display an auxiliary visual reference marker on a user device, such as a handheld device. The user may present the auxiliary reference point so that it is detected within the field of view of the autonomous robot's image sensor(s). The robot's navigation awareness control circuit decodes the auxiliary reference point. Upon recognizing the auxiliary reference point, the robot enters guided driving mode. In guided driving mode, the robot moves relative to the auxiliary reference point (e.g., toward the auxiliary reference point). The user may control the robot by making gestures on the user device while the auxiliary reference point is displayed, to move the robot in a specific direction (e.g., toward a fixed reference point).

[0012] In some examples, if a robot detects a fixed navigation reference while operating in guided driving mode, the robot automatically exits guided driving mode and re-enters autonomous driving mode. In such examples, the robot's navigation awareness control circuit performs the process of reorienting using the fixed reference and continuing to perform tasks in autonomous driving mode. In some examples, the user controls the switching of the robot from guided driving mode to autonomous driving mode. In some such examples, the robot remains in guided driving mode until user input is received, even if a fixed reference is detected by the robot. For example, the user can stop presenting non-fixed visual references, thereby switching the robot from guided driving mode to autonomous driving mode. In some examples, the user provides input via a user device or on the robot to switch the robot from guided driving mode to autonomous driving mode or free rotation mode. Thus, in some examples, the user can cause the robot to enter and remain in guided driving mode by presenting auxiliary references for movement or repositioning, regardless of whether or not the robot needs to reorient itself.

[0013] A second aspect of the present disclosure provides an apparatus comprising a memory, machine-readable instructions, and a processor circuit, wherein the processor circuit executes machine-readable instructions to: identify a first marker in image data corresponding to the output of an image sensor of an autonomous vehicle collected while the autonomous vehicle is in autonomous driving mode; switch the autonomous vehicle from autonomous driving mode to a second driving mode based on data associated with the first marker; cause the autonomous vehicle to perform a first maneuver in the second driving mode based on a first position of the first marker relative to the autonomous vehicle; determine location information of the autonomous vehicle in the environment; switch the autonomous vehicle from the second driving mode to autonomous driving mode, and after switching from the second driving mode, operate the autonomous vehicle in autonomous driving mode based on the location information.

[0014] A third aspect of the present disclosure provides a non-temporary machine-readable storage medium and includes instructions to cause a processor circuit to at least: detect a first reference based on image data corresponding to the output of an image sensor collected while the autonomous robot is in a first operating mode, wherein the first operating mode corresponds to an autonomous operating mode; in response to the detection of the first reference, switch the autonomous robot from the first operating mode to a second operating mode, move the autonomous robot in the second operating mode in a direction corresponding to the position of the first reference relative to the autonomous robot; generate navigation data for the autonomous robot, wherein the navigation data indicates the position of the autonomous robot in the environment; switch the autonomous robot from the second operating mode to the first operating mode; and, after switching from the second operating mode, drive the autonomous robot in the first operating mode based on the navigation data. [Brief explanation of the drawing]

[0015] [Figure 1] An exemplary system, including an autonomous robot and a navigation awareness control circuit, relating to the teachings of this disclosure, is illustrated. [Figure 2] Figure 1 is a block diagram of an exemplary navigation awareness control circuit. [Figure 3] Figure 2 is a flowchart illustrating exemplary machine-readable instructions and / or exemplary operations that may be performed by exemplary processor circuits to implement the navigation awareness control circuit. [Figure 4] Figure 3 is a block diagram of an exemplary processing platform, which includes a processor circuit structured to perform exemplary machine-readable instructions and / or exemplary operations in order to implement the navigation awareness control circuit of Figure 2. [Modes for carrying out the invention]

[0016] Generally, the same reference number is used throughout the drawings(s) and the accompanying specification to refer to the same or similar parts. The drawings are not to exact scale.

[0017] Figure 1 illustrates an exemplary system 100 for guiding an autonomous robot 102 in an environment 104 (e.g., a warehouse) as taught in this disclosure. The autonomous robot 102 may include, for example, an autonomous vehicle. In the exemplary system 100 of Figure 1, a user device 106 may be used by a user in the environment 104 to guide the robot 102, for example, if the robot 102 is unable to orient itself within the environment 104.

[0018] In autonomous driving mode, the exemplary robot 102 in Figure 1 moves within the environment 104 with or with limited user input control during the robot 102's movement. The exemplary autonomous robot 102 in Figure 2 includes one or more motors 108 (e.g., electric motors and / or other drive mechanisms) to cause the autonomous robot 102 to move via the robot's wheels 110. The autonomous robot 102 includes, for example, a motor control circuit 112 (e.g., hardware components and / or software components) to control the robot 102's speed.

[0019] The autonomous robot 102 includes a robot control circuit 114 for controlling the movement of the autonomous robot 102. In the example in Figure 1, the robot control circuit 114 is implemented by the processor circuit 116 of the robot 102. The robot control circuit 114 generates commands that control the movement of the robot 102 along a travel path to locations in the environment 104, for example. For example, the robot control circuit 114 generates commands that cause the robot 102 to change direction, move forward, adjust speed, etc.

[0020] The robot control circuit 114 defines the travel trajectory of the robot 102 when it is operating in autonomous driving mode. Commands generated by the robot control circuit 114 can be sent, for example, to the motor control circuit 112. The robot control circuit 114 includes a safety driving control circuit 118 that performs obstacle detection while the robot 102 is traveling and causes the robot 102 to perform maneuvers to avoid collisions.

[0021] The exemplary robot 102 in Figure 1 includes, for example, sensors for generating outputs while the robot 102 moves around the environment 104. The robot 102 in Figure 1 includes navigation sensors 120. Navigation sensors 120 may include, for example, accelerometers, gyroscopes, magnetometers, GPS receivers, etc. The exemplary robot 102 also includes image sensors 122 (for example, video cameras, still cameras, etc.). Image sensors 122 output, for example, image data (e.g., frames) representing the environment 104 while the robot 102 is moving. The outputs of the navigation sensors 120 and image sensors 122 may be analyzed by the robot control circuit 114 with respect to controlling the movement of the robot 102. The robot 102 may include other types of sensors, such as LiDAR sensors.

[0022] In some examples, the autonomous robot 102 includes a display screen 124 for providing visual output and receiving user input via the display screen 124, etc. In some examples, the autonomous robot 102 includes a speaker 126 for providing audio output. The exemplary robot 102 in Figure 1 includes a power source 128, such as a battery, for supplying power to components of the robot 102 that are communicably coupled via a bus 130.

[0023] The exemplary robot 102 in Figure 1 includes a navigation awareness control circuit 132. The navigation awareness control circuit 132 determines the navigation data (e.g., location, orientation) of the robot 102 relative to the environment 104 as the robot 102 travels within the environment 104. In some examples, the navigation awareness control circuit 132 executes an algorithm, such as a dead reckoning algorithm, to determine the position of the robot 102 in the environment 104 at a given time, based on the robot 102's previously known locations and outputs from navigation sensors 120 indicating the robot 102's speed and direction of travel over time. The navigation awareness control circuit 132 generates navigation data, for example, indicating the robot 102's position (e.g., coordinates), direction of travel, etc., in the environment 104 at a particular time. Navigation data generated by the navigation awareness control circuit 132 can be used by the robot control circuit 114 to enable the robot 102 to arrive at a specific location within the environment 104, for example, by causing the robot 102 to perform a specific maneuver (e.g., change direction) when traveling to that location.

[0024] In some examples, the navigation awareness control circuit 132 uses one or more fixed reference markers 134 (referred to herein as fixed reference(s) 134) disposed at corresponding known locations within the environment 104 to determine the position of the robot 102 relative to the environment 104. The fixed reference(s) 134 can include, for example, encoded markers (such as QR codes, AprilTags) that store coordinate data representing fixed (e.g., known, unchanging, permanent) locations of markers within the environment 104. In the example of FIG. 1, the navigation awareness control circuit 132 can recognize the fixed reference(s) 134 based on the image data output by the image sensor(s) 122. In response to recognizing one of the fixed references 134 within the field of view of the image sensor(s) 122, the navigation awareness control circuit 132 can decode the position information stored in the fixed reference 134 and identify, update, and / or confirm the position of the robot 102 relative to the environment 104 at a particular time. The navigation awareness control circuit 132 can verify or update the navigation data based on detecting another fixed reference 134 at a subsequent time during the travel of the robot 102 within the environment 104. Thus, the navigation data can be updated periodically during the travel of the autonomous robot 102 to maintain accurate position information of the robot 102.

[0025] In the example shown in Figure 1, the navigation awareness control circuit 132 is implemented by executable instructions executed on the processor circuit 116 of the autonomous robot 102. However, in other examples, the navigation awareness control circuit 132 is implemented by the processor circuit 136 of a user device 106 that communicates with the autonomous robot 102 (e.g., via a wired or wireless communication protocol), and / or by a cloud-based device 138 (e.g., one or more servers, processors, and / or virtual machines). In other examples, one or more components of the navigation awareness control circuit 132 are implemented by dedicated circuits located in the autonomous robot 102 and / or the user device 106. These components can be implemented in software, hardware, or any combination of software, firmware, and / or hardware.

[0026] The robot control circuit 114 operates the autonomous robot 102 based on the navigation data most recently determined by the navigation awareness control circuit 132. In some examples, the most recently generated navigation data may become stale or inaccurate due to an interruption in the ability of the navigation awareness control circuit 132 to determine the position of the robot 102. For example, sensor calibration errors or changes in the output of the navigation sensor(s) 120 over time, such as due to sensor drift, may affect the ability of the navigation awareness control circuit 132 to orient the robot 102 with respect to the environment 104. In some examples, the fixed reference(s) 134 may be blocked from the perspective of the image sensor(s) 122 of the robot 102 (e.g., due to inventory blocking one or more of the fixed references 134 within the warehouse). In such cases, the navigation awareness control circuit 132 may be unable to verify or update the position of the robot 102 and may move for a long period of time. In some examples, the robot 102 may lose its wireless communication capabilities, which may, for example, prevent the navigation awareness control circuit 132 from detecting and / or decoding information stored in the fixed reference(s) 134.

[0027] In some examples, the navigation awareness control circuit 132 causes the robot 102 to generate visual and / or audio outputs(s) indicating, for example, that its position has not been confirmed within a threshold time period, that the robot 102 is unable to find or follow a travel path to a certain location, or that the robot 102 has not reached that location within a threshold time period. In some examples, a user in environment 104 may determine, based on monitoring the robot 102 in environment 104, that the robot 102's navigation data is inaccurate over a given time period. For example, the robot 102 may not be aware that it has lost its way (for example, the robot 102 believes it is traveling on path "A" when it is actually traveling on path "B"). In such examples, the operator may manually intervene to control the robot 102 (for example, by stopping / pausing the robot 102's movement) and initiate the re-determination of the robot 102's position.

[0028] In some examples, the user can switch the robot 102 from autonomous driving mode to free rotation driving mode and manually move the robot 102 to, for example, a fixed reference 134, so that the navigation awareness control circuit 132 can reorient the robot 102 to the environment 104. For example, the user can provide input to the robot 102 to switch the motor(s) 108 to neutral so that the user can move the robot 102 (e.g., push, pull, rotate). When the robot 102 is positioned by the user so that one of the fixed references 134 is within the field of view of the image sensor(s) 122, the navigation awareness control circuit 132 can update the robot 102's navigation data using the fixed reference 134. The robot 102 can switch from free rotation driving mode to autonomous driving mode (e.g., via user input or automatically upon detection of a fixed reference 134). The robot 102 can continue autonomous operation based on the updated navigation data.

[0029] However, in some cases, it may not be ergonomically safe or ergonomic for a user to manually move the robot 102, for example, due to the weight of the robot 102 and / or the load carried by the robot 102, the size of the robot 102, the profile of the robot 102 (e.g., a low profile), etc. In other cases, the user may not want to manually move the robot 102 (for example, due to the user's health condition, general user preferences, etc.). In the examples disclosed herein, an auxiliary reference system is used to maneuver the robot 102, and in particular, to enable the navigation awareness control circuit 132 to update navigation data to relocate or reorient the robot 102 within the environment 104.

[0030] In the example shown in Figure 1, the auxiliary reference marker 140 (also called auxiliary reference 140) is presented on the display screen 142 of the user device 106 so that it is within the field of view of the image sensor(s) 122. Therefore, in some examples, the auxiliary reference 140 is a virtual reference.

[0031] In response to the detection and decoding of the auxiliary reference 140, the navigation awareness control circuit 132 causes the robot 102 to enter guided driving mode. In guided driving mode, the robot 102 moves on its own (for example, based on commands from the robot control circuit 114) and has functions such as obstacle detection and collision avoidance (based on commands generated by the safe driving control circuit), but the robot 102 operates semi-autonomously in that its travel trajectory is defined by the user. In the example in Figure 1, when the robot 102 is in guided driving mode, the robot 102's travel trajectory is based on the auxiliary reference 140. For example, the navigation awareness control circuit 132 causes the robot 102 to move in the direction corresponding to the position of the auxiliary reference 140 and to perform maneuvers to keep the auxiliary reference 140 within the field of view of the image sensor(s) 122.

[0032] As disclosed herein, a user can perform gestures while holding a user device 106 displaying an auxiliary reference 140 to direct the movement of the robot 102. Thus, the auxiliary reference 140 can be considered an unfixed reference, as its position depends on the user holding the user device 106, the user's movement of the user device 106, etc. In some examples, a user can use the auxiliary reference 140 to direct the robot 102 to one of the fixed references 134. In some examples, the navigation awareness control circuit 132 analyzes image data generated while the robot 102 is in guided driving mode to detect one of the fixed references 134. In response to the detection of a fixed reference 134, the navigation awareness control circuit 132 identifies the position of the robot 102 relative to the environment 104. Thus, the navigation awareness control circuit 132 updates or adjusts navigation data that can be used by the robot control circuit 114 when directing the movement of the robot 102. As disclosed herein, the navigation awareness control circuit 132 can switch the robot 102 from guided driving mode to autonomous driving mode in response to user input or based on rules (for example, when auxiliary criteria 140 are no longer detected in response to the detection of a fixed criterion 134) in order to resume autonomous driving based on updated data. The user device 106 may include, for example, a handheld device, such as a wearable device worn on the user's wrist. In the example of Figure 1, the auxiliary criteria application 144 is executed by the processor circuit 136 of the user device 106. The user can launch the auxiliary criteria application 144 to have one or more auxiliary criteria 140 presented on the display screen 142 of the user device 106.

[0033] Auxiliary criteria(s) 140 may include encoded data. In some examples, auxiliary criteria(s) 140 and fixed criteria(s) 134 may include different values ​​or ranges of values ​​using the same type of encoding (e.g., AprilTag with different values). In some examples, auxiliary criteria(s) 140 and fixed criteria(s) 134 may include different types of criteria. For example, fixed criteria(s) 134 may include AprilTag(s), and auxiliary criteria 140 may include QR codes(s). The encoded data of auxiliary criteria(s) 140 may include, for example, a value or command indicating that robot 102 should switch to guided driving mode. Thus, the data associated with auxiliary criteria 140 is different from the data associated with fixed criteria(s) 134 (e.g., known location data of fixed criteria(s) 134 in environment 104).

[0034] The auxiliary criteria application 144 can display the auxiliary criteria(s) 140 until user input to terminate the presentation of the auxiliary criteria(s) 140 is received on the user device 106. Alternatively, the auxiliary criteria application 144 can display the auxiliary criteria(s) 140 for a specific length of time (e.g., 3 minutes, 5 minutes).

[0035] The auxiliary reference application 144 can communicate (for example, wirelessly) with the navigation awareness control circuit 132. For example, a user can provide input(s) via the auxiliary reference application 144 indicating that the robot 102 should switch from guided driving mode to autonomous driving mode. Input(s) received by the auxiliary reference application 144 on the user device 106 can be transmitted to the navigation awareness control circuit 132.

[0036] The examples disclosed herein are primarily described in relation to an auxiliary criterion 140 as a virtual criterion presented via a user device 106, but in other examples the auxiliary criterion 140 may be a printed marker carried by the user (e.g., on a badge worn by the user). Thus, the examples disclosed herein are not limited to virtual criterions.

[0037] Figure 2 is a block diagram of a navigation awareness control circuit 132 for determining navigation data (e.g., position, direction of travel) for an autonomous robot, such as the autonomous robot 102 in Figure 1, relative to an environment (e.g., environment 104, warehouse). The navigation awareness control circuit 132 in Figure 2 can be instantiated by a processor circuit, such as a central processing unit, that executes instructions (e.g., creating instances that exist for any length of time, materialize, implement, etc.). Additionally or alternatively, the navigation awareness control circuit 132 in Figure 2 may be instantiated using alternative hardware, such as by an ASIC or FPGA structured to perform actions corresponding to instructions. Therefore, it should be understood that some or all of the circuit in Figure 2 may be instantiated simultaneously or at different times. Some or all of the circuit may be instantiated, for example, in one or more threads running simultaneously and / or serially on hardware. Furthermore, in some examples, some or all of the circuit in Figure 2 may be implemented by a microprocessor circuit that executes instructions to implement one or more virtual machines and / or containers.

[0038] The exemplary navigation awareness control circuit 132 in Figure 2 includes a position detection circuit 200, a reference detection circuit 202, a driving mode control circuit 204, and an inductive driving mode operation circuit 206. In some examples, one, more, or all of the position detection circuit 200, reference detection circuit 202, driving mode control circuit 204, and inductive driving mode operation circuit 206 are instantiated by a processor circuit configured to execute a position detection command and / or perform operations such as those represented by the flowchart in Figure 3.

[0039] The position detection circuit 200 manages the robot 102's navigation awareness of the environment 104. For example, while the robot 102 is moving, the position detection circuit 200 accesses the output of the navigation sensor(s) 120, such as speed and direction of travel. The position detection circuit 200 also accesses the output of the image sensor(s) 122 and performs image analysis to detect characteristics of the environment 104 in which the robot 102 is moving, so that it can recognize a specific location (for example, based on machine learning training).

[0040] The position detection circuit 200 executes one or more robot orientation algorithms 210 to determine the robot 102's navigation data 212 (e.g., position, direction of travel) relative to the environment 104 at a given time. The robot orientation algorithms 210 may include, for example, a dead reckoning algorithm in which the position detection circuit 200 estimates the robot 102's (e.g., current) position based on previously known positions of the robot 102 in the environment 104 and estimates of the robot 102's velocity and direction of travel over time (e.g., based on the output of navigation sensors 120). The robot orientation algorithms 210 and the navigation data 212 may be stored in a database 208. In some examples, the navigation awareness control circuit 132 includes the database 208. In some examples, the database 208 is located outside the navigation awareness control circuit 132 in a location accessible to the navigation awareness control circuit 132, as shown in Figure 2.

[0041] In some examples, the position detection circuit 200 executes a robot orientation algorithm 210 in response to detecting a fixed reference 134 in the environment 104. As disclosed herein, the reference detection circuit 202 of the exemplary navigation awareness control circuit 132 in Figure 2 can identify a fixed reference 134 and decode values ​​stored by the fixed reference 134, which may include coordinates corresponding to the location 134 of the fixed reference 134 in the environment 104. Based on the data decoded from the fixed reference 134, the position detection circuit executes a robot orientation algorithm 210 to determine navigation data 212 (e.g., position, direction of travel) to orient the robot 102 to the environment. For example, the position detection circuit 200 can derive the position of the robot 102 in the environment 104 by using the data decoded from one of the fixed references 134 to map the robot 102 to the location of the fixed reference 134 in the environment 104.

[0042] The position detection circuit 200 verifies or updates the navigation data 212 over time based on additional data received from, for example, navigation sensors 120 and / or image sensors 122, detection of another fixed criterion 134, etc. The robot control circuit 114 of the exemplary autonomous robot 102 in Figure 1 can communicate with the position detection circuit 200 to access the navigation data 212 for use in controlling the movement of the robot 102. In some examples, the robot control circuit 114 accesses the navigation data 212 directly from the database 208.

[0043] In some examples, if the position detection circuit 200 is unable to verify or update the robot 102's navigation data 212 after a threshold time period or a threshold number of attempts, the position detection circuit 200 may cause the robot 102 to generate an output(s) to warn, for example, a user(s) in the environment 104 that the robot 102 may be operating based on inaccurate or outdated navigation data 212. In some examples, the robot 102 determines that it has lost its way when sensor data (e.g., navigation data 212, data from navigation sensors(s) 120 and / or image sensors(s) 122, LiDAR sensor data, etc.) does not match expected data for a particular location on the map. In some examples, the position detection circuit 200, in response to data from the robot control circuit 114, outputs an alert(s) indicating that the robot 102 did not reach its destination within the expected time, or is not on a designated travel path. Alerts may include, for example, the activation of lights on robot 102, the presentation of a notification on robot 102's display screen 124, and / or audio output via speaker(s) 126. In some examples, alerts may include wireless notifications received via auxiliary reference application 144 and presented via user device 106's display screen 142. As disclosed herein, when position detection circuit 200 is unable to verify or update navigation data 212, the user may instruct auxiliary reference application 144 to present auxiliary reference 140 to guide robot 102.

[0044] The reference detection circuit 202 of the exemplary navigation awareness control circuit 132 in Figure 2 analyzes image data generated by sensor(s) 122 during the operation of the robot 102 to identify navigation references (e.g., fixed reference(s) 134, auxiliary reference(s) 140) within the field of view of image sensor(s) 122. The reference detection circuit 202 executes reference detection rules(s) 214 to identify reference(s) 134, 140. For example, reference detection rules(s) 214 can instruct the reference detection circuit 202 to access the image stream generated by image sensor(s) 122 and perform image analysis to recognize reference(s) 134, 140 in the image(s). Based on a reference detection rule(s) 214, the reference detection circuit 202 may periodically access image data, such as based on user input(s) received via an auxiliary reference application 144, in response to the position detection circuit 200 determining that the navigation data 212 has not been updated over a threshold time period. The reference detection rule(s) 214 are defined based on user input and may be stored in a database 208. In some examples, the reference detection rule(s) 214 instruct the reference detection circuit 202 to periodically scan image data to identify reference(s) 134, 140, causing the position detection circuit 200 to verify or update the navigation data 212, even if the position detection circuit 200 is unaware that the navigation data 212 may be old or inaccurate.

[0045] When one of the fixed criteria(s) 134 or auxiliary criteria(s) 140 is detected, the criterion detection circuit 202 decodes the data (e.g., values) stored in the criterion. In the case of fixed criteria(s) 134, the values ​​can identify the (e.g., persistent, known) location of each fixed criterion(s) 134 within the environment 104. In the case of auxiliary criteria(s) 140, the values ​​can indicate that the criterion is intended to assist the navigation of the robot(s) 102 and that the robot(s) 102 should enter an inductive driving mode (e.g., switch from autonomous driving mode to inductive driving mode). The criterion(s) can be stored in the database 208 as criterion value data 216.

[0046] In some examples, the environment 104 includes one or more autonomous robots 102 that travel within the environment 104. When an auxiliary criterion 140 is displayed via a user device 106, the user can maneuver the user device 106 to place the auxiliary criterion 140 within the field of view of the image sensor(s) 122 of the robot(s) 102 to which the auxiliary criterion 140 is intended. However, given time, the auxiliary criterion 140 may also be, additionally or alternatively, within the field of view of at least a portion of the image sensor(s) 122 of one or more other robot(s) 102 to which the auxiliary criterion 140 is not intended. To avoid interruptions in the travel of robot(s) 102 to which the auxiliary criterion 140 is not intended, a criterion detection rule(s) 214 can define criteria for identifying the auxiliary criterion 140 as intended for a particular robot(s) 102.

[0047] For example, a reference detection rule(s) 214 may indicate that the reference detection circuit 202 should recognize the auxiliary reference 140 as intended for a particular robot 102 when the auxiliary reference 140 is located within a portion or subset of the field of view of the image sensor(s) 122, or when the auxiliary reference 140 is captured in image data at least at a minimum size (e.g., pixel) value. In some examples, a reference detection rule(s) 214 may indicate that the reference detection circuit 202 should recognize the auxiliary reference 140 as intended for a particular robot 102 when the auxiliary reference 140 is detected in image data generated by the image sensor(s) 122 over a threshold duration.

[0048] The parameters defined by the reference detection rule(s) 214 for recognizing the auxiliary criterion 140 as intended for the first robot 102 prevent, for example, other robot(s) 102 in the environment 104 from responding to the auxiliary criterion 140, which is intended for the first robot but may be captured within the peripheral vision of the other robot(s) 122's image sensor(s) 122 (e.g., at the edge of the field of view of the other robot(s) 102's image sensor(s) 122, by the other robot(s) 102's secondary camera). In some examples, when the auxiliary criterion 140 is detected, the reference detection circuit 202 periodically checks that the auxiliary criterion 140 is still within the field of view of the image sensor(s) 122 and satisfies the reference detection rule(s) 214. As disclosed herein, in some examples, if the auxiliary criterion 140 is no longer within the field of view of the image sensor(s) 122, the driving mode control circuit 204 adjusts the driving mode of the robot 102 from an inductive driving mode to an autonomous driving mode.

[0049] In some examples, when the reference detection circuit 202 detects an auxiliary reference 140, the reference detection circuit 202 increases the frequency with which it scans image data generated by the image sensor(s) 122 to identify a fixed reference 134 in the environment 104, compared to the operation of the robot 102 when there is no auxiliary reference 140. By increasing the frequency with which the position detection circuit 200 scans for the fixed reference 134, which can be used to adjust the robot 102's navigation data 212, the amount of user involvement in orienting the robot 102 via the auxiliary reference 140 can be minimized. For example, based on the increased frequency with which it scans for image data for the fixed reference 134, the driving mode control circuit 204 can switch the robot 102 from guided driving mode to autonomous driving mode more quickly in response to the detection of the fixed reference 134.

[0050] The driving mode control circuit 204 of the exemplary navigation awareness control circuit 132 in Figure 2 causes the autonomous robot 102 to operate in autonomous driving mode, guided driving mode, or free rotation mode. As described in relation to Figure 1, in autonomous driving mode, the robot 102 moves without or substantially without user commands. In guided driving mode, the robot 102 operates semi-autonomously, in that the robot 102 moves on its own and has functions such as obstacle detection and collision avoidance, but the robot's travel trajectory is defined by the user through the presentation of auxiliary references 140 to the user device 106. As described herein, the user can move the robot relative to the auxiliary references 140 (e.g., move toward the auxiliary references 140) by making gestures while holding a device on which the auxiliary references 140 are displayed. In free rotation mode, the motor control circuit 112 switches the motor(s) 108 to a neutral state, and the robot 102 can be manually moved by the user (e.g., pushed, pulled).

[0051] In response to the detection of the auxiliary reference 140 by the reference detection circuit 202, the driving mode control circuit 204 switches the robot 102 from operating in autonomous driving mode to operating in guided driving mode. For example, the driving mode control circuit 204 outputs a command to the robot control circuit 114 that instructs the robot control circuit 114 to refrain from commanding the robot 102 to move along a travel path defined by the robot control circuit 114. More precisely, as disclosed herein, in guided driving mode, the robot control circuit 114 awaits commands from the guided driving mode operation circuit 206 regarding maneuvers to be performed by the robot 102, taking into account the position of the auxiliary reference 140 relative to the robot 102.

[0052] The driving mode control circuit 204 executes driving mode rules 218 to control the switching of driving modes. The driving mode rules 218 are defined by user input and can be stored in the database 208. In some examples, based on the driving mode rules 218, the driving mode control circuit 204 waits for the robot 102 to switch to the inductive driving mode until the reference detection circuit 202 recognizes the auxiliary reference 140 over a threshold time period. In some examples, the driving mode control circuit 204 causes the robot 102 to generate an output (e.g., audio output, visual output) indicating that the robot 102 is switching driving modes.

[0053] The driving mode rule(s) 218 ​​can also define criteria for switching from an inductive driving mode to another driving mode, such as an autonomous driving mode. The driving mode rule(s) 218 ​​can indicate that the robot 102 should switch from an inductive driving mode to an autonomous driving mode in response to the position detection circuit 200 confirming that the navigation data 212 has been updated. The driving mode rule(s) 218 ​​can also indicate that the robot 102 should switch from an inductive driving mode to an autonomous driving mode in response to the reference detection circuit 202 identifying a fixed reference 134 while the robot 102 is in an inductive driving mode. In some examples, based on the driving mode rule(s) 218, the driving mode control circuit 204 waits for a threshold amount of time before switching the robot 102 from guided driving mode to autonomous driving mode in response to the reference detection circuit 202 identifying a fixed reference 134 (for example, to verify that the position detection circuit 200 has successfully reoriented the robot 102 by updating the navigation data 212 using the fixed reference 134).

[0054] In some examples, the driving mode rule(s) 218 ​​instructs the driving mode control circuit 204 to switch the robot 102 from guided driving mode to another mode (e.g., autonomous driving mode, free rotation mode) in response to, for example, the auxiliary criterion 140 no longer being detected by the criterion detection circuit 202. For example, the user can provide input to the auxiliary criterion application 144 to stop presenting the auxiliary criterion 140. In response to the auxiliary criterion 140 no longer being displayed, the driving mode rule(s) 218 ​​may indicate, for example, that the robot 102 should switch to autonomous driving mode and resume autonomous operation (based on the expectation that, for example, when the auxiliary criterion 140 no longer is displayed, the robot 102 is guided to a fixed criterion 134, has updated its navigation data, and has re-determined the robot's position). In some examples, the driving mode rule(s) 218 ​​indicates that the robot 102 should switch from guided driving mode to autonomous driving mode when the auxiliary criterion 140 is no longer displayed, the criterion detection circuit 202 detects the fixed criterion 134, and / or when the position detection circuit 200 confirms that the navigation data 212 has been updated.

[0055] In some examples, the driving mode control circuit 204 switches the robot 102 from an inductive driving mode to another mode in response to user input in the user device 106 and / or the robot 102. In some examples, an auxiliary reference application 144 can enable the user to switch the robot 102 from an inductive driving mode to an autonomous driving mode or a free-rotation mode based on user input(s) in the user device 106. The ability to control the driving mode of the robot 102 via the auxiliary reference application 144 (or, for example, via the display screen 124 on the robot 102) can be based, for example, on user permission or setting. For example, a more experienced user may be granted permission to control the driving mode of the robot 102 using the auxiliary reference application 144. In such examples, the driving mode control circuit 204 adjusts the driving mode of the robot 102 based on the auxiliary reference application 144 or on user input(s) received on the robot 102.

[0056] In some examples, the user commands the robot 102 to switch from guided driving mode to free rotation mode. For example, the user may use an auxiliary reference 140 to switch the robot 102 from autonomous driving mode to guided driving mode in order to orient the robot 102 from an area in environment 104 that includes, for example, a narrow space, an obstacle, etc., which may prevent the user from accessing or easily accessing the robot 102. The driving mode control circuit 204 switches the robot 102 to guided driving mode in response to the detection of the auxiliary reference 140 by the reference detection circuit 202. As disclosed herein, the guided driving mode operation circuit 206 may command the robot 102 to move based on the position of the auxiliary reference 140 relative to the robot 102. When the robot 102 is in a location in environment 104 that the user can access the robot 102, the user may provide an input indicating that the robot 102 should switch to free rotation mode, for example, via an auxiliary reference application 144. In response to user input, the operating mode control circuit 204 commands the robot control circuit 114 to operate the robot 102 in free rotation mode. The user can manually move the robot 102, for example, to a fixed reference point in the environment 104, causing the position detection circuit 200 to realign the orientation of the robot 102.

[0057] In some examples, after the robot 102 has been operated via guided driving mode to reposition the robot 102 for purposes other than reorienting the robot 102, the user manually moves the robot 102 to reposition it. For example, if the robot 102 is trapped in a location but the navigation data 212 remains accurate, the user can use guided driving mode to maneuver the robot 102 out of that location and then manually move the robot 102 to reposition the robot 102 within the environment 104 without the position detection circuit 200 updating the navigation data 212. For example, the robot's travel path may be temporarily blocked by materials moved into the travel path (e.g., a passage) by another robot or another operator. In this case, the robot 102 may be manually moved while in free rotation mode to allow another action to be performed or the obstruction to be removed before the robot 102 can proceed to, for example, autonomous movement. In some examples, the robot 102 may be moved to a temporary storage area, such as a shipping crate, while in free rotation mode. In some examples, robot 102 may be positioned in a rapidly changing warehouse area, such as near a loading dock where pallet fields are being shipped or received. In this example, the free rotation mode may be used to reposition the robot away from unmapped or remapped areas.

[0058] When robot 102 is operating in guided driving mode, the guided driving mode operation circuit 206 of the exemplary navigation awareness control circuit 132 in Figure 2 generates commands to move robot 102 based on the position of auxiliary reference 140. The guided driving mode operation circuit 206 executes guided driving mode operation rules 220 to determine the movement of robot 102 while operating in guided driving mode. The guided driving mode operation rules 220 are defined based on user input and may be stored in the database 208. The guided driving mode operation circuit 206 communicates with the robot control circuit 114 to move robot 102 based on the commands generated by the guided driving mode operation circuit 206.

[0059] In some examples, the guided driving mode control circuit 206 determines the maneuvers that the robot 102 should perform to keep the auxiliary reference 140 within a defined portion of the field of view of the image sensor(s) 122 (for example, centering it within the field of view of the image sensor(s) 122). In some examples, the guided driving mode control circuit 206 determines the maneuvers that the robot 102 should perform to keep the auxiliary reference 140 at a certain distance x (for example, a fixed distance x or within a certain range of x). The guided driving mode control circuit 206 outputs commands to move the robot 102 forward / backward, left / right, etc., to keep the auxiliary reference 140 within a defined portion of the field of view of the image sensor(s) 122, within a fixed distance of the auxiliary reference 140, etc. The robot control circuit 114 executes the commands from the guided driving mode control circuit 206 to cause the robot 102 to perform the movement. As a result, the user can guide the robot 102 to a specific location in the environment 104 by having the user device 106 within the field of view of the image sensor(s) 122, thereby moving the robot 102 in a direction corresponding to the position of the auxiliary reference 140 (for example, toward the auxiliary reference 140). The user can, for example, walk while viewing the auxiliary reference 140 via the user device 106 and move the robot 102 based on commands generated by the guided driving mode operating circuit 206 (for example, to make it follow the auxiliary reference 140).

[0060] In some examples, the user moves the robot 102 in a specific direction by manipulating the user device 106 while the auxiliary reference 140 is presented on the display screen 142, thereby changing the position of the auxiliary reference 140 relative to the robot 102. For example, the user can perform gestures such as moving the user device 106 to the left, or raising or lowering the user device 106, in order to orient the robot 102 toward, for example, a fixed reference 134 in the environment 104. The guided operation mode operating circuit 206 moves the robot 102 toward the position of the auxiliary reference 140 defined by the user gesture(s). The guided operation mode operating circuit 206 can perform gesture recognition based on images captured by the user's image sensor(s) 122 as the user performs gestures to orient the robot 102. The guided operation mode operating rules(s) 220 may include image analysis algorithms, gesture recognition algorithms, and the like. The guided driving mode operating circuit 206 executes an algorithm to associate user gestures (multiple) with different robot maneuvers (e.g., forward / backward, left / right). Based on gesture recognition analysis, the guided driving mode operating circuit 206 determines how the robot 102 should move and outputs the corresponding command to be implemented by the robot control circuit 114. As a result, the robot control circuit 114 moves the robot 102 in a direction corresponding to the position of the auxiliary reference 140 relative to the robot 102.

[0061] When robot 102 is in guided driving mode, the safety driving control circuit 118 (Figure 1) remains active to detect obstacles and provide collision avoidance. Thus, in some examples, commands generated by the guided driving mode operating circuit 206 are modified by the safety driving control circuit 118 if, for example, the robot 102 would collide with another object in the environment 104 (e.g., another robot, inventory, a user in the warehouse) if it were moving in the direction determined by the guided driving mode operating circuit 206. Thus, guided driving mode effectively disables the autonomous driving trajectory of robot 102, although in some examples, the robot control circuit 114 may allow robot 102 to move autonomously for safety reasons.

[0062] In some examples, the navigation awareness control circuit 132 includes means for detecting position. For example, means for detecting position may be implemented by a position detection circuit 200. In some examples, the navigation awareness control circuit 132 includes means for detecting a reference. For example, means for detecting a reference may be implemented by a reference detection circuit 202. In some examples, the navigation awareness control circuit 132 includes means for controlling the driving mode. For example, means for controlling the driving mode may be implemented by a driving mode control circuit 204. In some examples, the navigation awareness control circuit 132 includes means for operating an inductive driving mode. For example, means for operating an inductive driving mode may be implemented by an inductive driving mode operating circuit 206. It should be understood that one, some, or all of the following may be instantiated using code executed by a microprocessor (such as that described below with reference to Figure 4): the navigation awareness control circuit 132, means for detecting position, position detection circuit 200, means for detecting a reference, reference detection circuit 202, means for controlling the driving mode, driving mode control circuit 204, means for operating the inductive driving mode, and / or the inductive driving mode operating circuit 206. Alternatively, they may be instantiated by any other combination of hardware, software, and / or firmware. It should also be understood that a single processor may be used to instantiate one or all of the different ones. An exemplary form of implementing the navigation awareness control circuit 132 of Figure 1 is illustrated in Figure 2, but one or more of the elements, processes, and / or devices illustrated in Figure 2 may be combined, divided, rearranged, omitted, removed, and / or implemented in any other way.

[0063] Figure 3 shows a flowchart representing exemplary machine-readable instructions that may be executed to configure a processor circuit to implement the navigation awareness control circuit 132 of Figure 2. The machine-readable instructions may be one or more executable programs or parts(s) of executable programs for execution by a processor circuit, such as the processor circuit 412 shown in the exemplary processor platform 400 described below in relation to Figure 4. The program may be embodied as software stored in one or more non-temporary computer-readable storage media, such as a compact disk (CD), floppy disk, volatile memory (e.g., any type of random-access memory (RAM)), or non-volatile memory (e.g., EEPROM®, FLASH® memory), associated with a processor circuit located within one or more hardware devices; however, the entire program and / or parts thereof may, alternatively, be executed by one or more hardware devices other than the processor circuit and / or embodied in firmware or dedicated hardware. The machine-readable instructions may be distributed across multiple hardware devices and / or executed by two or more hardware devices (e.g., server and client hardware devices). Furthermore, while the exemplary program is described with reference to the flowchart illustrated in Figure 3, many other methods for implementing the exemplary navigation awareness control circuit 132 may be used as alternatives. The machine-readable instructions described herein may be stored in any conventional format, such as compiled code or as an executable file. As described above, the exemplary operation of Figure 1 may be implemented using executable instructions (e.g., computer and / or machine-readable instructions) stored in one or more non-temporary computers and / or machine-readable media.

[0064] Figure 3 is a flowchart representing exemplary machine-readable instructions and / or exemplary actions 300 that can be executed and / or instantiated by a processor circuit to cause an autonomous robot (e.g., autonomous robot 102 in Figure 1) to generate navigation data to orient the robot within an environment (e.g., environment 104 in Figure 1). The machine-readable instructions and / or actions 300 in Figure 3 begin in block 302, and robot 102 is operating in autonomous driving mode (e.g., based on instructions from robot control circuit 114 in Figure 1).

[0065] In block 304, the reference detection circuit 202 of the exemplary navigation awareness control circuit 132 in Figure 2 determines, for example, whether an auxiliary reference 140 presented via the user device 106 has been identified as an intended reference for the robot 102. The reference detection circuit 202 detects the auxiliary reference 140 by analyzing image data output by the robot 102's image sensor(s) 122 and confirms that the auxiliary reference 140 is intended for the robot 102 based on the reference detection rule(s) 214.

[0066] In some examples, the reference detection circuit 202 is prompted to look for an auxiliary reference 140 in response to the position detection circuit 200 of the navigation awareness control circuit 132 indicating that the robot's navigation data 212 has not been updated within a threshold time period. In some examples, the reference detection circuit 202 periodically scans image data for the auxiliary reference 140 without a command from the position detection circuit 200 (and / or without the robot prior determination or recognition that it has lost its way). The reference detection circuit 202 decodes the data encoded in the auxiliary reference 140 to identify, for example, a command for the robot 102 to follow the auxiliary reference 140.

[0067] When the reference detection circuit 202 identifies and decodes the auxiliary reference 140, in block 306, the driving mode control circuit 204 generates a command to switch the robot 102 from autonomous driving mode to guided driving mode or semi-autonomous driving mode. The driving mode control circuit 204 communicates with the robot control circuit 114 to put the robot 102 into a guided driving mode in which the robot 102 moves on its own (for example, based on a command from the robot control circuit 114), but the robot 102's travel trajectory is defined by the user.

[0068] In block 308, the guided driving mode operation circuit 206 moves the robot 102 based on the position of the auxiliary reference 140 relative to the robot 102. For example, based on the guided driving mode operation rule(s) 220, the guided driving mode operation circuit 206 generates a command to move the robot 102 so that the auxiliary reference 140 is maintained at a specific position (for example, in the center of the field of view) relative to the field of view of the image sensor(s) 122, within a threshold distance of the robot 102.

[0069] In block 310, the guided driving mode operation circuit 206 determines whether the position of the auxiliary reference 140 relative to the robot 102 has changed. For example, the user can manipulate the position of the auxiliary reference 140 by making a gesture with the hand holding the user device 106 while the auxiliary reference 140 is presented on the display screen 142 of the user device 106 (for example, by moving the user device 106 to the right or left). In block 312, the guided driving mode operation circuit 206 moves the robot 102 in response to the change(s) of the position of the auxiliary reference 140. For example, the guided driving mode operation circuit 206 interprets the gesture made by the user based on the guided driving mode operation rule(s) 220. Based on the gesture(es) (e.g., move left, move right), the guided driving mode operation circuit 206 determines the operation that the robot 102 should perform and outputs a command to the robot control circuit 114 to perform the movement.

[0070] In block 314, the reference detection circuit 202 determines whether a fixed reference 134 in the environment 104 has been identified. For example, the reference detection circuit 202 identifies a fixed reference 134 by analyzing image data generated by the image sensor(s) 122 while the robot 102 is operating in guided driving mode. The reference detection circuit 202 decodes a value associated with the fixed reference 134, which may include the coordinates of the location of the fixed reference 134 in the environment 104.

[0071] If the reference detection circuit 202 does not identify the fixed reference 134, in block 316, the reference detection circuit 202 determines whether the auxiliary reference 140 is still being presented. If the reference detection circuit 202 continues to detect the auxiliary reference 140, control proceeds to block 308, where the robot 102 continues to move based on the position of the auxiliary reference 140.

[0072] When the reference detection circuit 202 identifies and decodes the fixed reference 134, the position detection circuit 200 executes a robot orientation algorithm 210 in block 318 based on the values ​​associated with the fixed reference 134 (e.g., position data). The position detection circuit 200 updates the robot 102's navigation data 212 to align (realign) the robot 102's orientation with respect to the environment 104.

[0073] In block 320, the driving mode control circuit 204 determines whether the robot 102 should switch from guided driving mode to autonomous driving mode. The driving mode control circuit 204 may determine that the robot should re-enter autonomous driving mode in response to, for example, the reference detection circuit 202 no longer detecting the auxiliary reference 140 (block 316), or the position detection circuit 200 updating the navigation data 212 (block 318). If the driving mode control circuit 204 determines that the robot 102 should switch from guided driving mode to autonomous driving mode (for example, based on the driving mode rule(s) 218, the auxiliary reference application 144, or user input(s) in the robot 102), in block 322, the driving mode control circuit 204 generates a command to operate the robot 102 in autonomous driving mode (for example, via the robot control circuit 114).

[0074] If the driving mode control circuit 204 determines that the robot 102 should not re-enter autonomous driving mode, in block 324, the driving mode control circuit 204 determines, for example, based on user input, whether the robot 102 should switch from guided driving mode to free rotation mode. In block 326, the driving mode control circuit 204 communicates with the robot control circuit 114 to put the robot 102's motor(s) 108 into a neutral state to allow the robot 102 to be moved manually. The robot 102 remains in free rotation mode until the driving mode control circuit 204 determines that the robot 102 should operate in autonomous driving mode (blocks 328, 330), or until the reference detection circuit 202 detects an auxiliary reference 140 (for example, another auxiliary reference 140) (block 332).

[0075] In some examples, the robot 102 remains in induction mode as long as the reference detection circuit 202 detects the auxiliary reference 140, and continues to move based on the position of the auxiliary reference 140 (blocks 324, 332). In response to continued identification of the auxiliary reference 140, or the identification of a different auxiliary reference 140, the driving mode control circuit 204 puts the robot 102 into induction mode (block 334), and control returns to block 308. The exemplary instruction 300 in Figure 3 terminates when the robot 102 is powered off (blocks 336, 338).

[0076] Figure 4 is a block diagram of an exemplary processor platform 400 structured to execute and / or instantiate the machine-readable instructions and / or actions of Figure 3 in order to implement the navigation awareness control circuit 132 of Figure 3. The processor platform 400 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a smartphone, a tablet computer, etc.), a personal digital assistant (PDA), an internet device, a headset (e.g., an augmented reality (AR) headset, a virtual reality (VR) headset, etc.), or other wearable device, or any other type of computing device.

[0077] The illustrated example processor platform 400 includes a processor circuit 412. The illustrated example processor circuit 412 is hardware. For example, the processor circuit 412 may be implemented by one or more integrated circuits, logic circuits, FPGAs, microprocessors, CPUs, GPUs, DSPs, and / or microcontrollers from any desired family or manufacturer. The processor circuit 412 may be implemented by one or more semiconductor-based (e.g., silicon-based) devices. In this example, the processor circuit 412 implements an exemplary position detection circuit 200, an exemplary reference detection circuit 202, an exemplary driving mode control circuit 204, and an exemplary inductive driving mode operating circuit 206.

[0078] The illustrated example processor circuit 412 includes local memory 413 (e.g., cache, registers, etc.). The illustrated example processor circuit 412 communicates with main memory, which includes volatile memory 414 and non-volatile memory 416, via bus 418. The volatile memory 414 may be implemented by synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS® dynamic random access memory (RDRAM®), and / or any other type of RAM device. The non-volatile memory 416 may be implemented by flash memory and / or any other desired type of memory device. Access to the main memory 414, 416 in the illustrated example is controlled by a memory controller 417.

[0079] The illustrated example processor platform 400 also includes an interface circuit 420. The interface circuit 420 may be implemented in hardware according to any type of interface standard, such as an Ethernet® interface, a Universal Serial Bus (USB) interface, a Bluetooth® interface, a Near Field Communication (NFC) interface, a Peripheral Component Interconnect (PCI) interface, and / or a Peripheral Component Interconnect Express (PCIe) interface. In the illustrated example, one or more input devices 422 are connected to the interface circuit 420. The input device(s) 422 allows the user to input data and / or commands to the processor circuit 412. The input device(s) 422 may be implemented, for example, by an audio sensor, a microphone, a camera (still or video), a keyboard, buttons, a mouse, a touchscreen, a trackpad, a trackball, an IsoPoint device, and / or a speech recognition system.

[0080] One or more output devices 424 are also connected to the interface circuit 420 of the illustrated example. The output devices 424 may be implemented by, for example, display devices (e.g., LED displays, LCD displays, etc.), haptic output devices, printers, and / or speakers. Thus, the interface circuit 420 of the illustrated example typically includes graphics driver circuitry such as a graphics driver card, a graphics driver chip, and / or a GPU.

[0081] The interface circuit 420 in the illustrated example also includes a network interface to enable data exchange with an external machine (e.g., any type of computing device) via a network 426. The processor platform 400 in the illustrated example also includes one or more mass storage devices 428 for storing software and / or data, such as an HDD, memory device, or SSD. The machine-readable instructions 432, which can be implemented by the machine-readable instructions in Figure 3, can be stored in the mass storage devices 428, volatile memory 414, non-volatile memory 416, and / or removable non-temporary computer-readable storage media such as a CD or DVD. It should be understood that the machine-readable instructions 432 can be downloaded via the interface circuit 420 and stored within the processor platform 400.

[0082] From the above, it will be understood that exemplary systems, methods, apparatus, and articles for facilitating the guidance of an autonomous robot using visual criteria have been disclosed. Examples disclosed herein use unfixed visual criteria (e.g., virtual criteria) presented via a user device to switch an autonomous robot from autonomous driving mode to guided driving mode, enabling the robot to be steered by a user in an environment. In examples disclosed herein, the autonomous robot responds by steering along a travel path defined by unfixed visual criteria (e.g., user gestures that change the position of virtual criteria relative to the robot). In guided driving mode, the exemplary autonomous vehicle disclosed herein can identify fixed visual criteria in the environment to update the robot's navigation data and thus orient the robot to the environment. Examples disclosed herein provide efficient control of the robot using auxiliary criteria, for example, in cases where the robot's ability to normally update navigation data using fixed criteria or techniques such as dead reckoning is interrupted. The exemplary robot resumes autonomous driving using accurate navigation data.

Claims

1. Image sensor and, Memory and Machine-readable instructions and Processor circuit and An autonomous robot comprising, the processor circuit, the autonomous robot The autonomous robot detects a first criterion based on image data corresponding to the output of the image sensor collected while the autonomous robot is in a first operating mode, wherein the first operating mode corresponds to an autonomous operating mode. In response to the detection of the first reference, the autonomous robot is switched from the first operating mode to the second operating mode, and the autonomous robot is moved in the second operating mode in a direction corresponding to the position of the first reference relative to the autonomous robot. The process involves generating navigation data for the autonomous robot, where the navigation data indicates the position of the autonomous robot within the environment. Switching the autonomous robot from the second operating mode to the first operating mode, and after switching from the second operating mode, driving the autonomous robot in the first operating mode based on the navigation data, An autonomous robot that executes the machine-readable instructions configured to perform the following actions.

2. The aforementioned processor circuit is The autonomous robot detects a second criterion based on image data corresponding to the output of the image sensor collected while the autonomous robot is in the second operating mode, To generate the navigation data of the autonomous robot based on the second criterion, An autonomous robot according to claim 1, configured to perform the following:

3. The autonomous robot according to claim 2, wherein the first criterion is that the autonomous robot is movable relative to the environment in which it is placed, and the second criterion is that it is fixed relative to the environment.

4. The autonomous robot according to any one of claims 1 to 3, wherein the first criterion is a hypothetical criterion.

5. The autonomous robot according to any one of claims 1 to 4, wherein the processor circuit is configured to switch the autonomous robot from a first operating mode to a second operating mode in response to the detection of a first criterion over a threshold time period.

6. The autonomous robot according to any one of claims 1 to 5, wherein the processor circuit is configured to switch the autonomous robot from a first operating mode to a second operating mode in response to the detection of a first reference within a portion of the field of view of the image sensor.

7. The autonomous robot according to any one of claims 1 to 6, wherein the processor circuit is configured to move the autonomous robot to maintain the first reference position with respect to the field of view of the image sensor while the autonomous robot is in the second operating mode.

8. The aforementioned processor circuit is While the autonomous robot is in the second operating mode, it detects a gesture based on the image data, wherein the gesture is associated with the first reference movement. The decision to control the vehicle is based on the aforementioned gesture, Moving the autonomous robot based on the aforementioned control, An autonomous robot according to any one of claims 1 to 7, configured to perform the following:

9. Memory and Machine-readable instructions and Processor circuit and A device comprising, the processor circuit, Identifying a first marker in image data corresponding to the output of the autonomous vehicle's image sensor, collected while the autonomous vehicle is in autonomous driving mode, Based on the data associated with the first marker, the autonomous vehicle is switched from the autonomous driving mode to the second driving mode, Based on the first position of the first marker relative to the autonomous vehicle, the autonomous vehicle is made to perform the first maneuver in the second driving mode, Determining the location information of the autonomous vehicle within the environment, Switching the autonomous vehicle from the second driving mode to the autonomous driving mode, and after switching from the second driving mode, operating the autonomous vehicle in the autonomous driving mode based on the location information, A device that executes the machine-readable instruction to perform the following actions.

10. The aforementioned processor circuit is Based on the image data, it is detected that the first marker has changed from the first position to the second position relative to the autonomous vehicle. Based on the second position of the first marker, the autonomous vehicle is made to perform a second maneuver in the second driving mode, The apparatus according to claim 9, which performs the following.

11. The aforementioned processor circuit is The autonomous vehicle is in the second driving mode when it detects a second marker based on image data corresponding to the output of the image sensor collected, Based on the data associated with the second marker, the location information of the autonomous vehicle is generated, The apparatus according to claim 9 or 10, which performs the following.

12. The apparatus according to claim 11, wherein the processor circuit switches the autonomous vehicle from the second driving mode to the autonomous driving mode in response to the detection of the second marker.

13. The apparatus according to any one of claims 9 to 12, wherein the processor circuit causes the autonomous vehicle to perform the first maneuver to maintain a first distance from the first marker.

14. The apparatus according to any one of claims 9 to 13, wherein the first marker is a virtual marker.

15. A non-temporary machine-readable storage medium, which is used in a processor circuit, A first criterion is detected based on image data corresponding to the output of an image sensor collected while the autonomous robot is in a first operating mode, wherein the first operating mode corresponds to an autonomous operating mode. In response to the detection of the first reference, the autonomous robot is switched from the first operating mode to the second operating mode, and the autonomous robot is moved in the second operating mode in a direction corresponding to the position of the first reference relative to the autonomous robot. The process involves generating navigation data for the autonomous robot, where the navigation data indicates the position of the autonomous robot within the environment. Switching the autonomous robot from the second operating mode to the first operating mode, and after switching from the second operating mode, driving the autonomous robot in the first operating mode based on the navigation data, A non-temporary machine-readable storage medium equipped with instructions to perform at least the following actions.

16. The instruction is given to the processor circuit, The autonomous robot detects a second criterion based on image data corresponding to the output of the image sensor collected while the autonomous robot is in the second operating mode, To generate the navigation data of the autonomous robot based on the second criterion, A non-temporary machine-readable storage medium according to claim 15, which causes the following to occur.

17. The non-temporary machine-readable storage medium according to claim 15 or 16, wherein the instruction causes the processor circuit to switch the autonomous robot from the first operating mode to the second operating mode in response to the detection of the first criterion over a threshold time period.

18. The non-temporary machine-readable storage medium according to any one of claims 15 to 17, wherein the instruction causes the processor circuit to switch the autonomous robot from the first operating mode to the second operating mode in response to the detection of the first criterion within a portion of the field of view of the image sensor.

19. The non-temporary machine-readable storage medium according to any one of claims 15 to 18, wherein the instruction causes the processor circuit to move the autonomous robot so as to maintain the first reference position with respect to the field of view of the image sensor while the autonomous robot is in the second operating mode.

20. The instruction is given to the processor circuit, While the autonomous robot is in the second operating mode, it detects a gesture based on the image data, wherein the gesture is associated with the first reference movement. The decision to control the vehicle is based on the aforementioned gesture, Moving the autonomous robot based on the aforementioned control, A non-temporary machine-readable storage medium according to any one of claims 15 to 19, which causes the following to occur.

Images