Controlling speed of autonomous vehicle using virtual envelope

Abstract

An example method includes obtaining information about a path to travel by an autonomous vehicle during movement of the autonomous vehicle through an environment; and generating a virtual envelope surrounding the autonomous vehicle and having at least two dimensions greater than the two corresponding dimensions of the autonomous vehicle. A length of the virtual envelope along the path is based on at least one of (i) a predefined duration during which the autonomous vehicle may travel along the path, or (ii) a duration during which the autonomous vehicle may travel along the path without stopping. A speed of the autonomous vehicle is based on the virtual envelope.

Description

Technical Field

[0001] This specification relates in its entirety to an example system configured to generate a virtual envelope around at least a portion of an autonomous vehicle and to use the virtual envelope to control the speed of the autonomous vehicle. Background Technology

[0002] Autonomous vehicles, such as mobile robots, are configured to move within environments such as warehouses. For example, an autonomous vehicle can use a mapping of the environment to plan a path or route through it. During movement along the path, the autonomous vehicle can determine its position within the environment and use that position to control its future movement. When multiple autonomous vehicles operate in the same environment, there is a chance of collisions between them. Summary of the Invention

[0003] An example method includes: obtaining information about the path an autonomous vehicle will travel during its movement through an environment; and generating a virtual envelope surrounding the autonomous vehicle and having at least two dimensions larger than two corresponding dimensions of the autonomous vehicle. The length of the virtual envelope along the path is based on at least one of: (i) a predefined duration during which the autonomous vehicle can travel along the path, or (ii) a duration during which the autonomous vehicle can travel along the path without stopping. The speed of the autonomous vehicle is based on the virtual envelope. The example method may include one or more of the following features, individually or in combination.

[0004] The autonomous vehicle can be a first autonomous vehicle, and the path can be a first path. Generating the virtual envelope may include: identifying the intersection of the first path and a second path that the second autonomous vehicle will travel during the second autonomous vehicle's movement through the environment; determining that the first autonomous vehicle will have to stop before the intersection; and making the length of the virtual envelope based on how much time the first autonomous vehicle can travel before stopping before the intersection. The method may include determining that the travel of the second autonomous vehicle takes precedence over the travel of the first autonomous vehicle, and therefore, the first autonomous vehicle will have to stop before the intersection. The travel of the second autonomous vehicle may take precedence over the travel of the first autonomous vehicle because the second autonomous vehicle is predicted to arrive at the intersection before the first autonomous vehicle.

[0005] The autonomous vehicle can be a first autonomous vehicle, and the path can be a first path. Generating the virtual envelope may include: identifying a position within a predefined distance of a second path that the first path is to be traveled by the second autonomous vehicle during its movement through the environment; determining that the first autonomous vehicle will have to stop before that position; and setting the length of the virtual envelope based on how much time the first autonomous vehicle can travel before stopping at that position. The method may include determining that the travel of the second autonomous vehicle takes precedence over the travel of the first autonomous vehicle, and therefore, the first autonomous vehicle will have to stop before that position. The travel of the second autonomous vehicle may take precedence over the travel of the first autonomous vehicle because the second autonomous vehicle is predicted to arrive at the position before the first autonomous vehicle.

[0006] Generating the virtual envelope may include: identifying areas in which the autonomous vehicle is prohibited from entering; and making the length of the virtual envelope based on its proximity to the areas.

[0007] Generating the virtual envelope may include: identifying a region in which the autonomous vehicle has primacy; and extending the virtual envelope into the region before one or more other autonomous vehicles enter the region.

[0008] Generating the virtual envelope may include dynamically updating the shape of the virtual envelope based on at least one of the autonomous vehicle's speed or obstacles in or within a predefined distance of the path.

[0009] The at least two dimensions of the virtual envelope may include a first dimension parallel to at least a portion of the path and a second dimension perpendicular to the first dimension. Generating the virtual envelope may include at least changing the size of the first dimension. Generating the virtual envelope may include combining polygons along the path to form the shape of the virtual envelope.

[0010] The virtual envelope surrounding the autonomous vehicle may have at least three dimensions larger than the three corresponding dimensions of the autonomous vehicle.

[0011] Example storage instructions of one or more non-transitory machine-readable storage media, executable to perform operations including: obtaining information about a path to be traveled by an autonomous vehicle during its movement through an environment; and generating a virtual envelope surrounding the autonomous vehicle and having at least two dimensions larger than two corresponding dimensions of the autonomous vehicle. The length of the virtual envelope along the path may be based on at least one of: (i) a predefined duration during which the autonomous vehicle may travel along the path, or (ii) a duration during which the autonomous vehicle may travel along the path without stopping. The speed of the autonomous vehicle may be based on the virtual envelope. One or more non-transitory machine-readable storage media may store instructions executable to perform any operations associated with the methods and variations thereof described above or elsewhere herein.

[0012] An example system includes an autonomous vehicle and one or more processing devices configured to execute instructions to perform operations including: obtaining information about the path the autonomous vehicle will travel during its movement through an environment; and generating a virtual envelope surrounding the autonomous vehicle and having at least two dimensions larger than two corresponding dimensions of the autonomous vehicle. The length of the virtual envelope along the path is based on at least one of: (i) a predefined duration during which the autonomous vehicle can travel along the path, or (ii) a duration during which the autonomous vehicle can travel along the path without stopping. The autonomous vehicle is configured to use the virtual envelope to control its speed. The system may include one or more of the following features (alone or in combination).

[0013] The autonomous vehicle may be a first autonomous vehicle, and the path may be a first path. The one or more processing devices may be configured to execute instructions to perform operations including obtaining information about a second path that a second autonomous vehicle will travel during its movement through the environment. Generating the virtual envelope may include: identifying the intersection of the first path and the second path; determining that the first autonomous vehicle will have to stop before the intersection; and setting the length of the virtual envelope based on how much time the first autonomous vehicle can travel before stopping before the intersection.

[0014] The one or more processing devices may be configured to execute instructions to perform operations including: determining that the travel of the second autonomous vehicle takes precedence over the travel of the first autonomous vehicle, and therefore, the first autonomous vehicle will have to stop before the intersection. The travel of the second autonomous vehicle may take precedence over the travel of the first autonomous vehicle because the second autonomous vehicle is predicted to arrive at the intersection before the first autonomous vehicle.

[0015] The autonomous vehicle may be a first autonomous vehicle, and the path may be a first path. The one or more processing devices may be configured to execute instructions to perform operations including obtaining information about a second autonomous vehicle to travel a second path during the second autonomous vehicle's movement through the environment. Generating the virtual envelope may include: identifying a position within a predefined distance of the first path on the second path; determining that the first autonomous vehicle will have to stop before that position; and setting the length of the virtual envelope based on how much time the first autonomous vehicle can travel before stopping at that position.

[0016] The one or more processing devices may be configured to execute instructions to perform operations including: determining that the movement of the second autonomous vehicle takes precedence over the movement of the first autonomous vehicle, and therefore, the first autonomous vehicle will have to stop before the location. The movement of the second autonomous vehicle may take precedence over the movement of the first autonomous vehicle because the second autonomous vehicle is predicted to arrive at the location before the first autonomous vehicle.

[0017] Generating the virtual envelope may include: identifying areas in which the autonomous vehicle is prohibited from entering; and making the length of the virtual envelope based on its proximity to the areas.

[0018] Generating the virtual envelope may include: identifying a region in which the autonomous vehicle has primacy; and extending the virtual envelope into the region before one or more other autonomous vehicles enter the region.

[0019] Generating the virtual envelope may include dynamically updating the shape of the virtual envelope based on at least one of the autonomous vehicle's speed or obstacles in or within a predefined distance of the path.

[0020] The at least two dimensions of the virtual envelope may include a first dimension parallel to at least a portion of the path and a second dimension perpendicular to the first dimension. Generating the virtual envelope may include at least changing the size of the first dimension.

[0021] Generating the virtual envelope may include combining polygons along the path to form the shape of the virtual envelope.

[0022] The virtual envelope surrounding the autonomous vehicle may have at least three dimensions larger than the three corresponding dimensions of the autonomous vehicle.

[0023] The one or more processing devices are part of a cluster management system external to the autonomous vehicle. The one or more processing devices may be configured to execute instructions to transmit data representing the virtual envelope to the autonomous vehicle. The autonomous vehicle may include an onboard control system configured to control the speed of the autonomous vehicle based on the envelope.

[0024] The one or more processing devices may be part of the onboard control system of the autonomous device.

[0025] Any two or more of the features described in this specification (including this summary section) may be combined to form specific embodiments not specifically described herein.

[0026] The systems, processes, devices, and variations thereof, or portions thereof described herein, including autonomous vehicles, may be implemented or controlled by a computer program product comprising instructions stored on one or more non-transitory machine-readable storage media, and these instructions may be executed on one or more processing devices. The systems, processes, devices, and variations thereof, or portions thereof described herein, including autonomous vehicles, may be implemented as devices, methods, or electronic systems, or portions thereof, which may include one or more processing means and memory storing executable instructions for implementing various operations. The systems, processes, devices, and variations thereof, or portions thereof described herein, including autonomous vehicles, may be configured, for example, by design, construction, arrangement, placement, programming, operation, activation, deactivation, and / or control.

[0027] Detailed descriptions of one or more specific embodiments are set forth in the accompanying drawings and the following detailed description. Other features and advantages will become apparent from the details, the drawings, and the claims. Attached Figure Description

[0028] Figure 1 This is a side view of an exemplary autonomous vehicle, showing possible internal and external components of an exemplary control system for an autonomous vehicle.

[0029] Figure 2 This is a perspective view of an exemplary autonomous vehicle showing an example of the placement of sensors and the sensor range provided by these sensors.

[0030] Figure 3 It is a top view of an exemplary mapping map that can be used for path planning and navigation of autonomous vehicles traveling through space.

[0031] Figure 4 This is a flowchart illustrating operations that may be included in an example process for generating a virtual envelope around at least a portion of an autonomous vehicle and using the virtual envelope to control the speed of the autonomous vehicle.

[0032] Figure 5 This is a top view of an example autonomous vehicle and its corresponding virtual envelope.

[0033] Figure 6 This is a top view of an example virtual envelope made of polygons.

[0034] Figure 7A and Figure 7B This is a top view of two autonomous vehicles using virtual envelopes to avoid collisions.

[0035] Figure 8 This is a top view of another example of two autonomous vehicles avoiding collisions by using virtual envelopes.

[0036] Figure 9A and Figure 9B This is a top view of another example of two autonomous vehicles avoiding collisions by using virtual envelopes.

[0037] Figure 10 This is a top view of an example of an autonomous vehicle using a virtual envelope to retain its entry into a room.

[0038] Figure 11A and Figure 11B It is a top view that uses the virtual envelope of autonomous vehicles to prevent entry into prohibited rooms.

[0039] Similar reference numerals in different figures indicate similar elements. Detailed Implementation

[0040] This paper describes an example system configured to control the movement of one or more autonomous vehicles in an environment. The system obtains information about the paths that one or more autonomous vehicles are configured (e.g., programmed) to travel in the environment. For example, the system receives information such as the expected or planned travel paths from the autonomous vehicles in the environment and generates virtual envelopes for the autonomous vehicles. The virtual envelopes are transmitted to the autonomous vehicles and are updated as the autonomous vehicles travel.

[0041] A virtual envelope corresponds to the autonomous vehicle's path and defines the space within which the autonomous vehicle travels. The virtual envelope expands or contracts based on the presence of objects in the autonomous vehicle's path. For example, the virtual envelope contracts when an object is present in the autonomous vehicle's path and expands when no object is present. The expansion and contraction of the envelope can occur along a continuum such that the distance between the autonomous vehicle and the object corresponds to the length of the virtual envelope. The expansion and contraction of the envelope can occur dynamically, such that the closer the autonomous vehicle is to the object, the more the virtual envelope contracts. The expansion and contraction of the envelope can occur in real time as the autonomous vehicle is moving. In this respect, in some specific implementations, considering the latency associated with processing, data transmission, hardware, etc., real time may not mean that actions are simultaneous, but may include actions that occur on a continuous basis or track each other in time.

[0042] The speed of an autonomous vehicle can be controlled based on the size of its virtual envelope. An example of the size of the virtual envelope is its length in the direction of travel. For instance, the longer the virtual envelope in the direction of travel, the higher the speed of the autonomous vehicle may be. Conversely, the shorter the virtual envelope in the direction of travel, the lower the speed of the autonomous vehicle may be. This correlation between size (e.g., the length of the virtual envelope in the direction of travel) and the speed of the autonomous vehicle allows it to stop or decelerate as it approaches an object. That is, when an autonomous vehicle approaches an object, its speed may decrease due to the shortening of its virtual envelope, making it easier for the autonomous vehicle to stop before colliding with the object. Conversely, when there are no objects in the path of the autonomous vehicle, its virtual envelope can be at its maximum size, indicating that the autonomous vehicle can operate at maximum speed, thereby reducing the time it takes for the autonomous vehicle to reach its destination.

[0043] Virtual envelopes can influence the operation of an autonomous vehicle at any time during its journey. They can be particularly useful when the vehicle's visual sensors cannot detect objects. For example, if two autonomous vehicles maintain the same speed, the system can determine that they will enter the same doorway at approximately the same time. The visual sensors on each vehicle may fail to detect the approach of the other. However, the virtual envelope can control the speeds of both vehicles to avoid collisions within the doorway.

[0044] The virtual envelope can have two or more dimensions, each larger than two or more corresponding dimensions of the autonomous vehicle. This configuration of the virtual envelope can be advantageous because it allows for some deviation from the autonomous vehicle's planned travel path. More specifically, in some examples, the autonomous vehicle generates information about the path it will take in its environment. This information may be based on, for example, the autonomous vehicle's destination and a mapping of the environment available to the autonomous vehicle (e.g., stored on or programmed into the autonomous vehicle). By using a virtual envelope larger than the autonomous vehicle, particularly in dimensions perpendicular to the direction of travel (e.g., the width of the autonomous vehicle), the autonomous vehicle can slightly deviate from its travel path while still remaining within the virtual envelope. Therefore, whenever the autonomous vehicle encounters a small obstacle that it needs to avoid, the system does not need to update the autonomous vehicle's virtual envelope.

[0045] Mapping maps typically include static objects, such as boundaries and landmarks in the environment. Mapping maps can be used by autonomous vehicles for path planning. Path planning may include determining a path or route through space to a destination. After determining the preferred path, the autonomous vehicle begins moving through space along the path located on the mapping map. The autonomous vehicle's path and speed are based on a virtual envelope. During this movement, the autonomous vehicle periodically or intermittently determines its position, orientation, or both position and orientation within space. This information allows the autonomous vehicle to confirm it is on the path, determine its position on the path, and determine if route corrections are needed to reach the destination. The autonomous vehicle uses elements in space to determine its position along the path by comparing elements detected by the autonomous vehicle using one or more sensors with the expected positions of those same elements on the mapping map. This information is transmitted to the system to update the virtual envelope and then transmitted back to the autonomous vehicle.

[0046] The operations described herein related to controlling autonomous vehicles using virtual envelopes can be implemented using one or more computing systems, such as the autonomous vehicle's control system and / or swarm management system. One or more computing systems may include hardware, software, or both hardware and software to perform mapping generation, path planning, localization, and virtual envelope generation and updating. In some implementations, all or part of the autonomous vehicle's control system may be "onboard," meaning that all or part of the autonomous vehicle's control system resides on the robot itself. In some implementations, at least a portion of the autonomous vehicle's control system may be remote, meaning that all or part (at least a portion) of the autonomous vehicle's control system is not located on the autonomous vehicle itself. In some implementations, the swarm management system is located remotely from the autonomous vehicle's control system. In some implementations, the swarm management system may be considered part of the autonomous vehicle's control system. Examples of autonomous vehicle control systems and swarm management systems are described below.

[0047] A non-limiting example of an autonomous vehicle configured to operate using the virtual envelope described herein is... Figure 1 Robot 10. In this example, robot 10 is a mobile robot and is referred to as "robot 10" or "robot". Robot 10 includes a body 12 with wheels 13 to enable robot 10 to move on surfaces 14 of the environment, such as the floor of a warehouse, factory, or other location. Robot 10 includes a support area 15 configured to support the weight of an object. In this example, robot 10 can be controlled to transport an object from one location to another. Robot 10 includes various detectors (also called sensors) for detecting elements near the robot. In some examples, elements may include living objects and static objects such as inanimate objects, boundaries, or landmarks.

[0048] In this example, robot 10 includes different types of visual sensors, such as a 3D camera, a 2D camera, and a light detection and ranging (LIDAR) scanner. A 3D camera is also called an RGBD camera, where R corresponds to red, G to green, B to blue, and D to depth. The 3D camera can be configured to capture video, still images, or both. It is important to note that the robot is not limited to this configuration or the use of these specific types of sensors. For example, the robot may include a single sensor, a single type of sensor, or more than two types of sensors. See also Figure 2 The robot 10 includes a 3D camera 16 at its front 17. In this example, the front of the robot faces the direction of travel. The rear of the robot faces the terrain that the robot has already traversed.

[0049] The robot 10 also includes a Light Detection and Ranging (LIDAR) scanner 19 located at its front. In operation, the LIDAR scanner outputs a laser beam that is reflected from an object in the environment. The time difference between the incident laser beam and the reflected laser beam is used to determine the distance to the object, and thus the object's position within the environment. The laser beam scans in two dimensions (2D), so the LIDAR detection is in a plane relative to the robot.

[0050] More specifically, since the LIDAR scanner is 2D, it will detect elements in plane 20 of the space through which the robot is controlled to traverse. Since the camera is 3D, it will detect elements in 3D volume 21 of the space through which the robot is controlled to traverse. The LIDAR scanner 19 is adjacent to the 3D camera 16 and points in the same general direction as the camera. Similarly, the 3D camera 16 is adjacent to the LIDAR scanner 19 and points in the same general direction as the LIDAR scanner. For example, the LIDAR scanner may be positioned just below the 3D camera, or the 3D camera may be positioned just below the LIDAR scanner, as... Figure 2 As shown in the example. In this configuration, both the 3D camera and the LIDAR scanner are configured to view at least a portion of the same area 22 in front of the robot during movement. The front of the robot may contain multiple 3D camera / LIDAR scanner assemblies, although only one is shown. The robot 10 may also include one or more 3D camera / LIDAR scanner assemblies 23 located at its rear 24. The robot 10 may also include one or more 3D camera / LIDAR scanner assemblies (not shown) located on its sides. The robot 10 may also include one or more 3D camera / LIDAR scanner assemblies (not shown) at two or more of its corners (e.g., at two diagonally opposite corners). Each 3D camera / LIDAR scanner may be configured to view a portion of the same area.

[0051] A 2D camera can be used as a replacement for or supplement to a 3D camera on robot 10. For example, in all the instances described herein, one or more 2D cameras may replace a 3D camera. To obtain 3D data of a region, two or more 2D cameras may be pointed at the same region and capture relevant 2D data to obtain 3D data. In the examples above, one or more 2D cameras and a LiDAR scanner may be configured to view at least a portion of the same region 22 in front of the robot during travel. Similarly, the 2D camera may be located at the rear or side of the robot.

[0052] In this regard, additional or alternative sensors may be used in some specific implementations. For example, the robot may include one or more one-dimensional (single-beam) optical sensors, one or more two-dimensional (2D) (scanning) laser rangefinders, one or more 3D high-definition LiDAR sensors, one or more 3D floodlight LiDAR sensors, one or more 2D or 3D sonar sensors, and / or one or more 2D cameras. Combinations of two or more of these types of sensors may be configured to detect both 3D and 2D information in the same area on the front, back, or side of the robot.

[0053] One or more sensors can be configured to continuously detect the distance between the robot and elements nearby. This enables path planning and safe guidance of the robot around or between detected objects. As the robot moves along the path, an onboard computing system continuously receives input from the sensors. If an obstacle obstructs the robot's trajectory, the onboard computing system is configured to plan a path around the obstacle. If an obstacle is predicted to obstruct the robot's trajectory, the onboard computing system is configured to plan a path around the obstacle. This information, constituting a deviation from the planned route through the environment, can be transmitted to a computing system (such as a swarm management system). The swarm management system can then update the robot's virtual envelope (described below) and transmit the updated virtual envelope back to the robot. The robot's speed is then controlled based on the virtual envelope, as described herein.

[0054] A LIDAR scanner, a 3D camera, and / or any other sensors on the robot constitute the robot's vision system. As mentioned above, each mobile robot traversing space may include such a vision system and may contribute data (such as visual data) for updating the robot's path and thus its virtual envelope.

[0055] Exemplary control systems for robots implement operations associated with the robot, such as mapping generation, path planning, and localization. In some implementations, the control system stores spatial mappings in computer memory (“memory”). The mappings may be stored in the memory on each robot or at any location accessible to the control system and the robot. For example, the mappings may be stored at a remote computing system, such as a swarm management system. For instance, the mappings may be stored at a remote server accessible to the robot, the control system, and / or the swarm management system. In some examples, remote access may include wireless access, such as via a computer network or a direct wireless link.

[0056] refer to Figure 3 Mapping map 30 may define the boundaries of the environment or space 30 traversed by the robot, such as walls 29 and doorways 28. Mapping map may include the locations of landmarks (such as pillars, corners, windows, poles, and other distinguishable permanent and non-permanent features of the space that serve as references for the robot during localization). Mapping map may include objects within the space (such as goods or containers). Mapping map may also include measurements indicating the dimensions of the space, measurements indicating the dimensions and locations of objects, boundaries, and landmarks, measurements indicating the distances between different objects, boundaries, and landmarks, and coordinate information identifying where objects, boundaries, and landmarks are located in the space. Figure 3 In the example, the robot plans a path or route through space, which is labeled as 31.

[0057] Return to reference Figure 1 In some examples, the robot's control system 40 may include onboard components 32 configured to perform path planning and localization based on a mapping map, control the robot's speed based on a virtual envelope, and provide updates to the swarm management system when the robot's path changes. Onboard components may include, for example, one or more processing devices 34, such as one or more microcontrollers, one or more microprocessors, programmable logic such as field-programmable gate arrays (FPGAs), one or more application-specific integrated circuits (ASICs), solid-state circuits, or any suitable combination of two or more of these types of electronic components. Onboard components may include, for example, a memory 35 storing machine-executable instructions that can be executed by one or more processing devices 35 to perform all or part of the functions of the onboard components attributable to the robot and / or control system described herein.

[0058] In some implementations, onboard components of the control system may communicate with a remote computing system, such as a swarm management system 38. The swarm management system 38 is remote, meaning it is not included on the robots. Components of the swarm management system 38 may be distributed in the same geographical location or, for example, across different geographical locations. Components of the swarm management system 38 may be distributed among different robots in space. Components of the swarm management system may include, for example, one or more processing devices 41, such as one or more microcontrollers, one or more microprocessors, programmable logic such as FPGAs, one or more ASICs, solid-state circuits, or any suitable combination of two or more of these types of electronic components. Components of the swarm management system 38 may include, for example, a memory 42 storing machine-executable instructions that can be executed by one or more processing devices 41 to perform all or part of the functions described herein belonging to the swarm management system.

[0059] A swarm management system can be configured to control one or more robots (such as those described herein) within a control environment. Each of the swarm management system and the robots may include a copy of the same spatial mapping or have access to it. The swarm management system can be configured to receive updates regarding the actual position and operational status of each robot in the swarm. As described herein, the swarm management system can be configured to perform global path planning over the entire swarm and generate a virtual envelope, which it outputs to the various robots, who use this virtual envelope to control their speed.

[0060] In some implementations, control systems, robot and cluster management systems can be configured to communicate via wireless communication systems, such as local area networks (LANs) with Wi-Fi, ZigBee, or Z-wave. Other networks that can also be used for communication between control systems, robots, and sensors include, but are not limited to, LoRa, NB-IoT (Narrowband Internet of Things), and LTE (Long Term Evolution). In some implementations, control systems, robot and cluster management systems can be configured to communicate via cellular networks, such as 5G cellular networks, configured to deliver peak data rates up to 20 gigabits per second (Gbps) and average data rates exceeding 100 megabits per second (Mbps), with latency between 8 and 30 milliseconds, and using an adaptive modulation and coding scheme (MCS) to maintain a low bit error rate (BLER), for example, less than 1%.

[0061] Figure 4An example process 46 is shown, which is configured to generate a virtual envelope around at least a portion (e.g., all or part) of robot 10 and use the virtual envelope to control the robot's speed. Process 26 includes example operations 47 that can be performed on an onboard component 32 of the robot's control system, and example operations 48 that can be performed by a swarm management system 38 or other remote components of the robot's control system.

[0062] Process 46 determines (47a) the path that robot 10 will take through the environment. For example, the robot can use a localization process to know its mapping of the environment (such as...). Figure 3 The robot can also program its destination into its onboard control system. Knowing its current position 50, its destination 51, and the environmental mapping 30, the robot can determine the path 31 from its current position to its destination 51. Figure 3 In making this determination, the robot may also consider obstacles (such as objects) within its field of vision. The path may be defined by data including a list of points between the current position and the destination. At each point, the path may include information about the robot's orientation and velocity. At each point, the path may also include the estimated time for the robot to reach the next point in the sequence between its current position and the destination.

[0063] Robot 10 transmits path-representing data (47b) to the cluster management system 38. The cluster management system receives path-representing data from the robot (48a). The cluster management system also receives path-representing data from one or more (e.g., all) other robots 52, 53, 54 in the environment before, during, or after receiving data from robot 10. Figure 3The cluster management system knows the mapping map 31 of the environment and knows the planned location of each robot in the environment at any given time. Using this information, the cluster management system identifies (48b) any obstacles in the planned path of robot 10. Obstacles may include potential collisions with other robots in the environment, which are determined based on the planned paths of robot 10 and other robots in the environment. In some examples, a potential collision may include the intersection of two robot paths, where each robot is expected to be at the same point in the environment at the same time based on the robot's planned path. In some examples, a potential collision may include two robot paths within a predefined proximity to each other, such that robots traveling along those paths will be unacceptably close to each other at some point in time. For example, their paths may not intersect, but the paths may be so close that the two robots will definitely collide. The size of the robots affects the degree to which robots can approach each other and collide or not. In another example of a potential collision, the paths of two robots may not intersect, but the paths may be so close that there is a possibility that the two robots may collide. For example, one or both of the robots may deviate slightly from their planned path due to obstacles or other factors in the environment. Given these potential deviations, the probability of a robot colliding is unacceptable. For example, if the robots' paths are within 30 cm, 40 cm, 50 cm, 60 cm, or less of each other, depending on the size of the robots, the probability of a collision is unacceptable. That is, the robots are close enough to each other at some point along their paths that if they deviate from their expected paths by 5%, 10%, 15%, 20%, etc., they might collide.

[0064] For each potential collision, the fleet management system determines (48c) which of the two or more robots involved in the potential collision has priority. In this case, priority may include which robot is entitled to proceed first if two (or more) robots are expected to be in a situation where a potential collision exists. Priority may be based on any suitable factors, such as whether a robot is carrying goods, where robots carrying goods are given priority over robots not carrying goods; which robot is moving faster, where robots moving faster are given priority over robots moving slower; which robot is expected to arrive at the location first, where robots arriving first are given priority over robots arriving later; which robot's task has a higher priority, where robots with higher priority tasks are given priority over robots with lower priority tasks; which robot has a shorter deadline to reach its destination, where robots with shorter deadlines are given priority over robots with longer deadlines, and / or other factors.

[0065] The swarm management system generates a virtual envelope for robot 10 (48d) based on the presence of potential collisions and whether the robot takes precedence over one or more other robots that may be involved in potential collisions. The virtual envelope indicates the robot's velocity at a predefined future time.

[0066] Figure 5 An example virtual envelope 55 of robot 10 is shown. As shown, the virtual envelope 55 extends around robot 10 and primarily along its planned direction of travel, indicated by arrow 59, at the front 57 of the robot. In some embodiments, the virtual envelope does not extend beyond the robot's rear 60; however, in some embodiments, the virtual envelope may extend outward from the rear 60 of the robot by a shorter distance than it extends at the front. This prevents another robot from traveling prematurely after a potential collision point. The size (e.g., length) of the virtual envelope in the direction of travel indicates the speed at which the robot can travel within a predefined duration (e.g., a time period). In some embodiments, this duration is 3 seconds (s), 4 seconds, 5 seconds, 6 seconds, or longer. In some implementations, the longer the virtual envelope, the faster the robot travels within a predefined duration, because a longer envelope indicates that there is no impending stop for the robot within the predefined duration, or that any stop within the predefined duration is relatively far from the robot's current position (e.g., close to 3 seconds if the predefined duration is 3 seconds). Therefore, the length of the virtual envelope can be correlated with the robot's predefined speed. For example, if the virtual envelope has a predefined maximum length, the robot can travel in the direction of travel at a predefined maximum speed for a predefined duration. If the virtual envelope has 50% of the predefined maximum length, the robot can travel in the direction of travel at 50% of the predefined maximum speed for a period less than the predefined duration (e.g., half the predefined duration). If the virtual envelope has 25% of the predefined maximum length, the robot can travel in the direction of travel at 25% of the predefined maximum speed for a period less than the predefined duration (e.g., a quarter of the predefined duration); and so on. Although in Figure 5 In the example below, the direction of travel is linear, but in other examples described below, the direction of travel can be non-linear.

[0067] The size (e.g., length) of the virtual envelope in the direction of travel indicates the speed at which the robot can travel without stopping for a predefined duration. For example, if the fleet management system knows the location where the robot must stop, the length of the virtual envelope will be based on the duration the robot may have traveled before reaching that location. Places where the robot may be required to stop could include doorways or the front of other robots.

[0068] The virtual envelope can also extend laterally (e.g., vertically) relative to the direction of travel 59. In this example, the virtual envelope extends relative to robot 10 in the direction of arrow 62, such that the size of the virtual envelope is also based on the robot's coverage area. For example, the virtual envelope may be 5% wider than the width 64 of robot 10, 10% wider than robot 10, 15% wider than robot 10, 20% wider than robot 10, etc. In some examples, the virtual envelope may extend outward by 10 cm on each side of robot 10, 15 cm on each side of robot 10, 20 cm on each side of robot 10, 30 cm on each side of robot 10, etc. In some specific implementations, the virtual envelope may not extend beyond the width 64 of robot 10 (not shown).

[0069] The advantage of extending the virtual envelope beyond the width 64 of robot 10 is that the robot has the flexibility to replan its path without reporting to the cluster management system. For example, if the robot detects an object such as a box within its path, it can move within the virtual envelope to avoid the box without calculating a new path and sending that path back to the cluster management system. This reduces the time it takes for the robot to reach its destination and the processing power required by the cluster management system.

[0070] refer to Figure 6 The virtual envelope 65 may be composed of multiple polygons 66 that at least partially overlap. Figure 6 In the example, rounded rectangles are used to construct the virtual envelope; however, any polygon can be used.

[0071] Return to reference Figure 4 The swarm management system 38 transmits (48e) the virtual envelope to the robot. The robot's onboard control system receives (47c) the virtual envelope and stores it in memory 35. The onboard control system of robot 10 controls (47d) the robot's speed over a certain duration (e.g., a predetermined duration such as 3 seconds or the time during which the robot can travel without stopping) based on the length of the virtual envelope in the direction of travel. For example, if the virtual envelope has 50% of a predefined maximum length, the robot can travel in the direction of travel at 50% of the predefined maximum speed for a predefined duration.

[0072] During travel, the robot's vision system continues to monitor its surroundings. If a relatively small obstacle exists in the direction of travel or the robot requires minor route corrections, it can maneuver freely within the virtual envelope without altering its path (47e). This maneuverability can be termed local replanning, as the robot can replan its path within the virtual envelope based on information, for example, from its vision system. As mentioned above, this maneuverability is due to the fact that the width of the virtual envelope is greater than the width of the robot. However, if the vision system detects a large object in the robot's path, the robot's onboard control system determines (47a) a new path, and process 46 follows... Figure 4 The robot proceeds to obtain an updated virtual envelope for the new path. If no path changes and time (e.g., a predetermined time or time before stopping) has not yet elapsed (47f), the robot's onboard control system continues to use the current virtual envelope to control (47d) the robot's speed. After time has elapsed (e.g., the robot has stopped or the predetermined time has passed), the robot's onboard control system determines (47a) a new path and process 46 as shown. Figure 4 The indicated path is used to obtain the updated virtual envelope for the new path.

[0073] The following describes the... Figure 4 Process 46 is an example of using a virtual envelope to control speed.

[0074] refer to Figure 7A and Figure 7B For example, the swarm management system 38 identifies the intersection 70 of two corresponding planned paths 71a and 72a for robots 71 and 72. This intersection is a potential collision point. When two robots approach each other... Figure 7A When intersecting at angle 70 as shown, depending on which robot has priority in this situation, the virtual envelope of the robots can be expanded (e.g., extended in the direction of travel) or contracted (e.g., shortened in the direction of travel) by the swarm management system. Figure 7A In this example, virtual envelopes 71b and 72b are at their maximum values ​​because no robot is within the stopping distance or the predefined duration of reaching intersection 70. Robot 71 has priority in this example. Therefore, as both robots move, their virtual envelopes are updated by the swarm management system based on their planned paths and / or based on input from their vision systems. Because robot 70 has priority, its virtual envelope does not change—for example, it maintains its current size across intersection 70, indicating that robot 10 can continue on its path 71 at its current speed. The dashed version, or virtual envelope 71b, shows the virtual envelope at a different point in time than the solid version of virtual envelope 71b. Conversely, robot 72 has no priority. Therefore, robot 72's virtual envelope shrinks, as... Figure 7BAs shown. The dashed version, or virtual envelope 72b, illustrates the virtual envelope at a different point in time than the solid version of virtual envelope 72b. This shorter virtual envelope 72b indicates the amount of time the robot can travel before coming to a stop before the intersection point 70. Therefore, in this example, the onboard control system of robot 72 slows down robot 72 based on the magnitude of the virtual envelope 72b in the direction of travel, potentially (but not necessarily) slowing it to a final stop before the intersection 72, in order to allow robot 71 to pass through the intersection point.

[0075] exist Figure 8 In another example shown, robots 80 and 81 approach a potential collision point 79. Although their paths 80a and 81a do not intersect, robots 80 and 81 will collide if both are moving at maximum speed, as determined by the swarm management system 38. In this example, robot 80 has priority. Because robot 80 has priority, its virtual envelope 80b does not change—for example, it retains its current size across intersection point 79, thus indicating that robot 80 can continue on its path 81 at its current speed. Conversely, robot 81 has no priority. Therefore, as... Figure 8 As shown, the virtual envelope 81b of robot 81 contracts due to the fact that virtual envelope 81b is shorter than virtual envelope 80b. This shorter virtual envelope 81b indicates the amount of time robot 81 can travel before coming to a stop. Therefore, the onboard control system of robot 81 slows down its speed before intersection 79 based on the size of virtual envelope 81b, possibly until it comes to a final stop, so as to allow robot 80 to pass through intersection point 79.

[0076] Figure 9A and Figure 9B Two robots 85 and 86 are shown, which are not within each other's visual range due to walls 87 along their respective directions of travel 85b and 86b. (Reference) Figure 9A The anticipated potential collision point is in region 88. In this example, robot 85 has priority. Therefore, robot 85's virtual envelope 85a is longer, indicating that robot 85 can continue on its current path, identified by arrow 85b, at a maximum speed or a speed greater than that of robot 86. Conversely, robot 86 has no priority. Therefore, robot 86's virtual envelope 86b is shorter, indicating the amount of time robot 86 can travel before stopping. (Reference) Figure 9B As robots 85 and 86 approach the potential collision point 88, robot 85 may continue along its path 85b without decelerating, as indicated by the length of its virtual envelope 85a. Conversely, robot 86 decelerates along its path 86b and may come to a stop, as indicated by how much its virtual envelope 86a has contracted.

[0077] In some specific implementations, virtual envelopes can be used to preserve space within the environment. Figure 10 In the example, robots 90 and 91 both move into room 92 through different entrances 93a and 93b. Robot 91 is closer to entering the room than robot 90. However, robot 90 takes precedence over robot 91, as defined by the swarm management system 38. That is, robot 90 has priority in moving into and within the room compared to robot 91. Therefore, the swarm management system can expand the virtual envelope 90a of robot 90 into the interior of room 92 in the direction of travel 90b of robot 90, and contract the virtual envelope 91a of robot 91 in the direction of travel 91b of robot 91. Thus, robot 91 can slow down or stop outside room 92, while robot 90 can continue to enter the room at its maximum speed or any suitable speed, even though robot 90 is closer to entering room 92 than robot 90.

[0078] In some implementations, virtual envelopes can be used to determine which of two robots can pass through a doorway or other passage first. For example, if two robots approach a doorway, the cluster management system can expand the virtual envelope of the robot that has priority in passing through the doorway. The virtual envelope of the other robot can contract to indicate that it must slow down or stop to allow the priority robot to proceed through the doorway.

[0079] In some implementations, virtual envelopes can be used to prevent robots from entering spaces where they are prohibited. Figure 11A and Figure 11B In the example, robot 95 is moving 95b towards entrance 94a of room 94. As robot 95 approaches the entrance, its virtual envelope 95b extends from... Figure 11A The length shown shrinks to Figure 11B The indicated length is used to slow down the robot and eventually bring it to a stop. Therefore, the swarm management system prevents robot 95 from entering room 94.

[0080] The technologies and their variations described in this article are not limited to those related to... Figure 1 and Figure 2 The autonomous vehicle described herein. For example, the technologies and variations thereof described herein can be used on any suitable mobile device, such as those described in U.S. Patent Publication No. 2021 / 0349468 (published November 11, 2021). Figure 1 , Figure 2 and Figure 3The mobile robot described herein, in its entirety, is incorporated herein by reference. The contents of U.S. Patent Publication No. 2021 / 0349468, relating to the description of autonomous vehicles, are incorporated herein by reference. In other examples, the techniques described herein and their variations can be used with self-driving cars. In other examples, the techniques described herein and their variations can be used with aerial drones. In the case of aerial drones or other devices not limited to driving along surfaces, the virtual envelope described herein can have three dimensions; that is, the envelope can be formed by a plurality of rectangular cuboids arranged sequentially and in three dimensions. The operation described herein will be the same except for the extension to three dimensions.

[0081] The example autonomous vehicles and systems described herein may include control systems and / or the processes described herein may be implemented using such control systems, which consist of one or more computer systems comprising hardware or a combination of hardware and software. For example, the autonomous vehicle, control system, or both may include various controllers and / or processing devices located at different points in the system to control the operation of its components. A central computer may coordinate the operations in the various controllers or processing devices. The central computer, controllers, and processing devices may execute various software routines to achieve the control and coordination of various automated components.

[0082] The example autonomous vehicles and systems described herein may be controlled at least in part using one or more computer program products, such as one or more computer programs tangibly embodied in one or more information carriers (such as one or more non-transitory machine-readable media), for execution or control of the operation of one or more data processing devices, such as programmable processors, computers, multiple computers and / or programmable logic components.

[0083] Computer programs can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for a computing environment. Computer programs can be deployed to execute on one or more computers located at a single site or distributed across multiple sites and interconnected via a network.

[0084] Actions associated with at least a portion of the robot can be performed by one or more programmable processors that execute one or more computer programs to perform the functions described herein. At least a portion of the robot can be implemented using dedicated logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) and / or ASICs (Application-Specific Integrated Circuits).

[0085] Processors suitable for executing computer programs include, for example, general-purpose microprocessors and special-purpose microprocessors, as well as any type of digital computer, any one or more processors. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The components of a computer include one or more processors for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include (or be operatively coupled thereto to receive data from or transfer data to or to it, or both) one or more machine-readable storage media, such as mass storage devices for storing data, for example, magnetic disks, magneto-optical disks, or optical disks. Machine-readable storage media suitable for embodying computer program instructions and data include all forms of non-volatile memory, including (by way of example) semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs.

[0086] In the specification and claims provided herein, the adjectives “first,” “second,” “third,” etc., do not specify priority or order. Rather, these adjectives are used only to distinguish the nouns they modify.

[0087] Any mechanical or electrical connection described herein may include a direct physical connection or an indirect connection that includes intermediate components.

[0088] The components of the different specific embodiments described herein can be combined to form other embodiments not specifically illustrated above. Multiple components can be excluded from the structures described herein without adversely affecting their operation. Furthermore, individual components can be combined into one or more single components to perform the functions described herein.

Claims

1. A method comprising: obtaining information about a path to be traveled by an autonomous vehicle during movement of the autonomous vehicle through an environment; and generating a virtual envelope that surrounds the autonomous vehicle and has at least two dimensions that are greater than two corresponding dimensions of the autonomous vehicle, wherein a length of the virtual envelope along the path is based on at least one of (i) a predefined duration that the autonomous vehicle is able to travel along the path, or (ii) a duration that the autonomous vehicle is able to travel along the path without stopping, and wherein a speed of the autonomous vehicle is based on the virtual envelope.

2. The method of claim 1, wherein the autonomous vehicle is a first autonomous vehicle and the path is a first path; and wherein generating the virtual envelope comprises: identifying an intersection of the first path and a second path to be traveled by a second autonomous vehicle during movement of the second autonomous vehicle through the environment; determining that the first autonomous vehicle will have to stop before the intersection; and making a length of the virtual envelope based on how much time the first autonomous vehicle is able to travel before stopping before the intersection.

3. The method of claim 2, further comprising: determining that travel of the second autonomous vehicle takes precedence over travel of the first autonomous vehicle, and as a result, the first autonomous vehicle will have to stop before the intersection.

4. The method of claim 3, wherein the travel of the second autonomous vehicle takes precedence over the travel of the first autonomous vehicle because the second autonomous vehicle is predicted to reach the intersection before the first autonomous vehicle.

5. The method of claim 1, wherein the autonomous vehicle is a first autonomous vehicle and the path is a first path; and wherein generating the virtual envelope comprises: identifying a location where the first path is within a predefined distance of a second path to be traveled by a second autonomous vehicle during movement of the second autonomous vehicle through the environment; determining that the first autonomous vehicle will have to stop before the location; and making a length of the virtual envelope based on how much time the first autonomous vehicle is able to travel before stopping before the location.

6. The method of claim 5, further comprising: determining that travel of the second autonomous vehicle takes precedence over travel of the first autonomous vehicle, and as a result, the first autonomous vehicle will have to stop before the location.

7. The method of claim 6, wherein the travel of the second autonomous vehicle takes precedence over the travel of the first autonomous vehicle because the second autonomous vehicle is predicted to reach the location before the first autonomous vehicle.

8. The method of claim 1, wherein generating the virtual envelope comprises: identifying an area where entry by the autonomous vehicle is prohibited; and making a length of the virtual envelope based on proximity to the area.

9. The method of claim 1, wherein generating the virtual envelope comprises: identifying an area where the autonomous vehicle has primacy; and extending the virtual envelope into the area before one or more other autonomous vehicles enter the area.

10. The method of claim 1, wherein generating the virtual envelope includes dynamically updating a shape of the virtual envelope based on at least one of a speed of the autonomous vehicle or an obstacle in the path or within a predefined distance of the path.

11. The method of claim 1, wherein the at least two dimensions include a first dimension parallel to at least a portion of the path and a second dimension perpendicular to the first dimension.

12. The method of claim 11, wherein generating the virtual envelope includes at least changing a size of the first dimension.

13. The method of claim 11, wherein generating the virtual envelope includes: combining polygons along the path to form a shape of the virtual envelope.

14. The method of claim 1, wherein the virtual envelope around the autonomous vehicle has at least three dimensions that are larger than three corresponding dimensions of the autonomous vehicle.

15. One or more non-transitory machine-readable storage media storing instructions executable to perform operations comprising: obtaining information about a path for an autonomous vehicle to travel during movement of the autonomous vehicle through an environment; and generating a virtual envelope around the autonomous vehicle and having at least two dimensions that are larger than two corresponding dimensions of the autonomous vehicle, wherein a length of the virtual envelope along the path is based on at least one of (i) a predefined duration that the autonomous vehicle is able to travel along the path, or (ii) a duration that the autonomous vehicle is able to travel along the path without stopping, and wherein a speed of the autonomous vehicle is based on the virtual envelope.

16. A system comprising: an autonomous vehicle; and one or more processing devices configured to execute instructions to perform operations comprising: obtaining information about a path for the autonomous vehicle to travel during movement of the autonomous vehicle through an environment; and generating a virtual envelope around the autonomous vehicle and having at least two dimensions that are larger than two corresponding dimensions of the autonomous vehicle, wherein a length of the virtual envelope along the path is based on at least one of (i) a predefined duration that the autonomous vehicle is able to travel along the path, or (ii) a duration that the autonomous vehicle is able to travel along the path without stopping; wherein the autonomous vehicle is configured to use the virtual envelope to control a speed of the autonomous vehicle.

17. The system of claim 16, wherein the autonomous vehicle is a first autonomous vehicle and the path is a first path; wherein the one or more processing devices are configured to execute instructions to perform operations comprising obtaining information about a second path to be traveled by a second autonomous vehicle during movement of the second autonomous vehicle through the environment; and wherein generating the virtual envelope includes: identifying an intersection of the first path and a second path; determining that the first autonomous vehicle will have to stop before the intersection; and causing a length of the virtual envelope to be based on how much time the first autonomous vehicle is able to travel before stopping before the intersection.

18. The system of claim 16, wherein the one or more processing devices are configured to execute instructions to perform operations comprising: determining that travel of the second autonomous vehicle takes precedence over travel of the first autonomous vehicle, and as a result, the first autonomous vehicle will have to stop before the intersection.

19. The system of claim 18, wherein the travel of the second autonomous vehicle takes precedence over the travel of the first autonomous vehicle because the second autonomous vehicle is predicted to arrive at the intersection before the first autonomous vehicle.

20. The system of claim 16, wherein the autonomous vehicle is a first autonomous vehicle, and the path is a first path; wherein the one or more processing devices are configured to execute instructions to perform operations comprising obtaining information about a second path to be traveled by a second autonomous vehicle during movement of the second autonomous vehicle through the environment; and wherein generating the virtual envelope comprises: identifying a location where the first path is within a predefined distance of the second path; determining that the first autonomous vehicle will have to stop before the location; and causing a length of the virtual envelope to be based on how much time the first autonomous vehicle can travel before stopping before the location.

21. The system of claim 16, wherein the one or more processing devices are configured to execute instructions to perform operations comprising: determining that travel of the second autonomous vehicle takes precedence over travel of the first autonomous vehicle, and as a result, the first autonomous vehicle will have to stop before the location.

22. The system of claim 21, wherein the travel of the second autonomous vehicle takes precedence over the travel of the first autonomous vehicle because the second autonomous vehicle is predicted to arrive at the location before the first autonomous vehicle.

23. The system of claim 16, wherein generating the virtual envelope comprises: identifying an area where entry by the autonomous vehicle is prohibited; and causing a length of the virtual envelope to be based on proximity to the area.

24. The system of claim 16, wherein generating the virtual envelope comprises: identifying an area where the autonomous vehicle has priority; and extending the virtual envelope into the area before one or more other autonomous vehicles enter the area.

25. The system of claim 16, wherein generating the virtual envelope comprises dynamically updating a shape of the virtual envelope based on at least one of a speed of the autonomous vehicle or an obstacle in the path or within a predefined distance of the path.

26. The system of claim 16, wherein the at least two dimensions comprise a first dimension parallel to at least a portion of the path and a second dimension perpendicular to the first dimension.

27. The system of claim 26, wherein generating the virtual envelope comprises changing at least a size of the first dimension.

28. The system of claim 26, wherein generating the virtual envelope comprises: combining polygons along the path to form a shape of the virtual envelope.

29. The system of claim 16, wherein the virtual envelope around the autonomous vehicle has at least three dimensions that are larger than three corresponding dimensions of the autonomous vehicle.

30. The system of claim 16, wherein the one or more processing devices are part of a fleet management system that is external to the autonomous device; wherein the one or more processing devices are configured to execute instructions to communicate data representing the virtual enclosure to the autonomous device; and wherein the autonomous vehicle comprises an onboard control system configured to control a speed of the autonomous vehicle based on the envelope.

31. The system of claim 16, wherein the one or more processing devices are part of an onboard control system of the autonomous device.

Images