System and method for automated path planning and travel for autonomous vehicles

Abstract

Processes for operating an autonomous vehicle (100) and corresponding systems and computer-readable mediums. A process (300) includes performing a path planning process (320) using vehicle position and perception information (352) to produce an iterative reference path (322), and performing a position control function (330) using the iterative reference path (322) and the vehicle position and perception information (352) to produce velocity and steering commands (332). The process includes controlling a vehicle propulsion system (340) to move the autonomous vehicle (100) along the iterative reference path (322) based on the velocity and steering commands (332). The process includes determining (350) updated vehicle position and perception information (352) that includes identification of any objects in or near the iterative reference path (322). The process includes using the updated vehicle position and perception information (352) for subsequent iterations of the path planning process (320).

Description

SYSTEM AND METHOD FOR AUTOMATED PATH PLANNING AND TRAVEL FOR AUTONOMOUS VEHICLESTECHNICAL FIELD

[0001] The present disclosure is directed, in general, to automated real-time and collision-free path planning processes and travel for autonomous vehicles including ground-travel vehicles such as robots and air-travel vehicles such as drones, including manned or unmanned vehicles.BACKGROUND OF THE DISCLOSURE

[0002] Quality inspection and control is essential for manufacturing industry to ensure higher productivity, safe operations and prevent catastrophic failures. An increasing number of companies today are using aerial drone technology for visual inspection, because it is a cost-conscious and effective way to inspect at heights and inaccessible areas for preventive maintenance planning, optimized production, and reduced downtime.

[0003] Current drone inspection processes are designed to inspect stationary objects (e.g., a building or a turbine housing) or slow-moving objects at far distances (e.g. parts on a conveyer belt). Such processes often require shutting down the industrial processes to allow the drone to be close enough to capture highly accurate inspection results. These processes are not possible during normal industrial activity because of the risk of collisions between the drone and objects in the environment.

[0004] Ground-based systems face similar problems. Automated Guided Vehicles (AGVs) that follow pre-defined tracks, and Autonomous Mobile Robots (AMRs) that navigate through the “shop” floor using maps and sensors, are the major technology drivers of the recent growth in robotics for logistics and other areas. Traditionally, an AGV / AMR (together, and including similar ground-based unmanned vehicles, referred toherein as a “ground vehicle”) is operated in a relatively static manner, repeating its predefined path, and if there are obstacles, the ground vehicle can stop and wait, or replan locally, on a very limited basis, to avoid obstacle. Essentially there is a global planner for an optimal path, and local obstacle avoidance to react to obstacles. Again, only limited deviation from the programmed path or adaption to obstacles is possible.

[0005] For both air- based and ground-based vehicles, improved systems are desirable.SUMMARY OF THE DISCLOSURE

[0006] Various disclosed embodiments include processes for operating an autonomous vehicle and corresponding systems and computer-readable mediums. A disclosed process includes performing a path planning process using vehicle position and perception information to produce an iterative reference path. The process includes performing a position control function using the iterative reference path and the vehicle position and perception information to produce velocity and steering commands. The process includes controlling a vehicle propulsion system to move the vehicle along the iterative reference path based on the velocity and steering commands. The process includes determining updated vehicle position and perception information that includes identification of any objects in or near the iterative reference path. The process includes using the updated vehicle position and perception information for subsequent iterations of the path planning process.

[0007] In various embodiments, the path planning process uses a Convex Feasible Set Algorithm. In various embodiments, the path planning process uses a convex optimization code generator. In various embodiments, the velocity and steering commands include manipulated variables including velocity commands and controlled variables including vehicle position and orientation. In various embodiments, the velocity and steering commands include thrust force information, torque information, and reference yaw angles. In various embodiments, the vehicle position and perception information includes simultaneous localization and mapping information. In various embodiments, the vehicle position and perception information includes at least one of position information, orientation information, velocity information, posture information, inertial measurement unit data, visual data, or simultaneous localization and mapping SLAM information. In various embodiments, the path planning process also uses visual data. In various embodiments, the autonomous vehicle is a ground-based robotic vehicle. In various embodiments, the autonomous vehicle is an aerial drone.

[0008] The foregoing has outlined rather broadly the features and technical advantages of the present disclosure so that those skilled in the art may better understand the detaileddescription that follows. Additional features and advantages of the disclosure will be described hereinafter that form the subject of the claims. Those skilled in the art will appreciate that they may readily use the conception and the specific embodiment disclosed as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. Those skilled in the art will also realize that such equivalent constructions do not depart from the spirit and scope of the disclosure in its broadest form.

[0009] Before undertaking the DETAILED DESCRIPTION below, it may be advantageous to set forth definitions of certain words or phrases used throughout this patent document: the terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation; the term “or” is inclusive, meaning and / or; the phrases “associated with” and “associated therewith,” as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like; and the term “controller” means any device, system or part thereof that controls at least one operation, whether such a device is implemented in hardware, firmware, software or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, and those of ordinary skill in the art will understand that such definitions apply in many, if not most, instances to prior as well as future uses of such defined words and phrases. While some terms may include a wide variety of embodiments, the appended claims may expressly limit these terms to specific embodiments.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] For a more complete understanding of the present disclosure, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, wherein like numbers designate like objects, and in which:

[0011] FIG. 1 illustrates an example of a vehicle in accordance with disclosed embodiments;

[0012] FIG. 2 illustrates an environment 200 in accordance with disclosed embodiments;

[0013] FIG. 3 illustrates an example of a process for operating an autonomous vehicle using real-time path planning in accordance with disclosed embodiments; and

[0014] FIG. 4 illustrates an example of a process for operating an autonomous vehicle using real-time path planning, in accordance with disclosed embodiments.DETAILED DESCRIPTION

[0015] FIGURES 1 through 4, discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged device. The numerous innovative teachings of the present application will be described with reference to exemplary non-limiting embodiments.

[0016] As described above, for both aerial drones and for ground-based AGVs and AMRs, current path-planning techniques only allow for limited adaptions from the preplanned travel path, and so cannot address the technological need to operate in a dynamic environment.

[0017] For drone usage, the features and status of many industrial processes under normal operation are quite different from the status when shut down for inspection using conventional techniques. For example, turbine sealing can only be detected when the turbine is running. Another example is that tiny cracks may only be able to be detected when the turbine blades are rotating at nominal speeds. In general, it is expensive in terms of both time and production to shut down and restart industrial processes. In other cases, it is often not possible to shut down a process just for drone inspection. Therefore, the limitations of infield drone inspection remain one of the major barriers to general- purpose industrial digital transformation.

[0018] Similar problems exist for current ground-based systems. Increasingly, there is a need to operate ground vehicles in highly dynamic environments where ground vehicle has to operate with dynamic paths, avoid dynamic obstacles, and even collaborate with humans. To accomplish this, the ground vehicle must be more aware of its environment. Further, like in the case of an aerial drone, a static global path is not effective for both safe and effective operations.

[0019] An unmanned vehicle can follow a predefined path, meaning the path will not change during runtime. While the vehicle runs on the path or follows the predesignated trajectory, it can employ different obstacle detection and avoidance techniques.

[0020] Some vehicles can employ a control or sensor-based safety zone. In these cases, the vehicle is equipped with two-dimensional (2D) or three-dimensional (3D) scanning devices, such as laser scanners. The laser scanner can measure the distance of the object to the vehicle in 2D or 3D space, and whenever an object is detected in the area of the configured shape surrounding the vehicle, the vehicle can react by slowing down, stopping, or reversing.

[0021] Some vehicles can employ perception or vision-based safety techniques. For example, artificial intelligence or computer-vision based machine perception techniques can be used for obstacle avoidance or to ensure safety. In some cases, for example, an RGBD camera mounted on the front of vehicle can “see” the environment, and various techniques can be used to process the image to detect humans, objects, or other obstacles. In various cases, convolutional neural network object detection processes can be used, or 3D vision techniques, such as a depth camera, LiDAR camera, or sphere vision, can be used to detect the object in the 3D space in front the vehicle, then the vehicle can react accordingly.

[0022] For both aerial drones and ground vehicles, safety is always the most important requirement in environments where humans may be present. Humans necessarily are unpredictable, so conventional path-planning techniques can be both ineffective and unsafe in many environments.

[0023] In conventional systems, the path-planning aspects and safety-response aspects are decoupled. The vehicle follows the predetermined path, and if it encounters obstacles, it can stop or make very-limited movements to address the obstacle. This approach is no longer sufficient in a more complex dynamic environment, such as a factory, commercial space, or other area in which people or other obstacles may cross or obstruct the preplanned ground or air path. Where the safe path of travel is constantly changing, the vehicle must be able to replan on the fly to adapt to the new environment.

[0024] Further, in an environment with complex structures and obstacles, the shape of the constraints can become very complex and nonlinear. Combined with dynamic moving obstacles, vehicle on-board path planning can be complex and computationally infeasible for real-time controls.

[0025] Disclosed embodiments address these issues with a novel real-time and collision- free path planning technique for vehicle control, whether an aerial vehicle or a ground vehicle. The disclosed processes enable the system to dynamically replan the travel path while considering both the optimal operation and safety.

[0026] The disclosed embedded vehicle control can also be integrated into a process control system. Disclosed processes can synchronize control between the vehicle and the process system to plan / replan the vehicle paths or trajectories according to the status of the process system to accomplish the required travel or inspection. In this way, the vehicle can dynamically plan its path to react to a safety related scenario, obstacle, or other issue quickly and efficiently. Disclosed processes can operate on vehicles with limited on-board processing capability.

[0027] FIG. 1 illustrates an example of a vehicle 100 in accordance with disclosed embodiments, which can be, for example any unmanned ground, water, or air vehicle, including a droid, robot, boat, plane, or submarine, and is intended to include the moving portion of a partially-stationary device, such as a robotic assembly arm or manipulator on an otherwise stationary robot. The block diagram of FIG. 1 is intended to illustrate major components of the vehicle 100 that may act as part of the processes described herein but is not intended to illustrate every feature or component of such vehicles, which may vary by implementation.

[0028] In this figure, vehicle 100 includes a power source 140, which can be any battery or other source of electrical power. Power source 140 is connected to controller 110, propulsion system 150, and sensor system 120, as well as other components of vehicle 100.

[0029] Controller 110 can be any processor, controller, application-specific integrated circuit (ASIC), or other device configured to perform processes designated to be performed by the controller herein, and specifically can be an embedded controller.

[0030] Propulsion system 150 can be any appropriate propulsion system for the vehicle (or portion of the vehicle) and can include related components for steering, moving, and otherwise changing the physical position, attitude, and direction of the vehicle 100. For example, in various embodiments, propulsion system 150 can include motors, wheels, tracks, and other components necessary for moving and steering a ground vehicle along the ground, rudders, propellers, and other components necessary for moving and steering a water vehicle on the surface of or through the water, and any rudders, elevators, propellers, motors and actuators for moving a robotic arm in 3D space, or other components necessary for moving and steering an aerial vehicle through the air. Propulsion system 150, in various embodiment, can control the vehicle in two or three dimensions of travel, and can control such factors as direction, trajectory, acceleration, braking, thrust, yaw, attitude, pitch, trim, turning, and others.

[0031] Sensor system 120 can be any appropriate system or device for sensing and / or identifying obstacles, such as a laser scanner, proximity sensor, a depth camera, a LiDAR camera, sphere vision, or others. Sensor system 120 can include any other components needed to sense and / or identify obstacles or other physical objects, including people, in the vicinity of the vehicle 100, including multiple sensors, processors, memories, or other components.

[0032] Controller 100 can be connected to or integrated with a memory 112 that can store any necessary code or data for performing processes as described herein.

[0033] Controller 100 can also be connected to communicate with a wireless communications device 130, such as a WiFi device, Bluetooth device, or other device. Controller can communicate via wireless communications device 130 with a control system as described herein.

[0034] FIG. 2 illustrates an environment 200 in accordance with disclosed embodiments. The example of FIG. 2 is used to illustrate that a vehicle 210, which can be any type of unmanned vehicle as described above for vehicle 100, can travel in an environment 200 along a path 240 represented by the arrows. Path 240 can be partially pre-planned, but, in accordance with disclosed embodiments, path 240 is dynamically generated as vehicle 100 travels in environment 200 in the vicinity of objects 230a, 230b, 230c, and 23 Od (collectively, objects 230). Objects 230 can each be any stationary or moving object or structure, including humans, in the environment 200, and of the objects 230 can be an object or structure to be inspected by the vehicle 210.

[0035] Vehicle 210 can also communicate, wired or wirelessly, with control system 220. Control system 220 can be implemented in any computer system, using standard processors, memories, and other components, configured and programmed to perform functions as described herein.

[0036] Environment 200 can be any 3D space, and path 240 can be any path in 2D or 3D space in the environment 200. For example, environment 200 can represent a ground topology for a ground vehicle 210 operating at ground level for path 240. In other cases, environment 200 can represent a 3D airspace for an aerial vehicle 210 operating in 3D airspace on a 3D path 240. In other cases, environment 200 can represent a 3D underwater space for a submarine vehicle 210 operating underwater on a 3D path 240.

[0037] Disclosed embodiments include an online dynamic path planning system for unmanned vehicles in a complex dynamic environment that balances a tradeoff between computational efficiency and optimality, and at the same time ensures that the vehicle travels safely in the environment.

[0038] One significant feature of various embodiments is a safety control process that can quickly respond to significant changes in the environment and plan a new path / trajectory on the fly that enables the vehicle to complete its objective safely and without collisions or unnecessary delay. In other systems, safety control functions are relatively slow and operate on a separate logical layer than motion control functions, which can be relatively fast. Disclosed embodiments can effectively combine safetycontrol functions and motion control functions into a single-layer process, as described herein, which operates as fast as other motion-control (only) processes. Disclosed embodiments provide the technical advantage of performing single-layer control with a very fast cycle time, such as 1kHz in some embodiments.

[0039] A path planning problem as addressed by disclosed embodiments is in general formulated as a nonlinear optimization problem. One significant challenge of such as solution is that the constraints are highly nonlinear, which requires computationally- expensive nonlinear solvers to generate the path.

[0040] To deal with nonlinear inequality constraints for safety in dynamic environments, disclosed embodiment can use an iterative convex optimization path-planning method for path planning and control when performing transportation tasks in a dynamic environment. Disclosed processes can generate reference paths with high precision and a large safety boundary in real time.

[0041] A disclosed planning and control processes leverages the Convex Feasible Set Algorithm (CFS), known to those of skill in the art, to efficiently search the non-convex feasible space for solutions by solving a sequence of convex optimizations constrained in the convex feasible sets. The CFS algorithm can tackle the inequality constraints and linearize the task equality constraints. Then the original nonlinear and non-convex optimization problem is converted into a quadratic program and can be solved in real time. In addition, the CFS algorithm is guaranteed to converge to local optima by iteratively minimizes the cost function in the convex feasible set and using the intermediate solutions as the reference points to generate the next convex feasible sets.

[0042] Disclosed embodiments can use the CVXGEN software tool to automatically generate embedded and real time control codes running on vehicle controllers. The CVXGEN software is a well-known software tool for code generation for convex optimization, available at time of filing under commercial license or under a free academic, non-commercial license. The generated codes are simple, flat, library-free C code that compiles into a reliable, high-speed solver for the convex optimization problems. The generated code is almost branch free, and so has highly predictableruntime behavior. The combination of regularization (both static and dynamic) and iterative refinement in the search direction computation yields reliable performance, even with poor quality data.

[0043] FIG. 3 illustrates an example of a process for operating an autonomous vehicle 100 using real-time path planning, in accordance with disclosed embodiments, that is particularly useful with ground vehicles such as AGVs. The process of FIG. 3 can be viewed as a functional diagram of a control architecture in accordance with disclosed embodiments.

[0044] At 310, a control system (such as control system 220) produces process states 312 and may produce an initial travel path 314. The control system sends to the vehicle, and the vehicle receives, the process states 312 and any initial travel path 314.

[0045] At 320, the vehicle, using the controller 110, receives the process states, any initial travel path 314, and vehicle position and perception information 352 (generated at 350, below), and performs a path planning process 320 based on the process states, any initial travel path, and vehicle position and perception information to produce an iterative reference path. In subsequent iterations, the previous iterative reference path 322 can be used in place of the initial travel path. The path planning process can use a Convex Feasible Set Algorithm and can use a convex optimization code generator, such as the CVXGEN software tool, to automatically generate embedded codes to run on the controller 110. The iterative reference path 322 avoids any obstacles identified in the vehicle position and perception information. While specific embodiments can use the Convex Feasible Set Algorithm and a convex optimization code generator such as the CVXGEN software tool as disclosed herein, other embodiments can use any lightweight solver for the functions described herein.

[0046] At 330, the vehicle, using the controller 110, the system performs a position control function using the iterative reference path 322 and vehicle position and perception information 352 (generated at 350, below) to produce velocity and steering commands 332. The velocity and steering commands can be in the form of commands for programmable drive controls, analog or digital signals for controlling a propulsionsystem, or otherwise. The velocity and steering commands direct the vehicle on a path that avoids the obstacles and directs the vehicle on a safe path. The velocity and steering commands can include manipulated variables such as velocity commands and controlled variables such as vehicle position and orientation.

[0047] At 340, the vehicle, using the controller 110 and based on the velocity and steering commands 332, controls the vehicle propulsion system to move the vehicle along the iterative reference path.

[0048] At 350, as the vehicle moves, the vehicle, using the controller 110 and based on sensor inputs, determines the vehicle’s updated vehicle position and perception information 352 that includes identification of any objects in or near its path. The vehicle position and perception information is returned to the path planning process (at 320) and the position control function (at 330) so the process can continue to perform real-time path planning and vehicle control. The vehicle position and perception information can include any of the sensor information described herein including simultaneous localization and mapping (SLAM) information, vehicle position information, vehicle environment information, and others.

[0049] Multiple approaches can be used with vision-guided drone control. One technique is to use robotic SLAM techniques. In this case, a drone relies on its own onboard sensors to autonomously navigate in unknown, hostile, and GPS-denied environments (such as ruined or semi-demolished buildings). Visual information and inertial measurement unit (IMU) measurements can be used to robustly obtain the pose estimation of a drone within a local map of the environment. In general, SLAM is computationally expensive. This implies that computational time of building the environment map is often too high to control a drone which has fast and unstable dynamics. Therefore, most of SLAM-based methods are limited to small workspaces that have definite image features.

[0050] Another approach is direct sensor-based control, such as using image-based visual servo control. Image-based visual servoing for a drone can be applied to a wider variety of problems, such as holding station, path following, obstacle avoidance, and landing. Potential problems of these approaches are the stability (e.g., the image Jocobian matrixof the process becomes singular) and convergence (e.g., the process gets trapped into local minimum) issues. Stability issues may cause a drone to lose control (e.g., to crash), and the convergence issue may control the drone to fly to a wrong place.

[0051] Disclosed embodiments can build an infield drone inspection system that synchronizes with the process control system to perform high-accuracy inspection at close distances to industrial processes. A real-time control algorithm is used to generate drone trajectory on the fly with a safety guarantee. One key advantage is that the disclosed process is able to perform drone inspection in dynamic environments. For example, a process as disclosed herein can quickly respond to significant changes in the vehicle’s environment and plan new path / trajectory on the fly to perform the inspection tasks.

[0052] Real-time path planning for arial vehicles can also performed using an iterative convex optimization motion planning method for trajectory planning and control when performing infield inspection and other tasks. The disclosed processes can generate reference trajectory with high tracking precision and large safety distance in real time.

[0053] As in the process above, the disclosed aerial vehicle trajectory control process leverages the CFS algorithm to efficiently search the non-convex feasible space for solutions by solving a sequence of convex optimizations constrained in the convex feasible sets. The CFS algorithm can tackle the inequality constraints and linearize the task equality constraints. Then the original nonlinear and non-convex optimization problem is converted into a quadratic program and can be solved in real time. In addition, the CFS algorithm is guaranteed to converge to local optima be iteratively minimizing the cost function in the convex feasible set and use the intermediate solutions as the reference points to generate the next convex feasible sets.

[0054] In various embodiments, the CVXGEN software tool is used to automatically generate embedded and real time control codes running on the vehicle controller. The combination of regularization (both static and dynamic) and iterative refinement in the search direction computation yields reliable performance, even with poor quality data.

[0055] FIG. 4 illustrates an example of a process for operating an autonomous vehicle using real-time path planning, in accordance with disclosed embodiments, that is particularly useful with aerial vehicles such as drones. The process of FIG. 4 can be viewed as a functional diagram of a control architecture in accordance with disclosed embodiments.

[0056] At 410, a control system, for example from a plant control system, produces process states and initial visual data from an image-based visual servoing process and may produce an initial travel path. The control system sends to the vehicle, and the vehicle receives, the process states, the initial visual data, and any initial travel path.

[0057] At 420, the vehicle, using the controller, receives the process states, the initial visual data, any initial travel path, and vehicle position and perception information (generated at 350, below), and performs a path planning process based on the process states, any initial travel path, the visual data, and vehicle position and perception information to produce an iterative reference path. In subsequent iterations, the previous iterative reference path can be used in place of the initial travel path. The path planning process can use a Convex Feasible Set Algorithm and can use a convex optimization code generator, such as the CVXGEN software tool, to automatically generate embedded codes to run on the controller. The iterative reference path avoids any obstacles identified in the vehicle position and perception information. The path planning process can also produce reference yaw angle information to be used at 440.

[0058] At 430, the vehicle, using the controller, the system performs a position control function using the iterative reference path and vehicle position and perception information (generated at 450, below) to produce velocity and steering commands. The velocity and steering commands can be in the form of commands for programmable drive controls, analog or digital signals for controlling a propulsion system, or otherwise. The velocity and steering commands direct the vehicle on a path that avoids the obstacles and directs the vehicle on a safe path. The velocity and steering commands can include thrust force information, torque information, reference yaw angles, and other information.

[0059] At 440, the vehicle, using the controller and based on the vehicle position and steering commands, and in some cases based on a reference yaw angle and IMU data, controls the vehicle propulsion system to move the vehicle through the air along the iterative reference path.

[0060] At 450, as the vehicle moves, the vehicle, using the controller and based on sensor inputs, determines the vehicle’s vehicle position and perception information of any objects in or near its path. The vehicle position and perception information is returned to the path planning process (at 420) and the position control function (at 430) so the process can continue to perform real-time path planning and vehicle control. The vehicle position and perception information can also include IMU data that is returned to the vehicle propulsion system (at 440). The vehicle position and perception information can include any of the sensor information described herein, position information, orientation information, velocity information, posture information, IMU data, visual data, simultaneous localization and mapping (SUAM) information, and others.

[0061] FIG. 5 illustrates an example of a process for operating an autonomous vehicle for safe inspection of a physical facility, such as a plant, using real-time path planning, in accordance with disclosed embodiments, which is particularly useful with aerial vehicles such as drones. The process of FIG. 5 can be viewed as a functional diagram of a control architecture in accordance with disclosed embodiments.

[0062] At 510, a control system signal processing function can produce goal positions based on inputs that can include sensor measurements such as visual information in red- green-blue (RGB) or red-green-blue- depth (RGBD) format, light detection and ranging (Uidar) information, and others.

[0063] At 512, a plant control system can produce process states and other information about the physical facility, such as its physical layout, the equipment present and its operating and physical state, the equipment or area to be inspected, the presence of humans or other dynamic elements, and other information. The control system sends to the vehicle, and the vehicle receives, the process states, the initial visual data, and any initial travel path.

[0064] At 520, the vehicle, using the controller, receives the process states, the goal position, and potentially other information such as initial visual data, any initial travel path, and any vehicle position and perception information, and performs a path planning process based on at least the process states and goal position. The path planning process can use a Convex Feasible Set Algorithm and can use a convex optimization code generator, such as the CVXGEN software tool, to automatically generate embedded codes to run on the controller. The path planning process can produce an iterative reference path / trajectory. The iterative reference path avoids any obstacles identified in the vehicle position and perception information. The path planning process can also produce reference yaw angle information to be used at 540.

[0065] At 530, the vehicle, using the controller, the system performs a safety position control function using the iterative reference path and vehicle state information (generated at 550, below) to produce velocity and steering commands. The velocity and steering commands can be in the form of commands for programmable drive controls, analog or digital signals for controlling a propulsion system, or otherwise. The velocity and steering commands direct the vehicle on a path that avoids the obstacles and directs the vehicle on a safe path. The velocity and steering commands can include thrust force information, torque information, reference yaw angles, desired roll / pitch angles, and other information. In particular, in this and other embodiments, the safety position control function (and position control functions in other embodiments) can use generated control codes and does not require a solver function as described at 520. For this reason, the iterative safety position control function (and position control functions in other embodiments) can react very quickly to potential safety issues and avoid any safety lapses.

[0066] At 540, the vehicle, using the controller and based on the vehicle position and steering commands, and in some cases based on the desired roll / pitch angles and IMU data, controls the vehicle propulsion system to move the vehicle through the air along the iterative reference path. In doing so, the vehicle can produce torque information.

[0067] At 550, as the vehicle moves, the vehicle, using the controller and based on sensor inputs, performs a tracking control function to determine the vehicle state information, which can include vehicle position and perception information of any objects in or near its path. This function can also be based on information such as the velocity and steering commands (from 530) and the torque information (from 540). The vehicle position and perception information is returned to the path planning process (at 520) and the position control function (at 530) so the process can continue to perform real-time path planning and vehicle control. The vehicle state information can also include IMU data that is returned to the vehicle propulsion system (at 540). The vehicle position and perception information can include any of the sensor information described herein, position information, orientation information, velocity information, posture information, IMU data, visual data, simultaneous localization and mapping (SLAM) information, and others.

[0068] Of course, those of skill in the art will recognize that, unless specifically indicated or required by the sequence of operations, certain steps in the processes described above may be omitted, performed concurrently or sequentially, or performed in a different order.

[0069] Those skilled in the art will recognize that, for simplicity and clarity, the full structure and operation of all systems suitable for use with the present disclosure is not being depicted or described herein. Instead, only so much of a system as is unique to the present disclosure or necessary for an understanding of the present disclosure is depicted and described. The remainder of the construction and operation of the systems and devices disclosed herein may conform to any of the various current implementations and practices known in the art.

[0070] It is important to note that while the disclosure includes a description in the context of a fully functional system, those skilled in the art will appreciate that at least portions of the mechanism of the present disclosure are capable of being distributed in the form of instructions contained within a machine-usable, computer-usable, or computer- readable medium in any of a variety of forms, and that the present disclosure appliesequally regardless of the particular type of instruction or signal bearing medium or storage medium utilized to actually carry out the distribution. Examples of machine usable / readable or computer usable / readable mediums include nonvolatile, hard-coded type mediums such as read only memories (ROMs) or erasable, electrically programmable read only memories (EEPROMs), and user-recordable type mediums such as floppy disks, hard disk drives and compact disk read only memories (CD-ROMs) or digital versatile disks (DVDs).

[0071] Although an exemplary embodiment of the present disclosure has been described in detail, those skilled in the art will understand that various changes, substitutions, variations, and improvements disclosed herein may be made without departing from the spirit and scope of the disclosure in its broadest form.

[0072] None of the description in the present application should be read as implying that any particular element, step, or function is an essential element which must be included in the claim scope: the scope of patented subject matter is defined only by the allowed claims. Moreover, none of these claims are intended to invoke 35 USC §112(f) unless the exact words "means for" are followed by a participle. The use of terms such as (but not limited to) “mechanism,” “module,” “device,” “unit,” “component,” “element,” “member,” “apparatus,” “machine,” “system,” “processor,” or “controller,” within a claim is understood and intended to refer to structures known to those skilled in the relevant art, as further modified or enhanced by the features of the claims themselves, and is not intended to invoke 35 U.S.C. §112(f).

Claims

WHAT IS CLAIMED IS:

1. A process (300) for operating an autonomous vehicle (100), the process (300) performed by the autonomous vehicle (100) and comprising: performing a path planning process (320) using vehicle position and perception information (352) to produce an iterative reference path (322); performing a position control function (330) using the iterative reference path (322) and the vehicle position and perception information (352) to produce velocity and steering commands (332); controlling a vehicle propulsion system (340) to move the autonomous vehicle (100) along the iterative reference path (322) based on the velocity and steering commands (332); determining (350) updated vehicle position and perception information (352) that includes identification of any objects in or near the iterative reference path (322); and using the updated vehicle position and perception information (352) for subsequent iterations of the path planning process (320).

2. The process of claim 1, wherein the path planning process (320) uses a Convex Feasible Set Algorithm.

3. The process of claim 1, wherein the path planning process (320) uses a convex optimization code generator.

4. The process of claim 1, wherein the velocity and steering commands (332) include manipulated variables including velocity commands and controlled variables including vehicle position and orientation.

5. The process of claim 1, wherein the velocity and steering commands (332) include thrust force information, torque information, and reference yaw angles.

6. The process of claim 1, wherein the vehicle position and perception information (352) includes simultaneous localization and mapping information.

7. The process of claim 1, wherein the vehicle position and perception information (352) includes at least one of position information, orientation information, velocity information, posture information, inertial measurement unit data, visual data, or simultaneous localization and mapping SLAM information.

8. The process of claim 1, wherein the path planning process (320) also uses visual data.

9. The process of claim 1, wherein the autonomous vehicle (100) is a ground-based robotic vehicle.

10. The process of claim 1, wherein the autonomous vehicle (100) is an aerial drone.

11. An autonomous vehicle (100) comprising at least a controller (110), configured to perform the process of any of claims 1-10.

Images