Control system and method for autonomous driving or remote control of vehicle platform

Abstract

The present invention discloses a control system and method for a vehicle platform, the system comprising: an on-board controller and an off-board controller that work together to provide autonomous driving navigation, perceived obstacle detection and avoidance in a field or similar area where the vehicle is deployed; and a user interface for user / vehicle interaction and control.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 894,685, filed on August 30, 2019, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention relates to a control system and method for a vehicle platform. In particular, the present invention relates to a control system and method for an autonomous driving or remotely controlled vehicle platform that can be used for agricultural applications. Background Art

[0004] In some prior art agricultural vehicles may be large platforms, particularly because they must include a cab with associated manual controls. Due to the large size of the vehicle, these types of vehicles may be associated with wheels that over-compact the soil, which can be detrimental to favorable growing conditions for plants or crops. These agricultural vehicles may also lack an integrated control system, which may result in large swaths of treated crops, spray patterns that are not parallel to one another, or patterns that do not track a target path plan (e.g., linear row segments, contour row segments, curved row segments, or spiral row segments, etc.). Additionally, some vehicles may not provide a system and method for controlling the vehicle for automated or remote operation. Therefore, it would be beneficial to develop a system and method for controlling a vehicle platform. Summary of the invention

[0005] According to one embodiment of the present invention, a system for controlling a vehicle platform is provided, which includes an on-board controller and an off-board controller, which work together to provide the vehicle platform with automatic driving or remote control navigation capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1A to Figure 1B is a block diagram of a control system for autonomous or remotely controlled vehicles.

[0007] Figure 2 is a schematic diagram of one embodiment of an autonomous or remotely controlled vehicle (eg, a water truck). DETAILED DESCRIPTION

[0008] According to one embodiment of the present invention, a system 100 for controlling a vehicle platform 11 is provided, which includes an onboard controller 101 and an off-board controller 102. In one embodiment, the system 100 also includes a data processor 150, a data storage device 153, and a data port 151 connected to a data bus 152. The data processor 150, the data storage device 153, and the data port 151 can communicate with each other via the data bus 152. The off-board control 102 can be connected to the data port 151 via a wireless link 154, such as a pair of wireless transceivers. The user interface 155 associated with the off-board controller 102 may include a display, a keypad, a touch screen display, a keyboard, a pointing device, or other devices that support an operator to remotely interact with the vehicle 11.

[0009] In one embodiment, data processor 150 includes a microcontroller, a microprocessor, a programmable logic array, a logic device, an arithmetic logic unit, a digital signal processor, an application specific integrated circuit, or other electronic device for inputting, outputting, processing, or manipulating data. Data storage device 153 may include electronic memory, non-volatile random access memory, magnetic storage device, disk drive, optical storage device, optical disk drive, or other suitable storage device or medium. Data storage device 153 may support electronic modules, store software instructions, or support data modules. For example, data port 151 may include a transceiver, a combination of a transceiver and a buffer memory, or a transmitter and a receiver.

[0010] like Figure 1A As shown in , an embodiment of the vehicle controller 101 includes an interface control module 103 , a posture estimation module 104 , a perception module 105 , a path navigation module 106 , an obstacle avoidance module 107 and a remote operation module 108 . Figure 1B The off-board controller 102 depicted in EMBODIMENT 1 includes a mission configuration module 110 , a static mission module 111 , a machine state module 112 , and an override control module 113 .

[0011] System 100 may be used to control a vehicle, for example Figure 2The vehicle platform 11 shown. In the embodiment disclosed in this specification, the vehicle platform 11 may include an automatic robotic machine that can apply fertilizers, herbicides, pesticides, seeds or other crop care inputs before crop planting and emergence and after crop emergence. The vehicle platform 11 or vehicle can be configured as a light vehicle, which is very suitable for reducing soil compaction during field operations because it eliminates the weight of the operator and the operator's cab. For many configurations, the robotic machine can reduce the weight of the climate control system, the infotainment system, and the various operator control devices associated with the cab. In some configurations, the track assembly of the vehicle can provide a ground pressure of less than 5 pounds per square inch (PSI), or in some configurations, it can even provide a ground pressure as low as 3PSI. The reduced ground pressure allows the vehicle platform 11 to operate under conditions where the soil structure is generally not allowed to enter the field because of concerns about soil compaction or damage. The control system 100 is not limited to agricultural vehicles and can be used to control other vehicle platforms, such as construction vehicles, forest vehicles, and lawn vehicles.

[0012] exist Figure 2 In the example shown in , the vehicle platform 11 includes a center body 10 having a first side 58 (e.g., front) and a second side 60 (e.g., rear) spaced a fixed, known distance from the first side 58. A plurality of adjustable legs 12 extend downwardly from the center body 10. Each adjustable leg 12 has a corresponding leg actuator (e.g., a linear actuator), an electric motor with a screw, or a hydraulic cylinder with an electric-hydraulic interface to adjust the corresponding vertical height of each adjustable leg 12. For example, the leg actuator can be used to maintain a horizontal attitude of the vehicle based on measurements from any of the following: a motion sensor, a first position determination receiver 62, or a second position determination receiver 64.

[0013] Each adjustable leg 12 terminates in a rotatable track assembly 68 (e.g., rotatable relative to the vertical axis 30 and rotatable relative to a horizontal axis associated with a drive track or belt) or a rotatable wheel. If the track assembly 68 is rotatable relative to the vertical axis to steer the vehicle, an actuator (e.g., a hydraulic actuator) or an electric motor is used at or above two or more track assemblies 68, such that the stator (of the electric motor or actuator) is associated with the upper portion of the leg 12 and the rotor (of the electric motor or actuator) is connected to the lower portion of the leg 12 or the corresponding tracked assembly. In some configurations, the track assembly 68 can be referred to as a track wheel, a continuous track tank track. Each track assembly 68 can be independently driven or rotated by a drive unit 70 (e.g., an electric motor or a hydraulic motor). In addition, in some configurations, the vehicle can be steered (e.g., steered differently) by applying different rotational energy (e.g., different rotational speeds) to the ground relative to different track units.

[0014] In one embodiment, a first position determination receiver 62 is associated with or near a first side 58 of a top portion of the vehicle 11; a second position determination receiver 64 is associated with or near a second side 60 of a top portion of the vehicle 11. The first position determination receiver 62 alone or in conjunction with the second position determination receiver 64 is configured to determine a position or angular orientation of the vehicle or center body 10 on a horizontal plane of the earth relative to a reference angle (e.g., magnetic north). For example, the first position determination receiver 62, the second position determination receiver 64, or both may include a satellite navigation receiver, such as a global navigation satellite system receiver (GNSS) or a global positioning system (GPS) receiver, wherein the satellite navigation receiver may have a wireless receiver for receiving correction signals, such as differential correction signals or precise point positioning (PPP) signals. In one embodiment, the position determination receivers (62, 64) may be supplemented with additional or supplemental sensors, such as dead reckoning sensors, odometers, gyroscopes, accelerometers, tilt, roll and yaw sensors, and / or inertial measurement units (IMUs) to facilitate determining a position or angular orientation in conjunction with the position determination receivers (62, 64). In some embodiments, additional sensors may be used, such as imaging systems, at least one stereo camera, radar, lidar, and other systems that acquire data about the environment surrounding the vehicle 11 .

[0015] exist Figure 2 In the illustrated embodiment, the center body 10 covers a tank 36 (e.g., a generally cylindrical tank) for containing a crop input, a fluid to be pumped, or a pressurized fluid (e.g., anhydrous ammonia). The crop may have an optional input port 51 with a cap that can be removed to fill the tank 36. The crop input or fluid refers to fertilizers, fungicides, insecticides, pesticides, herbicides, nitrogen, potassium, phosphorus, minerals, nutrients, soil amendments, chemicals, or other agronomic treatments of plants, seeds, roots, or soil. The tank 36 can contain a pressurized fluid, or a fluid to be pumped by a pump 39. In one configuration, the center body 10 has a central hollow area 40 associated with its underside, and wherein the central hollow area 40 is adapted to removably receive the tank 36 for containing a fluid, such as a crop input to be sprayed or a seed to be planted.

[0016] A plurality of supply lines 42 and conduits 61 are arranged to supply pumped or pressurized fluid from a tank 36 or a pump associated with the tank 36 to the respective nozzle assemblies. A manifold 44 connected to the vertical supply lines 42 distributes, switches or directs the pumped or pressurized fluid to one or more nozzles 48 of each nozzle assembly or to the entire nozzle assembly. The manifold 44 may include one or more valves, such as electromechanical valves or solenoid valves, for controlling the flow of the pumped or pressurized fluid to the respective nozzles 48 or nozzle assemblies. For example, in one configuration, each nozzle assembly includes a pair of nozzles 48 facing in opposite directions.

[0017] also, Figure 2 1 and 10 , the pivotable arm assembly 14 is shown, which includes a first arm 24, a second arm 26, a vertical rod 25, a collar 27, a support beam 21, and an optional frame for supporting the nozzle assembly. The support beam 21 extends in a lateral direction 65 that is substantially perpendicular to a longitudinal axis 67 of the central body 10 in a forward travel direction 66 of the vehicle platform 11. The arm actuator 22 is arranged to control the lateral position of the support beam 21. Thus, the nozzle assembly can be adjusted laterally without moving the vehicle 11 laterally to align the nozzle assembly with the plant row during the vehicle 11 passing adjacent to or through (e.g., passing parallel to) the field.

[0018] The onboard controller 101 can receive position data, motion data, and attitude data (e.g., yaw or heading) from the first position determination receiver 62, the second position determination receiver 64, the motion sensor, or other vehicle sensors. For example, the data storage device 153 can store (e.g., time-stamped) position data, motion data, and attitude data (e.g., yaw or heading) observed from the first position determination receiver 62, the second position determination receiver 64, and / or the motion sensor for transmission to the data processor 150 or any one or more modules via the data port 151.

[0019] The onboard controller 101 generates command data or command signals to send steering commands to the steering controller 122 to track the path plan, target heading or target yaw. The steering controller 122 in turn communicates with a steering system 120, such as an electro-hydraulic steering system or an electric steering system. In one example, the steering system 120 includes an electric motor that drives one or more tracks, wherein the direction of the vehicle can be controlled by differential steering relative to the tracks to steer or guide the vehicle according to a target heading provided by the onboard controller 101 or the data processor 150. In another example, the steering system 120 includes an actuator that rotates a portion, such as a bottom portion of a corresponding leg 12 about a vertical axis to steer or guide the vehicle 11 according to a target heading provided by the onboard controller 101 or the data processor 150 to cover a field or area with crop input, the target heading being consistent with the path plan or mission.

[0020] The onboard controller 101 may send command data or command signals to the steering controller 122, the brake controller 126, and the propulsion controller 130 via one or more data ports 151 or via the vehicle data bus 152 so that the vehicle 11 follows the path plan. The brake controller 126 is connected to a brake system 124, such as an electro-hydraulic brake system, an electric brake system, or a mechanical brake system. The brake controller 126 is connected to the data port 151. The propulsion controller 130 is connected to a propulsion unit 128, such as one or more electric motors, an internal combustion engine, or an internal combustion engine that provides rotational mechanical energy to a generator or alternator that provides electrical energy to one or more electric motors. The propulsion controller 130 is connected to the data port 151.

[0021] Reference again Figure 1A to Figure 1B In one embodiment, the system 100 accesses, creates or receives a path plan to guide the vehicle platform 11 along a target path when performing spraying, planting or another task in one or more fields. The system 100 also provides a user interface for human-vehicle interaction and control. In one embodiment, manual control is performed via an off-board controller 102. The on-board controller 101 is a central computing resource that is responsible for positioning the vehicle 11, enabling perception of obstacle detection, dynamic path generation and smoothing, and controlling and monitoring vehicle navigation.

[0022] like Figure 1AAs shown in FIG. 1 , the off-board controller 101 includes: an interface control module 103, which is the main control interface with the low-level machine controllers (120, 122, 124, 126, 128, and 130); and other modules of the on-board controller 101 and the off-board controller 102. The interface control module 103 maintains a state machine, performs synchronization between the low-level machine controllers (120, 122, 124, 126, 128, and 130) and the high-level controllers (101, 102), and serialization and deserialization of data transmitted through the data bus 152. The output of the interface control module directly affects the state and navigation of the vehicle 11.

[0023] The attitude estimation module 104 estimates the attitude of the vehicle 11 relative to the global coordinate system. In one embodiment, the attitude estimation is completed using an extended Kalman filter (EKF) assuming a uniform velocity model, and the state estimate is generated using differential position updates. The measurement updates of the EKF can be obtained from GPS data provided by the first position determination receiver 62 or the second position determination receiver 64. In alternative embodiments, the EKF can also use data from the IMU for predictive updates. The attitude estimation module 104 can also work with different motion model profiles of the vehicle, including Ackerman steering and articulated steering profiles.

[0024] The perception module 105 establishes a basis for examining and understanding the environment in which the vehicle 11 operates. This functionality is fundamental to safe autonomous driving operation. For example, the perception module 105 allows the vehicle platform 11 to identify positive and negative obstacles in its environment over time and pass critical navigation information to the decision-making components of the system. To achieve this functionality, the perception module 105 provides obstacle detection to: operate in geometric space without relying on image features, but does not exclude the incorporation of such information; consistently recognize several classes of objects, including people, vehicles, buildings, and negative obstacles, in the absence of crops or in conditions of crop emergence; and does not require "supervised learning" or the large amounts of data necessary for the underlying model.

[0025] In one embodiment, the perception module 105 detects the following targets: depth information is calculated from at least one stereo camera (or optionally a pair of stereo cameras) and a LiDAR point cloud, which is projected into a local reference coordinate system. These points are accumulated over time in a global coordinate system provided by the pose received from the pose estimation module 104 to provide temporal cohesion and resilience to temporary occlusion. Next, voxels or data points from the three-dimensional grid, from the relevant slices of the map are clustered using one of two methods to obtain a target score. In the first method, the score is determined from a rule-based task. In the second method, a machine learning process is used, and features are calculated on the three-dimensional points and may include density, height distribution, and location (empirically superior to normals, curvature, manifold projections, and edge features). The Gaussian mixture model (GMM) is iteratively re-centered using the K++ algorithm and the expectation maximization (EM) algorithm to produce soft clusters. Clusters are merged by minimizing differences. Using either method, rule-based non-maximum suppression assigns point clusters to objects. If objects are not currently visible, they decay over time. The returned objects exist in the map within the C-space extended view of the current trajectory and satisfy a set of simple rules, such as distance from the vehicle 11, minimum size from the surface, and minimum number of observations. Next, the surface model is updated using non-obstacle points. In some embodiments, a random sampling consensus algorithm (RANSAC) and least squares (LS) optimization can be performed. In an alternative embodiment, a Bayesian update is performed on the assumed Gaussian distribution family, using the new surface points discovered. Using this process, the perception module 105 provides a convex hull of the obstacle path, which is recorded and sent to the obstacle avoidance module 107.

[0026] The path navigation module 106 and the obstacle avoidance module 107 provide path planning and path correction. The path navigation module 106 tracks the vehicle 11 and sends navigation updates to the interface control module 103. This interface control module receives a list of waypoints from the off-board controller 102. That is, the path navigation module 106 performs waypoint bookkeeping and tracking. These waypoints then pass through a curve check test in turn, which calculates the required turning radius for these points. If the turning radius is less than the minimum turning radius of the machine, the points are passed to a Dubins smoothed path generator, which returns navigable waypoints. The path navigation module 106 then sends a buffer of waypoints to the interface control module 103 to perform the action.

[0027] The obstacle avoidance module 107 implements a plan to avoid obstacles. For example, the perception module 105 provides a convex hull of obstacles and a cost map of passability, which is determined by the obstacles detected in the voxel map and the occlusion scores. The cost map does not take into account the specific crops that the vehicle 11 passes through, for example, specific crops may have a lower passability cost than other obstacles, but this information is available to the system. If an obstacle is in the path of the vehicle 11, two planning methods to avoid the obstacle are implemented. In the first method, or the gradient descent-based method, gradient descent is used for the cost map, and a regularization term is used to ensure the motion constraints of the vehicle 11. In the second method, or the hybrid A* (Hybrid A Star) planning method, the cost map is regarded as converting a discrete space into a continuous graph using motion updates. All nodes that the vehicle 11 cannot execute are marked as high cost. Using Dubins motion updates, child nodes are calculated to the parent node position. In this embodiment, two heuristics are used: (1) Euclidean distance to the nearest threshold point on the row; and (2) a spacing heuristic that prefers nodes that implement a fixed spatial factor of the cell width and are assigned a lower cost than their neighbors. Each step to a waypoint is considered to be of uniform cost. Therefore, the planner ensures the shortest path due to the consistency and admissibility of the graph. In addition, the planner ensures that the vehicle 11 can execute the path because it takes into account the minimum turning radius of the child node exploration.

[0028] The remote operation module 108 sends machine status and local sensor (e.g., image) updates to the remote off-board controller 102. The remote operation module 108 is also responsible for maintaining and connecting to the off-board controller 102 by performing tasks. If the off-board controller 102 is out of reach, the vehicle can be stopped via the remote operation module 108. If stopped, the module 108 attempts to reestablish connection with the off-board controller 102. In one embodiment, the remote operation module 108 can also ensure that the vehicle 11 remains within the operating geofence. If for some reason, the vehicle 11 leaves the geofence, a stop message is triggered and an alert is sent to the off-board controller 102.

[0029] The onboard controller 101 also includes an architectural safety mechanism that is used to provide stable and safe operation of the vehicle 11. For example, in one embodiment, the architectural safety mechanism includes a parallel computing pipeline that enables redundancy and / or checks on system calculations. In another embodiment, the architectural safety mechanism includes a design that abstracts local components into a modular design, which allows algorithm components to be parameterized. Pose filtering, ground modeling, and classification algorithms are all examples of parameterized components, and subcomponents utilize the same design. Other features of the architectural safety mechanism include: low-overhead data transmission across distributed infrastructure, log support for various data including sensor data to reconstruct system performance offline, high-speed compression for image streaming to the user interface 155 (in this embodiment, the algorithm applies a random partial SVD to the image difference, with periodic keyframes to remove artifacts, and the random partial SVD balances the fidelity of the image in real time by adjusting the level of decomposition); a global state machine that monitors the autonomous state of the vehicle 11 and combines the readiness of various components with low latency; and multiple levels of redundancy in the safety system, including path integrity, path transmission, and digital fences.

[0030] Reference now Figure 1B The off-board controller 102 is responsible for initial configuration of the development mission, static path planning, and monitoring the status of the vehicle 11. The user interface 155 associated with the off-board controller 102 also allows the user to manually control the overload of the vehicle 11.

[0031] To develop a mission, the mission configuration module 110 allows the user to select the details of the application and mission parameters for performing navigation. The details may include identification of geo-fences, application rates, maximum navigation speeds, tool widths on the vehicle 11, turning thresholds for the vehicle 11, threshold conditions for refueling, and enabling / disabling data logging capabilities. Using the mission configuration module 110, the user can also locate the location of a gas station (or auxiliary station) on a map. For example, a user can upload GPS coordinates or manually select the location of a gas station on a map interface (e.g., Google Maps). The output of the mission configuration module 110 defines the navigation restrictions and an initial map of the area that the vehicle 11 is to pass through. The user can define the application area by uploading a shp file (shape-file) or manually selecting the area on a map interface built into the off-board controller 102.

[0032] The static mission module 111 uses the output of the mission configuration module 110 to generate customized waypoints that cover the desired area of ​​the site as well as potential refueling paths or exit points. The static mission module 111 has two components, a coverage component and an auxiliary component.

[0033] The overlay component of the static mission module 111 generates waypoints along the application area using a multi-plane raster scanning algorithm. The generated rows of waypoints are spaced apart according to the application unit width defined by the mission configuration module 110. In one embodiment, the user can also use the overlay component to define some rows of plants to be skipped. In some embodiments, the overlay component generates a semi-circular path at the end of each row of plants using the Dobbins path planning algorithm. The semi-circular path takes into account the minimum turning radius of the vehicle 11 to ensure that the generated path can be safely executed by the vehicle 11. The list of final waypoints can be checked and smoothed to illustrate the operating profile of the vehicle 11.

[0034] The auxiliary component of the static task module 111 generates a list of waypoints at decision nodes in the navigation map. The waypoints guide the vehicle 11 from a specific decision point to a refueling (auxiliary) location. Decision points are generated on the application map based on the exit row location and the Euclidean closeness to the auxiliary location. The generated waypoints allow the least crops to be crushed during navigation (for example, when the vehicle 11 turns at the end of a row). Cost heuristics are used to calculate the merge point to a particular headland. The auxiliary component can also define a return behavior after refueling. The return behavior is subject to the navigation area that can return to the last working row. In one embodiment, the return behavior is a "turn at the headland and return" behavior (defined using Dijkstra's Algorithm). In an alternative embodiment, the return behavior is a "light bulb turn and return" (defined using Dijkstra's Algorithm) behavior. The output of the static task module 111 is a list of waypoints classified as "normal" points and "auxiliary" points. The waypoint list may be sent to the vehicle 11 via the data port 151 and the wireless link 154 , such as a wireless serial communication device.

[0035] The machine status module 112 enables the off-board controller 102 to display certain machine information to the user. The on-board controller 101 transmits basic status information of the vehicle 11 and its various components through the remote operation module 108. The status information can be displayed to the user using interactive markers and status estimation graphics. The information displayed on the user interface 155 can include: vehicle health status (e.g., fault checks), vehicle status estimates (e.g., fuel, component temperatures, engine diagnostics, and similar data), mission status (e.g., completion status, mission bird's-eye view display, and vehicle location), image data, and other sensor data. The machine status module 112 also ensures that the off-board controller 102 is connected to the vehicle 11 during mission execution. If there is any communication interruption, the machine status module 112 attempts to reestablish the connection. In other embodiments, the machine status module 112 can send a distress signal if the vehicle 11 crosses a geo-fence, where the geo-fence defines the area in which the vehicle 11 should operate. The distress signal interrupts the mission and sends a stop signal. It is worth noting that this is a redundant feature because the on-board controller 101 performs similar functions.

[0036] The off-board controller 102 also includes an override control module 113 that enables a user to override the autonomous driving operation. For example, a user can override the autonomous driving control to instruct the vehicle 11 to perform actions related to core operations, mission operations, and path correction operations. For core operations, a user can instruct the vehicle to start a mission, stop, pause a mission, shut down the engine, and start the engine. For mission operations, a user can instruct the vehicle to resume the path from the last recorded application line, resume the mission from the vehicle's current position, force assistance at the next decision node, ignore outputs from the perception module 105, ignore obstacles, ignore state machine states, and follow breadcrumbs. In one embodiment, the override control message is sent to the on-board controller 101 as a 16-bit message, where the message is repeatedly sent until the correct execution of the override control message is observed. In alternative embodiments, other communication protocols may be used. For path correction operations, a user can mark general waypoints on a map for the vehicle 11 to follow and select the waypoints by clicking on the map interface. After activating the "follow breadcrumb navigation" control, the defined waypoints are assigned the highest execution priority. In one embodiment, the waypoints are passed through a Dobbins path planning algorithm to ensure that the resulting path is smooth and executable by the vehicle 11 .

[0037] Although the present disclosure has been described in detail with reference to the specific embodiments of the present disclosure, it will be appreciated by those skilled in the art that various changes and modifications may be made therein without departing from the spirit and scope of the embodiments. Therefore, the present disclosure is intended to cover modifications and variations of the present disclosure as long as they fall within the scope of the appended claims and their equivalents.

[0038] In addition, the features disclosed in the above description or claims or drawings, in their specific form or according to the mode for performing the disclosed functions or the method or process for obtaining the disclosed results, can be used alone or in any combination of these features to implement the present invention in their different forms. Specifically, one or more features of any embodiment described herein can be combined with one or more features of any other embodiment described herein.

[0039] Protection may also be sought for any features disclosed in any one or more of the publications cited in conjunction with the present disclosure and / or incorporated by reference.

Claims

1. A system for controlling a vehicle operating autonomously in an agricultural field, the vehicle having at least one sensor capable of providing data related to the environment of the vehicle, the system comprising: An on-board controller connected to a data port on the vehicle for receiving the data and formulating a navigation plan, the on-board controller comprising: an interface control module, the interface control module being used to provide instructions to the low-level controller to execute the navigation plan; A posture estimation module, the posture estimation module is used to estimate the posture of the vehicle relative to the global coordinate system; A perception module, the perception module is used to identify obstacles near the vehicle; a route navigation module for tracking the vehicle and sending navigation updates to the interface control module; and an obstacle avoidance module, the obstacle avoidance module being used to plan a path to avoid the obstacle identified by the perception module; An off-board controller for sending mission data and a waypoint set to the on-board controller, wherein the on-board controller formulates the navigation plan based on at least one of the mission data and the waypoint set, wherein the waypoint set defines a path covering the agricultural field determined in the mission data.

2. The system of claim 1, wherein the off-board controller comprises: A task configuration module, the task configuration module is used to receive task details from a user to generate a task output; A static machine module is used to receive the task output and generate a waypoint set used by the onboard controller to formulate the navigation plan.

3. The system of claim 2, wherein the off-board controller further comprises: A machine status module is configured to receive status data from the vehicle and display the status data on a user interface.

4. The system according to claim 2 or 3, wherein the off-board controller further comprises: An override control module is configured to send override data to the onboard controller, wherein the override data interrupts the autonomous driving operation of the vehicle. The system of claim 4 , wherein the override data comprises messages related to the mission data.

6. The system of claim 4, wherein the override data comprises a message that when received by the onboard controller causes the vehicle to stop.

7. The system according to claim 2, wherein the onboard controller further comprises: A remote operation module receives data from the at least one sensor and transmits the data to the off-board controller for display on a user interface.

8. The system of claim 1, wherein the low-level controller is selected from the group consisting of a steering controller, a braking controller, and a propulsion controller.

9. The system of claim 1, wherein the pose estimation module estimates the pose of the vehicle by using an extended Kalman filter that assumes a uniform velocity model.

10. The system of claim 9, wherein the pose is updated using data from the at least one sensor.

11. The system of claim 1 , wherein the perception module creates a map in a global coordinate system using data from the at least one sensor.

12. The system of claim 1, wherein the perception module recognizes objects in the environment without relying on image features.

13. The system of claim 2, wherein the mission details are selected from the group consisting of an application area, an application rate, a navigation speed, a vehicle width, a vehicle turning threshold, and a refueling threshold.

14. The system of claim 2, wherein the waypoints include auxiliary waypoints and normal waypoints.

15. The system of claim 1, wherein the off-board controller is connected to the data port via a wireless link.

16. The system of claim 1, wherein the at least one sensor is selected from the group consisting of an imaging system, a stereo camera, a lidar system, a position determination receiver, a satellite navigation receiver, a dead reckoning sensor, an odometer, an inertial measurement unit, and a pitch, roll, and yaw sensor.

17. The system of claim 1, wherein the data comprises at least one of position data, motion data, and posture data.

18. A method of controlling an autonomous vehicle operating in an agricultural field, the autonomous vehicle having an onboard controller, the method comprising: generating, using an off-board controller, mission data and a set of waypoints, wherein the mission data includes an initial map of an area that the autonomous vehicle will traverse, and wherein the set of waypoints defines a path covering the area; Receiving the mission data and waypoint set on a vehicle controller; formulating a path plan using at least one of the mission data and the waypoint set; determining a posture of the vehicle; navigating the path plan by tracking the set of waypoints; as well as Objects within the path plan are detected using sensors, wherein the sensors provide information about the environment surrounding the autonomous vehicle.

19. The method of claim 18, wherein detecting the object comprises: computing depth information from a stereo camera and a lidar point cloud having a plurality of points, wherein the plurality of points are projected into a local reference coordinate system; clustering the plurality of points over a period of time in a global coordinate system identified in the gesture; clustering voxels from global coordinate system voxel clusters; and A target score is determined based on the voxel clusters.

20. The method according to claim 19, further comprising: The object is identified using the object score.

Images