Managing workflow for autonomous work machine or fleet thereof

Abstract

An autonomous work vehicle is scheduled to perform work at the work site according to an initial work plan. The initial work plan estimates a scheduled end time for the work. Due to various inputs, such as a signal from a worker, an environmental condition, or the like, a completion time is determined that is different than the scheduled end time. During or after the autonomous work vehicle works the work site, a signal is sent to the autonomous work vehicle to complete work at an adjusted end time different than the scheduled end time based on the manual completion time. The initial work is plan adjusted such that the autonomous work vehicle finishes the work by the adjusted end time.

Description

MANAGING WORKFLOW FOR AUTONOMOUS WORK MACHINE OR FLEET THEREOFRELATED PATENT DOCUMENTS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 657,461, filed on June 7, 2024 and U.S. Provisional Application No. 63 / 787,388, filed on April 11, 2025, all of which are incorporated herein by reference in their entireties.SUMMARY

[0002] The present disclosure is directed to autonomous ground care machine platforms such as autonomous mowers. In one embodiment, a method involves deploying an autonomous work vehicle to a work site. The autonomous work vehicle is scheduled to perform work at the work site according to an initial work plan. The initial work plan estimates a scheduled end time for the work. The work site is worked or monitored by a worker while the autonomous work vehicle is performing the work. The worker ceases the work at an adjusted completion time different than the scheduled end time. During or after the autonomous work vehicle works the work site, a signal is sent to the autonomous work vehicle to complete work at an adjusted end time different than the scheduled end time based on the adjusted completion time. The initial work is plan adjusted such that the autonomous work vehicle finishes the work by the adjusted end time.

[0003] In another embodiment, a method involves deploying an autonomous work vehicle to a work site. The autonomous work vehicle is scheduled to work a region of the work site based on an initial work plan. An environmental condition is detected that affects the work region such that the environmental condition necessitates a change to the initial work plan. The change to the initial work plan includes at least one of a change in a completion time of the work region and a change in a work parameter used by the autonomous work vehicle as specified in the initial work plan. An updated work plan is formed that compensates for the change to the initial work plan, and the region is worked based on the updated work plan.

[0004] In another embodiment, a method involves deploying an autonomous work vehicle to perform work at a work site. A condition is detected that causes the autonomous work vehicle to halt the work. An electronic signal is sent to an operator indicating the the work was halted. A remote access session is facilitated that allows the operator to access sensor data pertaining to the autonomous work vehicle. Via the remote access, the operator is allowed to perform one of: resolve the condition to allow the autonomous work vehicle to continue to work; or command the autonomous work vehicle to continue to halt the work.

[0005] In another embodiment, a method involves deploying an autonomous work vehicle to perform work at a work site. A lighting condition is determined that affects an ability of the autonomous work vehicle to perform the work. An electronic signal is sent comprising instructions to change parameters of an automated lighting system separate from the autonomous work vehicle. The automated lighting system is operable to illuminate the work site, and the change is effected via the automated lighting system.

[0006] In another embodiment, a method involves selecting an autonomous work vehicle to work on a work site based on an initial work plan. A self-check evaluation is performed via a sensor of the autonomous work vehicle. Based on the self-check evaluation, predicting an impact on machine performance for the autonomous work vehicle is predicted. An adjusted work plan is prepared based on the initial work plan that accounts for the impact on the machine performance. The autonomous work vehicle is deployed to work the work site using the adjusted work plan.

[0007] These and other features and aspects of various embodiments may be understood in view of the following detailed discussion and accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The discussion below makes reference to the following figures, wherein the same reference number may be used to identify the similar / same component in multiple figures. The drawings are not necessarily to scale.

[0009] FIG. l is a schematic diagram of a system and work site according to various example embodiments;

[0010] FIG. 2 is a schematic diagram illustrating schedule coordination between an autonomous work vehicle and a human worker according to an example embodiment;

[0011] FIG. 3 is a schematic diagram illustrating schedule coordination between multiple autonomous work vehicles according to an example embodiment;

[0012] FIG. 4 and 5 are schematic diagrams illustrating adjustments between multiple schedules according to an example embodiment;

[0013] FIG. 6 is a schematic diagram illustrating peer-to-peer messaging between autonomous work vehicles;

[0014] FIG. 7 is a schematic diagram of communications between an autonomous work vehicle and various smart devices;

[0015] FIG. 8 is a flowchart of a method according to an example embodiment;

[0016] FIG. 9 is a schematic diagram illustrating diagnostic data collection according to an example embodiment;

[0017] FIGS. 10-14 are flowcharts of methods according to various embodiments;

[0018] FIG. 15 is a block diagram of various hardware and software components a system according to an example embodiment.DETAILED DESCRIPTION

[0019] In the following detailed description of illustrative embodiments, reference is made to the accompanying figures of the drawing which form a part hereof. It is to be understood that other equivalent embodiments, which may not be described and / or illustrated herein, are also contemplated.

[0020] The present disclosure relates generally to ground care machines, which may be variously referred to herein as ground care vehicles, ground maintenance machines, ground maintenance vehicles, and the like. Ground care machines, such as lawn and garden machines, are known for performing a variety of tasks. For instance, powered lawn mowers are used by both homeowners and professionals alike to maintain turf areas within a property or yard. The same or different machines may be used for maintenance on the turf areas (and sometimes away from the turf), which may involve performing anycombination of operations such as material collection (e g., plant matter, dirt, golf balls), spraying, dethatching, edging, rolling, towing, snow / ice treatment and removal, etc.[00211 Embodiments described herein include features of autonomous ground care machines that are used in a commercial environment. There are referred to interchangeably herein as autonomous work vehicles, autonomous work machines, work vehicles, machines, autonomous vehicles, vehicles, etc. While commercial ground care operations may still require a substantial amount of labor, it is anticipated that deployment of autonomous machinery may provide cost and safety benefits to landscaping contractors and other business operations that use outdoor equipment by reducing manual labor, as well as providing other benefits (e.g., increased precision for some tasks). While such technology may also be desirable for personal use, a commercial entity may be more willing to absorb higher up-front costs of autonomous technology in exchange for longterm labor cost savings and other advantages.

[0022] One issue encountered with autonomous vehicles is their ability to deal with unexpected conditions. For example, items such as large pieces of debris (e.g., tree branches) may fall into a mowing path, which at a minimum may cause a short delay to avoid, but is some cases may cause greater delays, e.g., causing the machine to get stuck on the debris. In other cases, ground conditions (e.g., unusually dry or wet soil, unusual turf growth pattern) that can slow down or speed up traversal may cause work to finish early or late.

[0023] In the case of a business, unexpected delays can lead to lost productivity, e.g., may make it difficult to finish on time and start the next paid job. Finishing the work early can also lead to lost productivity if it comes as a surprise to the work coordinator, who could have possibly used that extra time to do other paid work if known ahead of time. Because autonomous machines are less adaptable to changing conditions than a human worker, the resulting scheduling impacts may be more common for autonomous grounds work.

[0024] In embodiments described herein, a number of scenarios are described which impose temporal and / or physical constraints on normal or expected operation of an autonomous work machine. These constraints can result in interruptions or other changesin work, which can result in loss of productivity. Technical solutions are described that, in some cases, can predict or anticipate the conditions, thus allowing for mitigation to be performed even before work is started. Inevitably, unpredictable conditions will arise that can affect productivity of autonomous machines, and other technical solutions are described that can help mitigate problems in real-time, e.g., to reduce impact on schedule, quality of work, or other factors related to performing maintenance work such as grounds keeping.

[0025] In FIG. 1, a diagram shows a system according to an example embodiment. One or more autonomous work machines 104 are used to perform work in a work site 106. In this figure, the work machines 104 are part of a fleet 102, however the concepts described here are equally applicable to a single one of the autonomous work machines 104. The work site 106 in this example is part of a golf course, which includes at least mowable areas such as putting greens 108, fairways 109, rough 110 and tee box 111. Non- mowable areas include water hazards 112, trees / bushes 113, bunkers 114, trails 115, buildings 116, etc. These other areas may be workable by other machines, e.g., an autonomous sweeper may clean the trails 115.

[0026] The fleet 102 may include different machines to perform the same function, e.g., mowing, but having different capabilities. For example, the fleet 102 includes at least one autonomous rotary blade mower 104a and at least one autonomous reel mower 104b. Typically, the reel mower 104b will be used for regions such as the greens 108, whereas rotary mower 104a may be used for areas such as parts of the fairway 109 and / or the rough 110. The fleet 102 may also contain other autonomous machines (not shown) that perform a different function than work machines 104. The different function may be related to the function of other machines 104, such as a mobile collector that collects clippings from the mowers when their collection bins become full. The different function may be somewhat unrelated or independent, e.g., an autonomous fertilizer spreader that treats areas of the turf while other machines are mowing.

[0027] Generally, regions of the work site 106 may have been previously characterized sufficiently to enable autonomous operation of one or more autonomous work machines 104 of the fleet 102. This characterization may use, at least in part, usingaerial imagery or other wide-area data collection sensors. To gather more precise and reliable information, the work site 106 may be traversed by the same machine 104 that does the work, or an equivalent machine in a process sometimes referred to as training.

[0028] In some embodiments, training involves taking sensor readings during the traversal that allows characterizing the work site 106 in sufficient detail so that the work machines 104 can autonomously navigate and work in the site 106. For example, an autonomous mower that is targeted to cut grass in complex regions (e.g., hilly, many obstacles, irregular boundaries) may gather extensive data on feature location (e.g., landmarks), ground slope, ground traversal quality / difficulty, lighting, radio reception, etc. In contrast, an autonomous mower that is targeted to cut a football field may only need information on the peripheral boundary of the field, as it may be assumed that the field is a rectangle without significant obstacles that is easy to navigate and can be mowed in a simple pattern.

[0029] The training is used to gather and compile data 118 (e.g., maps, work plans) that can subsequently be used by the autonomous work machines 104 to work the site 106. The data 118 may be stored on the machines themselves and / or on a widely accessible computing infrastructure represented as data center 120, which may also be referred to as a cloud service, Internet service, local server, etc. Alternatively, a local device such as mobile device 122 used by operator 124 (also referred to herein as a worker) may be used to send the data 118 to the machines 104. The mobile device 122 may additionally or instead provide other functions, such as line-of-sight remote control of the vehicles 104, accessing data and sensors of individual vehicles 104, etc.

[0030] As autonomous work machines begin to be adopted, one common use case may be where a worker and an autonomous machine split up work at a work site. The worker, who may also be referred to herein as an operator, supervisor, etc., may deliver and / or stage the autonomous machine, which then performs a programmed task in a first region of the work site. While the autonomous machine is working, the worker can do another task on a second region of the work site. In other cases, the worker may roam along a non-defmed region supervising and supporting the operations but without performing any specific grounds work.

[0031] The regions worked by the worker and autonomous machine can be the same or overlap, e.g., the worker may cut grass at peripheral edges of a field while an autonomous mower mows the interior area of the field. In other cases, the worker may work a separate region nearby. In one scenario, the worker’s task is less predictable than that of the autonomous machine, therefore it is beneficial to provide an easy way to adjust the machine schedule to adapt conditions of the worker, e.g., such that both the worker and the machine can maximize work done in the time available.

[0032] In embodiments below, a staging area or point is described that generally corresponds to a convenient location to start and / or end a work session. The staging area may be explicitly defined by the operator, e.g., selection on a digital map. The staging area may be implicit, e.g., the first geolocation measured when the machine is powered on and commanded to start work. In some cases, different start and end staging areas may be defined. For example, it may be more convenient in some scenarios for the operator to drive a trailer from a start point to an end point than to have an autonomous machine navigate back to the start point. This can be extended to more than two staging areas, e.g., an intermediate staging area for work stoppages, lunch breaks, etc.

[0033] The staging area / areas may be useful in scheduling as described below, e.g., to calculate work completion times plus return to staging area times when schedules are altered. One or more predefined staging areas can also provide a convenient way to halt work and collect the machines. For example, if a storm is incoming, the operator may select a “stop work and return” command in which the machine takes the quickest path back to the staging area or to the closest of multiple staging areas if more than one is defined.

[0034] In FIG. 2, a diagram shows an example of an autonomous machine 104 that works a work site 106 in cooperation with a human worker 124 according to an example embodiment. The autonomous work vehicle is deployed to the work site 106, e.g., placed there via the worker, is temporarily or permanently stationed at the site, etc. The autonomous work vehicle 104 is scheduled to work a first region 200 of the work site according to an initial work plan 202, which includes at least a schedule 204 for the autonomous machine 104. The initial work plan 202 estimates a scheduled end time 203for work in the first region 200. In other words, the initial work plan 202 has information (e.g., work path, vehicle speed, start time) that allows estimating the scheduled end time 203, although the time value need not be expressly stored as data. For example the scheduled end time 203 may be calculated and continually updated based on actual progress of the vehicle 104. The initial work plan 202 may include other data not shown, such as a vehicle speed, work path, work implement speed, work implement setting (e.g., height of cut), etc.

[0035] The worker 124 works a second region 201 of the work site 106 while the autonomous machine 104 is working the first region 200. In some scenarios, the worker 124 may be working the first region 200 instead of the second region 201, and the work performed by the worker 124 need not be grounds work, e.g., supervision, resupply, etc. A schedule 205 is also shown for the worker 124, and this may be an expressly defined schedule, e.g., stored together with vehicle schedule 204, stored on a calendar application of the worker 124, etc. The worker schedule 205 may be implied, e.g., the worker 124 plans to work the same amount of time as in vehicle schedule 204. In either case, the worker 124 may be performing tasks that are less well-suited to autonomous work, such as trimming bushes / trees, mowing difficult to navigate regions, patching turf, and other miscellaneous tasks. In such a case, the worker 124 may wind up spending more or less time than planned. As a result, the worker 124 may want to extend or shorten the schedule 204 of the work vehicle 104, and do so with minimal effort.

[0036] In this scenario, while the autonomous work vehicle 104 is still working (current progress being indicated by progress bar 206), the worker 124 determines that he or she has finished or will finish working the additional region at an adjusted completion time (represented alternatively by dotted lines 208, 209) different than the autonomous end time 203. The change in the worker’s schedule 205 may be due to work-related reasons, e.g., the work may be easier or more difficult than anticipated, equipment failure, etc. The change may also or instead be due to conditions affecting the work site 106, e.g., inclement weather, use of the site by others, etc. The change may also or instead be due to conditions affecting the worker 124 that may or may not be work related, e.g., unsafe working temperatures, worker has another appointment to keep in the allotted time (e.g., has to pickup somebody early from an event), etc. In response to any of these conditions, the worker 124 may voluntarily or involuntarily have to finish working the second region 201 at a manual completion time 208, 209 different than the scheduled end time 203.

[0037] Once the adjusted completion time 208, 209 different than the scheduled end time 203 is determined, a signal is sent to the autonomous work vehicle to complete work at an adjusted end time different than the scheduled end time 203 based on the manual completion time 208, 209. For example, assume schedule 204 is estimated to run from 10:00AM to 2:00PM, and the worker 124 learns at 11 :00AM that he / she has to leave the site 106 at 1 :00PM. Thus, the worker 124 can send a signal to the autonomous work vehicle 104 to modify the work plan 202, so that both the worker 124 and the work vehicle 104 will finish at the same time.

[0038] As will be described in greater detail below, the change in completion time may result in unfinished work, different quality of work, etc., for at least the first work region 200. Nonetheless, this can still facilitate automatically adjusting the operations of the vehicle to efficiently use the shortened time, e.g., per a user preference. In contrast, under the same scenario where the worker learns at 12:45PM that the work needs to end at 1 :00PM, the options are likely more limited. In this latter scenario, the vehicle would 104 likely stop work soon or immediately after receiving the signal. In the 15 minutes remaining, the vehicle 104 would have enough time to return to a staging area / point, or some other location when work ends, e g., a charging station.

[0039] In this example, the first and second regions 200, 201 are physically separate. Even if the regions 200, 201 are close together, connected and / or overlapping, it will often be more convenient for the worker 124 to remotely signal a change in the schedule 204 (e.g., via a wireless signal) to the vehicle 104 rather than to physically access the vehicle 104 in order to effectuate a schedule change. For cases where the worker 124 meets up with the vehicle 104 at the termination of work, the travel time between the worker 124 and the vehicle 104 when the finishes work (e g., at staging area, charging station) may be taken into account when rescheduling work as described above. Note that even if the vehicle 104 moves to a charging station on the work site 106 after work is complete, the worker 124 may still wish to visit the vehicle 104 before leaving, e.g., toinspect for wear / damage, empty debris collectors, etc. The distance between any combination of the worker 124, vehicle 104, and regions 200, 201 may be used to calculate the travel time of the worker 124 at the completion of work.

[0040] In some cases, the adjusted end time 209 is later than the scheduled end time 203 by an extra time 210. In this case, the autonomous work vehicle 104 may use the extra time to work the first different region 200 to a higher quality than required by the work plan 202, and / or work other regions (not shown) of the work site 106. This may involve, for example, making additional passes with a mower, operating at a slower speed, etc. If the autonomous vehicle 104 determines that there is insufficient stored energy to work the extra time 209, the autonomous vehicle can work a portion of the extra time and return to a staging area (which may include an on-site charger or refueling station) and waits until the adjusted end time 209. In some cases, the autonomous work vehicle 104 may use the extra time 209 to reduce power consumption needed to finish the work, e.g., may move at a slower pace, reduce blade rotation speed, etc. In some embodiments where more than one autonomous machines (or a different worker) are being used together with the autonomous vehicle 104 at the work region 200, one of the machines (or the different worker) may be taken out of service (e.g., allowed to recharge, work a different job) from the work site if the remaining machines can finish the work by the adjusted end time.

[0041] In some cases, the adjusted end time 208 is earlier than the estimated end time by a shortage time 211. In that case, the autonomous work vehicle may perform at least one adjustment comprising: working a remaining portion of the work region to a lower quality than required by the work plan; and skipping a remaining portion of the work region. In such a case, the vehicle 104, worker 124 or other service may update a subsequent work plan of the work site to remediate the at least one adjustment. For example, a next available work slot in the calendar can be filled to finish any work that was uncompleted or not fully completed to the desired quality (e.g., perform a second mowing pass). In another embodiment, if a second autonomous work vehicle (not shown) and / or a different worker (not shown) is available but not currently working at the work site, the worker or machine can be added to assist autonomous work vehicle in finishing the work by the adjusted end time, without requiring a change in work quality.

[0042] The signaling from the worker 124 to change the schedule may occur in a number of ways. For example, a prearranged signal to halt the work may occur of the worker 124 enters a geofenced area, e.g., within the perimeter of the work region 200 or approaches the same. In other cases, the worker 124 may use a mobile device (e.g., device 122 in FIG. 1) to affect the change. In one embodiment, the schedule 204 may appear as within a calendar-type application on the mobile device, such that the worker 124 can change the extents of the work session, which will automatically contact the machine. This may work similar to features in electronic calendars that will automatically email to meeting participants a change to a meeting time. In another embodiment, a specialized application on the mobile device may provide similar functionality without a calendar view, e.g., sending a message in response to a user interface selection.

[0043] In another embodiment, the worker 124 may be able to contact the vehicle directly, e.g., via a voice call or text message. This can be sent directly to a phone number dedicated to the vehicle 104 and / or to a service entity that uses a machine identifier transmitted via a call or text to the number. In either case, voice commands or text commands can be converted to machine control commands. For example, a voice message of “I’ll be half an hour late, keep working until I arrive” can be converted to an in-process change to the work plan, including changes to work path, work parameters, completion time, machine / worker allocations, etc.

[0044] In scenarios above, the work plan is adjusted to account for an adjusted completion time, and this may be done automatically, e.g., via an algorithm operating on the vehicle 104 and / or a data center 120. Thus, once the worker 124 has commanded the change in completion time, the worker may want confirmation that the change was received, as well as what the changes are. Accordingly, the vehicle 104 and / or data center 120 may send an alert to the worker that indicates or describes the adjusted work plan. This may include, for an example a simple acknowledgement that the change has been accepted, and may also include a graphic or description of the plan, or instead a link that allows viewing the plan in detail if desired. This allows the worker to confirm the plan if it is acceptable, suggest another plan if not, cancel the change, etc.

[0045] In some embodiments, the adjustments to autonomous machine schedules may be due to delays or extra time discovered by machines in a fleet. In FIG. 3, a diagram shows an initial work plan of a fleet of two or more autonomous work vehicles 300a-c on a work site 302 according to an example embodiment. Each of the vehicles 300a-c has been scheduled to work a different region 302a-c of the work site 302 according to an initial work plan. The work plan includes a schedule 304 which includes a respective planned schedules 306a-c (shaded boxes) of the work vehicles 300a-c. Line 308 represents a current time t_c since the start of work (time=0) and bold lines 3 lOa-c represents a current progress of the respective machines 300a-c along each of the planned schedules 306a-c.

[0046] The location of lines 3 lOa-c relative to current time line 308 indicate how far off the machines 300a-c are from the planned. Specifically, machines 300a and 300c are behind schedule and machine 300b is ahead of schedule. Dashed lines 312a-c indicate current estimates of completion time for each machine 300a-c if present trends continue, and bold line 314 indicates an estimated end time of the initial work plan, e.g., a time when the operator expects to be on site collect the vehicles. Note that all of the initial schedules 306a-c are planned to finish early, e.g., the operator has padded the schedule to err on the side of the machines finishing early.

[0047] As indicated by currently estimated end time 312c, the autonomous work vehicle 300c will not finish by estimated end time 314 of the initial work plan. This will affect the entire fleet, assuming that the operator will expect all of the vehicles 330a-c to be done by time 314, e.g., to collect and / or redeploy the fleet. Thus, if one of the machines is currently ahead of or behind schedule by more than a threshold, this may signal to the fleet, a monitoring service, and / or the operator that an adjustment may be needed. In this case, time difference 316 represents a current delay greater than a threshold that is expected to prevent the fleet from finishing on time, and so an adjustment may be called for to remedy this. While the other machines are offset from their predicted schedules, the offset is small enough that no signaling of this is needed.

[0048] In one embodiment, the adjustment may just be that the vehicle 300c finishes as much as it can by time 314, leaving part of region 302c unworked. Another option, if available, is to increase an operating speed of vehicle 300c to catch up. Thisspeed-up may be at the expense of work quality of the unworked portions. Another option is to divert another of the work vehicles 300a-b to work region 302c to finish part of the work. In this example, vehicle 300b has ample time in the schedule to help out. This may involve dividing the work path used in region 302c such that a first part will be completed by machine 300c and a second part will be completed by machine 300b. This may also involve devising a traversal path of machine 300b from region 302b to region 302c to work the second part of region 302c.

[0049] Note that the adjustments described above do not take into account what is causing the autonomous work vehicle 300c to fall behind schedule. This could be due an environmental condition (e.g., soggy turf) that affects work region 302c such that the individual completion time 312c is changed from the planned completion time (before time 314). A vehicle condition may also be considered an environmental condition in some cases, e.g., vehicle is slipping on turf due to worn tires. In some cases, the autonomous work vehicle can detect such conditions before work or during a beginning stage of work, such that plan changes can be implemented in advance of the estimated work completion time.

[0050] In one example, once an environmental condition is detected that affects at least one work region (e.g., the condition could affect the work region itself or an autonomous work vehicle assigned to the region), a computing entity (e.g., a mobile device of the operator, a data center, one of the vehicles) can query the fleet to determine scheduling availability of others autonomous work vehicles. In response to the query, the entity can redistribute portions of the initial work plan to the other autonomous work vehicles according to the scheduling availability to compensate for the change of the individual completion time.

[0051] In some embodiments, the scheduling availability comprises an offset in current progress versus a predicted progress in the initial work plan. For example, vehicle 306b is ahead of schedule and so may be a candidate to compensate for another vehicle that is running late. Similarly, both vehicles 300a and 300b have gaps in their schedules (at the ends of the schedules 306a-b in this case) that could be part of the scheduling availability of these machines.

[0052] In another example, the scheduling availability includes or considers a remaining stored energy in different than what is needed to perform work specified in the initial work plan. In cases where a work vehicle has sufficient slack in the schedule to compensate for another vehicle, it should also have sufficient stored energy (e.g., battery energy or fuel remaining) to operate for the needed time. Other factors to be considered in the scheduling availability of vehicles is the suitability to do the work, e.g., type of work instrument (e.g., rotary versus reel cutter), ability to traverse targeted the work region (e.g., maximum allowable slope), ability to navigate in the region (e.g., including the needed sensors), etc.

[0053] In the illustrated example, the work plan schedule has enough spare time allocated such that one work vehicle can feasibly pick up some or all of the work of another of the vehicles while still completing in time. This may not always be the case, such that a compromise may have to be made with the work plan. In one case, the work completion time 314 can be changed (e.g., moved to a later time), possibly accompanied by a notification to the operator of this delay. In other cases, the redistributing of the portions of the individual work plan results in a change in a work quality without affecting a work completion time of the fleet. This may involve, for example, working at a faster speed, forgoing multiple passes, skipping work in less critical areas, assigning a different class of machine to do the work, etc. Any of these can also be communicated to the operator / worker via an alert or notification, and may also allow the operator to accept or reject the plan.

[0054] In the example shown in FIG. 3, a change in the actual performance versus the expected performance of the autonomous work vehicles results in an adjustment in the initial work plan. These variations can be recorded and used to adjust future predictions, e.g., work plans at the same site or at different sites. Over time, measurement of variations can give more accurate completion times (e.g., using statistical average) as well as variability (e.g., using standard deviation).

[0055] In other cases, there may be a change to the schedule itself that requires changing the initial work plan. For example, an operator who deploys the vehicles at the start of the work session and collects the vehicles at the end of the work session may havesituations arise in which the start or end time of the work will differ from the plan. In one scenario, the operator may also be doing manual work in the region, and finishes this work early or late. Thus the operator may wish to respectively curtail or extend the working schedules of the autonomous vehicles. In other cases, weather conditions (e.g., an oncoming severe storm), use conditions (e.g., somebody may need to use the work region), and the like, may cause the operator to request an early end to the work, or some other change to the schedule.

[0056] In FIG. 4, a diagram shows the schedule 304 of FIG. 3 under a different scenario. As in FIG. 3, the autonomous work vehicles 300a-c are currently working the different regions 302a-c, except in this case all vehicles are closely adhering to schedule (not required under this scenario). While the fleet is working the work site, a signal is sent to the autonomous work vehicles 300a-c to complete work at an adjusted end time 400 different than the estimated end time 314. The vehicles 300a-c (or some other computing entities) adjusts the initial work plan such that the fleet finishes working the work site by the adjusted end time 400.

[0057] In this particular example, the adjusted end time 400 is later than the estimated end time 314 by an extra time 402. The vehicles may adjust individual work plans in a number of different ways in response to this. The autonomous work vehicles may use the extra time 402 to work the different regions 302a-c to a higher quality than required by the work plan. This may involve, for example, making repeat passes over the work region, going at a slower pace, etc. In other embodiments, the autonomous work vehicles use the extra time to reduce power consumption needed to finish the work. Generally, moving at a slower speed can reduce power consumption as well as potentially providing a higher quality of work.

[0058] The ability to use the extra time 402 may depend on whether the autonomous machines have sufficient stored energy to work the extra time. If not, a selected machine with insufficient stored energy can work a portion of the extra time and return to a staging area and waits until the adjusted end time. If the vehicle is battery powered and there is a charger at the staging area, the extra time 402 may allow for apartial battery recharge on-site. A similar scenario may apply to internal combustion engines assuming a refueling facility is nearby.[00591 Insome cases, the signal to add the extra time 402 may be due to one or more work vehicles falling significantly behind schedule. Thus these vehicles may not need to make any adjustment except to complete work at the current pace. This may still allow other vehicles that are adhering to schedule to make changes to their individual work plans as described above.

[0060] In FIG. 5, a diagram shows the schedule 304 of FIGS. 3 and 4 under a different adjusted end time scenario. In this example, the adjusted end time 500 is earlier than the estimated end time 314 by a shortage time 502. The vehicles may adjust individual work plans in response to this in a number of different ways. For example, at least one autonomous work vehicle may work a remaining portion of the work region to a lower quality than required by the work plan, e.g., at a faster traversal speed. In other cases, at least one autonomous work vehicle may skip a remaining portion of the work region, e.g., stop work and return to a staging area by the adjusted end time. In the latter case, a subsequent work plan of the work site to remediate the skipped area. For example, any skipped regions may be added to the following work plan to be worked first. In another case, an additional work plan may be added based on a long term fleet schedule. For example, if work had to be stopped early on a Monday, and the region is not due to be worked until the next Monday, if the fleet is available in the meantime (e.g., the following Wednesday) a work plan may be added to a work calendar that finishes any skipped work from Monday, and may use a reduced-sized fleet to accomplish the make-up work.

[0061] In the embodiments above, the operations of setting up the work plan, modifying the work plan, tracking progress of the autonomous work machines, tracking issues affecting the autonomous work machine, tracking operator schedules, and the like, can be performed by a centralized entity such as the data center 120 or mobile device 122 in FIG. 1. In other embodiments, the autonomous work machines may be able to communicate with each other to accomplish some of these tasks without outside intervention.

[0062] The capabilities of the autonomous work machines to communicate with each other may vary based on the desired level of autonomy, the capabilities of the machines, and operator preferences. At one level, autonomous work machines may provide backup services to one another, such as providing data links, navigation data, etc., to other machines that are in a radio blackout region. At another level, the autonomous work machines may be able to reallocate portions of work between each other due to temporal or physical constraints as described elsewhere herein. At an ever higher level of autonomy, the operator may communicate an overall plan to the fleet, and the machines themselves may allocate the work using peer-to-peer communications and negotiations.

[0063] In FIG. 6, a diagram illustrates an example of peer-to-peer communications between a fleet of autonomous work machines according to an example embodiment. In this example, the fleet includes three work machines 600a-c. The machines 600a-c are interchangeable at least in some respects, such they can all handle a common set of tasks. Further, it may be assumed that the work plan is divisible into portions, such that each machine is assigned multiple portions. For example, of the fleet is working a single field, it may be divided into 30 sections, each estimated to take 5 minutes to complete. Each machine may be tasked to work 10 sections each, preferably assigning a contiguous region to each machine. Thus the fleet may be expected to finish the field in 50 minutes.

[0064] In this example, work machine 600a is experiencing a delay and will not finish in the target time frame. Accordingly, the work machine 600a sends a query 602 to the other machines 600b-c, indicating the task (Mow) and a description of the work (Field A, Section 3). Based on their current allocation of work and progress, the other machines 600b-c send responses to the query 602. Machine 600b sends response 604 indicating it is unable to take the work. Machine 600b sends a response 606 indicating that it can take the work, and also provides other indications of status, such as battery level indicator. The status indication allows the querying machine to rank responses, e.g., choosing the machine with the most available charge, that is closest to machine 600a, etc.

[0065] Assuming the machine 600a has gotten an acceptable response, it can redistribute the work to another machine and perform other actions, such as notifying an operator and / or monitoring service of the change in work. While this example showsmachine 600a sending queries for work that the machine cannot complete, similar queries could be sent if the machine has extra time such that it can help another machine. For example, the machine 600a may detect that some or all of its current region is too wet to mow, and so will delay or skip mowing that region. The machine 600a can query other machines to see how much work remaining they have, and then take on some portion of that work based on factors such as traversal distance needed to do the work, remaining fuel / battery energy, minimizing completion time of the entire fleet, etc.

[0066] Conditions that delay or stop autonomous work as described above can force a commercial operator to adjust schedules in order to account for lost time. As such contractors will likely have experience dealing with equipment failures or unexpected work conditions, an experienced operator will have the ability to adjust as needed. When an operator uses a fleet of autonomous vehicles, the ability to adjust schedules in response to unexpected event may be enhanced, and these adjustments may be automated to some degree, e.g., via server-to-client or peer-to-peer machine communications to quickly update the actions of an entire fleet if needed.

[0067] In addition to enabling communication with other like machines, the communication and processing capabilities described herein can allow an autonomous work machine to communicate with a wide variety of other devices. These devices, described broadly as the Internet of things (loT), may also be referred to as embedded devices, smart devices, etc., that include low-cost processors that be programmed to provide sophisticated, network-coupled functionality. An example of how an autonomous working machine can integrate with loT type devices is shown in the diagram of FIG. 7.

[0068] In FIG. 7, an autonomous work vehicle 700 is shown at or near a work region 702. One or more wireless network infrastructure devices 704 may provide connectivity for the autonomous work vehicle 700, although the autonomous work vehicle 700 may also be able to connect directly with other locally situated devices, e.g., via peer- to-peer networking, license-free radio links (e.g., 900MHz), optical data links, etc. The various device-to-device functionality (e.g., communications protocols, state tracking) can be performed by the autonomous work vehicle 700 and / or by an infrastructure-based service 706, e.g., a data center, an edge server, etc.

[0069] The loT devices shown in FIG. 7 include a pole mounted light 708, a path light 709, an unmanned aerial vehicle (UAV) 710 (commonly referred to as a drone), a gate 711 (e.g., locks and / or open / closing motor), sprinklers 712, and a camera 713 (e.g., security camera). Each of these devices 708-713 may be associated with one or more controllers, e.g., a local server and / or an infrastructure service that coordinates operations with the devices. For purposes of the following discuss, a reference to interacting with a particular loT device should be understood as direct interaction (e.g., sending commands to a processor on the device via a network connection made to a network adapter of the device) or indirect interaction via a service or other device. In some cases, the service can coordinate the operations of multiple loT devices and may include standard application program interfaces that accept generic commands which are translated to specific commands of the loT devices.

[0070] In one embodiment, the autonomous work vehicle 700 and / or service 706 can interact with the sprinklers 712 and other automated watering devices such as irrigation controllers, bubblers, moisture sensors, etc., that are collectively integrated into an irrigation system. These interactions can be used to determine watering timing, activation pattern, and drainage time at the work site. This can be used to, among other things: to determine whether to cut before or after watering; to schedule the autonomous work vehicle 700 according to irrigation schedule and irrigation factors above; reshuffle mowing schedule around changes to irrigation schedule; revise mowing schedule in view of the work vehicle’s operating status and rate and / or operating status of irrigation system; and match robot mover capacity to watering schedule and changes thereto.

[0071] In one embodiment, the system determines operation of an irrigation system that affects an ability of the autonomous work vehicle 700 to perform the work and sends an electronic signal with instructions to change a parameter of the irrigation system, which ultimately effects the changes. Among other things, this allows scheduling machines according to irrigation schedules, changing mowing schedules in response to a change in irrigation schedule (e.g., watering skipped due to rain the night before), revising mowing schedule in view of robot operating status and rate / operating status of irrigation system,and matching machine capacity to watering schedule and changes thereto (e.g., may select faster machines in fleet to work in regions with shorter timeframes between irrigation).[00721 In one embodiment, the autonomous work vehicle 700 and / or service 706 can interact with the lights 708, 709. The lights (and any associated light controllers) are herein referred to as an automated lighting system and may include any light and control elements that are separate from the autonomous work vehicle 700 and that are operable to illuminate the work site. For example, the machine or other entity can determine a lighting condition that affects an ability of the autonomous work vehicle to perform the work. An electronic signal is sent to the automated lighting system with instructions to change lighting parameters of the automated lighting system, which effects a change via the automated lighting system.

[0073] The lighting condition that affects the autonomous work vehicle may include a low light condition, in which case the instructions cause the automated lighting system to increase illumination at the work site. This may include infrared illumination which can enable machine vision via infrared camera sensors without requiring visible light. The lighting condition that affects of the autonomous work vehicle may include interference between a sensor of the autonomous work vehicle 700 and the automated lighting system. In such a case, the instructions cause the automated lighting system to deactivate or adjust the at least one light emitter (e.g., reduce intensity) of the automated lighting system. For example, the sensor interference may include an overload to an optical sensor of the autonomous vehicle. In cases where sensor interference occurs within a portion of the work site, and the deactivation or adjustment may only occur when the autonomous vehicle is navigating within the portion. In other cases, the adjustment comprises a change in wavelength emitted by the lighting system. For example, a narrowband fdter may be applied to a broadband illumination source to remove optical energy in the fdtered band that is interfering with a vehicle’s optical sensor. Changing from visible to infrared may also be considered a change in the emitted wavelength.

[0074] In one embodiment, the autonomous work vehicle 700 and / or service 706 can interact with the gate 711. For example, the autonomous work vehicle 700 and / or service 706 could signal changes to an existing schedule such that the gate is openedand / or unlocked at a particular time for access, which may be different than a prearranged time The work vehicle 700 and / or service 706 may also signal to allow the gate to be closed and / or locked if the work vehicle 700 cannot arrive at a prearranged time. These changes in gate signaling could also be performed for the benefit of a worker on-site. For example, the worker may not have the codes or other access credentials to access the gate 711 (e.g., a new hire, temporary worker) but the work vehicle 700 might be able to provide the access via a mobile phone app or voice commands as previously described. In this way, the work vehicle can serve as an authentication gateway for other devices and people in the work space, providing a central point for managing physical access credentials.

[0075] In one embodiment, the autonomous work vehicle 700 and / or service 706 can interact with the UAV 710 similar to interactions with the lights 708, 709. For example, the UAV 710 may have on-board illumination that allows it to act as a mobile light source for benefit of the work vehicle 700. In other embodiments, the UAV 710 may include a camera that can operate as an extension of the worker’s or operator’s eyes for remotely viewing the work region 702, e.g., to assess quality of work. In other cases, the UAV 710 may assist in remote troubleshooting, which is described in greater detail below.

[0076] As noted above, an issue affecting autonomous work machines is an unexpected condition in the work region or a condition affecting the work machine itself. For example, the machine may be programmed to stop work if certain conditions are detected, and can signal to an operator that the machine has been halted. In existing systems, the operator may have to physically access the machine to clear the condition, assuming it is something that can be cleared onsite. This can be an issue in a large work region, because physically accessing the machine can take a significant amount of travel time, and may force the human operator to stop any manual labor being performed. Thus productivity may be doubly impacted by a stoppage condition.

[0077] In many cases, a shutdown condition may be innocuous, such that the machine can safely continue to work. Nonetheless, such conditions should be first investigated by the operator before allowing the work to continue. For example, if the machine uses a geolocation service (e.g., global navigation satellite system, or GNSS) for navigation and experiences a signal blackout, it may be unable to determine its location tothe desired accuracy. The machine may be able to navigate from a known position during the blackout via dead reckoning using other sensors (e.g., wheel encoders, inertial navigation). However, dead reckoning is only sufficiently reliable for a known distance, such that if the geolocation signal is not reacquired after such distance is traversed, the machine may need to at least stop work implements (e g., blades) and may be programmed to halt movement completely, or to move at a considerably slowed pace in an attempt to recover primary navigation.

[0078] In another scenario, the autonomous machine may be operable at a high level of autonomy (e.g., level 5) in the work region, however the machine may encounter a situation where it is proximate to an object (e.g., person, vehicle, roadway) that results in a transition to a lower level of autonomy (e.g., level 4, supervisory mode). This may trigger a remote access action as described elsewhere herein, including remote access to machine cameras, UAV inspection, security camera inspection, etc. This may allow the operator to move the machine back to a location or situation where the higher level of autonomy can resume without significant interruption to either the autonomous machine or the operator.

[0079] Another shutdown situation that may occur relates to obstacles and stuck areas. Even where a machine has a strategy for getting unstuck or avoiding obstacles, situations will arise that the machine cannot handle on its own. For example, once the vehicle has detected that it has become stuck, it may perform a sequence of backups, turns, etc., in an attempt to extricate itself. In some cases, this may not be effective, even though a human looking at the obstacle could devise a maneuver that could extricate the vehicle. Another example is a large obstacle such as a tree branch that completely blocks part of the work region. The only way around the obstacle may be going through a previously designated keep-out area, e.g., a sidewalk on which people may be walking. In this case, a human operator could maneuver the machine around the obstacle along the sidewalk via remote control, allowing the machine to continue to operate autonomously thereafter.

[0080] In one embodiment, the operator (or somebody else with the appropriate credentials) can remotely access the autonomous work machine and attempt to remedy any conditions that is leading to a stop or delay. In FIG. 8, a flowchart shows a method according to an example embodiment. The method is implemented while performing 800work in the work region, wherein the machine regularly determines whether a halt condition 801 has been detected. The halt condition 801 may be a current or pending collision with an obstacle, loss of navigation capability (e.g., can’t determine current vehicle pose below a threshold), sensor failure or error, mechanical or electromechanical part out of operating range (e.g., drive current exceeds threshold), temperature exceeds limit, etc.

[0081] If the condition 801 is detected, then the autonomous machine slows or stops 802, e.g., to avoid any damage or other problems that might occur due to the condition. This may at least involve stopping any work implements (e.g., cutting blades) as well as halting or slowing movement. Other actions may also be taken, such as activating brakes, indicator lights, etc. After the autonomous machine has been secured (or in some case before or during the slowing / stopping), it will transmit 803 a signal regarding the condition. The signal may be transmitted to any combination of the operator, an on-site worker or supervisor, a monitoring service, etc. For purposes of this example, the term “operator” will be used to refer to any combination of entities that both receives the transmission 803, and deals with the transmission as described below.

[0082] The transmission 803 is sent via wireless signaling, which could use any combination of radio and optical transmissions. For example, the operator may have an Internet-connected mobile device and the autonomous device may also have an Internet data connection. This allows communicating with the operator via cellular or Internet protocols, e.g., a TCP / IP socket, SMS message, etc. The transmission 803 may include a minimal set of information (e.g., “Machine XYX has stopped”) or a more detailed set of information (e.g., “Machine XYZ has encountered an obstacle, retry maneuver limit exceeded, error code A125”).

[0083] In addition to the transmission 803, the machine may also gather information regarding the condition as indicated at block 804. This may include the information directly relevant to the condition 801 (e.g., particular sensor readings and / or coding instructions that triggered the condition), but may also include other state information not necessarily connected to the triggering condition. This state information may include navigation data such as machine pose, which includes location and orientationof the machine that is measured by any combination of an inertial measurement unit (IMU), a global navigation satellite system (GNSS) sensor, and a wheel encoder. Some sensor data may be used for navigation and for other purposes, such as camera images taken at the time the condition was detected, activation of bump or lift sensors, etc. Other data may also be part of the state information, such as measurements of ambient conditions (e.g., temperature, humidity), battery / fuel status, etc. Block 804 is shown as optional, because in some cases such data may not be gathered until the operator has remotely accessed the machine, assuming such information is needed at all.

[0084] At block 805, the operator connects to an operational module of the autonomous machine. Generally, an operational module allows remote control of the machine, and may act as a proxy for some or all physical controls on the machine, e.g., work implement on / off, move forward or reverse, steering, capture image from camera, etc. Assuming the autonomous machine has Internet access, this type of control can be implemented using an Internet connection, e.g., a secure TCI / IP socket, virtual private network (VPN), real-time messaging protocol (RTMP), real-time streaming protocol (RTSP), etc. In other embodiments, direct control may be established using a radio link or other local data transmission means. Such direct connections may also utilize Internet protocols (e.g., packet switched connections) but need not rely on Internet infrastructure (e.g., cellular network base stations and Internet routers).

[0085] In other embodiments, the operator may have access or use a UAV as shown in FIG. 7. This may be considered an extension of the remote access of the autonomous machine, which may provide an aerial view of the autonomous work machine. In some embodiments, the UAV may further be able to establish a line-of-sight data link with the autonomous machine, thereby acting as a data relay between the machine and an operator mobile device. In some cases, a fixed camera such as security camera 713 shown in FIG. 7 may be accessible and used for viewing the autonomous machine in some instances.

[0086] Based on what the operator finds after remote connection to the autonomous machine (or an intermediary device), the operator and / or software of the machine can make a determination on whether the operator can resolve the condition remotely, as indicated byblock 806. For example, the operator may be able to view camera images that indicate a suspected unsafe condition is erroneous, e.g., a reported obstacle that does not exist, but was reported due to a visual processing error or sensor error. In other cases, the condition may be validly detected, but one that the operator can clear remotely. For example, the autonomous machine may be stuck in position and unable to extricate itself, but a skilled operator might be able to do so remotely.

[0087] These remedial actions that can be performed by the operator are generally indicated by operator remote control block 807. The remote control can include any type of hardware or software settings. For example, some conditions may be cleared by the operator selecting a user interface element on a mobile device to clear an error, flag, register, or the like. Other conditions may involve the operator remotely controlling a motor, actuator or the like. The latter may also involve providing sensor feedback to the operator, e.g., a video feed, telemetry stream, etc.

[0088] If the condition is cleared or resolved remotely as indicated by block 808, then the work 800 can resume. If not, the machine can either remain halted or return to base as indicated at block 809. If the condition prevents movement or represents a hazard, then the decision at block 809 may be to remain in place and have a person visit the autonomous machine, e.g., for repair or collection. If the condition still permits the autonomous machine to navigate (e.g., work implement has failed but machine can still move), the machine may navigate back to a central collection point where the fleet will gather at the completion of work.

[0089] Because autonomous work machines may incorporate more sensors that typical non-autonomous machines, the autonomous machines can leverage the sensors for other purposes besides navigation. One example is self-health, in which the autonomous machine uses sensors to detect operational status and possible or actual machine faults. Such sensors may be dedicated to diagnostics (e.g., vibration sensors, temperature sensors) or may be primarily included for other purposes (e.g., navigation cameras, microphones). These sensors can be monitored during use, but also during downtime, as illustrated in the block diagram of FIG. 9.

[0090] In FIG. 9, a fleet of autonomous machines 900 is located in a facility 902, e.g., a workshop, garage, shed, or the like. The fleet need not all be in the same facility or even be indoors, as indicated by outdoor machine 901 which may be part of this fleet. For multiple machines, it may be convenient to co-locate them for purposes of charging, cleaning, and other routine operations when not being used for work. Further, the machines can dedicate the downtime to taking sensor readings and running diagnostics.

[0091] In one embodiment, the autonomous machines 900, 901 run overnight / downtime processes (self-diagnostics) to ensure they are ready for the next work session, e.g., the next morning. This could involve nonmoving onboard simulation to test software and circuits. In some embodiments, should the setup allow, this may include running traction and work implement motors (e.g., at reduced speeds) and making measurements (e.g., vibration, temperature, current draw) that characterizes performance of the motors and mechanical drives coupled thereto. Further, if the machines 900, 901 have cameras, they can do visual inspections of themselves and / or each other to determine anomalous conditions (e.g., buildup of mud or debris around moving parts). The facility 902 may also include fixed or mobile cameras 908 to collect similar information.

[0092] The checks run in the background by the machines 900, 901 may include firmware checks and updates for the autonomous machines. These checks may extend to the base stations (e.g., charging stations, cleaning stations) and may be used to detect low charging current, charging interruptions, and the like. The checks may also detect other conditions that may have a deleterious effect on the machines, such as standing water, fluid leakages, building door left ajar, etc. Generally, autonomous vehicles with cameras and other sensors can be adapted to learn these conditions over time, e.g., to establish baselines and determine anomalies. Since this occurs at a fixed facility 902, another machine (e.g., a local server 904 or offsite cloud service) can gather the data from the machines, such that the machine firmware does not need to have this capability.

[0093] The diagnostic sensor data may also include data gathered during a previous work session. Such sensors could include accelerometers or microphones to listen for abnormalities. Such abnormalities may include noise from failing bearings and on hydraulics, human and animal vocalizations. Other sensors could include thermocouples orinfrared cameras to identify thermal issues. Thermal issues could indicate problems with motors, cooling systems, bearings, onboard electronics, batteries, and the like.[00941 As indicated by dashed lines between the machines 900, 901 and the server 904, the data gathered by the autonomous machines 900, 901 can be connected by a centralized entity for further analysis. The data may include not only the result of current diagnostic routines, but log data gathered during a previous work session. One advantage of having a fleet of like vehicles is that trends that, by themselves, might not be considered anomalous, can be anomalous when compared with other like machines that are being used in like conditions. For example, if a machine is consistently drawing 15% more current at its work implement than the other machines, this may be an indication that further investigation is warranted even if the current draw is within allowable limits. Similarly, if all of the machines are drawing excess current in the range from 10%-20% over nominal, this may be due to environmental factors affecting all machines (e.g., temperature, humidity, robust turf growth) and less likely to be due to an electromechanical issue.

[0095] The data collected by the server 904 or cloud service can be used to prepare a report 906, e.g., on a daily basis. For example, this can be used to determine current expendables (e.g., maximum charge available, life of cutting blades) and estimate future expendables (evaluate whether machine can finish the job). In some cases, the data can generate alerts during the downtime, e.g., charger not operating. The server 904 or cloud service can also store large amounts of data to identify long term trends, which can be used to estimate maintenance cost, predict end of useful machine life, among other things.

[0096] The report 906 can be used for planning subsequent work sessions. For example, the report 906 may include a go or no-go status for each of the machines 900, 901. Any “no-go” machines may be investigated (e.g., via an alert sent to a worker or supervisor) to see if the condition can be cleared before work is scheduled to begin. In other embodiments, a relative health score can be used to allocate the division of work for the fleet. For example, a machine that is experiencing a reduced charge capacity, increased operational current, or the like, may be allocated a smaller division of work. This type of health report may utilize onboard diagnostics to drill down to extract state machine information, such as fault information, at a fine granular level. Such schemes are known toarrange and report this type of data, such as Simple Network Management Protocol (SNMP) for networked computer devices and Unified Diagnostic Services (UDS) in automotive applications.

[0097] In one or more embodiments, one or more of the autonomous work vehicles900, 901 are selected to work on a work site. Each of the autonomous work vehicles 900, 901 are scheduled to work respective one or more different regions of the work site based on an initial work plan. The vehicles 900, 901 and / or service element 904 performing selfcheck evaluations via respective sensors of the vehicles. Based on the self-check evaluations, impacts on machine performance are predicted for the respective vehicles 900,901. An adjusted work plan is prepared based on the initial work plan and accounting for the impacts on the machine performance. The autonomous work vehicles are then deployed to work the work site using the adjusted work plan. The adjusted work plan may be communicated to a worker or supervisor before being implemented, e.g., for review and acceptances.

[0098] The self-check evaluations could be indications of battery health, tire inflation level, fluid levels, hours of operation on tires, cutters, and the like. The impacts could vary based on the type of conditions detected. For example, if a driveshaft gets loose, this could lead to immobility and / or damage. In contrast, a failing cutting motor may only impact after-cut appearance, and so may still be allowed to perform some level of work, e.g., in applications and areas where appearance is not as critical.

[0099] A system that can optimize fleet deployment based on machine health can be adapted to provide a broad range of fleet optimization on other factors. One factor is the types of machines in the fleet. If machine capabilities are being over utilized or under utilized (e.g., a particular machine is working less than full time in a particular zone or tends to run late in a zone) then the zone assignments can be rearranged to more optimally utilize machine capabilities. Or a selection of machines available to do the work can be selected for that days work based on optimizing their as-designed capability, as well as current condition as indicated in report 906.

[0100] The conditions of the work region can affect fleet optimization. If the fleet management service can obtain data (e.g., via sensors, entry of worker observations)indicating variation of conditions this may be dealt with by changing the frequency of treatment, type of treatment, etc. For example, if grass growing faster than typical, then it may be scheduled for more frequent mowing. This could be extended to future predictions, such as long term weather forecasts that predict conditions that could accelerate or retard turf growth.

[0101] Another aspect of fleet optimization involves minimizing costs. While an initial outlay of autonomous machines may be defined by simulations or trial and error, after some time, the gathering of work data can provide more accurate measurement of cost effectiveness. For example, in some scenarios using smaller, cheaper machines may minimize total cost of ownership over some time period (e.g., two years), while for other scenarios fewer more expensive machines may minimize long term costs. These scenarios could consider the option of keeping a dedicated machine / fleet at each of a plurality of work regions versus transporting a machine / fleet to the multiple work regions. These optimizations can extend to other factors besides cost, such as quality of work, emissions, noise, public goodwill, etc.

[0102] In FIG. 10, a flowchart shows a method of scheduling an autonomous work vehicle according to an example embodiment. The method involves deploying 1000 an autonomous work vehicle to a work site. The autonomous work vehicle may be scheduled to work a first region of the work site according to an initial work plan. The initial work plan estimates a scheduled end time. A worker supervises the work site while the autonomous machine is working. In one example, a second region of the work site may be worked by a worker while the autonomous work vehicle is working the first region, or the worker may be doing other non-specific task in the first region, the second region, or elsewhere.

[0103] At block 1002, it is determined (e.g., by the worker, a supervisor, automated tracking application) that the work should cease at an adjusted completion time different than the scheduled end time. This may be due to the worker finishing or soon to be finishing working the second region. During or after the autonomous work vehicle works the first region, a signal 1003 is sent to the autonomous work vehicle to complete work at an adjusted end time different than the scheduled end time based on the manual completiontime. The initial work plan is adjusted 1004 such that the autonomous work vehicle finishes working the work site by the adjusted end time.[01041 The change to the work plan may, for example, cause the work vehicle to locate to a staging area at an adjusted time for collection. If the change provides extra work time, the autonomous work vehicle may use the extra time to work the first region to a higher quality than required by the work plan and / or adjusts work parameters reduce power consumption needed to finish the work. If, in such a scenario, the autonomous work vehicle determines that there is insufficient stored energy to work the extra time, the selected autonomous machine works a portion of the extra time and returns to a staging area and waits until the adjusted end time. If the work vehicle’s work time is cut short by the adjustment, a subsequent work plan of the work site can be adjusted to remediate the at least one adjustment, e.g., schedule another work session later that week to work or rework some parts.

[0105] In FIG. 11, a flowchart shows a method of scheduling an autonomous work vehicle according to another example embodiment. The method involves deploying 1100 an autonomous work vehicle to a work site. The autonomous work vehicle is scheduled to work a region of the work site based on an initial work plan. At block 1101, an environmental condition is detected that affects the work region such that the environmental condition necessitates a change to the initial work plan. The change includes at least one of a change in a completion time of the work region and a change in a work parameter used by the autonomous work vehicle as specified in the initial work plan. An updated work plan is formed 1102 that compensates for the change to the initial work plan and the region is worked 1103 based on the updated work plan.

[0106] In some embodiments, the environmental condition prevents at least part of the at least one work region from being presently worked (e.g., flooded region, downed branches). The environmental condition may cause work to proceed slower or faster than predicted (e.g., soft ground due to moisture, turf grown less than expected). The environmental condition may causes the individual completion time to be changed from the planned completion time by greater than a threshold amount, such that conditions that cause a completion time change less than the threshold amount do not trigger detection ofthe environmental conditions. Updating of the initial work plan may involve redistributing work to other autonomous work vehicles of a fleet and / or to a human worker. The updating of the initial work plan may involve changing a work quality without affecting a work completion time of the autonomous work vehicle.

[0107] In FIG. 12, a flowchart shows a method of operating an autonomous work vehicle according to an example embodiment. The method involves deploying 1200 an autonomous work vehicle to perform work at a work site. At block 1201, a condition is detected that causes the autonomous work vehicle to halt the work. In response, an electronic signal is sent 1202 to an operator indicating the work was halted. Remote data access is facilitated 1203 (e.g., via a data connection to the vehicle and / or a data connection to another device such as UAV or security camera) that allows the operator to access sensor data pertaining to the autonomous work vehicle. Via the remote data access, the operator is allowed 1204 to perform one of: resolve the condition to allow the autonomous work vehicle to continue to work; or command the autonomous work vehicle to continue to halt the work.

[0108] The condition detected at block 1201 may include at least one of an obstacle and a stuck condition. The sensor data may include camera data that allows the operator to view at least one of the obstacle and an area surrounding the autonomous work vehicle. The sensor data may instead or in addition include navigation data (e g., from GNSS, IMU, tilt sensor, bump / lift sensor, etc.) that allows the operator to determine a current orientation and location of the autonomous work vehicle. In addition to the remote access, the operator may be presented with a graphical representation of the autonomous work vehicle and an area surrounding the autonomous work vehicle based on the navigation data.

[0109] In these embodiments, allowing 1204 the operator to resolve the condition may involve remote piloting of the autonomous work vehicle and / or performing a remote reset of the autonomous work vehicle. In some cases, facilitating 1203 the remote data access may involve accessing a video signal via a first data channel to an unmanned aerial vehicle or security camera, the video signal including a view of the autonomous work vehicle. In such a case, resolving the condition or commanding the autonomous workvehicle comprises may involve utilizing a second data channel to the autonomous work vehicle.[01101 InFIG. 13, a flowchart illustrates a method of operating an autonomous work vehicle according to another example embodiment. The method involves deploying 1300 an autonomous work vehicle to perform work at a work site. At block 1301, a lighting condition is determined 1301 that affects an ability of the autonomous work vehicle to perform the work. In response, an electronic signal is sent 1302 including instructions to change parameters of an automated lighting system separate from the autonomous work vehicle. The automated lighting system is operable to illuminate the work site, and the electronic signal effects the change via the automated lighting system.[01U] In some cases, wherein the lighting condition is a low light condition, and which case the instructions cause the automated lighting system to illuminate the work site, which may involve increasing a current illumination level. In other cases, the lighting condition may include an interference between a sensor of the autonomous work vehicle and the automated lighting system, In such a scenario, the instructions cause the automated lighting system to deactivate or adjust the at least one light emitter of the automated lighting system. The interference in these cases may an overload to an optical sensor of the autonomous vehicle. In cases where the interference occurs within a portion of the work site, and the deactivation or adjustment may only occur when the autonomous vehicle is navigating within the portion of the work site. For example, the adjustment may include a change in wavelength emitted by the lighting system, e.g., to attenuate or eliminate and interfering wavelength but allow illumination using other wavelengths.

[0112] In FIG. 14, a flowchart illustrates a method of operating an autonomous work vehicle according to another example embodiment. The method involves selecting 1400 one or more autonomous work vehicles to work on a work site, each of the autonomous work vehicles scheduled to work respective one or more different regions of the work site based on an initial work plan. Self-check evaluations are performed 1401 via respective sensors of the one or more autonomous work vehicles. Based on the self-check evaluations, impacts on machine performance are predicted 1402 for the respective one or more autonomous work vehicles. An adjusted work plan is prepared 1403 based on theinitial work plan and accounting for the impacts on the machine performance. The one or more autonomous work vehicles are deployed to work the work site using the adjusted work plan.

[0113] In FIG. 15, a block diagram illustrates hardware and software components of a system according to various embodiments. The system utilizes one or more autonomous work vehicles 1500. The illustrated vehicle 1500 includes a system controller1501 that comprises one or more circuit boards that monitor and control various system functions. The system controller includes conventional computing hardware such as one or more processors 1502, e.g., central processing units, co-processors, logic circuits, etc. The processors 1502 are coupled to memory 1503 which may include any combination of volatile memory (e.g., dynamic random-access memory) and non-volatile memory (e.g., non-volatile solid-state memory). The processors 1502 access and execute one or more computer programs or routines stored in the memory, as well as storing and retrieving other data to / from memory such as factory and user settings, logging data, etc.

[0114] The processors 1502 are coupled to one or more input / output (I / O) interfaces 1504. The I / O interface 1504 facilitates communications between the processors1502 and various functional hardware modules described herein, separate lines (e.g., general input I / O, or GPIO lines) that send / receive analog and or digital signals that can be set and / or read by the processors 1502. Some or all of the hardware modules may be coupled to a common data bus, such as a CAN bus, I2C bus, Ethernet bus, etc. The data busses may include both commonly-coupled busses (e.g., with shared media access) or individual control lines.

[0115] The system controller 1501 is coupled to sensors 1505 which provide signals for control functions such as power control, navigation, implement control, user input processing, diagnostics, etc. The system controller 1501 is coupled to a power system 1506, which may include energy storage or generation devices (e.g., batteries, generators, alternators), power conditioning circuits (e.g., circuit breakers, voltage conversion, power smoothing) and power distribution (e.g., wires, power busses).

[0116] The system controller 1501 is coupled to a network interface 1507 that facilitates network communications with other devices (e.g., infrastructure service 1520and mobile device 1530). While the communications over the interface 1507 may use conventional wireless network protocols such as cellular data, Bluetooth, and WiFi, the communications may include proprietary protocols and / or point-to-point protocols that do not require a common network data transmission infrastructure.

[0117] Other hardware modules of the work vehicle 1500 include a user interface 1508 (e.g., control inputs, output devices such as lights and displays), a work unit 1509 (e.g., motors, clutches, brakes, cylinders, and the like that control a work implement), a drive / traction unit 1510 (e.g., motors, clutches, brakes and the like that control mobility wheels or treads). Note that the illustrated hardware components are provided for illustration and may be arranged differently than shown. For example, some sensors 1505 may be integrated into other modules and the signals generated by the sensors may be accessible by the system controller or access may be limited to a local controller or logic circuit of the device, e.g., a motor’s internal thermal cutoff temperature sensor. Note that a UAV may be considered a work vehicle under this scenario, although it may not include a work unit 1509 for performing ground work, and the drive / traction unit 1510 would include propellers and the like.

[0118] In the scenarios described elsewhere herein, the autonomous work vehicle 1500 interacts with an infrastructure service 1520 (e.g., local server, data center server) and / or a mobile device 1530 (e.g., smartphone, laptop). These devices are shown with their own processors 1521, 1531, memory 1522, 1532, and network interfaces 1523, 1533, as well as a user interface 1534 on the mobile device 1530.

[0119] The memory 1503, 1522, 1532 includes computer-readable instructions or applications that, when executed cause the processors 1502, 1521, 1531 to perform various calculations and / or issue commands. That is to say, the processors and memory may together define a computing apparatus operable to process input data and generate the desired output to one or more components / devices. For example, the apparatus may perform the methods in the flowcharts provided herein. Furthermore, a set of predefined standards (e.g., APIs, protocols) and / or a common code base can be used to ensure the machines in the system can intercommunicate. An example of this type of standard and / or code base is shown in scheduling and maintenance block 1540 in FIG. 15.

[0120] The blocks within the scheduling and maintenance block 1540 represent different data abstractions or functional modules that are used to implement software that is compatible between the work vehicle 1500, the infrastructure service 1520, and / or the mobile device 1530. A work scheduling module 1541 deals with date structures (e.g., as provided in Java™ Date and Calendar classes), conversion between calendar dates and machine time (e.g., epoch time), presentation of dates, schedules, and events, etc.

[0121] A work planning module 1542 includes knowledge of work tasks (e.g., mowing, fertilizing, debris collection), work site geometry (e.g., boundaries and obstacles defined using a common reference such latitude / longitude), path planning (e.g., path geometry), machine capabilities (e.g., type of work, speed, turning radius) and other concepts that allow giving the work vehicle precise instructions to perform a specific, well- defined task and self-evaluate progress and problem issues related to the task. In order to interact with work scheduling 1541 and other modules (e.g., fleet management 1545), the work planning module 1542 may define fungible blocks of works (e.g., a work unit) that enable assembling larger tasks into smaller segments, which can facilitate flexible rescheduling and reassignment of work between machines and / or between a person and a machine.

[0122] A remote access module 1543 allows the service 1520 and / or mobile device 1530 to obtain sensor data (and possibly other data, such as software and settings) from the work vehicle 1500, as well obtaining similar data from loT devices 1525. The remote access module 1543 also provides the ability to send commands to the work vehicle 1500 and / or loT devices 1525 to perform hardware functions (e.g., actuate a motor) and software functions (e.g., change settings, reboot). The remote access module 1543 may also include a suite of networking protocols to facilitate robust and secure remote access, e.g., virtual private networks, encryption, mesh networking, etc.

[0123] A diagnostics module 1544 includes data structures used to define machine status, from low level performance data (e.g., error codes, system voltages, sensor readings) to higher level status (e.g., go, no-go). The data structures may be hierarchal and resemble a hierarchy of machine communications and control (e.g., motor to motor controller to system controller). The diagnostics module 1544 may also include facilitiesfor estimating failures (e.g., mathematical models of machine failures) and other statistical and scientific algorithms for analyzing data from a fleet of the work vehicles 1500.[01241 A fleet management module 1545 uses specific information that is provided by two or more work vehicles 1500, such as model / serial number, hours of operation, diagnostics, location, etc., in order to manage the maintenance and deployment of a fleet. This may include tracking statistics such as individual and collective machine usage, cost estimates, financial forecasting, work availability forecasting, etc. The collection of such data may be enhanced by the autonomous functionality (e.g., processors, sensors, network communications) of the work vehicles 1500 such that it frees operators and managers from performing tedious or repetitious tasks such as filling out forms in order to obtain detailed fleet data.

[0125] It is noted that the terms “have,” “include,” “comprises,” and variations thereof, do not have a limiting meaning, and are used in their open-ended sense to generally mean “including, but not limited to,” where the terms appear in the accompanying description and claims. Further, “a,” “an,” “the,” “at least one,” and “one or more” are used interchangeably herein. Moreover, relative terms such as ’’left,” “right,” “front,” “fore,” “forward,” “rear,” “aft,” “rearward,” “top,” “bottom,” “side,” “upper,” “lower,” “above,” “below,” “horizontal,” “vertical,” and the like may be used herein and, if so, are from the perspective shown in the particular figure, or while the machine is in an operating configuration. These terms are used only to simplify the description, however, and not to limit the interpretation of any embodiment described. As used herein, the terms “determine” and “estimate" may be used interchangeably depending on the particular context of their use, for example, to determine or estimate a position or pose of a vehicle, boundary, obstacle, etc.

[0126] Further, it is understood that the description of any particular element as being connected to or coupled to another element can be directly connected or coupled, or indirectly coupled / connected via intervening elements.

[0127] Unless otherwise indicated, all numbers expressing feature sizes, amounts, and physical properties used in the specification and claims are to be understood as being modified in all instances by the term “about.” Accordingly, unless indicated to thecontrary, the numerical parameters set forth in the foregoing specification and attached claims are approximations that can vary depending upon the desired properties sought to be obtained by those skilled in the art utilizing the teachings disclosed herein. The use of numerical ranges by endpoints includes all numbers within that range (e.g., 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5) and any range within that range.

[0128] The various embodiments described above may be implemented using circuitry, firmware, and / or software modules that interact to provide particular results. One of skill in the arts can readily implement such described functionality, either at a modular level or as a whole, using knowledge generally known in the art. For example, the flowcharts and control diagrams illustrated herein may be used to create computer-readable instructions / code for execution by a processor. Such instructions may be stored on a non- transitory computer-readable medium and transferred to the processor for execution as is known in the art. The structures and procedures shown above are only a representative example of embodiments that can be used to provide the functions described hereinabove.

[0129] Note that any components described herein using terms such as “processor,” “controller,” “logic circuit,” “CPU,” or the like may be implemented using a plurality of discrete units operating together. For example, a processer that performs a series of steps or operations may be construed as two or more processors operating cooperatively to perform the steps. Similarly, other processing hardware such as memory and input-output may perform the described functions with multiple discrete units operating cooperatively or being coordinated by another unit, e.g., by a central processor or processors.

[0130] The foregoing description of the example embodiments has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the embodiments to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. Any or all features of the disclosed embodiments can be applied individually or in any combination and are not meant to be limiting, but purely illustrative. It is intended that the scope of the invention be limited not with this detailed description, but rather determined by the claims appended hereto.

Claims

CLAIMS:

1. A method comprising: deploying an autonomous work vehicle to a work site, the autonomous work vehicle scheduled to autonomously perform work at the work site according to an initial work plan, the initial work plan estimating a scheduled end time for the work; working or monitoring the work site by a worker while the autonomous work vehicle is performing the work; determining that the worker will cease the work at an adjusted completion time different than the scheduled end time; during or after the autonomous work vehicle works the work site, signaling to the autonomous work vehicle to complete work at an adjusted end time different than the scheduled end time based on the adjusted completion time; and via a processor, adjusting the initial work plan such that the autonomous work vehicle autonomously finishes the work by the adjusted end time.

2. The method of claim 1, further comprising causing the autonomous work vehicle to locate to a staging area at the adjusted end time for collection by the worker.

3. The method of claim 1 or 2, wherein the adjusted end time is later than the scheduled end time by an extra time.

4. The method of claim 3, wherein the autonomous work vehicle uses the extra time to perform the work to a higher quality than required by the work plan.

5. The method of claim 3, wherein if the autonomous work vehicle determines that there is insufficient stored energy to work the extra time, the autonomous work vehicle works a portion of the extra time and returns to a staging area and waits until the adjusted end time.

6. The method of claim 3, wherein the autonomous work vehicle uses the extra time to reduce power consumption needed to finish the work.

7. The method of claim 1 or 2, wherein the adjusted end time is earlier than the scheduled end time by a shortage time.

8. The method of claim 7, wherein the autonomous work vehicle performs at least one adjustment comprising: performing a remaining portion of the work to a lower quality than required by the work plan; and skipping a remaining portion of the work region.

9. The method of claim 8, further comprising updating a subsequent work plan of the work site to remediate the at least one adjustment.

10. The method of any previous claim, wherein adjusting the initial work plan comprises adding at least one of a second autonomous work vehicle, the worker, and a different worker to assist the autonomous work vehicle in finishing the work by the adjusted end time.

11. The method of any previous claim, further comprising sending an alert to the worker that indicates or describes the adjusted work plan.

12. The method of any previous claim, wherein the worker is performing manual work at the work site while the work vehicle is performing the work, and wherein the adjusted completion time is due to the worker finishing earlier or later than planned.

Images