Image notification for autonomous grounds maintenance machines

Abstract

Autonomous machines and methods for operating autonomous machines are described. A controller of the autonomous machine may direct navigation of the autonomous machine within a work region using a set of wheels and a propulsion controller of the autonomous machine. The controller may determine that an operational obstruction to grounds maintenance has been encountered during navigation. In response to the determination that the operational obstruction has been encountered, the controller may capture one or more images using a vision system of the autonomous machine. Additionally, the controller may determine that the operational obstruction is unknown. In response to the determination that the operational obstruction is unknown, the controller may generate an unknown obstruction notification including the one or more images and transmit the unknown obstruction notification to an external device.

Description

IMAGE NOTIFICATION FOR AUTONOMOUS GROUNDS MAINTENANCEMACHINESRELATED PATENT DOCUMENTS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 446,588, filed on February 17, 2023, which is incorporated herein by reference in its entirety.

[0002] The present disclosure relates to autonomous machine navigation. In particular, the present disclosure relates to autonomous machine navigation for grounds maintenance machines.

[0003] In recent years, robotic lawn mowers adapted to autonomously mow have become more prevalent. These mowers typically operate unattended within a predefined boundary of a property, relying upon various boundary detection capabilities to ensure the mower is constrained to specific work areas within the predefined boundary.SUMMARY

[0004] Embodiments of the present disclosure relate to navigation for autonomous machines within a work region and obstruction detection. In particular, embodiments of the present disclosure are directed to determination of new or otherwise unknown obstructions to performance of ground maintenance. The techniques of the present disclosure allow autonomous machines to determine that an unknown obstruction has been encountered and provide an image notification to allow a user to identify what the obstruction is or identify a source of the unknown obstruction. Additionally, the notification may allow the user to provide feedback to improve future obstacle detection and avoidance.

[0005] In one aspect, the present disclosure relates to an autonomous machine to autonomously provide grounds maintenance in a work region. The autonomous machine may include a housing coupled to a maintenance implement, a set of wheels supporting the housing over a ground surface, a propulsion controller operably coupled to the set of wheels, a vision system comprising one or more cameras adapted to capture image data, and a controller comprising one or more processors. The controller may be operatively coupled to the vision system and the propulsion controller. The controller may be configured to direct navigation ofthe autonomous machine within the work region and determine that an operational obstruction to grounds maintenance has been encountered. In response to the determination that the operational obstruction has been encountered, the controller may be further configured to capture one or more images of the work region using the vision system. The controller may also be configured to determine that the operational obstruction is an unknown operational obstruction and generate, in response to the unknown operational obstruction, an unknown obstruction notification including the one or more images. The controller may also be configured to transmit the unknown obstruction notification to an external device.

[0006] In another aspect, the present disclosure relates to a method for operating an autonomous machine in a work region. The method may include directing navigation of the autonomous machine within the work region using a set of wheels and a propulsion controller of the autonomous machine and determining that an operational obstruction to grounds maintenance has been encountered. The method may further include capturing one or more images of the work region using a vision system of the autonomous machine in response to the determination that the operational obstruction has been encountered. Still further, the method may include determining that the operational obstruction is an unknown operational obstruction and generating, in response to the unknown operational obstruction, an unknown obstruction notification including the one or more images. The method may further include transmitting the unknown obstruction notification to an external device.

[0007] The summary is not intended to describe each embodiment or every implementation of the present disclosure. A more complete understanding will become apparent and appreciated by reference to the following detailed description and claims taken in view of the accompanying figures of the drawing.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Illustrative embodiments will be further described with reference to the figures of the drawing, wherein:

[0009] FIG. l is a perspective view of an autonomous grounds maintenance machine with a vision system in accordance with one embodiment of the present disclosure.

[0010] FIG. 2 is a diagrammatic elevation side view of an autonomous grounds maintenance machine of FIG. 1.

[0011] FIG. 3 is a plan view of a work region within a boundary that may be operated within using the machine of FIG. 1 in accordance with one embodiment of the present disclosure.

[0012] FIG. 4 is a plan view of a zone within the work region of FIG. 3 and an example of pathing of the machine of FIG. 1 within a boundary defining the zone in accordance with one embodiment of the present disclosure.

[0013] FIG. 5 is a plan view of a work region that includes an exclusion zone and a transit zone that may be operated within using the machine of FIG. 1 in accordance with one embodiment of the present disclosure.

[0014] FIG. 6 is a plot of elevation versus distance along a travel path in accordance with one embodiment of the present disclosure.

[0015] FIG. 7 is a plot of grade versus distance along the travel path of FIG. 4.

[0016] FIG. 8 is an overhead view of the machine of FIG. 1 performing an obstacle avoidance method in accordance with one embodiment of the present disclosure.

[0017] FIG. 9 is an overhead view of the machine of FIG. 1 performing another obstacle avoidance method in accordance with one embodiment of the present disclosure.

[0018] FIG. 10 is a schematic representation of various systems of the machine of FIG. 1 in accordance with one embodiment of the present disclosure.

[0019] FIG. 11 is a schematic representation of sensors providing data to a navigation system that communicates with a platform of the machine of FIG. 1 in accordance with one embodiment of the present disclosure.

[0020] FIG. 12 is as schematic representation of sensor data input and sensor fusion processing in a sensor fusion module for use with the navigation system of FIG. 11 in accordance with one embodiment of the present disclosure.

[0021] FIG. 13 is a flowchart illustration of one example of a method of operating an autonomous machine within a work region.

[0022] FIG. 14 is a diagrammatic illustration of a graphical depiction of an unknown obstruction notification.

[0023] The figures are rendered primarily for clarity and, as a result, are not necessarily drawn to scale. Moreover, various structure / components, including but not limited to fasteners, electrical components (wiring, cables, etc.), and the like, may be shown diagrammatically orremoved from some or all of the views to better illustrate aspects of the depicted embodiments, or where inclusion of such structure / components is not necessary to an understanding of the various illustrative embodiments described herein. The lack of illustration / description of such structure / components in a particular figure is, however, not to be interpreted as limiting the scope of the various embodiments in any way.DETAILED DESCRIPTION

[0024] In the following detailed description of illustrative embodiments, reference is made to the accompanying figures of the drawing which form a part hereof. It is to be understood that other embodiments, which may not be described and / or illustrated herein, are certainly contemplated.

[0025] All headings provided herein are for the convenience of the reader and should not be used to limit the meaning of any text that follows the heading, unless so specified. Moreover, unless otherwise indicated, all numbers expressing quantities, and all terms expressing direction / orientation (e.g., vertical, horizontal, parallel, perpendicular, etc.) in the specification and claims are to be understood as being modified in all instances by the term “about.” The term “and / or” (if used) means one or all of the listed elements or a combination of any two or more of the listed elements. The term “i.e ” is used as an abbreviation for the Latin phrase id est and means “that is.” The term “e.g.,” is used as an abbreviation for the Latin phrase exempli gratia and means “for example.”

[0026] Embodiments of the present disclosure provide autonomous machine navigation methods and systems to autonomously navigate and operate within a boundary of a work region, particularly for grounds maintenance, such as lawn mowing. The autonomous machine may be configured in different modes to carry out various navigation functionality, such as training mode, offline mode, and online mode. The autonomous machine may define one or more boundaries of a work region using a vision system and a non-visi on-based sensor, for example, instead of using a boundary wire. The autonomous machine may correct a position or orientation within the work region, which is determined or estimated using one or more non- vision-based sensors, by using a position or orientation determined by the vision system. Training the autonomous machine may be performed during a training mode, which may include one or more phases, such as a touring phase and a mapping phase.

[0027] Some aspects described herein relate to defining a boundary of a work region using a vision system and a non-vision-based sensor. Some aspects of the present disclosure relate tocorrecting an estimated position within the work region using a vision system. The vision system may utilize one or more cameras. Images may be recorded by directing the autonomous machine along a desired boundary path (e.g., during a training mode). Algorithms may be used to extract features, to match features between different images, and to generate a three- dimensional point cloud (3DPC, or 3D point cloud) corresponding to at least the work region (e.g., during an offline mode). Positions and orientations of the autonomous machine during image recording may be determined for various points in the 3DPC, for example, based on the positions of various points in the 3DPC and positions of the corresponding features in the recorded images. Positions and orientations may also be recovered directly during generation of the point cloud. At least the position information may be used to determine a boundary for the work region for subsequent navigation of the autonomous machine in the work region. During operation (e.g., during an online mode), the vision machine may record operational images and determine a vision-based position and orientation of the autonomous machine. The vision-based position may be used to update, or correct errors in, a determined or estimated position based on non-vision-based sensors. Various aspects described herein relate to utilizing limited computing resources while achieving suitable navigation of the work region. The processing of recorded images may occur during an offline mode, for example, when the autonomous machine is charging overnight. The vision system may be used at a low refresh rate to complement a high refresh rate non-vision-based navigation system.

[0028] While described herein in illustrative examples as an autonomous mower, such a configuration is illustrative only as systems and methods described herein also have application to other autonomous machines including, for example, commercial mowing products (e.g., riding fairway or greens mowers that are driven by a user), other ground working machines or vehicles (e.g., debris blowers / vacuums, aerators, dethatchers, material spreaders, snow throwers, weeding machines for weed remediation), indoor working vehicles such as vacuums and floor scrubbers / cleaners (e.g., that may encounter obstacles), construction and utility vehicles (e.g., trenchers), observation vehicles, and load transportation (e.g., including people and things, such as people movers and hauling equipment). Furthermore, the autonomous machines described herein may employ various one or more types of navigation, such as random, modified random, or specific path planning, to carry out their intended functionality.

[0029] It is noted that the terms “have,” “include,” “comprises,” and variations thereof, do not have a limiting meaning, and are used in their open-ended sense to generally mean “including, but not limited to,” where the terms appear in the accompanying description andclaims. Further, “a,” “an,” “the,” “at least one,” and “one or more” are used interchangeably herein. Moreover, relative terms such as “left,” “right,” “front,” “fore,” “forward,” “rear,” “aft,” “rearward,” “top,” “bottom,” “side,” “upper,” “lower,” “above,” “below,” “horizontal,” “vertical,” and the like may be used herein and, if so, are from the perspective shown in the particular figure, or while the machine 100 is in an operating configuration (e.g., while the machine 100 is positioned such that wheels 106 and 108 rest upon a generally horizontal ground surface 103 as shown in FIG. 2). These terms are used only to simplify the description, however, and not to limit the interpretation of any embodiment described.

[0030] As used herein, the terms “determine,” “estimate,” and “generate” may be used interchangeably depending on the particular context of their use, for example, to determine or estimate a position or pose of the mower 100 or a feature, particularly with respect to data.

[0031] While the construction of the actual grounds maintenance machine is not necessarily central to an understanding of embodiments of this disclosure, FIGS. 1 and 2 illustrate one example of an autonomous grounds maintenance machine (e.g., an autonomously operating vehicle, such as an autonomous lawn mower 100) of a lawn mowing system. FIG. 1 shows a perspective view of the mower 100 and, for simplicity of description, FIG. 2 shows the mower 100 illustrated schematically. The mower 100 may include a housing 102 (e.g., frame or chassis with a shroud) that carries and / or encloses various components of the mower as described below. The mower 100 may further include ground support members, such as wheels, rollers, or tracks. In the illustrated embodiment, ground support members shown includes one or more rear wheels 106 and one or more front wheels 108, that support the housing 102 upon a ground (grass) surface 103. As illustrated, the front wheels 108 are used to support a front end portion 134 of the mower housing 102 and the rear wheels 106 are used to support the rear end portion 136 of the mower housing.

[0032] One or both rear wheels 106 may be driven by a propulsion system (e.g., including one or more electric wheel motors 104) to propel the mower 100 over the ground surface 103. In some embodiments, the front wheels 108 may freely caster relative to the housing 102 (e.g., about vertical axes). In such a configuration, mower direction may be controlled via differential rotation of the two rear wheels 106 in a manner similar to a conventional zero-turnradius (ZTR) riding mower. That is to say, the propulsion system may include a separate wheel motor 104 for each of a left and right rear wheel 106 so that speed and direction of each rear wheel may be independently controlled. In addition, or alternatively, the front wheels 108 could be actively steerable by the propulsion system (e.g., including one or more steer motors105) to assist with control of mower 100 direction, and / or could be driven by the propulsion system (i.e., to provide a front-wheel or all-wheel drive mower).

[0033] An implement (e.g., a grass cutting element, such as a blade 110) may be coupled to a cutting motor 112 (e.g., implement motor) carried by the housing 102. When the motors 112 and 104 are energized, the mower 100 may be propelled over the ground surface 103 such that vegetation (e.g., grass) over which the mower passes is cut by the blade 110. While illustrated herein using only a single blade 110 and / or motor 112, mowers incorporating multiple blades, powered by single or multiple motors, are contemplated within the scope of this disclosure. Moreover, while described herein in the context of one or more conventional “blades,” other cutting elements including, for example, disks, nylon string or line elements, knives, cutting reels, etc., are certainly possible without departing from the scope of this disclosure. Still further, embodiments combining various cutting elements, e.g., a rotary blade with an edgemounted string trimmer, are also contemplated.

[0034] The mower 100 may further include a power source, which in one embodiment, is a battery 114 having a lithium-based chemistry (e.g., lithium-ion). Other embodiments may utilize batteries of other chemistries, or other power source technologies (e.g., solar power, fuel cell, internal combustion engines) altogether, without departing from the scope of this disclosure. It is further noted that, while shown as using independent blade and wheel motors, such a configuration is illustrative only as embodiments wherein blade and wheel power is provided by a single motor are also contemplated.

[0035] The mower 100 may further include one or more sensors to provide location data. For instance, some embodiments may include a global positioning system (GPS) receiver 116 (or other position sensor that may provide similar data) that is adapted to estimate a position of the mower 100 within a work region and provide such information to a controller 120 (described below). In other embodiments, one or more of the wheels 106, 108 may include encoders 118 that provide wheel rotation / speed information that may be used to estimate mower position (e.g., based upon an initial start position) within a given work region. The mower 100 may also include a sensor 115 adapted to detect a boundary wire, which could be used in addition to other navigational techniques described herein.

[0036] The mower 100 may include one or more front obstacle detection sensors 130 and one or more rear obstacle detection sensors 132, as well as other sensors, such as side obstacle detection sensors (not shown). The obstacle detection sensors 130, 132 may be used to detect an obstacle in the path of the mower 100 when travelling in a forward or reverse direction,respectively. The mower 100 may be capable of mowing while moving in either direction. As illustrated, the sensors 130, 132 may be located at the front end portion 134 or rear end portion 136 of the mower 100, respectively.

[0037] The sensors 130, 132 may use contact sensing, non-contact sensing, or both types of sensing. For example, both contact and non-contact sensing may be enabled concurrently or only one type of sensing may be used depending on the status of the mower 100 (e.g., within a zone or travelling between zones). One example of contact sensing includes using a contact bumper protruding from the housing 102, or the housing itself, that can detect when the mower 100 has contacted the obstacle. Non-contact sensors may use acoustic or light waves to detect the obstacle, sometimes at a distance from the mower 100 before contact with the obstacle (e.g., using infrared, radio detection and ranging (radar), light detection and ranging (lidar), etc.).

[0038] The mower 100 may include one or more vision-based sensors to provide localization data, such as position, orientation, or velocity. The vision-based sensors may include one or more cameras 133 that capture or record images for use with a vision system. The cameras 133 may be described as part of the vision system of the mower 100. Types of images include, for example, training images and / or operational images.

[0039] The one or more cameras may be capable of detecting visible light, non-visible light, or both. The one or more cameras may establish a total field of view of at least 30 degrees, at least 45 degrees, at least 60 degrees, at least 90 degrees, at least 120 degrees, at least 180 degrees, at least 270 degrees, or even at least 360 degrees, around the autonomous machine (e.g., mower 100). The field of view may be defined in a horizontal direction, a vertical direction, or both directions. For example, a total horizontal field of view may be 360 degrees, and a total vertical field of view may be 45 degrees. The field of view may capture image data above and below the height of the one or more cameras.

[0040] In some embodiments, the mower 100 includes four cameras 133. One camera 133 may be positioned in each of one or more of directions including a forward direction, a reverse direction, a first side direction, and a second side direction (e.g., Cardinal directions relative to the mower 100). One or more camera directions may be positioned orthogonal to one or more other cameras 133 or positioned opposite to at least one other camera 133. The cameras 133 may also be offset from any of these directions (e.g., at a 45 degree or another non-right angle).

[0041] The mower 100 may be guided along a path, for example, in a manual manner using handle assembly 90. In particular, manual direction of the mower 100 may be usedduring a training mode to learn a work region or a boundary associated with the work region. The handle assembly 90 may extend outward and upward from a rear end portion 136 of the mower 100.

[0042] The camera 133 positioned in a forward direction may have a pose that represents the pose of the autonomous machine. The pose may be a six-degrees of freedom pose, which may include all position and orientation parameters for a three-dimensional space (see also description related to FIG. 6). In some embodiments, the position and orientation of the cameras may be defined relative to a geometric center of the mower 100 or relative to one of the edges of the mower 100.

[0043] Sensors of the mower 100 may also be described as either vision-based sensors and non-vi si on-based sensors. Vision-based sensors may include cameras 133 that are capable of recording images. The images may be processed and used to build a 3DPC or used for optical odometry (e.g., optical encoding). Non-vision-based sensors may include any sensors that are not cameras 133. For example, a wheel encoder that uses optical (e.g., photodiode), magnetic, or capacitive sensing to detect wheel revolutions may be described as a non-vision-based sensor that does not utilize a camera. Wheel encoding data from a wheel encoder may be also described as odometry data. In some embodiments, non-vision-based sensors do not include a boundary wire detector. In some embodiments, non-vision-based sensors do not include receiving signals from external system, such as from a GPS satellite or other transceiver.

[0044] Optical encoding may be used by taking a series or sequence of images and comparing features in the images to determine or estimate a distance traveled between the images. Optical encoding may be less susceptible to wheel slippage than a wheel encoder for determining distance or speed.

[0045] In addition to the sensors described above, other sensors now known or later developed may also be incorporated into the mower 100.

[0046] The mower 100 may also include a controller 120 adapted to monitor and control various mower functions. The controller 120 may include a processor 122 that receives various inputs and executes one or more computer programs or applications stored in memory 124. The memory 124 may include computer-readable instructions or applications that, when executed, e.g., by the processor 122, cause the controller 120 to perform various calculations and / or issue commands. That is to say, the processor 122 and memory 124 may together define a computing apparatus operable to process input data and generate the desired output to one or more components / devices. For example, the processor 122 may receive various input dataincluding positional data from the GPS receiver 116 and / or encoders 118 and generate speed and steering angle commands to the one or more wheel motors 104 to cause the drive wheels 106 to rotate (at the same or different speeds and in the same or different directions). In other words, the controller 120 may control the steering angle and speed of the mower 100, as well as the speed and operation of the cutting blade.

[0047] In general, GPS data generated based on data from the GPS receiver 116 (FIG. 1) may be used in various ways to facilitate the determining a pose of the mower 100. In some embodiments, GPS data may be used as one of the non-vision-based sensors to help determine non-vi si on-based pose data. The non-vision-based pose data may be updated or corrected using vision-based pose data. GPS data may also be used to facilitate updating or correcting an estimated pose, which may be based on non-vision-based pose data and / or vision-based pose data. In some embodiments, the GPS data may be augmented using a GPS-specific correction data, such as real-time kinematics (RTK) data. GPS-RTK data may provide a more accurate or precise location that corrects for anomalies in GPS timing compared to nominal GPS data.

[0048] Reference herein may be made to various parameters, data, or data structures, which may be handled in a controller 120, for example, by being processed by a processor 122 or stored in or retrieved from a memory 124.

[0049] The controller 120 may use the processor 122 and memory 124 in various different systems. In particular, one or more processors 122 and memory 124 may be included in each different system. In some embodiments, the controller 120 may at least partially define a vision system, which may include a processor 122 and memory 124. The controller 120 may also at least partially define a navigation system, which may include a processor 122 and memory 124 separate from the processor 122 and memory 124 of the vision system.

[0050] Each system may also be described as having its own controller 120. For example, the vision system may be described as including one controller 120 and the navigation system may be described as having another controller 120. As such, the mower 100 may be described as having multiple controllers 120. In general, as used herein, the term “controller” may be used to describe components of a “system” that provide commands to control various other components of the system.

[0051] In addition, a communication system 140 may be provided to permit the mower 100 / controller 120 to operatively communicate (e.g., via a wireless radio 117) with a communication network such as a wireless network 142, thereby allowing communication (e.g., bidirectional communication) between the mower and other devices. For example, thewireless network 140 may be a cellular or other wide area network, a local area network (e.g., Institute of Electrical and Electronics Engineers (IEEE) 802.11 local “Wi-Fi” network), or a personal area or peer-to-peer network (“P2P,” e.g., “Bluetooth” network). Other devices may communicate over the wireless network with the mower 100, including, for example, a remote computer 144, which may be configured as a mobile / cellular phone, tablet, desktop computer, notebook computer, or wearable computer. Preferably, the wireless network 142 is connected to the internet so that the user / remote computer 144 may interact with the communication system 140 regardless of the user’s location. Moreover, connection of the wireless network 140 to the internet allows communication with most any other remote computer including, for example, an internet-based (e.g., cloud-based) server 146.

[0052] The communication system 140 may also permit communication over the wireless network with a base station 258. Although not specifically illustrated, the communication system 140 may include conventional network hardware including gateways, routers, wireless access points, etc. (not shown).

[0053] While illustrated as using a centralized communication network (e.g., wherein each device connects to a central network), other embodiments may utilize a decentralized or ad-hoc network, wherein communication occurs directly between devices. Still further, while illustrated as primarily utilizing wireless communication protocols, such a configuration is not limiting as for example, various devices (e.g., the base station 258) could connect to the communication network or other devices using wired connections without departing from the scope of this disclosure.

[0054] In view of the above, it will be readily apparent that the functionality of the controller 120 may be implemented in any manner known to one skilled in the art. For instance, the memory 124 may include any volatile, non-volatile, magnetic, optical, and / or electrical media, such as a random-access memory (RAM), read-only memory (ROM), nonvolatile RAM (NVRAM), electrically-erasable programmable ROM (EEPROM), flash memory, and / or any other digital media. While shown as both being incorporated into the controller 120, the memory 124 and the processor 122 could be contained in separate modules.

[0055] The processor 122 may include any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field- programmable gate array (FPGA), and / or equivalent discrete or integrated logic circuitry. In some embodiments, the processor 122 may include multiple components, such as any combination of one or more microprocessors, one or more controllers, one or more DSPs, oneor more ASICs, and / or one or more FPGAs, as well as other discrete or integrated logic circuitry. The functions attributed to the controller 120 and / or processor 122 herein may be embodied as software, firmware, hardware, or any combination of these. Certain functionality of the controller 120 may also be performed in the cloud or other distributed computing systems operably connected to the processor 122.

[0056] In FIG. 2, schematic connections are generally shown between the controller 120 and the battery 114, one or more wheel motors 104, blade motor 112, optional boundary wire sensor 115, wireless radio 117, and GPS receiver 116. This interconnection is illustrative only as the various subsystems of the mower 100 could be connected in most any manner, e.g., directly to one another, wirelessly, via a bus architecture (e.g., controller area network (CAN) bus), or any other connection configuration that permits data and / or power to pass between the various components of the mower. Although connections with some of the sensors 130, 132, 133 are not shown, these sensors and other components of the mower 100 may be connected in a similar manner. The wireless radio 117 may communicate over a cellular or other wide area network (e.g., even over the internet), a local area network (e.g., IEEE 802.11 “Wi-Fi” radio), or a peer-to-peer (P2P) (e.g., BLUETOOTH™) network with a mobile device 140 (e.g., mobile computing device, mobile computer, handheld computing device, smartphone, cellular phone, tablet, desktop, or wearable computer, smartwatch, etc.). In turn, the remote computer 144 may communicate with other devices over similar networks and, for example, may be used to connect the mower 100 to the internet.

[0057] In some embodiments, various functionality of the controller or controllers 120 described herein may be offloaded from the mower 100. For example, recorded images may be transmitted to a remote server 142 (e.g., in the cloud) using the wireless radio 117 and processed or stored. The images stored, or other data derived from processing, may be received using the wireless radio 117 and be stored on, or further processed by, the mower 100.

[0058] FIGS. 3 and 4 show a work region 200 or a containment zone 202, 210 within the work region 200. A boundary may be defined, or determined, around the work region 200. The mower 100 may cover the work region 200 (e.g., traversed to mow the work region) using various methods. In some embodiments, the mower 100 may traverse random, semi-random, or planned paths within the work region 200. In some embodiments, other boundaries around the containment zones 202, 210 may be defined within the boundary of the work region 200 depending on the method used to cover the work region 200. For example, the containment zones 202, 210 may be travelling containment zones or static containment zones.

[0059] FIG. 3 shows one example of covering a work region 200 with the mower 100 using a plurality of zones 202, 210 (e.g., containment zones). The work region 200 may represent an outdoor area or maintenance area, such as a lawn. The mower 100 may be operated to travel through the work region 200 along a number of paths to sufficiently cut all the grass in the work region 200. The mower 100 may recharge as needed, for example, when transitioning between zones 202, 210. A recharging base or base station (similar to 258 at FIG. 5) may be located within or along the work region 200.

[0060] A boundary may be used to define the work region 200. The boundary may be defined manually, or automatically, using a training mode of the mower 100. In addition, some of the boundary may also be defined using a fixed property boundary or other type of boundary. In some embodiments, the boundary may be defined by directing the mower 100 along the work region 200, in particular, along a desired boundary path of the work region 200.

[0061] Boundaries may be defined relative to the work region 200 for different purposes. For example, a boundary may be used to define a containment zone, such as for zone 202, zone 210, or work region 200. In general, the mower 100 may be directed to travel within a boundary for a containment zone for a period of time. Another boundary may be used to define an exclusion zone. An exclusion zone may represent an area of the work region 200 for the mower 100 to avoid or travel around. For example, an exclusion zone may contain an obstacle (such as a landscaped garden) or problem area (such as a steep slope). Another boundary may be used to define a transit zone, which may also be described as a transit path. In general, a transit zone is a zone connecting two other zones, such as a path connecting different containment zones. A transit zone may also be defined between a point in the work region and a “home” location or base station. A maintenance task may or may not be performed in the transit zone. For example, the mower 100 may not mow grass in a transit zone. In an example involving a yard divided by a driveway, a transit zone may include the entire driveway, or at least a path across the driveway, between two grassy parts of a lawn for the mower 100 to traverse.

[0062] The work region 200 may be mapped with a terrain map. For example, the terrain map may be developed during a teaching mode of the mower, or during subsequent mowing operations. Regardless, the terrain map may contain information about the terrain of the work region 200, for example, elevation, grade, identified obstacles (e.g., permanent obstacles), identified stuck areas (e.g., areas the mower has gotten stuck whether due to grade or othertraction conditions), or other information that may facilitate the ability of the mower 100 to traverse the work region.

[0063] The coordinate system 204 is shown for illustrative purposes only. The resolution of points stored in the terrain map may be sufficient to provide useful elevation and / or grade information about the terrain in the work region 200 (e.g., on the order of feet or decimeters). For example, the resolution of points may correspond to spacing between points being less than or equal the width of the mower 100. In some cases, different functions of path planning may use different levels of resolution. For example, path planning that maps containment or exclusion zones may have the highest resolution (e.g., on the order of centimeters). In other words, the resolution of points proximate to, adjacent to, or near irregular boundaries or obstacles may have a finer granularity.

[0064] The mower 100 may start coverage of the work region 200, e.g., starting at a boundary of the work region. The mower 100 may determine a first zone 202. The zone 202 may be located adjacent to a boundary of the work region 200 or, as illustrated, may be located further within the work region. In one embodiment, the zone 202 covers the entire work region 200.

[0065] In another embodiment, the zone 202 does not cover the entire work region 200. When the mower 100 is finished mowing the zone 202, the mower may start another zone (e.g., zone 210, which may be dynamic or fixed) to continue mowing.

[0066] The mower 100 may determine a starting coordinate 206, or starting point, within the first zone 202. For example, the starting coordinate 206 may be selected from the highest elevational point within the zone 202 or somewhere at the edge of the zone 202. The mower 100 may rotate, if needed, to orient itself toward the starting coordinate 206 from its current position at the boundary of the work region 200. The mower 100 may propel itself toward the starting coordinate 206.

[0067] After arriving at the starting coordinate 206, the mower 100 may begin travelling through the zone 202 to cut grass within the zone. As described below, the mower 100 may use randomly-generated destination waypoints within the zone. In addition, or in the alternative, the mower 100 may use a planned pattern with planned waypoints within the zone. Such pattern mowing may use planned waypoint creation to cover the zone.

[0068] When the mower 100 arrives at a final destination waypoint 208, the mower is finished cutting grass within the current zone 202. The mower 100 may determine a next zone 210 (which may or may not be immediately adjacent to the zone 202) and a next starting point212 within the next zone. The mower 100 may orient itself and begin travelling to the next starting point 212. The path 220 from a final destination waypoint 208 in a zone 202 or toward a next starting point 212 in a next zone 210 may be described as a “go to goal” path (e.g., which may traverse a transit zone).

[0069] Once the mower 100 arrives at the next starting point 212, the mower 100 may begin travelling through the next zone 210. The process of generating and working travelling containment zones may be repeated a number of times to provide sufficient coverage of the work region 200.

[0070] In FIG. 4, one method 222 of covering a zone 202 is shown as an overhead view illustrating a sequence of paths for taking the mower 100 through at least part of the zone. The path of the mower 100 shown may be applicable, for example, to operation of the mower 100 when a boundary defines a containment zone around zone 202 within the boundary of the work region 200 (FIG. 3).

[0071] In the illustrated embodiment, the mower 100 travels from starting point 224 to destination waypoint 226. After reaching destination waypoint 226, the mower 100 may determine a second destination waypoint 228, rotate XI degrees, and travel toward the second destination waypoint. This sequence of rotating and travelling may continue to reach third destination waypoint 230, fourth destination waypoint 232, and final destination waypoint 234 (e.g., using rotations X2, X3, and X4, respectively). Although only a few destination waypoints 226, 228, 230, 232, 234 are shown in this illustration, the mower 100 may travel to several more waypoints in order to sufficiently cover the zone 202. In some embodiments, the mower 100 may select the smallest angle available to rotate and orient itself toward the next destination waypoint (e.g., 90 degrees counter-clockwise instead of 270 degrees clockwise).

[0072] FIG. 5 shows one example of a work region 250 including a transit zone 252, or transit path, extending across an exclusion zone 254, such as a driveway. The mowing area, or static containment zones 256, of the work region 250 may be located on each side of the driveway, but no mowing area connects these two sides. To train the transit zone 252, the mower 100 may first be placed at the desired starting point (see solid line representation of mower 100 in FIG. 3). A handle assembly (not shown) may be in a manual mode position. A training phase or mode may then be initiated using the remote computer 144 (FIG. 1). Once initiated, the mower 100 may be pushed or driven along the desired transit zone 252. Once the desired path is traversed (see broken line and mower 100 in FIG. 5), the operator may end the training session and save the transit zone. During autonomous mower operation, the mower100 will only cross from one side of the driveway, or exclusion zone 254, to the other using the defined transit zone 252. Multiple transit zones could be trained across any one exclusion zone.

[0073] Once all boundaries (including exclusion zones) and transit zones are taught, a map of the work region may be presented to the user on the remote computer 144 so that the operator can confirm that all boundaries (including exclusion zones) and transit zones are properly accounted for. The operator may then confirm that the boundaries and transit zones are properly represented before autonomous mowing operation may begin. In some embodiments, the operator may be able to delete and / or modify boundaries and transit zones using the mobile device during this review.

[0074] Transit zones may be used to define how the mower 100 gets from one portion of the work region to another (or to an isolated second work region). For example, transit zones may be configured to direct the mower: to a particular mowing area; across an exclusion zone such as a sidewalk, patio, or driveway that bifurcates the work region; or through a gate of a fenced yard. The mower will generally not enter into an exclusion zone unless a transit zone is trained through the exclusion zone. Moreover, the mower may not typically mow while moving along some of these transit zones.

[0075] Not all exclusion zones may include a transit zone. For example, some exclusion zones may be defined around obstacles that the mower 100 cannot traverse. A transit zone may not be defined across such an exclusion zone.

[0076] A base station 258 may be provided and positioned in or near the work region 250. The base station 258 may be connected to a source of electrical power, which may be stationary or portable. The base station 258 provides a storage location for the mower when not operating, and further includes self-engaging electrical connections to permit the mower to autonomously return to the base station 258 and recharge its battery 114 (FIG. 1) when needed.

[0077] In addition to transit zones, when the mower 100 selects a next destination waypoint, the mower may use a predetermined terrain map to determine an orientation or direction for the mower. In particular, the terrain map may be used to determine an orientation or direction for the path to the next destination waypoint.

[0078] In FIG. 6, one illustration of data that may be contained in a terrain map is shown as elevation plot 300 that maps elevation to distance along one illustrative path 302 with an elevation curve 308. The elevation data for a work region may be determined in various ways. In one example, the first time the mower 100 traverses a work region, the mower 100 may collect elevation data. As illustrated, elevation plot 300 corresponds to a cross-section of aterrain map. Populating the entire terrain map based on a plurality of cross-sections may take time (e.g., more than one pass through the map). The mower 100 may fill in parts of the terrain map not yet measured using two-dimensional interpolations of elevation data that has already been measured. Also, although the mower 100 may collect elevation data in discrete samples, the plot 300 may be generated using a smoothing function, such as a line fitting algorithm, to extrapolate a smooth curve for the elevation curve 308 as illustrated.

[0079] The mower 100 may analyze the elevation curve 308 along the path 302 for various information. In some embodiments, the mower 100 may select a highest elevation within a zone as a starting coordinate, or waypoint in a new travelling containment zone.

[0080] The mower 100 may use the elevation data along path 302 to determine a grade curve 312 that corresponds to the path 302 of elevation plot 300 shown as plot 310 in FIG. 7, which is another illustration of data that may be contained in the terrain map such as, for example, elevation data of a cross-section of the terrain map. As illustrated, the plot 310 maps grade (in degrees) to distance along the path 302. The grade curve 312 may be calculated based on the discrete elevation data or on the smooth elevation curve 308. The plot 310 may be generated using a smoothing function, such as a line fitting algorithm, to extrapolate a smooth curve for the grade curve 312. The grade curve 312 may be calculated as a plurality of slopes or using a differential function of the elevation curve 308.

[0081] Additionally, or alternatively, the mower 100 may determine and store gradient directly, for example, based on measuring pitch and roll data. The grade curve 312 may be computed, for example, using a combination of elevation data and gradient stored by the mower 100.

[0082] In general, the rear wheels 106 (FIG. 1) of the mower 100 will have more traction when travelling uphill in a forward direction compared to travelling downhill going forward. Likewise, the mower 100 generally has more traction when travelling downhill in a reverse direction compared to travelling uphill in reverse. The opposite may be true for a front-wheel drive mower. The techniques described herein orient the rear- wheel drive mower 100 in a forward direction to provide traction to the rear wheels 106 when going uphill and orient the mower in a reverse direction when going downhill. As described herein, the mower 100 may only rotate when the uphill or downhill grade is steep enough to cause a problem for the mower (e.g., problem areas). That is, some uphill or downhill slopes with shallow grades may not require the mower 100 to be oriented in the forward or reverse direction, respectively, sothe mower may traverse some shallow uphill grades in the reverse direction or some shallow downhill grades in a forward direction.

[0083] In some embodiments, the mower 100 may be configured to determine or store mass property data (e.g., mass distribution data) and pitch and / or roll data. For example, pitch and / or roll may be measured to generate the data. Any suitable technique may be used to determine mass property data and pitch and / or roll data, such as those known to a person of ordinary skill in the art having the benefit of this disclosure. Non-limiting examples of techniques for determining mass property data include using 3D computer-assisted design / drafting (CAD) tools or using weigh scales. Pitch and / or roll data may be determined by a navigation system of the mower 100. The navigation system may be configured to fuse sensor data from one or more sensors to provide pitch and / or roll data. Non-limiting examples of on-board sensors include: inertial measurement units (IMUs), wheel sensors, global positioning systems (GPS), or any other devices suitable for determining position and / or orientation of the mower 100. The orientation of the mower 100 may include, for example, pitch, roll, and yaw / heading.

[0084] Using the mass property data and the pitch and / or roll data, the mower 100 may be configured to determine how weight is distributed to each of its wheels and determine a maximum traction force available at each wheel to execute a maneuver. The mower 100 may apply limited wheel torque to one or more wheels so that the maximum traction forces are not exceeded during the maneuver. For example, if the mower 100 is on a slope, the mower may determine which drive wheel 106 has less traction and limit the acceleration of this drive wheel accordingly. In another example, when the mower 100 is accelerating on up or down a hill, torque may be limited to both wheels.

[0085] In one or more embodiments, the mower 100 may determine that certain maneuvers (e.g., traversing forward or in reverse or pivot turning) should not be executed based on a particular pitch and / or roll in the pitch and / or roll data. In one example, the mower 100 may have a high roll angle due to the presence of a hill and may determine that a maneuver to pivot turning to face uphill at this high roll angle should not be executed. In general, the mower 100 may be configured to avoid slip proactively, rather than only reacting to a slip event, which may facilitate avoiding situations where the mower cannot recover (e.g., becomes stuck).

[0086] The grade curve 312 may be used to identify various points or areas of the terrain along the path 302. For example, one or more local maxima 304 (FIG. 4) and one or more local minima 306 (FIG. 4) may be determined (e.g., where the grade curve equals zero). The mower100 may select one of the local maxima 304 or minima 306 along the path 302, or within the entire zone, as a starting coordinate. In particular, the mower 100 may select a local maximum 304 or minimum 306 having the smallest local grade (e.g., flattest area).

[0087] The local grade may be determined by comparing a plurality of grades stored in a terrain map and identifying the grade having the smallest absolute magnitude. The local grade may be calculated as a vector based on stored elevation data. To find the smallest local grade, the terrain map may be searched for cells in the zone. Then, those cells in the zone may be compared to identify the smallest stored or calculated gradient.

[0088] The local maxima 304 and minima 306 may be associated with rotatable areas 314. In general, rotatable areas 314 are associated with a range of grades along the path 302 that encompass flat areas and small slopes. Rotatable areas 314 may include local maxima 304 and minima 306 as shown by comparison of FIGS. 4 and 5. An ideal rotatable area 314 may include a zero-grade point (e.g., a maxima or minima of elevation).

[0089] Problem areas 316 may be associated with a range of high magnitude grades (including highly positive and highly negative grades) along the path 302 that encompass the steepest slopes as can be seen by comparison of FIGS. 4 and 5. The problem areas 316 are associated with grades that the mower 100 should not traverse without considering the orientation of the mower.

[0090] In some cases, the problem areas may include impassable areas. Impassable areas may be defined by grades that the mower cannot traverse without excessive difficulty. The mower may avoid, or skip over, the current path 302 if an impassable area (e.g., too steep a grade) is determined to lie along the path. For example, the mower may detect a grade that exceeds a threshold in its path and stop before reaching the grade or soon after reaching the grade. The threshold may be determined based on, for example, a jurisdictional or industrially- accepted standard. The mower may attempt to move to an area with a lower grade within a time window. When the time window expires, the user may manually provide instructions, commands, or guidance to move the mower.

[0091] The mower 100 may determine one or more rotatable areas 314 based on the grade curve 312. The rotatable areas 314 may be determined using one or more grade thresholds, which may include an upper grade threshold 318 (e.g., positive grade) and a lower grade threshold 319 (e.g., negative grade) for rotatable areas. Where the magnitude of the grade curve 312 does not exceed either of the grade thresholds 318, 319, the mower 100 may designate that portion of the path 302 as a rotatable area 314. For the illustrated grade curve312, three rotatable areas 314 are shown along the path 302 associated with portions of the grade curve 312 that are not higher than the upper grade threshold 318 for rotatable areas and are not lower than the lower grade threshold 319 for rotatable areas.

[0092] The mower 100 may determine one or more problem areas 316 based on the grade curve 312. The problem areas 316 may be determined using one or more grade thresholds, which may include an upper grade threshold 320 (e.g., positive grade) and a lower grade threshold 321 (e.g., negative grade). In general, the thresholds 318, 319 for rotatable areas and the thresholds 320, 321 for problem areas are different but, in some embodiments, may be close in value or even the same. Where the magnitude of the grade curve 312 exceeds one of the grade thresholds 320, 321, the mower 100 may designate that portion of the path 302 as a problem area 316. For the illustrated grade curve 312, two problem areas 316 are shown along the path 302 associated with portions of the grade curve 312 that are higher than the upper grade threshold 320 for problem areas or are lower than the lower grade threshold 321 for problem areas.

[0093] In some embodiments, the absolute values of the grade curve 312 may be used and compared with a grade threshold. Where the absolute value of the grade curve 312 does not exceed the one grade threshold (e.g., grade threshold 318) for rotatable areas, that portion of the path 302 may be designated as a rotatable area 314. Where the absolute value of the grade curve 312 does exceed the one grade threshold (e.g., grade threshold 320) for problem areas, that portion of the path 302 may be designated as a problem area 316.

[0094] The mower 100 may use the identified rotatable areas 314 and problem areas 316 along the path 302 to determine planned rotations of the mower along the travel path 302. In particular, when the mower 100 identifies problem areas 316 along the travel path 302, the mower may analyze whether the travel path includes prior rotatable areas 314 that the mower will traverse before reaching the problem area. The prior rotatable areas 314 may be used to orient the mower 100 in preferred directions to traverse the problem areas 316. The mower 100 may traverse the travel path 302 based on the planned rotations. In one embodiment, the planned rotations may be determined before the mower 100 begins to traverse the travel path 302 to a destination waypoint within the current zone or a starting coordinate in a new zone. In another embodiment, the planned rotations may be determined during traversal of the travel path 302 but before the mower 100 reaches the problem area 316 or the prior rotatable area 316.

[0095] For each problem area 316, the mower 100 may identify the preferred direction to propel the mower through the problem area. For example, if the problem area 316 is associated with a positive grade (and the mower 100 being a rear- wheel drive machine), the mower may determine that the preferred direction is a forward direction through the problem area. Likewise, if the problem area 316 is associated with a negative grade, the mower may determine that the preferred direction is a reverse direction through the problem area.

[0096] Then, the mower 100 may determine the direction that the mower will traverse the rotatable area 314 that is located before, or prior to, the mower reaching the problem area 316. If the direction that the mower 100 will traverse the prior rotatable area 314 is different than the preferred direction through the problem area, then the mower will rotate (e.g., 180 degrees) in the prior rotatable area 314. If the direction that the mower 100 will traverse the prior rotatable area 314 is the same as the preferred direction through the problem area, then the mower will not rotate in the prior rotatable area 314.

[0097] In some embodiments, the controller 120 (FIG. 1) may be used to use and store various plots, curves, and data associated with or determined from the terrain map, including coordinates, elevations, grades, thresholds, rotatable areas, problem areas, travel paths, zones, work regions, and planned rotations.

[0098] One example of using planned rotations for the grade curve 312 is schematically illustrated in FIG. 5 using portions A, B, and C (shown aligned to the grade curve 312 of plot 310), which correspond to parts of the path 302. To begin, the mower 100 decides to travel down the portion A along the path 302 in a forward direction because the mower had determined that the first problem area 316 (left-most problem area) has a positive grade. The mower 100 does not rotate in the first rotatable area 314 (left-most rotatable area) and travels through the first problem area 316 (left-most problem area). Then, the mower 100 reaches the second rotatable area 314 (middle rotatable area) at point B and rotates 180 degrees. The mower 100 then continues down the portion C along the path 302 in a reverse direction because the mower had determined that the second problem area 316 (right-most problem area) has a negative grade.

[0099] Once rotations are planned for all problem areas 316 along the travel path 302, the mower 100 may be propelled along the travel path based on the planned rotations. The predetermined terrain map may not include, however, obstacles that have been placed along the travel path 302 that may impede the progress of the mower 100.

[0100] Planning rotations may be particularly useful for downhill grades at a boundary of a zone or work region. In general, rear-wheel drive mowers are susceptible to becoming stuck after stopping at a downhill boundary and attempting to move away from the boundary. By planning rotations, the mower 100 is configured to use an orientation (e.g., backward or forward) that provides the most traction to move uphill away from the boundary. Using planned rotations may be more beneficial to work regions in which the mower 100 encounters more turns, or boundaries (e.g., using travelling containment zones to cover a work region).

[0101] In FIG. 8, one method 330 for modifying a travel path 332 to deal an obstacle 334 is shown. As the mower 100 propels along the path 332, the mower uses the sensor 130 to detect the obstacle 334. The sensor 130 uses non-contact sensing to detect the obstacle 334 before the mower 100 contacts the obstacle. Upon detection of the obstacle 334, the mower 100 rotates to avoid the obstacle 334 and travel along a detour 336 around the obstacle (e.g., wide of the obstacle). Then, the mower 100 continues onward toward the planned destination waypoint after avoiding the obstacle. In some embodiments, the detour 336 may be analyzed for problem areas, similar to a planned travel path.

[0102] In FIG. 9, another method 340 for modifying the travel path 332 to deal with the obstacle 334 is shown. The method 340 is similar to method 330 in many respects, except that the sensor 130 uses contact sensing to detect the obstacle 334 upon contact of the mower 100 with the obstacle 334. The mower 100 rotates to avoid the obstacle 334 and travels along detour 342 around the obstacle (e.g., wide of the obstacle). In general, the detour 342 may take a path that is tighter to the obstacle 334 than detour 336. The mower 100 then continues onward toward the planned destination waypoint after clearing the obstacle. In some embodiments, the detour 342 may be analyzed for problem areas, similar to a planned travel path.

[0103] In some embodiments, the mower 100 may choose to use either method 330, 340. The sensor 130 may include both contact and non-contact sensing capabilities that are selectable by the mower 100. In one example, the mower 100 may select method 340 when travelling within a zone (e.g., to a destination waypoint within a current zone) to improve coverage of the zone. In another example, the mower 100 may select method 330 when travelling between zones (e.g., to a destination waypoint within a next zone), for example, because mowing while travelling between zones is not typically used to cover the work region.

[0104] Although the mower 100 is shown rotating counter-clockwise to avoid the obstacle 334 in FIGS. 8 and 9, the mower may also rotate in a clockwise direction to avoid the obstacle.The decision to rotate clockwise or counter-clockwise may be based on open area available in the work region. For example, the mower 100 may rotate toward the direction with more open area available. Additionally, or alternatively, the decision to rotate clockwise or counterclockwise may be based on which orientation provides sufficient, or the most, traction.

[0105] The mower 100 may encounter various types of obstacles 334 (e.g., artificial or natural). Some obstacles may not be detectable by the sensor 130, for example, during a teaching or training mode. The mower 100 may detect some obstacles (e.g., a playground slide raised off the ground) by becoming stuck or by receiving user input (e.g., user-defined exclusion zones relatable to the terrain map) indicating the location of the obstacle. If the mower 100 gets stuck at a certain location in the terrain map, the mower may be configured to remember the location in the terrain map and conditions (e.g., pitch, roll, and / or heading). In some embodiments, the mower 100 may automatically identify and create an exclusion zone with or without permission of the user. When the terrain map is used for path planning, any path that includes the stuck location may be treated as an exclusion zone, an obstacle, or a grade that is too steep to traverse. The terrain map or other data structure may be updated to reflect the presence of such exclusion zones, obstacles, or grades. In other words, the mower 100 may avoid the same location and / or conditions leading to becoming stuck on subsequent planned paths. In this manner, methods 330, 340 may be used to avoid the obstacle 334 even when the sensor 130 does not directly detect the obstacle.

[0106] In FIG. 10, schematic connections between various systems are shown that may be defined by the mower 100 (FIGS. 1-3). A vision system 402 may be operably coupled to a navigation system 404. The navigation system 404 may be operably coupled to the propulsion system 406.

[0107] The navigation system 404 may record non-vision-based data during a training mode while the vision system 402 records images, such as training images. Although the mower 100 may be directed manually by a user, in some embodiments, the navigation system 404 may autonomously direct the machine during the training mode. The vision system 402 may include one or more cameras to record, or capture, images. In some embodiments, a controller of the vision system 402 may provide position and / or orientation data to the navigation system 404 based on the recorded images, which may be used to facilitate navigation of the mower 100. For example, the vision system 402 may provide an estimated position and / or orientation of the mower 100 to the navigation system 404 based on visionbased sensor data.

[0108] In some embodiments, the navigation system 404 may primarily use a position and / or orientation based on non-vision-based sensor data for navigation. For example, non- vision-based sensor data may be based on an output from an inertial measurement unit or wheel encoder. During a training mode and / or an offline mode, for example, a controller of the navigation system 404 may determine a boundary using non-vision-based sensor data, and the vision-based data, for subsequent navigation of the autonomous machine in the work region. During an online mode, for example, a controller of the navigation system 404 may determine a pose based on vision-based pose data, non-vision-based pose data, or both. In some embodiments, a pose may be determined based on non-vision-based sensor data and update the pose based on the vision-based pose data. The navigation system 404 may compare the visionbased position and / or orientation to the non-vision-based position and / or orientation to correct for errors and update the position, which may be described as sensor fusion. In some embodiments, sensor data other than vision-based sensor data may be used to correct for errors and update the position, such as GPS data.

[0109] A controller of the navigation system 404 may command the propulsion system 406 based on an updated pose. For example, a corrected or updated position and / or orientation may be used by the navigation system 404 to provide propulsion commands to a propulsion system 406. The propulsion system 406 (e.g., propulsion hardware) may be defined to include, for example, motors 112, 104 and wheels 106, 108 (FIG. 1) and / or any related drivers (e.g., motor controllers or microchips).

[0110] The mower 100 may include a plurality of modes or states. The mower 100 may be configured in a training mode, an offline mode, and an online mode. The mower 100 may switch between the various modes, which may also be described as configurations or states. Some functionality of the mower 100 may be used during certain modes, for example, to optimally utilize computing resources.[OHl] As used herein, the term “training mode” refers to a routine or state of an autonomous machine (e.g., mower 100) for recording data for later or subsequent navigation of the machine in a work region. During the training mode, the machine may traverse the work region without performing maintenance functions. For example, a training mode of an autonomous lawn mower may include directing the mower to traverse along some or all of the work region (e.g., along a desired boundary path), or a zone within the work region (e.g., containment zone or exclusion zone), and may or may not use a mowing blade in the zone orwork region. In some cases, the mower may be manually directed using a handle in the training mode. In other cases, the mower may be autonomously directed by the navigation system.

[0112] As used herein, the term “offline mode” refers to a routine or state of an autonomous machine (e.g., mower 100) for charging a portable power supply or processing data recorded during an online mode or training mode. For example, an offline mode of an autonomous lawn mower may include docking the mower in a charging station overnight and processing data recorded during a training mode or an online mode.

[0113] As used herein, the term “online mode” refers to a routine or state of an autonomous machine (e.g., mower 100) for operating in a work region, which may include traversing the work region and performing maintenance functions using a maintenance implement. For example, an online mode of an autonomous lawn mower may include directing the mower to cover or traverse the work region, or a zone within the work region, and using a mowing blade in the zone or work region to cut grass.

[0114] In general, the mower 100 may interact with the remote computer 144 (FIG. 1) during, for example, the training mode and / or the online mode.

[0115] In some embodiments, while a user manually directs the mower 100 during the training mode, the remote computer 144 may be used to provide training speed feedback. The feedback may indicate whether the user is moving the autonomous machine too quickly during training using, e.g., a color-coded dashboard.

[0116] In some embodiments, the remote computer 144 may be used to inform the user about certain areas, zones, or portions of the work region where the images acquired were not sufficient. For example, an error in a certain area may be detected and the remote computer 144 may inform the user of where the area is and may even direct the user along a path to record additional images to correct the detected error.

[0117] In some embodiments, the remote computer 144 may be used to select the type of boundary or zone for training: containment zone, exclusion zone, or transit zone.

[0118] In some embodiments, the remote computer 144 may be used to provide real-time zone shape feedback. The zone shape may or may not be tied to a real-world scale and orientation. For example, a map based on sensor data may be used to provide the zone shape feedback to the remote computer 144.

[0119] The mower may provide the time-to-completion estimate via an application running on the mobile device, or via periodic notifications (e.g., text messages) provided to the mobile device.

[0120] In FIG. 11, schematic representations of various systems of an autonomous machine (e.g., mower 100 of FIGS. 1-3) are shown. Sensors 420 may be operably coupled to the navigation system 404 to provide various sensor data, for example, to be used during an online mode. The vision system 402 (e.g., vision controller) and the navigation system 404 (e.g., navigation controller) may each include its own processor and memory. Various modules of the navigation system 404 are shown to implement various functionality to navigate the autonomous machine. The navigation system 404 may be operably coupled to a platform 460 to control physical actions of the autonomous machine.

[0121] The sensors 420 may include sensors associated with the navigation system 404, vision system 402, or both. The navigation system 404 and the vision system 402 may both include the same type of sensors. For example, the systems 402, 404 may each include an inertial measurement unit (IMU).

[0122] As used herein, the term “platform” refers to structure of the mower (e.g., mower 100 of FIGS. 1-3) that support the sensors 420 and the navigation system 404. For example, the platform 460 may include a propulsion system 406 (e.g., motors and wheels), the housing 102 (FIG. 1), the cutting motor 112 (FIG. 1), and the maintenance implement 110 (FIG. 1), among other possible components. In some embodiments, the entire autonomous machine may be described as being on the platform 460.

[0123] In the illustrated embodiment, the sensors 420 include the vision system 402 and non-vision-based sensors 422. Sensor data from the sensors 420 may be provided to a sensor fusion module 430. In particular, the vision system 402 may provide an estimated vision-based pose containing position and orientation parameters to the sensor fusion module 430. Non- vision-based sensors 422 may include, for example, an IMU and / or a wheel encoder. The sensor fusion module 430 may provide an estimated pose of the autonomous machine based on sensor data from the sensors 420. In particular, the sensor fusion module 430 may estimate a non-vision-based pose based on data from non-vision based sensors 422, which may be corrected or updated using a vision-based pose estimate determined based on data from visionbased sensors of the vision system 402.

[0124] As used herein, the term “pose” refers to a position and an orientation. The pose may be a six-degrees of freedom pose (6DOF pose), which may include all position andorientation parameters for a three-dimensional space. Pose data may include a three- dimensional position and a three-dimensional orientation. For example, the position may include at least one position parameter selected from: an x-axis, a y-axis, and a z-axis coordinate (e.g., using a Cartesian coordinate system). Any suitable angular orientation representations may be used. Non-limiting examples of angular orientation representations include a yaw, pitch, and roll representation, a Rodrigues’ representation, a quaternions representation, and a direction cosine matrix (DCM) representation may also be used alone or in combination. In one example, the orientation may include at least one orientation parameter selected from a yaw (e.g., vertical z-axis orientation), a pitch (e.g., a transverse y-axis orientation), and a roll (e.g., a longitudinal x-axis orientation).

[0125] A path planning module 440 may receive the estimated pose of the autonomous machine from the sensor fusion module 430 and use the estimated pose for autonomous navigation. Other information, or data, may be received by the path planning module 440 to facilitate navigation. An obstacle detection module 432 may provide information regarding the presence of an obstacle in the work region and the position of the obstacle based on sensor data from the sensors 420. The navigation system 404 may also define and update a map 434, or navigation map, of at least the work region. The map 434 may define or be updated to define one or more of containment zones, exclusion zones, transit zones, and mowing history, each of which may be provided to the path planning module 440 to facilitate navigation. Mowing history may also be provided to a scheduling management module 436. The scheduling management module 436 may be used to inform the path planning module 440 of various tasks for the autonomous machine, such as when to start mowing the work region during the week. Also, the path planning module 440 may perform both global path planning (e.g., determining zones within the work region) and local path planning (e.g., determining waypoints or starting points).

[0126] A propulsion controller 450 may receive data from the path planning module 440, the sensor fusion module 430, and the sensors 420, which may be used by the propulsion controller 450 to provide propulsion commands to the propulsion system 406. For example, the propulsion controller 450 may determine a speed or traction level based on data from the sensors 420. The path planning module 440 may provide one or more waypoints or starting points to the propulsion controller 450, which may be used to traverse the some or all the work region. The sensor fusion module 430 may be used to provide rate or speed data, accelerations, positions, and orientations of the autonomous machine to the propulsion controller 450. The propulsion controller 450 may also determine whether the autonomous machine is traversingthe path determined by the path planning module 440 and may facilitate correcting the path of the machine accordingly.

[0127] Other information, or data, related to the maintenance functionality of the autonomous machine may be provided to the propulsion controller 450 to control a maintenance implement, such as a cutting blade for mowing. For example, a motor drive current for the cutting blade motor may be provided to the propulsion controller 450. The propulsion controller 450 may also provide maintenance commands, for example, to control a maintenance implement on the platform 460.

[0128] In FIG. 12, shows one example of implementing the sensor fusion module 430 using sensor data from the sensors 420. Any suitable sensor data from various sensors 420 may be used. As illustrated, the sensors 420 include an inertial measurement unit 470, a wheel encoder 472, and the vision system 402.

[0129] Inertial measurement data from the inertial measurement unit 470 may be used by a pose determination module 474. The pose determination module 474 may provide an estimated pose of the autonomous machine based at least in part of the inertial measurement data. In particular, the pose determination module 474 may provide at least one of an estimated position and orientation. In some embodiments, the pose determination module 474 may even provide one or more velocities (e.g., speed or rate).

[0130] A Kalman filter 482 may be used to provide pose estimation data to the pose determination module 474, which may also be used to provide an estimated pose of the autonomous machine. In particular, the Kalman filter 482 may provide at least one of an estimated delta position, delta velocity, and delta orientation. As used herein, the term “delta” refers to a change in a variable or parameter. In some embodiments, output data from the Kalman filter 482 may be used to correct errors in a pose estimated based on data from the inertial measurement unit 470. The pose determination module 474 may provide a corrected, or updated, pose in the sensor fusion output 484.

[0131] The Kalman filter 482 may receive information, or data, based on output from the wheel encoder 472 and the vision system 402. The wheel encoder 472 may provide wheel speeds 476 to the Kalman filter 482. The vision system 402 may provide optical odometry 478 and vision position correction 480. Optical odometry 478 may utilize images and determine information about movement of the autonomous machine, such as a distance that the autonomous machine has traveled. In general, optical odometry 478 may be used to determine a change in position, a change in orientation, a linear velocity, an angular velocity, or anycombination of these. Any suitable optical odometry algorithms available to one of ordinary skill in the art may be used depending on the particular autonomous machine and application. The vision position correction 480 provided by the vision system 402 may include a visionbased pose data, for example, a vision-based pose estimate.

[0132] The pose determination module 474 may receive or process data from the Kalman filter 482 at a low refresh rate and use low rate updates. Data from the inertial measurement unit 470 may be received or processed at a high refresh rate and use high rate updates faster than the Kalman filter data. Output from the sensor fusion output 484 may feed back into the Kalman filter 482 as an input to facilitate Kalman filter operation. In other words, the pose determination module 474 may provide an estimated pose at a higher rate than the output of the Kalman filter 482 or the Kalman filter inputs (wheel speeds 476, optical odometry 478, or vision position correction 480). For example, the vision position correction 480 may be performed at various rates on the order of one to four times per minute (e.g., about 1 / 10 Hz or 1 / 100 Hz), whereas the pose determination module 474 may provide a pose on the order of 6000 times per minute (e.g., about 100 Hz). In some embodiments, the higher rate may be an order of magnitude that is one, two, three, four, five, or even six times the lower rate.

[0133] In some embodiments (not shown), the Kalman filter 482 may be included in the pose determination module 474. In some embodiments, the Kalman filter 482 may use a high refresh rate.

[0134] Although the mower 100 may possess the obstacle identification, obstacle avoidance, mapping, and navigation capabilities described herein, the mower may encounter new or otherwise unknown operational obstructions that the mower lacks the capability to autonomously identify and / or circumvent. When unknown operational obstructions are encountered, it may be beneficial for the mower 100 to receive additional input, training, and / or intervention. Accordingly, a method or process to provide a notification indicative of the unknown operational obstruction may improve the performance of the mower 100 and allow for additional input, training, and / or intervention to improve obstacle detection, obstacle avoidance, and work region navigation.

[0135] A method 500 for operating an autonomous machine (e.g., mower 100) in a work region (e.g., work region 200) is depicted in FIG. 13. The method 500 may be carried out or otherwise executed by a controller (e.g., controller 120) of the autonomous machine. In other words, the controller 120 may be configured to carry out the method 500. Additionally, the controller may utilize various systems (e.g., the vision system 402, the navigation system 404,the propulsion system 406) and sensors (e.g., sensors 420, fusion sensor 430, etc.) to carry out the method 500. Although described with regard to mower 100, the method 500 may be carried out by any suitable autonomous machine, autonomous ground maintenance device, or autonomous mower.

[0136] The method 500 may include directing navigation of the autonomous machine within the work region using a set of wheels 106, 108 (see FIG. 2) and a propulsion controller 450 (see FIG. 11) of the autonomous machine at 502. Navigation of the autonomous machine within the work region may include using a maintenance implement 110 (see FIG. 2) to perform work within the work region. Additionally, the navigation system 404 (see FIG. 11) may be used to navigate the work region according to various embodiments described herein.

[0137] The method 500 may include determining that an operational obstruction to grounds maintenance has been encountered at 504. As used herein, the term “operational obstruction to grounds maintenance” may refer to any obstacle or condition within the work region that prevents the autonomous machine from performing some portion of a grounds maintenance routine, method, or process. Operational obstructions to grounds maintenance may include, for example, any obstacle or condition that prevents movement of the autonomous machine to a portion of the work region, prevents the autonomous machine from achieving a desired pose, prevents the autonomous machine from navigating along a desired path, prevents the maintenance implement 110 from performing work, prevents the autonomous machine from moving, prevents any of the wheels 106, 108 from propelling the autonomous machine, or any other obstacle or condition that prevents the autonomous machine from performing some portion of a grounds maintenance routine, method, or process.

[0138] Determining that the operational obstruction to grounds maintenance has been encountered may include determining that a portion of the work region is inaccessible to the autonomous machine. The controller 120 may determine that the portion of the work region is inaccessible in response to failure of one or more attempts to enter the portion of the work region. For example, the controller 120 may instruct the propulsion controller to navigate into the portion of the work region from at multiple locations without success. Accordingly, the controller 120 may determine that the portion of the work region is inaccessible or otherwise obstructed.

[0139] Determining that the operational obstruction to grounds maintenance has been encountered may include determining, based on the one or more sensor signals, that movement in a desired direction did not occur in response to directing navigation of the autonomousmachine in the desired direction. The controller 120 may direct navigation of the autonomous machine in the desired direction using the set of wheels 106, 108 and the propulsion controller 450 of the autonomous machine. The controller 120 may also receive one or more sensor signals from one or more sensors (e.g., sensors 420) of the autonomous machine. Based on the one or more sensor signals, the controller 120 may determine that movement in the desired direction did not occur. For example, a signal from the GPS 116 receiver may indicate that a position of the autonomous machine remains static while the wheel speeds 476 are greater than 0. Further, for example, a signal from the inertial measurement unit 470 may indicate that the autonomous machine is stationary after commands to move have been sent to the propulsion controller 450. Still further, for example, signals from the GPS receiver 116 or inertial measurement unit 470 may indicate that the autonomous machine has moved in a direction other than the desired direction.

[0140] Determining that the operational obstruction to grounds maintenance has been encountered may include determining that the autonomous machine is immovable from a current location. The controller 120 may determine that the autonomous machine is immovable from the current location based on one or more sensor signals. For example, the controller may direct navigation in a series of directions and one or more signals received from the sensors 420 may indicate that a position of the autonomous machine remains static.

[0141] The method 500 may include capturing one or more images of the work region using a vision system 402 (see FIGS. 10 and 11) in response to the determination that the operational obstruction has been encountered at 506. The one or more images may be captured using one or more of the cameras 133 (see FIG. 1). In general, the number of images and the process for capturing the one or more images may depend on the operational obstruction encountered by the autonomous machine. In one or more embodiments, the one or more images may include an image captured by each of the cameras 133 to capture the total field of view of the cameras 133. For example, if the autonomous machine is unable to move in any direction, the total field of view of the cameras 133 may be captured in an effort to identify a source of the operational obstruction to grounds maintenance. In one or more embodiments, the one or more images may include a series of images taken by one or more of the cameras. For example, if the autonomous machine is unable to access a portion of the work area, the autonomous machine may be navigated along the perimeter of the portion of the work area while capturing images to provide the series of images. In one or more embodiments, the one or more images may include at least one image captured by a single one of the cameras 133. For example, if the autonomous machine is unable to move in a particular direction the singleone of the cameras 133 most closely aligned with the particular direction may capture at least one image.

[0142] The method 500 may include determining that the operational obstruction is an unknown operational obstruction at 508. As used herein, the term “unknown operational obstruction” may refer to an operational obstruction to grounds maintenance that does not correlate to any obstacles or conditions known to the autonomous machine, does not fit any criterion that may allow the autonomous machine to identify a source of the obstacle or condition, or an operational obstruction that was unanticipated by the autonomous machine. In other words, an unknown operational obstruction may refer to an obstacle or condition that the autonomous machine cannot determine the source of, categorize, or anticipate.

[0143] Determining that the operational obstruction is an unknown operational obstruction is unknown may include determining that a source of the operational obstruction is unidentifiable based on sensor signals received from the sensors 420. The sensor signals may include images provided by the cameras 133, other signals provided by the vision system 402, signals from the inertial measurement unit 470, signals from the wheel encoder 472, etc.

[0144] Determining that the operational obstruction is an unknown operational obstruction may include comparing one or more sensor signals to a known obstruction profile and determining that the one or more sensor signals do not correspond to a known obstruction indicated by the obstruction profile. The known obstruction profile may include the map of the work region, boundary information, grade thresholds, problem areas, locations of known obstacles, known features, etc. For example, movement of the autonomous machine may be restricted by an unknown object that does not match any known obstacles or features and may not be sensed by the obstacle detection sensors 130, 132. Accordingly, the signals provided by the sensors 420 may not correspond to any known obstruction included in the known obstruction profile. Further, for example, movement of the autonomous machine may be restricted by changes to the terrain (e.g., formation of a hole, wall, or incline) that do not correspond to the terrain map included in the map of the work region.

[0145] Determining that the operational obstruction is an unknown operational obstruction may include determining that movement or operation of the autonomous mower is impaired by an unanticipated obstacle or condition. In other words, one or more sensor signals (e.g., signals provided by any one of sensors 130, 420, etc.) may indicate that movement of the autonomous mower in one or more directions or use of the working implement (e.g., mower blade 110) is restricted unexpectedly. Unanticipated obstacles or conditions may refer to obstacles that arenot detected in a manner that allows modification of the travel path 332 using methods 330 and 340 of FIGS. 8 and 9. Accordingly, unanticipated obstacles or conditions may be encountered by the autonomous machine 100 in a manner that may restrict or impair mobility or work of the autonomous machine.

[0146] The method 500 may include generating, in response to the determination that the operational obstruction is unknown, an unknown obstruction notification including the one or more images at 510. Additionally, the unknown obstruction notification may include a title, a message, or a link. The unknown obstruction notification may include a push notification. In general, push notifications may display or “pop-up” on a receiving device (e.g., remote computer 144) even when a corresponding application is not opened. Accordingly, a user can receive the unknown obstruction notification in a timely manner without the need to regularly open or check a particular application.

[0147] The method 500 may include transmitting the unknown obstruction notification to an external device 520 (see FIG. 14) at 512. The unknown obstruction notification may be transmitted using the wireless radio 117. Additionally, the unknown obstruction notification may be transmitted via the wireless network 142 and / or the internet. The external device 520 may include the remote computer 144, the server 146, the base station 258, or other device connected to the wireless network 142. The unknown obstruction notification may be transmitted for display on the external device 520, or a display operatively coupled to the external device.

[0148] An example unknown obstruction notification 524 as displayed on the external device 520 is depicted in FIG. 14. The external device 520 may include or be operatively coupled to a display 522. In one or more embodiments, the display 522 may be a graphical user interface. The unknown obstruction notification may include a title 526, the one or more images 528, a message 530, and a link 532.

[0149] The title 526 may indicate that an unknown operational obstruction has been encountered by the autonomous machine. For example, the title 526 may include text (e.g., “unknown obstruction”) or other symbols that indicate an unknown obstruction has prevented the autonomous machine from providing or completing a grounds maintenance task.

[0150] The one or more images 528 may be displayed in any suitable manner or arrangement. The one or more images 528 may be arranged, for example, in a stacked or tiled arrangement. Furthermore, a single image of the one or more images 528 may be displayed on the display 522 until the single image or link 532 are selected by a user. In response toselection of the single image or link 532, any additional images of the one or more images 528 may be displayed on the display 522.

[0151] The message 530 may provide additional information about the unknown obstruction. For example, the message 530 may indicate a type of obstruction such as, for example, that the autonomous mower cannot move, that the autonomous mower cannot move in a particular direction, that the autonomous mower cannot reach or navigate to a portion of the work area, etc. The message 530 may also include one or more error messages indicating whether any systems of the autonomous mower are in an error state or condition. Still further, the message 530 may include sensor data that indicates an obstruction but does not conform to any known obstruction or criterion regarding known obstacles or conditions.

[0152] The link 532, when selected, may direct the user to an application or webpage to allow the user to view additional information about the unknown obstruction. For example, the link 532 may allow the user to view additional images. Further, for example, the link 532 may allow the user to view camera images or video provided by the cameras 133 in real time. Still further, for example, the link 532 may direct the user to an app or application that allows the user to attempt to control the autonomous machine and / or investigate the unknown obstruction further. In one embodiment, the link 532 includes a uniform resource locator (URL).

[0153] Additionally, the link 532, when selected, may allow the user to provide information about the unknown obstruction. For example, the link 532 may allow the user to identify the unknown obstruction in the one or more images. Further, the link 532 may allow the user to identify or select a type of obstruction such as, for example, an object, a terrain condition, a weather condition, etc. The information provided by the user in regard to the unknown obstruction may be used to improve obstacle detection and avoidance of the autonomous mower. The information provided by the user may be correlated with sensor signals, a position of the autonomous mower 100, or other data to allow the unknown obstacle to be identified and avoided in the future.

[0154] The invention is defined in the claims. However, below there is provided a non- exhaustive list of non-limiting examples. Any one or more of the features of these examples may be combined with any one or more features of another example, embodiment, or aspect described herein.

[0155] Example Exl : An autonomous machine to autonomously provide grounds maintenance in a work region, the autonomous machine comprising: a housing coupled to a maintenance implement; a set of wheels supporting the housing over a ground surface; apropulsion controller operably coupled to the set of wheels; a vision system comprising one or more cameras adapted to capture image data; and a controller comprising one or more processors and operatively coupled to the vision system and the propulsion controller, the controller configured to: direct navigation of the autonomous machine within the work region; determine that an operational obstruction to grounds maintenance has been encountered; capture one or more images of the work region using the vision system in response to the determination that the operational obstruction has been encountered; determine that the operational obstruction is an unknown operational obstruction; generate, in response to the unknown operational obstruction, an unknown obstruction notification comprising the one or more images; and transmit the unknown obstruction notification to an external device.

[0156] Example Ex2: The autonomous machine as in example Exl, wherein, to determine that the operational obstruction to grounds maintenance has been encountered, the controller is further configured to determine that a portion of the work region is inaccessible to the autonomous machine.

[0157] Example Ex3 : The autonomous machine as in any one of the previous examples, wherein the autonomous machine comprises one or more sensors and, to determine that the operational obstruction to grounds maintenance has been encountered, the controller is further configured to: direct navigation of the autonomous machine in a desired direction using the set of wheels and the propulsion controller of the autonomous machine; receive one or more sensor signals from the one or more sensors; and determine, based on the one or more sensor signals, that movement in the desired direction did not occur in response to directing navigation of the autonomous machine in the desired direction.

[0158] Example Ex4: The autonomous machine as in any one of the previous examples, wherein to determine that the operational obstruction to grounds maintenance has been encountered, the controller is further configured to determine that the autonomous machine is immovable from a current location.

[0159] Example Ex5: The autonomous machine as in any one of the previous examples, wherein to determine that the operational obstruction to grounds maintenance has been encountered, the controller is further configured to determine that movement or operation of the autonomous mower is impaired by an unanticipated obstacle or condition.

[0160] Example Ex6: The autonomous machine as in any one of the previous examples, wherein, to capture one or more images of the work region, the controller is configured tocapture at least one image corresponding to a direction of desired movement of the autonomous machine.

[0161] Example Ex7: The autonomous machine as in any one of the previous examples, wherein, to capture one or more images of the work region the controller is configured to capture a series of images along a perimeter of an inaccessible portion of the work region.

[0162] Example Ex8: The autonomous machine as in any one of the previous examples, wherein the autonomous machine further comprises one or more sensors and, to determine that the operational obstruction is unknown, the controller is further configured to: receive one or more sensor signals from the one or more sensors; compare the one or more signals to a known obstruction profile; and determine that the one or more sensor signals do not correspond to a known obstruction indicated by the obstruction profile.

[0163] Example Ex9: The autonomous machine as in any one of the previous examples, wherein the autonomous machine further comprises one or more sensors and, to determine that the operational obstruction is unknown, the controller is further configured to determine that a source of the operational obstruction is unidentifiable based on sensor signals received from the one or more sensors.

[0164] Example ExlO: The autonomous machine as in any one of the previous examples, wherein the controller is configured to transmit the unknown obstruction notification to a mobile device comprising a graphical user interface.

[0165] Example Exl 1 : The autonomous machine as in any one of the previous examples, wherein the unknown obstruction notification comprises a push notification.

[0166] Example Exl2: The autonomous machine as in any one of the previous examples, wherein the unknown obstruction notification comprises the one or more images, a title, a message, and a link.

[0167] Example Exl3 : A method for operating an autonomous machine in a work region, the method comprising: directing navigation of the autonomous machine within the work region using a set of wheels and a propulsion controller of the autonomous machine; determining that an operational obstruction to grounds maintenance has been encountered; capturing one or more images of the work region using a vision system of the autonomous machine in response to the determination that the operational obstruction has been encountered; determining that the operational obstruction is an unknown operational obstruction; generating, in response to the unknown operational obstruction, an unknownobstruction notification comprising the one or more images; and transmitting the unknown obstruction notification to an external device.

[0168] Example Exl4: The method as in example Exl3, wherein determining that the operational obstruction to grounds maintenance has been encountered comprises determining that a portion of the work region is inaccessible to the autonomous machine.

[0169] Example Exl5: The method as in any one of examples Exl3 or Exl4, wherein determining that the operational obstruction to grounds maintenance has been encountered comprises: directing navigation of the autonomous machine in a desired direction using the set of wheels and the propulsion controller of the autonomous machine; receiving one or more sensor signals from one or more sensors of the autonomous machine; and determining, based on the one or more sensor signals, that movement in the desired direction did not occur in response to directing navigation of the autonomous machine in the desired direction.

[0170] Example Exl6: The method as in any one of examples Exl3 to Exl5, wherein determining that the operational obstruction to grounds maintenance has been encountered comprises determining that the autonomous machine is immovable from a current location.

[0171] Example Exl7: The method as in any one of examples Exl3 to Exl6, wherein determining that the operational obstruction to grounds maintenance has been encountered comprises determining that movement or operation of the autonomous mower is impaired by an unanticipated obstacle or condition.

[0172] Example Exl8: The method as in any one of examples Exl3 to Exl7, wherein capturing the one or more images of the work region comprises capturing at least one image corresponding to a direction of desired movement of the autonomous machine.

[0173] Example Exl9: The method as in any one of examples Exl3 to Exl8, wherein capturing the one or more images of the work region comprises capturing a series of images along a perimeter of an inaccessible portion of the work region.

[0174] Example Ex20: The method as in any one of examples Exl3 to Exl9, wherein determining that the operational obstruction is unknown comprises: receiving one or more sensor signals from one or more sensors of the autonomous machine; comparing the one or more signals to a known obstruction profile; and determining that the one or more sensor signals do not correspond to a known obstruction indicated by the obstruction profile.

[0175] Example Ex21 : The method as in any one of examples Exl3 to Ex20, wherein determining that the operational obstruction is unknown comprises determining that a sourceof the operational obstruction is unidentifiable based on sensor signals received from one or more sensors of the autonomous machine.

[0176] Example Ex22: The method as in any one of examples Exl3 to Ex21, wherein the unknown obstruction notification is transmitted to a mobile device comprising a graphical user interface.

[0177] Example Ex23: The method as in any one of examples Exl3 to Ex22, wherein the unknown obstruction notification comprises a push notification.

[0178] Example Ex24: The method as in any one of examples Exl3 to Ex23, wherein the unknown obstruction notification comprises the one or more images, a title, a message, and a link.

[0179] Example Ex25: The autonomous machine as in any one of examples Exl to Exl2, wherein the vision system is configured to estimate a pose of the autonomous machine.

[0180] Example Ex24: The method as in any one of examples Exl3 to Ex24, wherein determining that an operational obstruction to grounds maintenance has been encountered is based on an estimated pose of the autonomous machine, the estimated pose determined by the vision system.

[0181] Thus, various embodiments of autonomous machine navigation are disclosed. Although reference is made herein to the accompanying set of drawings that form part of this disclosure, one of at least ordinary skill in the art will appreciate that various adaptations and modifications of the embodiments described herein are within, or do not depart from, the scope of this disclosure. For example, aspects of the embodiments described herein may be combined in a variety of ways with each other. Therefore, it is to be understood that, within the scope of the appended claims, the claimed invention may be practiced other than as explicitly described herein.

[0182] All references and publications cited herein are expressly incorporated herein by reference in their entirety into this disclosure, except to the extent they may directly contradict this disclosure.

[0183] All scientific and technical terms used herein have meanings commonly used in the art unless otherwise specified. The definitions provided herein are to facilitate understanding of certain terms used frequently herein and are not meant to limit the scope of the present disclosure.

[0184] The recitation of numerical ranges by endpoints includes all numbers subsumed within that range (e.g. 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5) and any range within that range. Herein, the terms “up to” or “no greater than” a number (e.g., up to 50) includes the number (e.g., 50), and the term “no less than” a number (e.g., no less than 5) includes the number (e.g., 5).

[0185] The terms “coupled” or “connected” refer to elements being attached to each other either directly (in direct contact with each other) or indirectly (having one or more elements between and attaching the two elements). Either term may be modified by “operatively” and “operably,” which may be used interchangeably, to describe that the coupling or connection is configured to allow the components to interact to carry out at least some functionality (for example, a propulsion controller may be operably coupled to a motor driver to electrically control operation of the motor).

[0186] Reference to “one embodiment,” “an embodiment,” “certain embodiments,” or “some embodiments,” etc., means that a particular feature, configuration, composition, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Thus, the appearances of such phrases in various places throughout are not necessarily referring to the same embodiment of the disclosure. Furthermore, the particular features, configurations, compositions, or characteristics may be combined in any suitable manner in one or more embodiments.

[0187] As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” encompass embodiments having plural referents, unless the content clearly dictates otherwise. As used in this specification and the appended claims, the term “or” is generally employed in its sense including “and / or” unless the content clearly dictates otherwise.

Claims

CLAIMSWhat is claimed is:

1. An autonomous machine to autonomously provide grounds maintenance in a work region, the autonomous machine comprising: a housing coupled to a maintenance implement; a set of wheels supporting the housing over a ground surface; a propulsion controller operably coupled to the set of wheels; a vision system comprising one or more cameras adapted to capture image data; and a controller comprising one or more processors and operatively coupled to the vision system and the propulsion controller, the controller configured to: direct navigation of the autonomous machine within the work region; determine that an operational obstruction to grounds maintenance has been encountered; capture one or more images of the work region using the vision system in response to the determination that the operational obstruction has been encountered; determine that the operational obstruction is an unknown operational obstruction; generate, in response to the unknown operational obstruction, an unknown obstruction notification comprising the one or more images; and transmit the unknown obstruction notification to an external device.

2. The autonomous machine as in claim 1, wherein, to determine that the operational obstruction to grounds maintenance has been encountered, the controller is further configured to determine that a portion of the work region is inaccessible to the autonomous machine.

3. The autonomous machine as in claim 1 , wherein the autonomous machine comprises one or more sensors and, to determine that the operational obstruction to grounds maintenance has been encountered, the controller is further configured to: direct navigation of the autonomous machine in a desired direction using the set of wheels and the propulsion controller of the autonomous machine; receive one or more sensor signals from the one or more sensors; anddetermine, based on the one or more sensor signals, that movement in the desired direction did not occur in response to directing navigation of the autonomous machine in the desired direction.

4. The autonomous machine as in any one of claims 1-3, wherein to determine that the operational obstruction to grounds maintenance has been encountered, the controller is further configured to determine that the autonomous machine is immovable from a current location.

5. The autonomous machine as in any one of claims 1-3, wherein to determine that the operational obstruction to grounds maintenance has been encountered, the controller is further configured to determine that movement or operation of the autonomous mower is impaired by an unanticipated obstacle or condition.

6. The autonomous machine as in any one of claims 1-3, wherein, to capture one or more images of the work region, the controller is configured to capture at least one image corresponding to a direction of desired movement of the autonomous machine.

7. The autonomous machine as in any one of claims 1-3, wherein, to capture one or more images of the work region the controller is configured to capture a series of images along a perimeter of an inaccessible portion of the work region.

8. The autonomous machine as in any one of claims 1-3, wherein the autonomous machine further comprises one or more sensors and, to determine that the operational obstruction is unknown, the controller is further configured to: receive one or more sensor signals from the one or more sensors; compare the one or more signals to a known obstruction profile; and determine that the one or more sensor signals do not correspond to a known obstruction indicated by the obstruction profile.

9. The autonomous machine as in any one of claims 1-3, wherein the autonomous machine further comprises one or more sensors and, to determine that the operational obstruction is unknown, the controller is further configured to determine that a source of the operational obstruction is unidentifiable based on sensor signals received from the one or more sensors.

10. The autonomous machine as in any one of claims 1-3, wherein the controller is configured to transmit the unknown obstruction notification to a mobile device comprising a graphical user interface.

11. The autonomous machine as in any one of claims 1-3, wherein the unknown obstruction notification comprises a push notification.

12. The autonomous machine as in any one of claims 1-3, wherein the unknown obstruction notification comprises the one or more images, a title, a message, and a link.

13. The autonomous machine as in any one of claims 1-3, wherein the vision system is configured to estimate a pose of the autonomous machine.

14. A method for operating an autonomous machine in a work region, the method comprising: directing navigation of the autonomous machine within the work region using a set of wheels and a propulsion controller of the autonomous machine; determining that an operational obstruction to grounds maintenance has been encountered; capturing one or more images of the work region using a vision system of the autonomous machine in response to the determination that the operational obstruction has been encountered; determining that the operational obstruction is an unknown operational obstruction; generating, in response to the unknown operational obstruction, an unknown obstruction notification comprising the one or more images; and transmitting the unknown obstruction notification to an external device.

15. The method as in claim 14, wherein determining that the operational obstruction to grounds maintenance has been encountered comprises determining that a portion of the work region is inaccessible to the autonomous machine.

16. The method as in claim 14, wherein determining that the operational obstruction to grounds maintenance has been encountered comprises:directing navigation of the autonomous machine in a desired direction using the set of wheels and the propulsion controller of the autonomous machine; receiving one or more sensor signals from one or more sensors of the autonomous machine; and determining, based on the one or more sensor signals, that movement in the desired direction did not occur in response to directing navigation of the autonomous machine in the desired direction.

17. The method as in any one of claims 14-16, wherein determining that the operational obstruction to grounds maintenance has been encountered comprises determining that the autonomous machine is immovable from a current location.

18. The method as in any one of claims 14-16, wherein determining that the operational obstruction to grounds maintenance has been encountered comprises determining that movement or operation of the autonomous mower is impaired by an unanticipated obstacle or condition.

19. The method as in any one of claims 14-16, wherein capturing the one or more images of the work region comprises capturing at least one image corresponding to a direction of desired movement of the autonomous machine.

20. The method as in any one of claims 14-16, wherein capturing the one or more images of the work region comprises capturing a series of images along a perimeter of an inaccessible portion of the work region.

21. The method as in any one of claims 14-16, wherein determining that the operational obstruction is unknown comprises: receiving one or more sensor signals from one or more sensors of the autonomous machine; comparing the one or more signals to a known obstruction profile; and determining that the one or more sensor signals do not correspond to a known obstruction indicated by the obstruction profile.

22. The method as in any one of claims 14-16, wherein determining that the operational obstruction is unknown comprises determining that a source of the operational obstruction isunidentifiable based on sensor signals received from one or more sensors of the autonomous machine.

23. The method as in any one of claims 14-16, wherein the unknown obstruction notification is transmitted to a mobile device comprising a graphical user interface.

24. The method as in any one of claims 14-16, wherein the unknown obstruction notification comprises a push notification.

25. The method as in any one of claims 14-16, wherein the unknown obstruction notification comprises the one or more images, a title, a message, and a link.

26. The method as in any one of claims 14-16, wherein determining that an operational obstruction to grounds maintenance has been encountered is based on an estimated pose of the autonomous machine, the estimated pose determined by the vision system.

Images