Defining the boundaries of an autonomous machine's working space

Abstract

The boundaries of a work area within which the autonomous machine will operate are defined. Dead zones within the work area, where loss of wireless geolocation services is known or predicted, are defined. A traverse pattern within the boundaries is automatically generated. The autonomous machine performs the work by executing the traverse pattern while using wireless geolocation services to navigate outside the dead zone. If a dead zone is encountered before, during, or after executing the traverse pattern, The machine prioritizes location inputs that do not rely on wireless geolocation services and operates within dead zones.

Description

[Technical Field]

[0001] RELATED PATENT DOCUMENTS This application claims the benefit of U.S. Provisional Application No. 63 / 406,524, filed September 14, 2022, and U.S. Provisional Application No. 63 / 425,746, filed November 16, 2022, both of which are incorporated herein by reference in their entireties. Summary of the Invention [Means for solving the problem]

[0002] The present disclosure is directed to an apparatus and method for assisting in the training or programming of an autonomous work vehicle. In one embodiment, the method includes defining a boundary for a work area in which an autonomous machine will operate. Dead zones are defined within the work area where loss of wireless geolocation service is known or predicted. A traverse pattern is automatically generated within the boundary, and the traverse pattern encounters the dead zone. The autonomous machine performs the traverse pattern while using wireless geolocation services to navigate outside the dead zone. When a dead zone is encountered before, during, or after performing the traverse pattern, the machine performs the work in the dead zone using a local navigation mode that does not rely on wireless geolocation services.

[0003] These and other features and aspects of various embodiments can be understood in view of the following detailed description and accompanying drawings.

[0004] In the following description, reference will be made to the following figures, in which the same reference numerals may be used to identify similar / identical components in multiple figures. The drawings are not necessarily to scale. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 is an illustration of an autonomous work vehicle in accordance with an illustrative embodiment.

[0006] [Figure 2] FIG. 2 is an illustration of a work area utilized by an autonomous work vehicle in accordance with an illustrative embodiment. [Figure 3] FIG. 3 is an illustration of a work area utilized by an autonomous work vehicle in accordance with an illustrative embodiment.

[0007] [Figure 4] FIG. 4 is a diagram illustrating communication regarding route planning with an operations center in accordance with an illustrative embodiment.

[0008] [Figure 5] FIG. 5 is a diagram illustrating boundary identification in accordance with an example embodiment.

[0009] [Figure 6] FIG. 6 is a flowchart illustrating a method according to an example embodiment. [Figure 7] FIG. 7 is a flowchart illustrating a method according to an example embodiment. [Figure 8] FIG. 8 is a flowchart illustrating a method according to an example embodiment.

[0010] [Figure 9] FIG. 9 is a diagram illustrating automatic boundary detection using an electronic map, according to an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Detailed Description In the following detailed description of exemplary embodiments, reference is made to the accompanying drawings, which form a part hereof. It should be understood that other equivalent embodiments not described and / or shown herein are also contemplated.

[0012] The present disclosure relates to autonomous work vehicles. Generally, autonomous work vehicles can traverse a work area equipped with work implements that perform repetitive and / or monotonous tasks. Examples of such tasks include mowing, snow removal, spreading solid or liquid (e.g., salt, fertilizer, seeds, water, herbicides, pesticides), soil treatment (e.g., aeration), cleaning, applying markings or coatings, etc. Autonomous vehicles can be self-powered (e.g., internal combustion engine, battery, fuel cell) and self-guided. Self-guiding a machine can still involve human input, such as first defining the task to be performed and then directing the machine to perform the task.

[0013] Embodiments described herein relate to navigation methods and systems for autonomous machines that navigate and operate within the boundaries of a work area, such as lawn mowing, snow removal, surface treatment, animal control, security, etc. The autonomous machines may be configured in various modes to perform various navigation functions, such as training mode, offline mode, and online mode. The autonomous machines may use a vision system and geolocation services to define one or more boundaries of the work area.

[0014] In FIG. 1 , a simplified perspective view illustrates some features of an autonomous work vehicle 100 according to one or more embodiments. An example of an autonomous work vehicle 100 is an autonomous lawnmower, although other autonomous work vehicles may have similar components arranged in a similar manner. As shown in this figure, the autonomous work vehicle 100 may include a housing 102 (e.g., a frame or chassis with a shroud) that carries and / or encloses various components of the lawnmower, as described below. The autonomous work vehicle 100 may further include ground traverse members, such as wheels, rollers, or tracks. In the illustrated embodiment, the ground traverse members include one or more rear wheels 106 and one or more front wheels 108, which support and move the housing 102 over a ground (e.g., grass) surface 103. As shown, the front wheels 108 are used to support a front end portion 110 of the housing 102, and the rear wheels 106 are used to support a rear end portion 111 of the housing.

[0015] One or both rear wheels 106 may be driven by a propulsion system (e.g., including one or more electric wheel motors 104) to propel the autonomous work vehicle 100 over the surface 103. In some embodiments, the front wheels 108 are free to caster (e.g., about a vertical axis) relative to the housing 102. In such a configuration, the direction of the mower can be controlled via differential rotation of the two rear wheels 106 in a manner similar to a conventional zero-turn-radius (ZTR) riding lawn mower. For example, the propulsion system may include separate wheel motors 104 for each of the left and right rear wheels 106 so that the speed and direction of each rear wheel can be controlled independently. Additionally or alternatively, the front wheels 108 may be actively steerable by the propulsion system (e.g., including one or more steering motors 105) to assist in controlling the direction of the autonomous work vehicle 100 and / or may be driven by the propulsion system (i.e., to provide a front-wheel drive or all-wheel drive mower).

[0016] An implement (e.g., a grass-cutting element such as blade 113) may be coupled to an implement motor 112 (e.g., a cutting motor) carried by housing 102. When motors 112 and 104 are energized, autonomous work vehicle 100 may be propelled over surface 103, and vegetation (e.g., grass) passed over by the mower is cut by blade 113. While illustrated herein using only a single blade 113 and / or motor 112, mowers incorporating multiple blades powered by single or multiple motors are contemplated. Additionally, while described herein in the context of one or more conventional "blades," other cutting elements may include, for example, disks, nylon string or line elements, knives, cutting reels, etc. Additionally, embodiments combining various cutting elements are also contemplated, for example, combining a rotary blade with an edge-mounted string trimmer.

[0017] Autonomous work vehicle 100 may further include a power source, which in one embodiment is battery 114 having a lithium-based chemistry (e.g., lithium ion). Other embodiments may utilize batteries of other chemistries or other power source technologies entirely (e.g., solar power, fuel cells, internal combustion engines). Additionally, while shown using separate blade and wheel motors, it should be noted that such a configuration is exemplary only, as embodiments in which blade and wheel power are provided by a single motor are also contemplated.

[0018] Autonomous work vehicle 100 may further include one or more sensors 116 for providing position data. For example, some embodiments may include a global positioning system (GPS) receiver (or other position sensor that may provide similar data) adapted to estimate the position of autonomous work vehicle 100 within a work area and provide such information to controller 120 (described below). In other embodiments, one or more of wheels 106, 108 may include encoders 118 that provide wheel rotation / speed information that may be used to estimate the position of the mower within a given work area (e.g., based on an initial starting position). Sensors 116 may include a boundary detector, for example, via ground-penetrating radar, sonar, lidar (laser range finder), radio frequency identification (RFID), etc. The boundary detector may be used in addition to other navigation techniques described herein.

[0019] Sensors 116 may include forward obstacle detection sensors, rearward obstacle detection sensors, side obstacle detection sensors, or other proximity detectors. The obstacle detection sensors may be used to detect obstacles in the path of autonomous work vehicle 100 as autonomous work vehicle 100 travels in a forward or reverse direction. Autonomous work vehicle 100 can mow while moving in either direction. Although not shown, the obstacle detection sensors may be located at front end portion 110 or rear end portion 111 of autonomous work vehicle 100, respectively.

[0020] Obstacle detection sensors may use contact sensing, non-contact sensing, or both types of sensing. For example, both contact sensing and non-contact sensing may be enabled simultaneously depending on the state of autonomous work vehicle 100 (e.g., movement within or between zones), or only one type of sensing may be used. An example of contact sensing includes using a contact bumper protruding from housing 102, or the housing itself, that can sense when autonomous work vehicle 100 comes into contact with an obstacle. Non-contact sensors can use acoustic or light waves to detect obstacles, sometimes at a distance from autonomous work vehicle 100 before contact with the obstacle, for example, using infrared, radio detection and ranging (radar), light detection and ranging (lidar), acoustic detection and ranging (sonar), etc.

[0021] Autonomous work vehicle 100 may include one or more vision-based sensors to provide localization data such as position, orientation, and speed. The vision-based sensors may include one or more cameras 122 that capture or record images for use in the vision system. Cameras 122 may be described as part of the vision system of autonomous work vehicle 100. Types of images may include, for example, training images and / or operational images.

[0022] The one or more cameras 122 can detect visible light, non-visible light, or both. The one or more cameras 122 can establish a full field of view around the autonomous machine (e.g., autonomous work vehicle 100) of at least 30 degrees, at least 45 degrees, at least 60 degrees, at least 90 degrees, at least 120 degrees, at least 180 degrees, at least 270 degrees, or at least 360 degrees. The field of view can be defined horizontally, vertically, or in both directions. For example, the full horizontal field of view can be 360 ​​degrees, and the full vertical field of view can be 45 degrees. The field of view can capture image data above and below the height of the one or more cameras.

[0023] In some embodiments, autonomous work vehicle 100 includes four cameras 122. Each camera 122 may face a different direction, including a forward direction, a reverse direction, a first lateral direction, and a second lateral direction (e.g., a reference direction relative to autonomous work vehicle 100). One or more camera directions may be positioned orthogonal to the direction of one or more other cameras 122, or may be positioned in the opposite direction to the direction of at least one other camera 122. Cameras 122 may also be offset from any of these directions (e.g., at 45 degrees or another non-perpendicular angle). In some embodiments, fewer than four cameras 122 may be used to generate images from four different directions. For example, a single camera may rotate about a vertical axis to obtain four different images in the forward, reverse, and lateral directions.

[0024] Autonomous work vehicle 100 may be guided along a path by, for example, pushing, driving, or towing using manual controls such as steering wheel assembly 134. For example, manual direction of autonomous work vehicle 100 may be used during a training mode to learn the work area and / or boundaries associated with the work area. Steering wheel assembly 134 may extend outward and upward from rear end portion 111 of autonomous work vehicle 100. Depending on different vehicle configurations, other types of manual controls may be used, such as a steering wheel and pedals for a passenger vehicle, a wired or wireless controller, a towing handle, etc.

[0025] Forward-facing camera 122 may have a pose that represents the pose of the autonomous machine. The pose may be a six-degree-of-freedom pose and may include all position and orientation parameters in three-dimensional space. In some embodiments, the position and orientation of the camera may be defined relative to the geometric center of autonomous work vehicle 100 or relative to one of the edges of autonomous work vehicle 100.

[0026] The sensors of the autonomous work vehicle 100 may be described as either vision-based sensors or non-vision-based sensors. Vision-based sensors may include cameras 122 that can record images. The images may be processed and used to construct a 3D point cloud (3DPC) and / or for optical odometry (e.g., optical encoding). Non-vision-based sensors may include any sensor other than cameras 122. For example, wheel encoders 118, which use optical (e.g., photodiodes), magnetic, and / or capacitive sensing to detect wheel rotation, may be described as a non-vision-based sensor that does not utilize a camera. Wheel-encoded data from wheel encoders may also be described as odometry data. In some embodiments, non-vision-based sensors do not include a boundary detector. In some embodiments, non-vision-based sensors receive signals from wireless geolocation services, such as from GPS satellites or other transceivers. These sensors may also be grouped as providing localization input that is independent of wireless geolocation services. These localization inputs include vision-based inputs, encoders, boundary detectors, bump / collision sensors, wireless proximity sensors, and more.

[0027] The autonomous work vehicle 100 may also include a controller 120 operable to monitor and control various mowing functions. As seen in the block diagram at the bottom of FIG. 1 , the controller 120 may include one or more processors 124 that receive various inputs and execute one or more computer programs or applications stored in a memory 126. One or both of the processor 124 and the memory 126 are coupled to input / output (I / O) circuitry 128. The I / O circuitry facilitates on-board communication between peripherals 130, such as network adapters, user interfaces, sensors, and the like. The I / O circuitry 128 may also facilitate communication with off-board devices, such as motor controllers, sensors, cameras 122, lights (not shown), and the like. Off-board communication may be achieved via a controller area network (CAN), an I / O bus, or a network interface (I / O). 2 Bus media and protocols such as inter-integrated circuit (C), universal serial bus (USB), etc. may be used.

[0028] The memory 126 may include any volatile, non-volatile, magnetic, optical, and / or electrical media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), electrically erasable programmable ROM (EEPROM), flash memory, and / or any other digital media. Although both the memory 126 and the processor 124 are shown as being integrated into the controller 120, they may be included in separate modules.

[0029] Processor 124 may include any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), and / or equivalent discrete or integrated logic circuitry. In some embodiments, processor 124 may include multiple components, such as any combination of one or more microprocessors, one or more controllers, one or more DSPs, one or more ASICs, and / or one or more FPGAs, as well as other discrete or integrated logic circuitry. Functionality attributed herein to controller 120 and / or processor 124 may be embodied as software, firmware, hardware, or any combination thereof. Certain functions of controller 120 may also be performed in a cloud or other distributed computing system operably connected to processor 124.

[0030] The memory 126 may include computer-readable instructions or applications that, when executed by, for example, the processor, cause the controller 120 to perform various calculations and / or issue commands. The processor 124 and the memory 126 together may define a computing apparatus operable to process input data and generate desired output for one or more components / devices.

[0031] Controller 120 may use processors and memories in a variety of different systems. In particular, one or more processors and memories may be included in each of the different systems. In some embodiments, controller 120 may at least partially define a vision system, which may include a processor and memory. Controller 120 may also at least partially define a navigation system, which may include a processor and memory separate from the processor and memory of the vision system.

[0032] Each system may also be described as having its own controller 120. For example, a vision system may be described as including one controller 120, and a navigation system may be described as having another controller 120. Thus, an autonomous work vehicle 100 may be described as having multiple controllers 120. In general, as used herein, the term "controller" may be used to describe a component of a "system" that provides commands to control various other components of the system.

[0033] The following embodiments relate to training an autonomous machine to traverse a work area. Typically, the machine has a predefined work area that is traversed to perform work. In outdoor applications, a user typically assists the autonomous machine in learning boundaries and off-limits areas. While outdoor boundaries such as fences, sidewalks, and trees can be detected by the autonomous machine's sensors, other boundaries, such as property lines, may not have features that can be detected by the autonomous machine and therefore must be taught or input to the machine by the user. Additionally, a work area may be composed of multiple non-contiguous sub-areas, each of which not only requires a defined boundary but also requires the definition of safe traverse paths between the areas, and the user may be best able to select these traverse paths.

[0034] Current autonomous work machines are expected to require some degree of user input to define their work area, but this type of instruction can be complex or tedious for casual users. For example, a user can guide an autonomous machine along the perimeter of a work area to define boundaries and collect navigation data. For machines that include vision-based navigation capabilities, this can include capturing images during the guided traverse that can be compiled into a navigation database. This navigation database can include image features and three-dimensional point clouds (3DPCs), which can be referenced during autonomous operation by comparing live image data with corresponding data in the database.

[0035] User-guided traversal can be effective in boundary definition because it allows the user to closely assess conditions that may not be considered until actually guiding an autonomous machine. For example, areas with potential obstacles or getting stuck, such as sinkholes, narrow passages, or steep slopes, may appear harmless at first glance, but may become problematic when manually pushing a machine through these areas. On the other hand, a disadvantage is that user-guided traversal of a machine can be time-consuming and tedious in very large work areas or work areas with few obstacles and simple boundaries.

[0036] Another method for defining work areas is through the use of electronic / digital maps. Digital maps typically combine cartographic data (e.g., geological reference points) and imagery (e.g., satellite or aerial photography). Digital maps display man-made or natural geographic features (e.g., roads, buildings, property lines, rivers, hills, etc.) in two-dimensional (2D) or three-dimensional (3D) renderings on a computer screen. Using digital maps, users can define boundaries and no-go areas using computer input, such as graphic tools to define lines, curves, closed shapes, and other graphic elements. The extents of these graphic elements can be converted into geographic coordinates (e.g., latitude and longitude) that are used to locate the actual extent of the site. Geolocation systems, such as GPS and real-time kinetic (RTK) positioning, are used on autonomous machines to sense their location and allow them to traverse paths defined through the digital map.

[0037] Using digital maps to define work area features, while useful for large areas, has its own inherent drawbacks. For example, it may not be as accurate as a user-guided manual traverse by an autonomous machine. While some geolocation systems are fairly accurate (e.g., within a few centimeters), there can still be errors in area selection on maps that may not have the resolution to show small obstacles or features. Also, computer-aided drafting (CAD)-type interfaces used to define boundaries and other features can be difficult to master for less computer-savvy users. Placing precise points on a map, such as with finger input, can also be difficult on mobile devices with touchscreens.

[0038] Some autonomous devices, such as robotic vacuum cleaners, may be able to define their own working space through autonomous exploration because boundaries (such as walls or furniture) may be relatively easy to detect and the penalties for errors (such as getting stuck or falling down stairs) may not be severe. This may not be the case for outdoor working areas, which may have more complex hazards that are difficult to overcome.

[0039] In the embodiments described below, methods and systems are described that can use different techniques to define a working area, combining different aspects of the boundary definition techniques described above, so that the working area can be conveniently defined with sufficient accuracy to allow an autonomous machine to subsequently operate within the area without supervision. These methods and systems take into account navigation accuracy as well as the relative advantages and disadvantages of different navigation modes (e.g., geolocation, image-based localization, dead reckoning, etc.).

[0040] For purposes of the following discussion, various concepts of work area 200 will be described as shown in the diagram of FIG. 2 . A boundary is defined or identified around work area 200. In some embodiments, autonomous work vehicle 100 may traverse random, semi-random, or planned paths to perform work in work zones 202-204 within work area 200. Work zones 202-204 may represent outdoor areas or maintenance areas, such as lawns. Autonomous work vehicle 100 may travel through work area 200 along multiple paths that sufficiently cover the areas within work zones 202-204, for example, to mow all of the grass within each zone 202-204. Autonomous work vehicle 100 may be recharged as needed, such as when transitioning between zones 202-204. Recharging bases, docking stations, or base stations 205 may be located within or along work area 200.

[0041] Boundaries may be used to define work area 200 and various zones 202-204 within work area 200. Boundaries may be defined manually or automatically using a training mode of autonomous work vehicle 100. Additionally, portions of boundaries may also be defined using fixed property boundaries or other types of boundaries. In some embodiments, boundaries may be defined by guiding autonomous work vehicle 100 along work area 200, such as along a desired boundary path of work area 200 in training mode.

[0042] Other boundaries may be used to define exclusion zones (also called no-go zones). Exclusion zones may represent areas of work domain 200 that autonomous work vehicle 100 is to avoid or detour around. For example, exclusion zones may include obstacles 206-208 (such as landscaped yards) or problem areas (such as steep slopes). Exclusion zones may also represent areas that are traversable but not traversed during work, such as zones of dirt, concrete, gravel, etc. Other boundaries may also be used to define transition zones 210-212, which may be described as transition paths. These zones 210-212 may also be defined as paths independent of boundaries, as indicated by the dashed lines in zones 210-212.

[0043] Generally, transition zones 210-212 or transition paths are zones or paths that connect two other zones, such as transition zone 211 connecting work zones 202 and 203. Transition path 213 connecting work zones 203 and 204 is also shown, but does not necessarily have designated zone boundaries. Transition zones may also be defined between a point within a work area and a "home" location or recharger (e.g., base station 205). Maintenance tasks may or may not be performed in transition zones. For example, autonomous work vehicle 100 may not mow grass in transition zones.

[0044] Work area 200 may be mapped in whole or in part using a topographical map. For example, the topographical map may be created during a training mode of the mower or during a subsequent mowing operation. In either case, the topographical map may include information about the terrain of work area 200, such as elevation, slope, identified obstacles (e.g., permanent obstacles), identified stuck areas (e.g., areas where the mower is stuck due to slope or other traction conditions), or other information that may aid autonomous work vehicle 100 in its ability to traverse the work area.

[0045] The resolution of the points stored in the topographic map may be sufficient to provide useful elevation and / or slope information about the terrain of the work area 200 (e.g., on the order of feet or decimeters). For example, the resolution of the points may correspond to a spacing between points that is equal to or less than the width of the autonomous work vehicle 100. In some cases, different functions of the path planning may use different levels of resolution. For example, a path planner mapping a work zone or exclusion zone may have the highest resolution (e.g., on the order of centimeters). In other words, points that are close to, adjacent to, or near irregular boundaries or obstacles may have a finer resolution.

[0046] Autonomous work vehicle 100 may begin covering work area 200, for example, from the boundary of the work area indicated by point 214 or from charger 205. Autonomous work vehicle 100 may determine a first zone 202. Zone 202 may be located adjacent to the boundary of work area 200 or may be located further inward of work area 200, as shown. In other embodiments, zone 202 may cover the entire work area 200. Once autonomous work vehicle 100 has finished mowing zone 202, the mower may begin mowing another zone (e.g., zone 202, which may be dynamic or stationary).

[0047] In one or more embodiments, it may be desirable to rely as much as possible on wireless geolocation (e.g., GPS, RTK) for the autonomous work vehicle 100 to navigate the work area 200. Wireless geolocation does not require as much power and / or processing power as visual navigation, and in some cases has more than sufficient accuracy. One problem encountered with wireless geolocation is that there may be areas where wireless geolocation does not work due to radio signal interference. An example work area 300 illustrating this is shown in the diagram of FIG. 3.

[0048] In this working area 300, a single working zone 302 is shown with a defined boundary 304. This boundary 304 identifies the area in which the autonomous machine 100 will operate. Other boundaries (e.g., exclusion zones) may also be defined that govern the operation of the autonomous machine 100 within the working area 300, but these other zones are not shown here. One or more dead zones 305, 306 are defined in the working area 300. Within the dead zones 305, 306, loss of wireless geolocation service is known or predicted. There are various methods for determining the dead zones 305, 306, which are described in more detail below. For purposes of this example, the geometry of the work location boundary 304 and the dead zones 305, 306 is known, but a traverse path through the working zone 302 has not yet been generated.

[0049] In general, given boundary 304, definitions of dead zones 305, 306, the working footprint of autonomous machine 100, and the type of traverse path (e.g., column, row, concentric circle, custom pattern, random), traverse pattern 310 can be automatically generated within the boundary. Traverse pattern 310 can encounter dead zones 305, 306 and plot a path through them, or can avoid explicitly defining a path through them. Figure 3 shows a portion of traverse pattern 310 that has encountered dead zone 306 and is assumed to continue as shown, e.g., in a column direction, throughout zone 302, eventually encountering dead zone 305 as well.

[0050] Navigation through the dead zones 306 while performing the traverse pattern 310 can be accomplished in a number of ways. In one embodiment, the traverse pattern 310 may be plotted through the dead zones 306, with breakpoints 312 indicated at the boundaries of the dead zones 306. In other embodiments, the traverse pattern 310 may be modified with segments 314 that traverse the outer boundaries of the dead zones 306. In this latter case, the dead zones 306 may be treated the same as exclusion zones for path generation purposes. The difference in this case from exclusion zones is that the autonomous machine will eventually operate through the dead zones 305, 306 (or may operate through the dead zones 305, 306 before or during the operation of the traverse pattern 310).

[0051] Once the entire traverse pattern 310 is defined, the autonomous machine 100 executes the traverse pattern 310, optionally training in the work zone 302 and ultimately working in the work zone 302. The traverse pattern 310 is followed while traveling using wireless geolocation services. If the autonomous machine 100 encounters a dead zone 306 before, during, or after executing the traverse pattern 310, it prioritizes local navigation mode (e.g., giving it more weight if already in use, switching it on if not in use, relying solely on it, etc.) and operates within the dead zone using a path 316 connecting breakpoints 312 in the traverse pattern 310. The local navigation mode can include any type of navigation that does not rely on wireless geolocation services, such as GPS or RTK. For example, visual navigation using locally stored 3DPCs may be used. The local navigation data uses the same geographic coordinates used by the geolocation service and can be fixed to the GPS / RTK coordinate system, allowing the machine to estimate its geolocation within the dead zone even if GPS or RTK communication is lost. Reserving resource-intensive local navigation modes, such as visual navigation, for specific areas such as dead zones reduces the amount of machine training required, simplifies setup, and also makes more efficient use of power and computer resources during operation.

[0052] If the machine is configured to work through the dead zone 306 via paths 316, 317, the autonomous machine 100 can switch to local navigation mode at the edge of the dead zone 306 (indicated by breakpoint 312) and attempt to remain on the path 316 that joins the breakpoint 312. Once the path 316 within the dead zone 306 is complete, the autonomous machine 100 continues to work by executing the traverse pattern 310 while navigating using wireless geolocation services. This can be repeated for other paths 317 that cross the boundary of the dead zone 306 connecting other breakpoints in the traverse path 310.

[0053] In some cases, the dead zones 306, 305 may indicate intermittent loss of connectivity to geolocation services. In such cases, reliance on local navigation mode may occur only when connectivity is lost or quality drops below a threshold, and not at specific locations such as the fixed breakpoint 312. Furthermore, if higher accuracy geolocation (e.g., RTK) is lost but lower accuracy geolocation (e.g., GPS) is available, reliance on local navigation mode may be necessary only when higher accuracy is needed. For example, GPS may be sufficient for navigation using random patterns, resulting in a primary reliance on GPS for detecting boundaries or obstacles.

[0054] If traverse pattern 310 uses avoidance segment 314 to avoid dead zone 306, autonomous machine 100 can complete the entire traverse pattern 310 without entering dead zones 305-307, then drive back and work through dead zones 305-307. Alternatively, autonomous machine 100 may work through dead zones 305-307 before traverse pattern 310. In either case, wireless geolocation services may be used to drive to the edge of dead zones 305-307, and then local navigation mode may be used to work through dead zones 305-307. Dead zones 305-307 may be worked using any suitable pattern, such as concentric circular paths 318 shown in dead zone 305 or random paths 319 shown in dead zone 307.

[0055] Note that there can be many variations in the order in which dead zones are worked. For example, if traverse pattern 310 with avoidance segment 314 is used, dead zones 305-307 can be worked the first time autonomous machine 100 encounters the boundary of dead zone 306 (e.g., upper breakpoint 312), transitions to local navigation mode, works dead zone 306 in the mode described above, and then resumes traverse pattern 310 at the same point where it was entered. Entering and working through dead zone 306 can occur any time autonomous machine 100 is at or near the boundary of dead zone 306, not just during the initial encounter.

[0056] When traversing the dead zones 305, 306, the autonomous machine 100 may use navigation modes such as dead reckoning, IMU, visual navigation, radio beacons, and LIDAR. The autonomous machine 100 may rely on stored maps (e.g., 3DPC, image features) previously mapped by the machine through autonomous or guided exploration and registered to a coordinate system (e.g., terrestrial latitude and longitude) used to map the entire work area. The autonomous machine 100 may have preferred or predefined routes through the dead zones 305, 306 and be guided along those routes by comparison of sensor data with stored map data. Routes may be modified based on changing conditions, e.g., temporary obstacles or stuck areas detected through cameras, touch sensors, wheel spin detectors, etc. Similar obstacle adaptations may also be used while traversing the traverse pattern 310 outside the dead zones 305, 306, but may not require reference to an internal map or other data.

[0057] The location and size of the dead zone can be identified in many ways. For example, the autonomous machine 100 can directly identify the dead zone while traveling through the work area 300 and determining that the signal strength (or some other measure of quality) of the wireless geolocation service is below a threshold. This may occur during training or during work, and the autonomous machine 100 may be operating in an autonomous, semi-autonomous, or user-guided mode. This loss of signal strength can define a point on the boundary of the dead zone, and the autonomous machine 100 can move away from (e.g., back up) or pass through the dead zone until the wireless geolocation signal quality is above the threshold. Once at least three points on the boundary of the dead zone (not on a straight line) have been determined, a triangular area can be plotted to represent the dead zone. This triangular shape may expand as more points are discovered.

[0058] Another method that may determine the location and size of dead zones is the analysis of digital map data. This map data may include, but is not limited to, the locations of terrestrial wireless geolocation service transmitters, topographical maps including the topology of land and / or permanent structures (e.g., buildings, towers), the transmission locations of potentially interfering wireless transmitters, and analysis of 2D images to identify blocking or interfering structures (e.g., trees, wireless towers). Such map data may be used to predict the location and size of dead zones. Other map data may include direct measurements of signal strength (or other measures of transmission quality) made during surveys or collected from autonomous machines, mobile devices, etc.

[0059] Typically, the prediction of dead zones may be performed by a centralized service with powerful computers that has access to many data sources. For purposes of this disclosure, this service will be referred to as an operations center (or robot operations center). An operations center is generally a network-accessible data service that can communicate with users and autonomous machines and that runs code specifically written to manage the setup and operational behavior of the autonomous machines.

[0060] At least one predicted dead zone can be used by the autonomous machine before it actually traverses the work area even once, for example, during training. For example, if a dead zone is on or near a boundary, the machine's behavior can change depending on how the boundary is learned by the machine. If the autonomous machine is being driven (e.g., pushed) by a user on a boundary, the autonomous machine can begin recording visual navigation images before reaching the predicted dead zone and while traversing the predicted dead zone. If radio geolocation is actually lost while traversing the predicted dead zone, the visual data can be flagged for further processing later, for example, to form a 3PDC of the area. If radio geolocation is not lost while traversing the predicted dead zone, the image can be retained or discarded, and the result can be sent back to the operations center as feedback to the prediction algorithm, for example, used as error to update a machine learning module.

[0061] If a dead zone is inside the work area, it can be used similarly when an inside boundary is traversed under user supervision. If the inside is traversed using autonomous navigation during training, the machine can slow down before entering the predicted dead zone in case radio geolocation is lost. If radio geolocation is actually lost while traversing the predicted dead zone, the machine can attempt to navigate the dead zone using position-specific inputs that do not rely on radio geolocation services. In other cases, the machine can back out or otherwise attempt to avoid the dead zone while navigating autonomously. The machine can inform the user at the time or later that the dead zone may require further manual training. The machine can perform other operations related to manual navigation, such as collecting images in anticipation of the dead zone or sending a signal back to the operations center if the dead zone is not found in the predicted location.

[0062] The above features and operations related to dead zones can be applied to any type of zone where work is required, but where difficulties are known or predicted and such difficulties may be unrelated to geolocation. These are broadly referred to herein as "smart zones," meaning that more sophisticated navigation modes and / or routes may be required to traverse the zone. A more detailed description of smart zones is provided in U.S. Patent No. 10,932,409, issued March 2, 2021. Within a smart zone, a vehicle travels in a random pattern within a restricted zone that is smaller in size than the work area. The restricted zone can be moved or driven across the work area, so that over time, the restricted zone will travel across substantially the entire working surface of the work area. Smart zones, as described above, can be used to cover or work through dead zones. In one or more other embodiments, smart zones may be used to cover other zones that are expected to be difficult based on the number of obstacles per unit area, the steepness of the gradient within the zone, the ratio of obstacle area to working area, the number of non-boundary turns that occur within a predefined route, the operator definition, the number of times an autonomous machine has previously been stuck in the zone, the estimated time per unit area to work the zone, etc.

[0063] FIG. 4 shows a diagram detailing how an operations center 400 can assist in defining a work area and generating work routes within the work area. User devices 402 can communicate with the operations center 400 over a wide area network. The operations center 400 can provide map data 404 to the device, which can be used to display the work area on an electronic display 403. The map data 404 can be provided directly or through a third party, for example, using a network mapping application program interface. The map data 404 can include vector graphics, aerial photography (which is understood to include satellite imagery), infrastructure overlays (e.g., road and trail boundaries, building geometry), property boundaries, topological overlays, etc. The map data 404 can be derived from public and / or private sources.

[0064] In addition to map data 404, operations center 400 may also provide access to hint data 406 developed specifically for autonomous vehicle operations, such as hint data 406 to assist in the manual and automatic definition of autonomous vehicle work paths. For example, hint data 406 may include predictions of radio geolocation dead zones or smart zones, as described above. Other hint data 406 may include predictions of boundaries, work areas, problem areas, etc. Hint data 406 may include closed shapes that are overlaid on a map to indicate problem areas or dead zones. Other hint data 406 may be provided by image processing algorithms that can predict geometry (e.g., boundaries of a work area), thereby reducing the amount of user input for defining the aspects of the work area.

[0065] For example, in robotic mowing applications, work boundaries are often visible where the grass meets the borders of sidewalks, fences, gravel, etc. These boundaries can be easily detected on aerial photographs by variations in color, brightness, texture, etc. Workable regions and problem areas can also be identified within these boundaries. For example, green areas with relatively little local variation in color / brightness may indicate grass, while green areas with greater local variation in color / brightness may indicate trees, shrubs, or other foliage.

[0066] A user can, for example, use a mouse or touchscreen input to draw on a boundary on the electronic display 403, using color intensity variations as a guide. Image processing algorithms (e.g., edge detection) can accelerate this process, so that the user can be presented with predicted boundary segments based on edge detection in the aerial imagery. Thus, the actual boundary 407 displayed on the display can be drawn by the user or detected by the image processing algorithm. In the latter case, the user can select the boundary 407 to use (or the entire area encompassed by the boundary) by selecting one or more graphic elements (e.g., polylines) on the display 403, and can edit such selected elements using electronic drawing-type tools to correct for algorithmic idiosyncrasies. A similar process can be used to define exclusion zones, such as boundaries 409 and 411, which in this example correspond to a baseball field and a basketball court.

[0067] For example, machine learning algorithms can be trained to recognize known object types, such as trees, bushes, playground equipment, utility poles, and equipment, from aerial imagery. In such cases, operations centers can automatically flag areas containing these detected objects as potentially unworkable or problematic. Similar analysis can be used to detect ground cover, such as grass, dirt, rock, concrete, and asphalt, to guide automatic geometry definition and selection. Convolutional neural networks are known to be effective for this type of image classification and can accept non-image data, such as topological data, in their training and prediction.

[0068] After the user is able to view and add to the graphical representation of the boundary 407 via at least an electronic display, boundary data 408, as well as other optional data such as no-go area data 410 (e.g., defining an exclusion zone within the boundary 407), may be communicated to the operations center 400. These data 408, 410 may be subject to checks, such as ensuring the boundary is a closed shape, identifying and resolving intersecting shapes, etc. This may involve additional communications between the operations center 400 and the user device 402, not shown here.

[0069] Once the geometry of the work area is sufficiently defined and approved by the user, the operations center 400 can define fill pattern data 412, which can be sent to the user device 402 and rendered as a work path 413. To generate the fill pattern data 412, the operations center 400 can use other data (not shown), such as the model number or serial number of the autonomous machine, which can be used to define the machine's geometry and other performance data, such as turning radius, top speed, battery range, etc. Other data used to generate the fill pattern data 412 can include the mode and / or attachments used with the machine (e.g., mowing, edging, trash collection), the location of charging stations, etc. This additional data may be entered manually via the user device 402 during a session and / or retrieved from a user account.

[0070] After the fill pattern data 412 is transmitted to the user device 402, the user may have another opportunity to edit the work path 413, as indicated by the modification data 414. For example, the fill pattern data 412 may include multiple pattern types (e.g., horizontal, vertical, diagonal, random), and the user may decide which one to use. Furthermore, multiple different work paths may be used to work an area within a single session, for example, to create a crisscross mowing pattern. In other cases, different patterns may be used at different times, such as alternating patterns weekly to achieve even mowing or patterns for special events (e.g., holiday-themed patterns).

[0071] Note that the boundary data 408 and fill pattern data 412 may both be estimates at this stage. Even with carefully placed graphical elements, there may be errors of up to one meter between the location registered on the map and the actual location on the ground. Therefore, the estimated geometry created by this map can be considered a general guideline. In some cases, the machine may not need to approach the boundary during operation (e.g., not get closer to the boundary than the expected placement error during boundary creation), in which case the operation traverse path can be generated and verified without further refinement of the boundary definition. In other cases, a more accurate representation of the boundary may be desired, which can be achieved by having the autonomous machine traverse the work area. For example, a user may place the autonomous machine in the work area and instruct the autonomous machine or an operations center, for example, to have the machine traverse any of the previously defined boundaries 407, 409, 411. This is described below as the boundary verification phase.

[0072] When traversing the approximate boundary during the verification phase, the autonomous machine may be under user supervision. In one example, the user may directly control the autonomous machine, for example, via a wired or wireless controller (e.g., a custom controller, a smartphone), by riding or pushing the machine, or by walking the boundary and having the machine track (e.g., using visual and / or proximity detection sensors) or record the boundary. Manual traversal of the boundary may not be necessary for large, featureless boundaries, but manual assistance may still be used for small or complex areas. Other methods described herein can be used to verify larger, less complex boundaries.

[0073] For example, in one or more embodiments, an autonomous machine may attempt to traverse a previously defined estimated boundary, either remaining as defined without adjustment, or using the defined estimated boundary while making corrections based on sensor inputs, such as bump sensors for obstacle detection and visual sensors for tracking boundary features (e.g., line-following mode). In this case, the user simply follows the controller and provides inputs as needed. For example, the user may stop the machine if it is too far from the desired boundary, is about to hit an obstacle, etc. Following this machine stop, manual corrections can be made to correct for the error, and the machine may resume traversing the estimated route and next closest point. Such corrections can also be made without stopping the machine, for example, while in motion.

[0074] Once a more accurate boundary is defined during the boundary verification phase, the measurement data can replace the estimated version stored on the autonomous machine and / or at the operations center. Saving the verified boundary can also trigger the re-creation of the automatic traverse path(s) 413 to conform to the verified boundary. After the automatic traverse path has been reshaped to correspond to the verified boundary, the path can be verified with or without the work implement. This work path verification can be the final step before allowing full autonomous operation, for example, to discover further issues with the work area, e.g., stuck areas, dead zones.

[0075] Note that predicted dead zones or problem areas can also be defined as part of the boundary definition and path generation described in connection with FIG. 4 . While dead zone boundaries may not require validation, the machine may need to collect navigation data for those zones (e.g., before or during work path validation) so that they can be traversed later if geolocation is lost. In one embodiment, for example, prior to work path validation, a user can initiate a training phase to drive the machine to the location of predicted dead zones or problem areas and collect additional data. This additional data may include 3DPC construction of dead zones using cameras and SLAM, random traversal to identify obstacles, etc. Data collection in dead zones can be performed under user guidance using techniques similar to boundary validation described above, such as direct control, user following, autonomous movement with user corrections, etc.

[0076] During the boundary verification and / or final testing of the traverse path, additional dead zones and / or problem areas may be discovered, and these dead zones and / or problem areas may then be traversed under user guidance to gather additional navigation and mapping data. For example, known or predicted problem areas or dead zones may be mapped by the autonomous machine under user guidance before allowing the final testing of the traverse path. If other problem areas or dead zones are discovered during the final testing of the traverse path, the machine may be stopped and the user may request additional mapping supervision. Note that a smart zone may be run within a dead zone to map obstacles in that area. This may enable future path planning options, for example, by discovering where all trees are located. This assumes that all exclusion zones have been mapped within the dead zone, allowing the mower to travel freely within the area.

[0077] It should be noted that the embodiment described in Figure 4 may include allowing a user to first analyze the work area via a digital map and manually and / or automatically define at least a portion of the boundary geometry, and then verify this geometry using a machine traversal over the work area. Figure 5 illustrates another method of identifying the boundary, using the same work area as shown in Figure 4.

[0078] In this example, the user drives / moves the autonomous machine to the work area. At this point, the autonomous machine may be in network communication with the operations center, but this is not necessary. Once the user places the autonomous machine in a workable location in the work area, the user initiates random capture mode. In random capture mode, the autonomous machine moves forward until one of the following occurs: the machine hits an obstacle; the user stops the machine when it reaches a boundary or hazard / obstacle (e.g., a deadman's switch); or the user ends random capture mode. In the first two cases, the autonomous machine turns a random amount (e.g., an angle greater than 60°) and continues moving forward on the new path until it stops again. The user can take over and record at any time, for example, by directly controlling the machine; during a radio geolocation outage, the user can be prompted or requested to take over; in this case, a dead zone is registered and the autonomous machine can take video recording to build a navigation database (e.g., 3DPC) for the area. A similar sequence of events can occur if the machine becomes stuck while traversing a path.

[0079] While traveling, the autonomous machine records its location and may also record other data, such as geolocation system signal strength, signs of travel difficulty (e.g., high motor current, lower or higher than desired speed), etc. Even if this collected data does not result in a dead zone or problem area, it can be valuable data for machine learning purposes. In Figure 5, path 500 shows an example of a path that may be traversed and recorded in this manner.

[0080] As more points are captured, a map of the area is created and updated. This map can be saved locally and / or sent to an operations center. Map data can also be downloaded to the user's device. For example, turnaround points on the path 500 can be connected to form an estimated boundary 502. Once the user is satisfied with the map definition, the user can exit capture mode. The user can still edit the map. For example, they can add extensions 504 to the boundary using graphics tools, such as stretching the estimated boundary 502 or drawing new segments. This allows for areas not traversed during the random capture but that the user wishes to cover during the work cycle. These boundaries can then be used to generate the fill, as described above. Depending on the scope of the random capture, validation of the boundary and / or fill path may not be necessary, for example, if enough area is covered during the random capture, resulting in a reasonably calculated map of the work area.

[0081] 6-8 illustrate, through flowcharts, the generation of a task map according to various exemplary embodiments. FIG. 6 illustrates a method for task map generation via a user device according to an exemplary embodiment. The method includes supporting user selection of a boundary estimate, where the user clicks to specify the boundary on the device (600), e.g., a standalone device or via a service on an operations center. The user can optionally mark (601) large no-go areas (e.g., water hazards, concrete slabs, etc.) within the boundary using similar user interface mechanisms. The operations center proposes (602) training patterns, which can also be generated locally on the user device, to provide a boundary estimate that is validated in the field using an autonomous machine.

[0082] Upon arriving at the site, the user drives (603) the autonomous machine, pushing, moving, or otherwise guiding it to the boundary. Under user supervision (e.g., a deadman switch to halt execution, active position adjustments and corrections), the machine autonomously navigates along the boundary (e.g., using geolocation services). To reduce the amount of work required of the user, the user can make a one-time correction and the machine can attempt to derive the remaining boundary segments on its own. For example, if the estimated boundary is parallel to the actual boundary but offset, the user can remotely bias or shift the machine's traversal boundary, such as by 5 cm to the left of the boundary. The machine makes the adjustment, allowing the user to visually confirm the boundary alignment, and the adjustment is maintained until a new waypoint is reached. After reaching the new waypoint, the bias is reset to zero and the machine again traverses the estimated boundary.

[0083] If a trouble area is detected (604), the autonomous machine can automatically stop and notify the user. The user remotely controls (605) (or directly controls) the autonomous machine along the boundary until the autonomous machine can continue autonomously. The vision system can capture the necessary data during control (605), which can involve, for example, passing through the affected area multiple times from different directions to collect enough data.

[0084] Once the boundary is traversed (block 606 returns "yes"), the adjustments are recorded and re-uploaded to the operations center (or locally coupled computing device) (607). In the next phase of training, the operations center (or locally coupled computing device) generates (608) and provides a training fill pattern. Next, the user positions the autonomous machine in the work area (e.g., along the boundary), and similar to the boundary definition phase in which the machine executes the training fill pattern, in areas where geolocation fails during the execution of the fill pattern, the user can manually continue the pattern, similar to step 605. If an obstacle is encountered, the user may guide the machine around the obstacle or use the machine's own obstacle avoidance algorithm. In another embodiment, the mower can attempt to map obstacles within the work area instead of attempting to avoid them. For example, the perimeter around a tree or the edge of a retaining wall can be automatically mapped by bumping into or sensing the perimeter around the object. If the machine's own obstacle avoidance algorithm is sufficient to handle most obstacles, the user can have the machine traverse the fill pattern itself. These results (adjustments, obstructions, etc.) are uploaded to, for example, an operations center or computing device and used to refine the fill pattern for working the area (609).

[0085] In Figure 7, a flow diagram illustrates an example of random workspace mapping according to an example embodiment. The user drives / moves the robot to the workspace (700) and initiates random capture mode. The machine moves forward (701) until one of the following events (702) is detected: the machine hits an obstacle, or the user stops the machine, such as when it reaches a boundary or hazard. If the operator's stop in block 703 is not accompanied by a command to end random capture, the machine turns to a new direction (704) and moves forward again (701).

[0086] Once the user exits the random capture mode (block 703 returns "yes"), the traversed locations are uploaded (705) to an operations center or local computing device. Note that this upload 705 can occur continuously or intermittently while traversing the route, instead of at the end of random capture. The location data is used to generate (706) an area map (e.g., boundaries or work areas, exclusion zones, dead zones) that can be presented to the user (e.g., sent to the user's mobile device). This can be followed by user modification of the map, such as by additional traversal of selected areas, editing the map via an electronic display, etc. As shown in FIG. 6, additional steps may also be performed, such as fill pattern generation 608 and refinement 609.

[0087] FIG. 8 illustrates a flow diagram of an example of cloud-assisted work area mapping in accordance with an illustrative embodiment. A user drives / moves an autonomous machine to a work area (800) and communicates to an operations center (e.g., using a mobile device) that the machine is on or within the boundary of the work area (801). The machine's coordinates are uploaded to the operations center (802), and the operations center uses a service to obtain an aerial image of the area (803). A detection algorithm is used to identify (804) the boundaries of different regions (e.g., grass, sidewalk, asphalt, foliage, etc.), and an estimated training pattern is generated. FIG. 9 illustrates an example of automatically detected boundary regions 900-905, with different shading that may indicate different characteristics of each region. The boundary and / or training pattern is downloaded to the autonomous machine or user device. The user confirms (805) the boundary / training pattern and instructs (806) the autonomous machine to navigate the generated boundary. As with other embodiments, a user can fine-tune (807) the autonomous machine's movement to more closely follow physical boundaries.

[0088] As with the previous embodiment, areas within the work area where geolocation is not available are traversed by operator control (using automatic image capture for visual training), dead reckoning, and / or random navigation. Collided obstacles are automatically traversed and the boundary adjusted. Wall-following algorithms or operator control can also be used to avoid obstacles. Once the boundary is complete, a training pattern is executed and obstacles are automatically traversed and mapped. The perimeter of the obstacle is either using a wall-following algorithm or operator control. The visually mapped area is generated on the autonomous machine for later execution (e.g., 3PPC generation). The boundary adjustment / mapping can be uploaded to the operations center to generate a fill pattern.

[0089] While the disclosure is not so limited, an understanding of various aspects of the disclosure will be gained through the description of specific illustrative examples provided below. Various modifications of the illustrative examples, as well as additional embodiments of the disclosure, will become apparent herein.

[0090] Example 1 is a method including: defining a boundary of a work area within which an autonomous machine operates; defining a dead zone within the work area where loss of wireless geolocation services is known or predicted; automatically generating a traverse pattern within the boundary; causing the autonomous machine to perform a task by executing the traverse pattern while traveling outside the dead zone using the wireless geolocation services; and, when the dead zone is encountered before, during, or after execution of the traverse pattern, prioritizing position determination input that does not rely on the wireless geolocation services and performing the task within the dead zone.

[0091] Example 2 includes the method of example 1, where the localization input includes image-based localization using a three-dimensional point cloud. Example 3 includes the method of example 1 or 2, where the autonomous machine navigates along multiple random paths while in the dead zone. Example 4 includes the method of any one of examples 1-3, where the dead zone is encountered during execution of the traverse pattern, and the method further includes, upon completion of operation in the dead zone, continuing execution of the traverse pattern while navigating using the wireless geolocation service.

[0092] Example 5 includes the method of any one of examples 1-3, wherein the dead zone is encountered before or after execution of the traverse pattern and the wireless geolocation service is used to navigate into the dead zone before or after completion of the traverse pattern. Example 6 includes the method of any one of examples 1-5, wherein defining the boundary includes navigating an autonomous machine along the boundary under user supervision and defining the boundary based on a path traversed during the journey based on the boundary estimate.

[0093] Example 7 includes the method of example 6, where navigating the boundary under user supervision includes pushing, driving, or towing the autonomous machine. Example 8 includes the method of any one of examples 1-5, where defining the boundary includes assisting a user in selecting an estimate of the boundary via an image on an electronic map of the work area, moving the autonomous machine to the work area, autonomously navigating the autonomous machine along the estimate of the boundary under user supervision, and defining the boundary based on a path traversed during navigation based on the estimate of the boundary.

[0094] Example 9 includes the method of example 8, wherein autonomously navigating the autonomous machine along the boundary estimate under the supervision of the user includes the user correcting the autonomous machine so that the traversed path coincides with a local boundary while the autonomous machine moves along the boundary estimate. Example 10 includes the method of example 8, further including, in response to identifying a problem area while autonomously navigating the autonomous machine along the boundary estimate, stopping autonomous navigation and assisting a user in guiding the autonomous machine through the problem area.

[0095] Example 11 includes the method of example 10, where, during the user guidance of the autonomous machine through the problem area, the autonomous machine records camera images stored on the autonomous machine, which are used to geolocate features in the camera images, and the geolocated features are used for subsequent navigation in a local navigation mode when operating in or near the problem area. Example 12 includes the method of example 10, where the problem area is identified based on the autonomous machine's inability to autonomously navigate through obstacles in the problem area. Example 13 includes the method of example 10, where the problem area is automatically identified based on image analysis of the electronic map at an operations center.

[0096] Example 14 includes the method of Example 8, in which supporting a user's selection of the boundary estimate via the image on the electronic map of the work area includes performing image analysis of the image to identify workable areas having similar image characteristics; displaying the workable areas as overlays on the electronic map; and receiving one or more user selections of the workable areas, wherein geometries of the one or more workable areas are used to define the boundary estimate.

[0097] Example 15 includes the method of example 14, where the image analysis further identifies a problem area within the work space, the method further including: stopping autonomous navigation while the autonomous machine is autonomously navigating along the boundary estimate to assist a user in guiding the autonomous machine through the problem area. Example 16 includes the method of example 15, where the problem area is identified based on identifying an inoperable region having characteristics corresponding to a known obstacle type. Example 17 includes the method of example 1, where the dead zone of the work space is automatically defined by image analysis of an electronic map.

[0098] Example 18 includes the method of any one of examples 1-17, where defining the dead zone includes autonomously navigating the autonomous machine along the traverse pattern while the autonomous machine is operating, and defining a geometry of the dead zone in response to loss of the wireless geolocation service in the dead zone. Example 19 includes the method of example 18, further including halting autonomous navigation and assisting a user of the autonomous machine in navigating the dead zone in response to loss of the wireless geolocation service in the dead zone. Example 20 includes the method of example 18, further including responsive to loss of the wireless geolocation service in the dead zone, the autonomous machine recording a camera image that is stored on the autonomous machine, the camera image being used for subsequent navigation through the dead zone in a local navigation mode when operating in the dead zone.

[0099] Example 21 includes the method of example 18, wherein a geometry of the dead zone is transmitted to an operations center and associated with a geometry of the working area, and wherein the geometry of the dead zone and the geometry of the working area are used to train a predictive algorithm.Example 22 includes the method of any one of examples 1-21, wherein defining the dead zone includes autonomously navigating the autonomous machine along the traverse pattern during operation of the working area of ​​the autonomous machine, and defining the geometry of the dead zone in response to loss of the wireless geolocation service in the dead zone.Example 23 includes the method of example 22, further including recording a camera image that is stored on the autonomous machine while attempting to traverse the dead zone, the camera image being used for subsequent navigation through the dead zone in a local navigation mode when operating in the dead zone.

[0100] Example 24 includes the method of any one of Examples 1 to 23, wherein defining the boundary includes: moving the autonomous machine to the work area; autonomously driving the autonomous machine along a plurality of random paths, each of the random paths terminating when an obstacle is encountered or when a user commands the autonomous machine to stop; identifying an estimated boundary based on endpoints of the plurality of random paths; presenting the estimated boundary to a user via an electronic map of the work area; and defining the boundary based on a selected one of the estimated boundaries that is acceptable to the user.

[0101] Example 25 includes the method of example 24, wherein a problem area within the boundary is also defined based on the selected estimated boundary. Example 26 includes the method of example 24, wherein defining the dead zone includes defining a geometry of the dead zone in response to loss of the wireless geolocation service along one of the random paths. Example 27 includes the method of example 26, further including recording, by the autonomous machine, camera images that are stored on the autonomous machine while traveling through the dead zone, the camera images being used for subsequent navigation through the dead zone in a local navigation mode when operating in the dead zone. Example 28 includes the autonomous machine of any one of examples 1-27, comprising a processor operable to perform the method of any one of examples 1-27.

[0102] While the disclosure is not so limited, an understanding of various aspects of the disclosure will be gained through the description of specific illustrative examples provided below. Various modifications of the illustrative examples, as well as additional embodiments of the disclosure, will become apparent herein.

[0103] It should be noted that the terms "have," "include," "comprises," and variations thereof do not have a limiting meaning and are used in their open-ended sense when the terms appear in the accompanying description and claims, generally to mean "including, but not limited to." Furthermore, "a," "an," "the," "at least one," and "one or more" are used interchangeably herein. Additionally, relative terms such as "left," "right," "front," "fore," "forward," "rear," "aft," "rearward," "top," "bottom," "side," "upper," "lower," "above," "below," "horizontal," "vertical," and the like may be used herein, and when used, are from the perspective shown in a particular figure or while the machine is in an operational configuration. As used herein, the terms "determine" (identify) and "estimate" may be used interchangeably depending on the particular context of their use, for example, to determine or estimate the position or orientation of a vehicle, boundary, obstacle, etc.

[0104] Unless otherwise indicated, all numerical values ​​expressing size, quantity, and physical properties of features used in the specification and claims are understood to be modified in all instances by the term "about." Accordingly, unless indicated to the contrary, the numerical parameters set forth in the foregoing specification and appended claims are approximations that may vary depending upon the desired properties sought to be obtained by those of ordinary skill in the art utilizing the teachings disclosed herein. The use of numerical ranges by endpoints includes all numbers within that range and any range within that range (e.g., 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5).

[0105] The various embodiments described above may be implemented using circuits, firmware, and / or software modules that interact to provide particular results. Those skilled in the art will readily be able to implement such described functionality at the modular level or as a whole using knowledge generally known in the art. For example, the flowcharts and control diagrams illustrated herein may be used to create computer-readable instructions / code for execution by a processor. Such instructions may be stored on a non-transitory computer-readable medium and transferred to a processor for execution, as known in the art. The structures and procedures illustrated above are merely representative of embodiments that may be used to provide the functionality described herein.

[0106] The foregoing description of example embodiments has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the embodiments to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. Any or all features of the disclosed embodiments may be applied individually or in any combination and are intended to be purely illustrative, not limiting. It is intended that the scope of the invention be determined not by this detailed description, but rather by the appended claims.

Claims

1. Defining the boundaries of the workspace in which the autonomous machine operates, Encountering a dead zone within the work area where loss of wireless geolocation service has been determined or predicted, During traversing through the dead zone, the autonomous machine captures camera images, and compiles these camera images into a navigation database that includes geolocation-determined image features from the camera images and is used for subsequent image-based localization. Automatically generate a traverse pattern within the aforementioned boundary, The autonomous machine is instructed to perform the task by executing the traverse pattern while traveling outside the dead zone using the wireless geolocation service. Before, during, or after the execution of the traverse pattern, if the dead zone is encountered, priority is given to the image-based positioning, and the operation is performed within the dead zone. Methods that include...

2. The method according to claim 1, further comprising predicting the dead zone by analyzing digital map data before the autonomous machine traverses the work area, wherein the traverse through the dead zone includes a user-guided traverse for collecting navigation images.

3. The method according to claim 1 or 2, wherein the autonomous machine travels along a plurality of random paths while within the dead zone.

4. The method according to claim 1, wherein the dead zone is encountered during the execution of the traverse pattern, and the method, once the work in the dead zone is completed, continues the execution of the traverse pattern to perform the work while driving using the wireless geolocation service.

5. The method according to claim 1, wherein the dead zone is encountered before or after the execution of the traverse pattern, and the wireless geolocation service is used to drive into the dead zone before or after the completion of the traverse pattern.

6. Defining the aforementioned boundary means The autonomous machine is driven along the boundary under the supervision of the user, The boundary is defined based on the path traversed during travel, which is based on the estimation of the boundary. The method according to claim 1, including the method described in claim 1.

7. The method according to claim 6, wherein driving the boundary under the user's supervision includes pushing, driving, or towing the autonomous machine.

8. Defining the aforementioned boundary means To support user selection for boundary estimation via an image on an electronic map of the work area, Moving the autonomous machine to the work area, To make the autonomous machine autonomously move along the estimated boundary under the supervision of the user, The boundary is defined based on the path traversed during travel, which is based on the estimation of the boundary. The method according to claim 1, including the method described in claim 1.

9. The method according to claim 8, wherein allowing the autonomous machine to autonomously travel along the estimated boundary under the supervision of the user includes the user correcting the autonomous machine so that the traversed path coincides with the local boundary while the autonomous machine is moving along the estimated boundary.

10. The method according to claim 8, further comprising stopping autonomous driving and assisting user guidance of the autonomous machine through the problem area in response to identifying a problem area while the autonomous machine is autonomously driving along the estimated boundary.

11. The method according to claim 10, wherein, during user guidance of the autonomous machine through the problem area, the autonomous machine records the camera image stored on the autonomous machine, which is used to identify the geographic location of features within the camera image, and the geographically located features are used for subsequent driving in local navigation mode when performing the work in or near the problem area.

12. The method according to claim 10, wherein the problem area is identified on the basis that the autonomous machine is unable to autonomously navigate through an obstacle in the problem area.

13. The method according to claim 10, wherein the problem area is automatically identified based on image analysis of the electronic map at the operation center.

14. Assisting the user in estimating the boundary via the image of the electronic map of the work area is, The image analysis of the aforementioned image is performed to identify a workable region with similar image characteristics, The aforementioned workable area is displayed as an overlay on the electronic map, The method includes receiving a user selection of one or more of the workable areas, the geometry of which is used to define the boundary estimation, the image analysis further identifies a problem area within the work area, the method further includes stopping the autonomous driving while the autonomous machine is autonomously driving along the boundary estimation, and assisting user guidance of the autonomous machine through the problem area, the problem area being identified based on identifying an unworkable area having features corresponding to a known type of obstacle. The method according to claim 8.

15. The method according to claim 1, wherein prioritizing image-based localization includes using both the wireless geolocation service and the image-based localization, and giving greater weight to the image-based localization.

16. The autonomous machine comprising a processor capable of performing the method according to claim 1 by the autonomous machine.

Images