Robotic lawnmower path planning and obstacle recognition

Abstract

The invention discloses a robotic lawn mower and a computer-implemented method for distinguishing feature types of lawn areas. The method comprises the following steps: detecting non-lawn characteristics on a forward path of the robot mower; controlling the robotic mower to at least partially bypass the non-lawn feature along a periphery of the non-lawn feature; recording one or more features of the bypassing non-lawn feature; whether the type of the non-lawn feature is a boundary of the lawn area or a feature completely included within the lawn area is distinguished according to the bypassing feature.

Description

Technical Field

[0001] The present application relates to a device, system and method for autonomously controlling a robot lawn mower, and in particular to a device, system and method for distinguishing obstacle and / or boundary types for a robot lawn mower. Background Art

[0002] A robot or robotic lawn mower is an autonomous robot that is used to cut grass on a lawn or field. With a robotic lawn mower, humans don’t have to physically mow the grass, which can be a mundane and tedious task.

[0003] Figure 1 An example of a lawn area in which a robotic lawn mower can operate is shown. The robotic lawn mower 1 can be placed in a charging station 2 and charged, and mows within a lawn area 3 designated by boundary lines or guide lines 4. Before the robotic lawn mower 1 is started, the boundary lines or guide lines 4 need to be physically installed. The robotic lawn mower 1 will detect the boundary line 4 and change direction (usually randomly) to avoid it.

[0004] The use of the boundary line 4 is inefficient, and the user needs to carefully set the boundary line 4 along the perimeter of the lawn area 3 and any obstacles / hazards within the lawn area 3. This is a time-consuming and laborious task. In addition, the boundary line 4 needs to be maintained throughout the life cycle of the robotic lawn mower 1.

[0005] Route planning where the robotic lawn mower randomly changes direction is also inefficient because the random direction may be towards an obstacle / hazard or another boundary. By using random directions in route planning, the robotic lawn mower 1 may take longer to mow the lawn in lawn area 3.

[0006] It has been realized that it would be beneficial to provide a robotic lawn mower that does not require boundary lines and can plan paths so that the robotic lawn mower can work more efficiently. Summary of the invention

[0007] This summary is intended to introduce some concepts in a simplified form that are further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0008] In a first aspect, the present disclosure provides a method for controlling a robotic lawn mower, the method comprising: detecting a non-lawn feature on a forward path of the robotic lawn mower; controlling the robotic lawn mower to at least partially bypass the non-lawn feature along the periphery of the non-lawn feature; recording one or more features of bypassing the non-lawn feature; and distinguishing, based on the bypass feature, whether the type of the non-lawn feature is the boundary of a lawn area or a feature that is completely contained in the lawn area.

[0009] Once the type of non-lawn feature is identified, it can be stored in the memory of the robotic lawn mower so that the path planning of the robotic lawn mower to mow the lawn or move within the lawn area can take into account the location and / or type of the non-lawn feature identified within the lawn area. The robotic lawn mower can partially bypass the non-lawn feature to distinguish it, which means that it does not need to complete a full bypass around the non-lawn feature. It can also completely bypass the non-lawn feature. The perimeter can be an inner perimeter if it is a boundary, and an outer perimeter if it is a feature completely contained within the lawn area. The feature completely contained within the lawn area can be an obstacle or a hazard.

[0010] Preferably, the detour characteristics include a detour time.

[0011] Preferably, the detour time is indicated by a timer of a predetermined time length, and the method further comprises: starting the timer at the start time of controlling the robotic lawn mower to detour the non-lawn feature around the periphery of the non-lawn feature; if the timer expires, distinguishing the non-lawn feature as a boundary of a lawn area.

[0012] Using a detour time in this manner, particularly a timer of a predetermined length, can improve efficiency. If the lawn area is large, detours around the boundary of the lawn area can be a time-consuming task for the robotic lawn mower. Automatically distinguishing the non-lawn feature as the boundary of the lawn area upon expiration of the timer means that the robotic lawn mower does not need to detour the entire non-lawn feature.

[0013] Preferably, the method further comprises setting a predetermined duration of the timer by controlling the robotic lawn mower to perform a calibration orbiting motion of a feature completely within the lawn area, wherein the calibration orbit comprises: controlling the robotic lawn mower to orbit along the periphery of the feature completely within the lawn area; recording the calibration time required to complete a full orbit around the feature; and setting the predetermined duration of the timer to be equal to or greater than the calibration time.

[0014] By automatically calibrating the timer so that the time length is equal to or greater than the calibration time required to bypass the obstacle or hazard, the timer can be automatically set to provide an effective method of distinguishing the boundary of the lawn area. In particular, if the non-lawn feature used for calibration is the largest of the non-lawn features that are completely contained within the lawn area, the time length of the timer can be set to be greater than the time required to bypass the largest feature to ensure that the timer expiration time indicates the boundary of the lawn area, while keeping the timer time low to maximize efficiency.

[0015] Preferably, the method further comprises: recording a detour time starting from a start time of controlling the robot lawn mower to detour the non-lawn feature around the perimeter of the non-lawn feature; dynamically comparing the detour time elapsed since the start time with a time threshold; and if the elapsed detour time is equal to or greater than the time threshold, distinguishing the non-lawn feature as a boundary of a lawn area.

[0016] This is an alternative to a timer that compares the time since the detour started to a time threshold in real time.

[0017] Preferably, the method includes dynamically comparing the detour time elapsed since the start time with a plurality of time thresholds, wherein each time threshold corresponds to the respective size of the non-lawn feature; and distinguishing the type of the non-lawn feature and the size of the non-lawn feature according to which of the plurality of time thresholds is exceeded in the process of detour the non-lawn feature.

[0018] By using multiple thresholds, not only can the types of non-lawn features be distinguished, but also the sizes of non-lawn features can be distinguished within a certain range according to the time threshold between which the robotic lawn mower completes the detour around the non-lawn features.

[0019] The time thresholds are preferably user configurable or adjustable. A user may set one or more time thresholds and adjust them as features of a lawn area are introduced, moved or modified.

[0020] The orbit feature preferably includes a direction of rotation, indicating whether the orbit is clockwise or counterclockwise.

[0021] The rotational direction of the orbit describes the overall turning direction of the robotic lawn mower when orbiting the non-lawn feature. For example, an inertial measurement unit (IMU) can be used to measure this direction, which is both quick and efficient. Another method of measuring this direction is to use visual simultaneous localization and mapping (V-SLAM). In the case of using a camera, a VSLAM algorithm or technology can be used to record the rotation angle based on the position change of a specific feature in a series of images taken by the camera, thereby recording the rotation direction of the camera.

[0022] The rotation direction is preferably the cumulative rotation direction during the entire process of circling the non-lawn feature. The rotation direction describes the overall rotation direction of the robotic lawn mower during the entire process of circling the non-lawn feature.

[0023] Preferably, the detour characteristic comprises a grass distribution direction, the grass distribution direction starting from the robotic lawn mower and pointing to a part of the environment detected by the robotic lawn mower that contains a lawn.

[0024] Thus, the direction of grass distribution can be determined by any sensor suitable for detecting one or more locations of grass and non-grass in the vicinity of the robotic lawn mower. Although the direction of grass distribution is preferred, the present invention can relate to any surface cover feature other than grass, such as other vegetation and plants. One purpose of the lawn cover direction is to help distinguish the type of non-lawn feature, which is discussed in further preferred features below. Therefore, it is not a requirement of the present disclosure to give priority to the general direction of lawn. Other surface cover features may also be used, including material or man-made features, such as concrete or roads.

[0025] Preferably, the distribution direction of the grass points to a part of the environment where the robotic lawn mower is located with the highest ratio of grass to non-grass.

[0026] Preferably, the method comprises periodically determining and recording, by the robotic lawn mower, a distribution direction of grasses, thereby recording a plurality of distribution directions of grasses.

[0027] Preferably, in the process of circling the non-lawn feature, the distribution direction of the grass is substantially to the left or to the right relative to the forward direction of the robotic lawn mower.

[0028] Preferably, distinguishing the type of non-lawn feature according to the detour feature includes: if the rotational movement indicates a clockwise detour of the non-lawn feature: if the distribution direction of the grass basically points to the right side of the robot mower relative to the forward movement of the robot mower, then distinguishing the type of the non-lawn feature as a boundary of a lawn area; if the distribution direction of the grass basically points to the left side of the robot mower relative to the forward movement of the robot mower, then distinguishing the type of the non-lawn feature as a feature completely contained in the lawn area; and if the rotational movement indicates a counterclockwise detour of the non-lawn feature, then distinguishing the type of the non-lawn feature as a boundary of a lawn area: if the distribution direction of the grass basically points to the left side of the robot mower relative to the forward movement of the robot mower, then distinguishing the type of the non-lawn feature as a boundary of a lawn area; if the distribution direction of the grass basically points to the right side of the robot mower relative to the forward movement of the robot mower, then distinguishing the type of the non-lawn feature as a feature completely contained in the lawn area.

[0029] Preferably, in the process of controlling the robot lawn mower to circle the perimeter of the non-lawn feature, the distribution direction of the grass is determined by: acquiring sensor data from a sensor of the robot lawn mower, wherein the sensor data represents information indicating the presence of grass in the environment where the robot lawn mower is located; processing the sensor data to determine the distribution direction of the grass, wherein the distribution direction of the grass indicates a direction from the robot lawn mower to a part of the environment where the robot lawn mower is located with the most grass as indicated by the sensor data.

[0030] Preferably, the sensor data is an image acquired from a camera of the robotic lawn mower; wherein processing the sensor data comprises: segmenting the image into lawn and non-lawn areas; and determining the grass distribution direction as a relative direction from the position of the robotic lawn mower in the image to the position of the segmented lawn area.

[0031] Thus, the method can use the robotic lawn mower, such as a robotic lawn mower including a camera and an inertial measurement unit (IMU), to detect features of the bypass, including the direction of rotation and the prevailing direction of the grass. Therefore, using this method, relatively simple and inexpensive hardware can be used to distinguish non-lawn features. In addition, there is no need to use an IMU to detect bypass features, as this can be achieved using VSLAM technology by monitoring, recording and tracking a series of images captured by the camera.

[0032] Preferably, the method further comprises: capturing a plurality of images by a camera of the robot lawn mower; segmenting each of the plurality of images into lawn and non-lawn areas; determining a grass distribution direction of each of the plurality of images to obtain a plurality of grass distribution directions; and determining an overall grass distribution direction from the plurality of grass distribution directions; wherein distinguishing the type of non-lawn features according to the detour features comprises using the overall grass distribution direction.

[0033] Using multiple grass distribution direction measurements to form an overall grass distribution direction can improve the reliability of the method and reduce the risk of falsely distinguishing non-grass features.

[0034] Preferably, the method further includes: generating a map from multiple images, including aggregating lawn and non-lawn areas in each of the multiple images onto the map to form one or more aggregated areas; determining a current position of the robotic lawn mower on the map; determining an area with the most lawn from the one or more aggregated areas; determining an exit direction from the current position of the robotic lawn mower to the area with the most lawn on the map; and, upon distinguishing that the non-lawn feature is a feature completely contained within the lawn area: controlling the robotic lawn mower to leave the periphery of the non-lawn feature along the exit direction.

[0035] Using the same data collected to distinguish between non-lawn feature types to determine exit direction is highly efficient because it allows both tasks to be accomplished without further use of sensors. Determining exit directions along the perimeter of non-lawn terrain based on areas with the most grass also improves grass mowing efficiency because after exiting the perimeter of non-lawn terrain, the grass is more likely to be mowed without interference from obstacles.

[0036] Preferably, the orbit feature further comprises a predetermined initial rotation direction, and upon detection of the non-lawn feature, the robotic lawn mower is configured to turn in that direction.

[0037] Preferably, distinguishing the type of non-lawn feature according to the characteristics of the circular motion includes: if the predetermined initial rotation direction is substantially consistent with the rotation direction, distinguishing the type of the non-lawn feature as a boundary of a lawn area; if the predetermined initial rotation direction is opposite to the rotation direction, distinguishing the type of the non-lawn feature as a feature completely contained in the lawn area.

[0038] Therefore, the type of non-turf feature can be distinguished from several different features of the bypass. If multiple features are used, the reliability of the distinction can be improved.

[0039] Preferably, the method further comprises, before controlling the robotic mower to go around the periphery of the non-lawn feature and substantially at the periphery of the non-lawn feature, controlling the robotic mower to rotate in a predetermined initial rotation direction so that the robotic mower faces a direction tangential to the periphery of the non-lawn feature; and controlling the robotic mower to go around the periphery of the non-lawn feature starting from the direction tangential to the periphery of the non-lawn feature.

[0040] Preferably, the method further comprises: controlling the robotic lawn mower to completely circumvent the non-lawn feature around the perimeter of the non-lawn feature; and detecting whether the robotic lawn mower has circumvented the entire area of ​​the non-lawn feature.

[0041] Preferably, detecting whether the robot mower has bypassed the entire non-lawn feature comprises recording an initial orientation of the robot mower, and then controlling the robot mower to bypass the non-lawn feature around the periphery of the non-lawn feature; in the process of bypassing the non-lawn feature, periodically recording an updated orientation of the robot mower; and comparing the updated orientation of the robot mower with the initial orientation of the robot mower until the comparison result indicates that the robot mower has bypassed the entire area of ​​the non-lawn feature.

[0042] The orientation can be detected by the same sensor as the rotation direction described above, such as an IMU or a camera using VSLAM technology. Therefore, it can be determined whether a complete circumvention of a non-turf feature is detected using data related to the rotation direction.

[0043] Preferably, the initial heading heading and the updated heading heading correspond to an initial yaw angle value and an updated yaw angle value of the robotic lawn mower, respectively.

[0044] In the best case, when the difference between the initial yaw angle value and the updated yaw angle value is at least three hundred and sixty degrees, the comparison result indicates that the robotic mower has bypassed the entire non-lawn feature.

[0045] Preferably, the method comprises determining an orbiting rotational direction by comparing an initial heading orientation of the robotic lawn mower and an updated heading orientation of the robotic lawn mower.

[0046] In a second aspect, the present disclosure provides a computer-readable medium having instructions stored thereon, which, when executed by a processor, can cause the processor to perform the method of the first aspect.

[0047] In a third aspect, the present disclosure provides a computer program, which, when executed by a processor, causes the processor to execute the method of the first aspect.

[0048] In a fourth aspect, the present disclosure provides a robotic lawn mower, comprising a storage module, one or more sensors, and a processor communicatively connected to the storage module and the one or more sensors, wherein the processor is configured to execute the method of the first aspect.

[0049] The one or more sensors include a camera for capturing images of the robotic mower's forward path.

[0050] The one or more sensors also include an inertial measurement unit for calculating yaw values.

[0051] In a fifth aspect, the present disclosure provides a system comprising: the robot lawn mower of the fifth aspect, and a charging station for charging the robot lawn mower.

[0052] Portions of the methods described herein may be performed by software in machine-readable form on a tangible storage medium, for example in the form of a computer program comprising computer program code adapted to perform all of the steps of any of the methods described herein when the program is run on a computer, the computer program being embodied on a computer-readable medium. Examples of tangible (or non-transitory) storage media include disks, thumb drives, memory cards, etc., but do not include propagating signals. The software may be adapted to be executed on a parallel processor or a serial processor so that the method steps may be performed in any suitable order or simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Embodiments of the present invention will be described with reference to the following drawings:

[0054] Figure 1 is a schematic diagram illustrating an example of a lawn area where a robotic lawn mower can work;

[0055] Figure 2 is a schematic diagram of a lawn area, in which a robotic lawn mower may operate according to an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of a robot lawn mower according to an embodiment of the present invention;

[0057] Figure 4 is a flow chart illustrating a method for boundary tracking and non-lawn feature identification according to one embodiment of the present invention;

[0058] Figure 5 is a flow chart illustrating a second method for boundary tracking and non-lawn feature identification according to one embodiment of the present invention;

[0059] Figure 6 is a flow chart illustrating a path planning method according to an embodiment of the present invention;

[0060] Figure 7 is a flow chart illustrating a further path planning method according to one embodiment of the present invention;

[0061] Figure 8 Photos of typical obstacles are shown, along with corresponding drawings showing detected grass and non-grass areas; and

[0062] Fig. 9 A photographic schematic of a typical obstacle is shown, along with a corresponding map drawn in accordance with the present invention showing detected lawn and non-lawn areas and a planned path to escape the obstacle.

[0063] Common reference numerals are used in the Figures to indicate similar features. Specific embodiments

[0064] The embodiments of the present invention described below relate to an apparatus and method for controlling a small autonomous vehicle, such as a robotic lawn mower.

[0065] Figure 2 A schematic overview of a robot mower 101 in a typical lawn area 103 is shown according to one embodiment of the present invention. The lawn area 103 includes lawn (grass) and may include obstacles / hazards 104 therein. The robot mower 101 is configured to cut lawn (grass) within the lawn area 103 without leaving the lawn area 103 and without hitting / entering any obstacles / hazards 104. When the robot mower 101 is inactive, it is configured to charge at a charging station 102. The lawn area 103 does not need to be delineated by guide lines or boundary lines, as the robot mower 101 is able to navigate in the lawn area 103 without these features. The lawn area 103 is defined by a boundary 103a. The boundary 103a refers to the actual boundary of the lawn.

[0066] Embodiments of the present invention include methods for detecting non-lawn features, distinguishing the non-lawn features to determine whether they are obstacles / hazards 104 or boundaries 103a of a lawn area 103, and controlling a robotic lawn mower to avoid these features. Figure 3 A schematic diagram of a robot lawn mower 101 according to the present invention is shown. The robot lawn mower 101 includes a body 1011, wheels 1012, a power source (not shown), a mowing implement (not shown), and a control module 1013. The robot lawn mower is preferably a differential drive robot, which allows the robot to have a zero turning radius. This allows the robot lawn mower to turn in any situation and helps to avoid the robot lawn mower 101 from getting stuck. The control module 1013 includes a computer, or a processor configured to perform calculations and execute software, and a memory. The control module 1013 includes or otherwise communicates with one or more sensors. The one or more sensors are configured to provide information about lawn mower parameters to the control module 1013, such as the absolute / relative position of the robot lawn mower 101; the absolute / relative position of obstacles and / or lawn area boundaries; the absolute / relative position of the charging station 102; the absolute / relative position of the lawn within the lawn area 103 that was recently cut or trimmed by the robot lawn mower 101; humidity; temperature; robot lawn mower kinematics; robot lawn mower power, etc. Such sensors may include inertial measurement units (IMUs), cameras, cadence sensors, speed sensors, accelerometers, GPS sensors, ultrasonic sensors, radars, infrared sensors, laser sensors, lidars, and the like. It is understood that the control module may include any of these types of sensors, and may also be communicatively coupled with any of these types of sensors. The control module 1013 of the robot mower 101 may be further configured to include an imaging sensor and / or communicate with an imaging sensor (such as a camera device). The imaging sensor is configured to capture images of the environment in which the robot mower is located, preferably on the path that the robot mower is about to move. It is not difficult to understand that the imaging sensor has parameters such as field of view and maximum frame rate. It is best to use a wide-angle camera so that the field of view includes at least a partial side view and a view of the front or upcoming path. For example, the imaging sensor may be a Sony IMX327 or IMX390 with a field of view of 130 degrees.

[0067] The imaging sensor is configured to routinely capture images corresponding to a front view of the robotic mower 101, thereby capturing a stream of images illustrating the environment in front of the robotic mower 101 or in the path of the robotic mower 101. It should be understood that although a front view is referred to herein, the wide angle field of view may include images such as a side view or a rear view of the robotic mower.

[0068] The control module 1013 is configured to process each captured image using a semantic segmentation algorithm, segmenting the captured image into regions or areas, and identifying these regions or areas as including lawn or not including lawn. Regions that do not include grass can be referred to as "non-lawn features" in which the robotic lawn mower 101 cannot enter and / or start the mowing tool. The semantic segmentation algorithm is applied to each captured image in the captured image stream by software stored on a local memory connected to the control module 1013. However, it should be understood that any one or more of the computer-implemented processes discussed herein can be remotely executed via a network, server, or similar device.

[0069] It is well known that semantic segmentation algorithms can be trained based on various techniques. Applying semantic segmentation algorithms on the captured images results in segmented images where the segmented images are segmented into different regions or areas. These regions are then classified as either lawn or non-lawn.

[0070] The robotic lawn mower can use imaging sensors and the above-mentioned image segmentation algorithm to identify obstacles and boundaries, and use them in conjunction with the boundary tracking process to provide the ability to distinguish or differentiate obstacles and boundaries. Therefore, in an embodiment of the present invention, the robotic lawn mower 101 can be used to distinguish between the boundary 103a of the lawn area and obstacles / hazards 104 that are completely within the boundary 103a. Obstacles / hazards 104 that are completely within the boundary 103a can be regarded as "islands" within the lawn area.

[0071] As described above, obstacles / hazards 104 or "islands" may be distinguished from the boundary 103a by a boundary tracking process. The boundary tracking process may form part of a more general path planning algorithm for the robotic lawn mower 101. The path planning algorithm describes how to instruct the robotic lawn mower 101 to move within the lawn area 103 to mow the lawn and avoid the boundary 103a and obstacles / hazards 104.

[0072] Figure 4 A first embodiment of a boundary tracking process, flowchart 400, is shown.

[0073] In a first step 401, the robotic lawn mower 101 is configured to cut the lawn while moving forward until a non-lawn feature, such as an obstacle / hazard 104 or a boundary 103a, is detected in the forward path of the robotic lawn mower 101. The detection of these features can be performed by any one or more sensors on the robotic lawn mower 101, and / or using the image processing discussed above. The method for determining that there are non-lawn features in the forward path of the robotic lawn mower 101 is to determine that the area in front of the robotic lawn mower 101 is "non-lawn" by using an image segmentation algorithm in the case of using an imaging sensor, or by other logic in the case of using other sensors for detection. This determination does not distinguish the type of non-lawn feature detected at this time. It should be understood that although the robotic lawn mower 101 is referred to as mowing in this example, this is not a necessary condition, and the robotic lawn mower 101 can also move without mowing, such as returning to the charging station 102.

[0074] In a second step 402, the robotic mower 101 is configured to continue moving toward the detected non-lawn feature until a minimum acceptable distance from the non-lawn feature is reached. The minimum acceptable distance is configurable and is set to prevent the robotic mower 101 from colliding with or otherwise encroaching or passing over the non-lawn feature. Once the robotic mower 101 reaches the minimum acceptable distance, the robotic mower 101 is configured to rotate in a set direction and then record the initial orientation of the robotic mower 101. The set direction may be right or left, for example, a direction relative to the non-lawn feature when the robotic mower 101 initially faces the non-lawn feature. If the set direction is left, then after reaching the minimum acceptable distance from the non-lawn feature, the robotic mower 101 is configured to turn left so that the robotic mower 101 is disposed adjacent to the non-lawn feature in a direction such that the forward direction of the robotic mower 101 is tangential to the non-lawn feature. The initial orientation may be recorded by any suitable sensor, such as an IMU or a compass. In the case of using an IMU, the IMU yaw is recorded after the robotic mower is turned in a set direction so that it is facing tangentially to the non-lawn features. The initial heading is recorded into memory and stored on the robotic mower 101. For example, the initial heading may be 60 degrees.

[0075] In a third step 403, the robotic mower 101 is configured to circumvent the perimeter of the non-lawn feature. To ensure that the robotic mower 101 travels along the perimeter of the non-lawn feature, the robotic mower 101 is configured to maintain a minimum acceptable distance from the non-lawn feature. To assist in this process, the robotic mower 101 is configured to use one or more sensors and / or imaging sensors discussed above. In particular, one or more sensors are used to measure the distance from the robotic mower 101 and / or the direction relative to the non-lawn feature. When an imaging sensor is used for this purpose, it can be ensured that the frame captured by the imaging sensor includes at least a portion of the non-lawn feature. In other words, it can be ensured that the non-lawn feature appears in the field of view of the imaging sensor. The imaging sensor can be fixed relative to the robotic mower 101. In this case, the field of view of the imaging sensor is fixed relative to the direction of the robotic mower 101. It will be self-evident that the real-world distance can be determined based on the distance between pixels in the image. In another example, as the robotic mower 101 moves along the perimeter of a non-lawn feature, an ultrasonic, infrared, or radar sensor is used to calculate the distance of the robotic mower 101 to the non-lawn feature. The non-lawn feature that appears in the frame captured by the imaging sensor, or the distance obtained from one or more other sensors, is then used to inform the control of the robotic mower to follow the perimeter of the non-lawn feature. In particular, controlling the robotic mower 101 to move along the perimeter of the non-lawn feature may include detecting a virtual perimeter in the image or sensor data that corresponds to the actual perimeter of the non-lawn feature, determining the direction of the virtual perimeter, and adjusting the heading of the robotic mower 101 accordingly. Other control methods may also be used, such as maintaining a set or predetermined distance during the detour and / or staying within a set or predetermined distance from the detected non-lawn feature.

[0076] In the fourth step 404, the accumulated rotation of the robot mower 101 while circling the non-lawn feature is recorded. The specific method is to compare or subtract the current orientation with the initial orientation stored in the memory. The robot mower 101 can periodically record the current orientation until the accumulated difference between the current orientation and the initial orientation is equal to or greater than 360 degrees. The direction of rotation is also recorded as part of the bearing record. The direction of rotation can be determined using any suitable sensor. For example, the direction of rotation is determined by an IMU sensor. For example, when starting to record at a 60-degree orientation, if you move along the perimeter of the non-lawn feature until the IMU yaw value reaches 420 degrees, it indicates a clockwise (or accumulated right) rotation. If the IMU yaw accumulated value reaches -300 degrees, it indicates the opposite counterclockwise (or accumulated left) rotation. During the process of circling the non-lawn feature, the IMU yaw data can be sampled and recorded regularly to determine the accumulated rotation direction of the circling in this way. Once it is detected that the accumulated rotation amplitude is 360 degrees (+360 degrees or −360 degrees), the method will proceed to the fifth step 405 .

[0077] In the fifth step 405, the direction of the detected feature's orbit is first determined based on the IMU data recorded in the fourth step 404. A negative rotation of -360 degrees indicates a counterclockwise (or cumulative left-hand) rotation. A positive rotation indicates a clockwise (or cumulative right-hand) rotation. This information is used together with the information indicating the initial set direction that the robotic mower 101 rotated in the second step 402.

[0078] If the set direction is a left turn, for example, the robotic lawn mower 101 is configured to turn left in the second step 402, the clockwise movement of the detected feature indicates that the detected feature is an obstacle / hazard 104 within and surrounded by the lawn area 103. In this case, the detected feature indicates an area within the lawn area 103 that cannot be mowed. Figure 2 As shown, this may be an obstacle / hazard 104. Conversely, when the set direction is left, the counterclockwise rotation indicates that the detected feature is the outer boundary 103a of the lawn area 103. In this case, the detected feature represents the edge of the lawn area 103, and the robotic lawn mower 101 cannot mow here.

[0079] If the direction set in the second step 402 is a right turn, then a clockwise detour indicates that the detected feature is the outer boundary 103a of the lawn area 103. A counterclockwise detour using a right turn indicates that the detected feature is an obstacle / hazard 104 within and surrounded by the lawn area 103.

[0080] In this way, the robotic lawn mower 101 can utilize the set direction knowledge and the accumulated rotation direction knowledge to distinguish the types of non-lawn features.

[0081] In an optional sixth step 406, once the type of non-lawn feature is determined to be an external boundary 103 or an internal hazard / obstacle 104, a path is planned to exit the perimeter of the non-lawn feature. The relative position / area / perimeter of the non-lawn feature and / or the type of the non-lawn feature (boundary 103a or obstacle 104) may be recorded in memory so that the robotic mower 101 can identify the non-lawn feature and take its position and size into account when planning future mowing paths in the lawn area 103. More details on the non-lawn feature perimeter exit will be provided later in the description.

[0082] It should be understood that although the flowchart 400 and the method described therein are explained with reference to the use of an IMU, the above-mentioned functions of the IMU can be performed by one or more other sensors or technologies. For example, a camera device can be used in conjunction with a VSLAM technology or algorithm. The camera can capture a series of images, and the VSLAM algorithm records the cumulative rotation and orbiting direction of the robot mower 101 based on the relative changes in the position and / or orientation of the features recorded and tracked through the series of images. It should be understood that the IMU functions mentioned elsewhere in this description can also be replaced by a camera that uses the same or similar VSLAM technology to determine the rotation, position and orientation of the robot mower 101.

[0083] Figure 5 A second implementation of the boundary tracking process is shown. The second implementation can be used as Figure 4 The second embodiment is similar to the first embodiment and includes several identical / similar steps. For completeness, these steps are listed below as part of the following method 500.

[0084] In a first step 501, the robotic lawn mower is configured to move forward until a non-lawn feature is detected. Figure 4 As with the first embodiment described above, the type of non-lawn feature is unknown at this stage and may be a boundary 103a or an obstacle / hazard 104 completely contained within the boundary 103a. The robotic mower may be mowing or may be moving without mowing. The non-lawn feature may be detected by an imaging sensor or any other sensor of the robotic mower. The robotic mower 101 is configured to continue moving toward the detected non-lawn feature until a minimum acceptable distance is reached from the non-lawn feature. The minimum acceptable distance is configurable and is set to prevent the robotic mower 101 from colliding with or otherwise encroaching or crossing over the non-lawn feature.

[0085] In the second step 502, the robotic mower is configured to move along the perimeter of the non-lawn feature, maintaining a minimum acceptable distance. As described above, in the first embodiment of the second step 402, this operation can be performed using an imaging sensor. In particular, after the imaging sensor detects the non-lawn feature in the first step 501, the robotic mower can be instructed to turn so as to keep at least a portion of the non-lawn feature within the field of view of the imaging sensor. It will be appreciated that standard image processing and computer vision algorithms can be used to detect and track the non-lawn feature in different frames and perspectives of the imaging sensor. For example, if the non-lawn feature is initially visible to the right of the imaging sensor's field of view, the robotic mower can be configured to gradually rotate along the perimeter so that the detected non-lawn feature remains on the right side of the imaging sensor's field of view. It will be appreciated that other sensors, such as sensors that can infer the distance between the robotic mower 101 and the non-lawn feature, can also be used to help track the perimeter of the non-lawn feature. It should be noted that, unlike the first embodiment of the method 400 described above, the second embodiment does not require the robotic mower 101 to initially turn in a set direction. In this embodiment, the robotic mower may turn to any initial orientation so as to face a tangent direction to the non-lawn feature to begin perimeter tracking.

[0086] Before traveling around the perimeter of the non-lawn feature, the robotic mower 101 is configured to record the initial heading, orientation, or angle of the robotic mower 101. This can be done using an IMU sensor, recording the IMU yaw at the beginning of the walk. For example, the initial recorded orientation can be 60 degrees. The initial heading is recorded into memory and stored on the robotic mower 101.

[0087] In the third step 503, the robotic lawn mower 101 is configured to continuously record the ratio of lawn to non-lawn on the left and right sides of the continuous stream of images recorded by the imaging sensor as the robotic lawn mower 101 circumnavigates the perimeter of the non-lawn feature. These ratios may be recorded in a reference database on a frame-by-frame basis. The ratio of lawn to non-lawn may be determined using similar computer vision techniques as described above, including image segmentation and classification. Specifically, each image captured by the imaging sensor may be segmented into lawn / non-lawn regions, and then the ratios of these regions may be recorded. Two ratios may be recorded for each frame of the image: a first ratio of lawn to non-lawn, corresponding to the left half of the captured image; and a second ratio of lawn to non-lawn, corresponding to the right half of the captured image. Each of these first and second ratios may have a value recorded in the reference database for each frame. For example, in frame 2, the first ratio of lawn to non-lawn may be 5, and the second ratio of lawn to non-lawn may be 0.5. These values ​​indicate that there is more grass in the left side of the frame than in the right side of the frame. Although one ratio or first and second ratios are described herein, it should be understood that any comparison mechanism or logic may be used to post-process the captured image to determine whether there is more grass on the left or right side of the captured frame. For example, the image post-processing technique may return a simple result indicating right or left (R or L), corresponding to the side with the most grass. The indicator R or L may then be stored in a reference database according to the corresponding frame number. Preferably, the ratio may be used to determine the direction of grass distribution, e.g., if the ratio indicates that grass is more on the right side of the image, then the direction of grass distribution is from the position of the robotic lawn mower to the right side of the image.

[0088] In a fourth step 504, the accumulated rotation angle of the robotic mower 101 is recorded as the robotic mower 101 travels around the perimeter of the non-lawn feature. The accumulated rotation angle may be described as a direction of rotation (e.g., clockwise or counterclockwise). As described in the second step 502, this may be detected by an IMU sensor that is used to record the initial orientation / bearing / angle of the robotic mower 101. The accumulated rotation is calculated by comparing or subtracting the current orientation from the initial orientation stored in memory. The robotic mower 101 may periodically record the current orientation until the accumulated difference between the current orientation and the initial orientation is equal to or greater than 360 degrees. The direction of rotation is also recorded as part of the orientation record. The direction of rotation may be determined using any suitable sensor. For example, the direction of rotation is determined by an IMU sensor. For example, when starting to record at a 60 degree orientation, if one moves along the perimeter of the non-lawn feature until the IMU yaw value accumulates to 420 degrees, this indicates a clockwise (or accumulated right) rotation. If the IMU yaw accumulated value reaches -300 degrees, it indicates an opposite counterclockwise (or accumulated left-hand) rotation. During the process of circling the non-turf feature, the IMU yaw data can be sampled and recorded regularly to determine the accumulated rotation direction of the circling. Once the accumulated rotation amplitude is detected to be 360 ​​degrees (+360 degrees or -360 degrees), the method will continue to the fifth step 505.

[0089] In the fifth step 505, the robot mower 101 is configured to distinguish the non-lawn feature as an outer boundary 103a of the lawn area 103 or an "isolated island" hazard / obstacle 104 contained in the lawn area 103. To this end, the robot mower 101 uses the accumulated rotation direction information in the fourth step 504, as well as information about the prevalence of lawn / non-lawn on the left or right side of the recording frame or the prevalence direction of lawn in the reference database. If the recorded ratio or L and R direction indicators (for example) stored in the reference database continue to show that there are more non-lawns on the side where the robot mower gradually turns to bypass the non-lawn feature according to the accumulated rotation direction, the non-lawn feature is determined to be an obstacle / hazard and an island. In other words, if the prevailing direction of the grass always points away from the rotation center indicated by the accumulated rotation direction, then the non-lawn feature is distinguished as an island obstacle / hazard. On the other hand, if the growth direction of the lawn always basically points to the rotation center, then the non-lawn feature is distinguished as a boundary 103a.

[0090] For example, if the robot mower gradually turns left (counterclockwise) around the periphery of the non-lawn feature based on the rotation direction during the detour indicated by the IMU yaw data, and the ratio or L and R indicators recorded in the reference database continue to show that the non-lawn area is more on the left side of the captured image, it indicates that the robot mower is navigating around the periphery of the island obstacle / hazard 104 in the lawn area 103.

[0091] When the recorded ratios consistently indicate that the non-lawn area is larger on the side opposite to the side where the robotic mower gradually turns around the non-lawn feature according to the cumulative rotation direction, it can be determined that the non-lawn feature is the outer boundary 103a of the lawn area 103.

[0092] In other words, when the turning direction during the detour corresponds to the direction in which non-lawn is most dense in the ratio recorded in the reference database, the non-lawn feature is identified as the island obstacle / hazard 104. Conversely, when the turning direction during the detour is inconsistent with the direction in which non-lawn is most common in the recorded ratio, the non-lawn feature is distinguished as the boundary 103a of the lawn area. For example, if the IMU yaw data shows that the cumulative angle change compared to the initial orientation is -360 degrees, this means that the robot mower is turning counterclockwise, indicating a cumulative left turn. If the recorded ratio also shows that non-lawn appears more on the left side of the frame, it means that the robot mower has been detour around the periphery of the island obstacle / hazard 104. Conversely, if the IMU yaw data shows that the cumulative angle change compared to the initial orientation is +360°, this means that the robot mower has turned in a clockwise direction, indicating a cumulative right turn. If the recorded ratio also shows that non-lawn appears more on the left side of the frame, it means that the robot mower has been detour around the inner periphery of the boundary 103a of the lawn area 103.

[0093] According to method 500, either the prevalence of non-lawn or the prevalence of lawn (ie, the presence of non-lawn or the prevalence of lawn) may be used. It will be readily appreciated that using lawn prevalence direction instead of non-lawn prevalence direction means that the reverse of the above logic is applied.

[0094] Furthermore, it should be understood that the rotational direction is only required to be recorded in the method 500 if the lawn to non-lawn ratio recorded in the third step 503 does not provide absolute direction data. In other words, the lawn:non-lawn ratio or grass distribution direction is only required when the initial orientation or direction is recorded in the second step 502 and the rotational direction is recorded in the fourth step 504 relative to the orientation of the robotic mower 101. For example, if the ratio records an L or R direction indicator, it is necessary to determine which direction is left and which direction is right relative to the non-lawn features based on the orientation of the robotic mower 101.

[0095] In some examples, recording the ratio of lawn to non-lawn includes recording the absolute orientation of lawn to non-lawn. In this case, the type of non-lawn feature can be distinguished by accumulating each recorded orientation and determining whether they point inward or outward relative to the path taken by the robotic mower along the perimeter of the non-lawn feature. If the orientation of the lawn is accumulated to indicate an inward direction, the non-lawn feature can be distinguished as a boundary 103a. If the orientation of the grass distribution on the lawn is accumulated to indicate an outward direction, the non-lawn feature can be identified as an island obstacle / hazard 104. For example, Fig. 9 A grass distribution direction 912 pointing outward relative to the path taken by the robotic mower 101 is shown.

[0096] Determining whether grass / non-grass is more prevalent in the frames captured on one side compared to the other side can be done in several different ways. For example, a "winner takes all" approach can be used, where the number of frames with more grass or non-grass on the left side is compared to the number of frames with more grass or non-grass on the right side. The greater number of the two determines the overall distribution rate. Alternatively, the recorded ratios can be averaged to determine an average ratio, and thus the distribution rate. It goes without saying that other techniques can also be used to determine the distribution rate, which can work by detecting and focusing on the distribution rate of non-grass areas rather than grass.

[0097] In an optional sixth step 506, once the type of non-lawn feature is determined to be an external boundary 103 or an internal hazard / obstacle 104, a path is planned to exit the perimeter of the non-lawn feature. The relative position / area / perimeter of the non-lawn feature and / or the type of the non-lawn feature (boundary 103a or obstacle 104) may be recorded to a memory so that the robotic mower 101 can identify the non-lawn feature and take its position and size into account when planning future mowing paths in the lawn area 103. More details on the non-lawn feature perimeter exit will be provided later in the description.

[0098] Thus, the first and second embodiments of the border tracking process are similar. Figure 5 The second embodiment does not require the robotic mower 101 to move in a set direction, but instead uses sensor data (e.g., images from an imaging sensor) to determine the relative positions of lawn and non-lawn relative to the accumulated rotation direction, thereby determining whether the detected non-lawn feature is a boundary 103a or an obstacle 104.

[0099] It is understood that the method 400 and the method 500 can be used in combination to provide three parameters (accumulated rotation direction, initial setting direction and ratio data) for distinguishing the type of non-turf features. Such three parameters bring redundancy to the method, making it more reliable.

[0100] As a further option, when tracking the perimeter of the non-lawn feature begins in method 400 or 500, the robotic mower 101 may start a timer. The timer is configured to time the boundary tracking process, including the formation of any mapping of lawn and non-lawn areas. If the timer exceeds a maximum threshold, the robotic mower 101 may perform an operation. The operation may include distinguishing the non-lawn feature according to the timer. For example, the timer may be two minutes, so that if the boundary tracking process exceeds two minutes, the robotic mower can distinguish that the non-lawn feature is the boundary 103a of the lawn area 103 when the two-minute timer expires. Since the robotic mower 101 does not need to go around the entire boundary 103a to distinguish, the timer running in this manner can improve the efficiency of the boundary tracking process. The timer may be a 30 second, 1 minute, 2 minute, or 5 minute timer. The timer may be set according to a predetermined speed of the robotic mower 101. The timer may also be set according to a predetermined maximum internal obstacle / hazard perimeter length, so that once the robotic mower travels to that length at a predetermined speed, the timer is configured to expire. The timer may be user configurable so that the user can set the timer based on the relative size of obstacles and hazards 104 in a particular lawn area 103. In this way, the timer will expire soon after the period of time required to circumvent the largest obstacle / hazard 104 in the lawn area.

[0101] The timer may also be automatically set during a calibration procedure automatically performed by the robotic mower 101. During the calibration procedure, the robotic mower 101 may be positioned by a user or the like to face or otherwise approach the largest obstacle / hazard 104 in the lawn area 103. The robotic mower 101 may then perform the below-the-boundary process 400 or 500. During this calibration process, the time required to completely bypass the largest obstacle / hazard 104 is recorded and stored so that the robotic mower 101 has a reference time for the largest obstacle / hazard 104 in the lawn area 103. The timer is timed based on the reference time. For example, the timer may be set equal to the sum of the reference time and a buffer time, where the buffer time is additional time to ensure that the largest obstacle / hazard 104 is bypassed before the timer expires. For example, the buffer time may be 5 seconds, 10 seconds, or 30 seconds.

[0102] As described above, once a map of lawn and non-lawn areas is generated, expiration of the timer may cause the robotic mower to exit the demarcation boundary 103a in the area with the most lawn.

[0103] Although timers and their expiration times are discussed herein, it is understood that the time required for the detour may also be recorded. The recorded elapsed time may be dynamically compared to one or more thresholds, where the thresholds indicate or otherwise distinguish between the category of the obstacle / hazard (e.g., small / medium / large) and / or the boundary 103a. If the threshold corresponding to the boundary 103a is passed within the recorded time, the robotic mower 101 may end the boundary tracking process 400 or 500 as described above to maximize the efficiency of performing non-lawn feature differentiation.

[0104] As a further option, at the beginning of each perimeter tracking operation performed in the above method, the control module 1013 can record the "initial position" of the robot mower 101 and store it in memory. The initial position can be the absolute position of the robot mower 101, or the position of the robot mower relative to a detected non-lawn feature. The position can be recorded by using a GPS sensor, etc., or by recording the relative position relative to other features in the robot mower's environment. For example, the robot mower 101 can communicate with the charging station 102 and store the distance and direction from the charging station 102 to the current position of the robot mower 101.

[0105] At the end of each method, the initial position can be used to verify whether the robotic mower 101 has circumnavigated the entire non-lawn feature, because the robotic mower 101 will arrive at the same position after completing a circle around the non-lawn feature. This means that the robotic mower 101 will move along the perimeter of the non-lawn feature until it automatically moves around the entire non-lawn feature once, including a cumulative rotation of three hundred and sixty degrees, and then returns to the recorded initial position.

[0106] At this point, the robotic lawn mower 101 is configured to identify or match its current position with the recorded initial detection position to verify that the non-lawn feature has been bypassed. The matching of the current position with the initial position can be performed using one or more sensors for recording the initial position and storing it in a memory. This can be done using a position-related sensor such as a GPS sensor.

[0107] Where an imaging sensor is used for this purpose, image processing may be performed at predetermined time intervals to compare images captured by the imaging sensor at interval positions with images taken from the initial position. At each time interval, each image is compared to the initial image until a match or high similarity between the images is determined. Verification of the return to the initial position may also require that the robotic mower 101 be in the same orientation or position as recorded at the initial position. This may be determined, for example, using a compass or IMU.

[0108] Once one or more boundaries 103a and / or obstacles 104 are determined.

[0109] Once a non-lawn feature is identified by method 400 or method 500, the type of the non-lawn feature may be recorded in memory and its location may be stored. The location of the non-lawn feature may be stored relative to a reference point (e.g., charging station 102) or may be stored absolutely based on a positioning sensor (e.g., a GPS sensor). The robotic mower may plan a path to cover and cut the entire lawn area 103 based on the locations of the detected non-lawn features. If more obstacles / hazards are detected within the lawn area 103, the path may be modified. The path may be modified by repeating method 400 or 500 for newly detected obstacles / hazards in the lawn area 103, and then determining a new path based on the detected obstacles / hazards.

[0110] Now refer to Figures 7 to 9 The relationship between the above boundary tracking process and the overall function and path planning of the robotic lawn mower 101 is described.

[0111] The boundary tracking process is part of a more general path planning algorithm for directing the robotic lawn mower 101 how to move and / or mow. The path planning algorithm may be implemented by software or computer programs that are stored in the robot's memory and executed by a processor of the robotic lawn mower 101. The memory and processor may form part of the control module 1013.

[0112] The path planning algorithm includes instructions or logic for the robot mower 101 to mow the lawn area 103 and return to the charging station 102. These behaviors represent two operating modes of the robot mower 101: the first mode instructs the robot mower 101 to actively mow the lawn area 103; the second mode instructs the robot mower 101 to return, stop mowing, and return to the charging station 102. Whether the robot mower 101 is in the first mode or the second mode may depend on parameters such as battery life and / or time of day. For example, when the battery charge of the robot mower 101 is below a threshold, or when the time of day reaches a threshold, the control module 1013 may place it in the second mode, thereby instructing the robot mower 101 to stop mowing and return to the charging station 102.

[0113] Figure 6 A flow chart of a method 600 corresponding to the second mode of operation is shown. In the second mode, the path planning algorithm instructs the robotic mower 101 to return to the charging station 102.

[0114] In the first step 601, the robot mower 101 receives an instruction to return to the charging station 102. The instruction is executed by the control module 1013. When executing the instruction, the robot mower 101 will stop mowing and go to the boundary 103a of the lawn area 103. The location of the boundary 103a may not be stored or known to the robot mower 101, so the direction of travel may be random. In this regard, since the robot mower 101 does not need to store location data related to the boundary 103a or the charging point 102, a large storage module or GPS / location data is not required.

[0115] In the second step 602, the robotic lawn mower 101 detects non-lawn features on the path of the robotic lawn mower 101 through one or more sensors such as imaging sensors. After the non-lawn features are detected, the above-mentioned reference Figure 4 and Figure 5 This can be performed by the control module 1013. This will cause the robotic lawn mower 101 to perform the above method, such as Figure 4 and Figure 5 shown.

[0116] In the third step 603, the result of the boundary tracking process is determined, that is, whether the non-lawn feature is an "island" obstacle or hazard 104 or a boundary 103a of the lawn area 103. If the non-lawn feature is determined to be an "island" obstacle / hazard 104, the method proceeds to the fourth step 604. If the non-lawn feature is determined to be a boundary 103a of the lawn area 103, the method proceeds to the fifth step 605.

[0117] In a fourth step 604, the robotic mower 101 is instructed and configured to exit the boundary following process, thereby exiting the perimeter of the island obstacle / hazard 104. The robotic mower 101 may exit in a random direction, or the robotic mower 101 may store the initial orientation or direction when detecting features on the lawn in the second step 602, and continue to proceed in the initial orientation or direction when exiting the perimeter of the non-lawn feature in the fourth step 604. To ensure that the initial orientation or direction is traversable by the robotic mower 101, the robotic mower is configured to rotate in the initial orientation and observe the upcoming (forward) environment using one or more sensors (e.g., imaging sensors) to ensure that the path in the initial orientation or direction is free of non-lawn features. If the orientation or direction is not clear, the robotic mower 101 will continue to perform the boundary following process until the orientation or direction is clear. Therefore, the robotic mower 101 may periodically check whether the orientation or direction is clear.

[0118] Once the robotic mower 101 escapes the perimeter of the non-lawn feature, the method will return to the second step 602 when the robotic mower 101 detects another non-lawn feature.

[0119] In a fifth step 605, after determining that the non-lawn feature is the boundary 103a of the lawn area 103, the robotic mower 101 is instructed and configured to continue along the perimeter of the boundary 103a of the lawn area 103 to reach the charging station 102. The robotic mower 101 can detect the presence of the charging station 102, and vice versa, using any suitable technology or tool. For example, the robotic mower 101 can include a charging station detection sensor, such as a radio frequency identification (RFID) tag, and / or similarly, the charging station 102 can include an RFID tag. The robotic mower 101 and / or the charging station 102 may also include a sensor configured to detect contact between the robotic mower 101 and the charging station 102.

[0120] Figure 7 A flow chart of a method 700 corresponding to a first mode of operation is shown. In the first mode, the path planning algorithm instructs the robotic mower 101 to mow the lawn area 103 on a path.

[0121] In the first step 701, the robot lawn mower 101 is instructed to mow along a path formed or set by a path planning algorithm implemented in software in the control module 1013. The path planning algorithm may include a general path planning algorithm, such as an algorithm that randomly selects a mowing path for the robot lawn mower 101. However, the path planning algorithm further includes the following steps: Figure 4 and Figure 5 The boundary tracing process of either or both graphs in .

[0122] In a second step 702, one or more sensors of the robotic mower 101 detect non-lawn features in the path of the robotic mower and initiate a boundary following process. At this point, according to the first or second embodiment, the robotic mower is configured to perform the above-mentioned boundary following process.

[0123] In a third step 703 , the result of the boundary tracking process is determined, ie, whether the non-lawn feature is distinguished as an “island” obstacle or hazard 104 , or a boundary 103 a of the lawn area 103 .

[0124] In the fourth step 704, once the type of non-lawn feature is distinguished as an outer boundary 103a or an "island" obstacle 104, the robotic mower 101 is configured to leave the perimeter of the non-lawn feature. The robotic mower 101 can leave the perimeter at any suitable location. However, in the first mode of operation, the robotic mower 101 leaves the perimeter in a tangential direction of "most grass", which is more efficient in mowing. This means that the direction in which the robotic mower 101 leaves the perimeter is the direction in which the most "grass" is detected during the boundary tracking process of the perimeter area of ​​the non-lawn feature.

[0125] To determine this direction, the robotic mower 101 can detect and determine lawn and non-lawn areas within the field of view of the imaging sensor as the robotic mower 101 circumnavigates the non-lawn feature during the implementation of the boundary tracking process discussed above. This occurs in the third step 503 of method 500. Because the imaging sensor provides a forward view, each frame captured by the imaging sensor effectively captures a view of the non-lawn feature environment that is tangent to the perimeter of the non-lawn feature. The robotic mower 101 can record the grass and non-grass areas in each frame captured while traveling around the perimeter of the non-lawn feature, and combine these areas in the frame stream to create a map around the non-lawn feature. The map shows the surrounding environment of the non-lawn feature. It will be appreciated that the detection of lawn and non-lawn areas in these surrounding environments can be performed using standard image segmentation and classification techniques discussed above.

[0126] After distinguishing the type of non-lawn terrain, the robotic mower will be configured to determine the area with the highest lawn ratio in the stored map, or the area with the highest lawn to non-lawn ratio. The robotic mower 101 will then leave the perimeter of the non-lawn feature in the direction of the determined area. If the area is blocked by non-lawn features, the robotic mower 101 can be instructed to continue tracking the perimeter of the non-lawn feature according to the boundary tracking process until the robotic mower can reach the area in a straight line. Alternatively, the robotic mower can be configured to exit from the perimeter of the non-lawn object to the area with the most lawn that can be directly reached in a straight line from the current position of the robotic mower 101. By directing the robotic mower 101 to the area with the most grass, the efficiency of the mower is improved because it is less likely to be frequently disturbed by more non-lawn features and more grass can be cut in a shorter time.

[0127] Figure 8 An example photograph 800 of an obstacle 804 and a corresponding map 802 of lawn and non-lawn areas are shown.

[0128] Fig. 9 A second example photo 900 of an obstacle 904 is shown, along with a corresponding map of lawn and non-lawn areas 902. A schematic diagram of the robotic lawn mower 101 is superimposed on the map 902. Arrows 906 indicate the direction from the robotic lawn mower 101 to the area on the map with the most lawn or the highest ratio of lawn to non-lawn.

[0129] Fig. 9 Example directions used in methods 400 and 500 are also described. In particular, Fig. 9 An initial rotation direction 908, an orbital rotation direction 910, and several recorded grass distribution directions 912 are shown. It should be understood that the grass distribution direction 912 may be a non-grass distribution direction, in which case Fig. 9 The direction of the middle arrow 912 will be opposite.

[0130] exist Fig. 9 In the example of , the initial rotation direction 908 is opposite to the rotation direction of the orbit 910. The initial rotation direction 908 is clockwise, while the rotation direction of the orbit 910 is counterclockwise. According to the method 400, this means that the non-turf feature 904 is classified as an obstacle / hazard.

[0131] The pre-grass direction 912 always radiates outward from the rotation center of the rotation direction 910. According to the method 500, this indicates the presence of an obstacle / hazard. If the distribution direction of the grass is recorded as a grass / non-grass ratio or L and R indicators, in Fig. 9 In the example above, the direction of the grass distribution will always be represented by a high ratio or consistent R direction indicator on the right side of the image.

[0132] Thus, the methods and robotic mower 101 described herein provide both a method for distinguishing types of non-lawn features and a method for leaving or escaping from non-lawn features in order to maximize mowing. Distinguishing types of non-lawn features and recording the results helps provide the robotic mower 101 with information about its environment that can be used for path planning and charging station return operations. One benefit of these methods is that each method can be performed without complex sensors. In particular, each of the above methods can be performed with imaging sensors and / or IMUs.

[0133] In the described embodiments of the present invention, the control module 1013 may be implemented as any form of computing and / or electronic device. It should be understood that, although it is mentioned above that the control module 1013 of the robotic lawn mower 101 is responsible for the methods described herein, these methods may be executed or calculated through interaction with an external server, computer or satellite that communicates with the robotic lawn mower 101.

[0134] The robotic lawn mower includes a computing and / or electronic device that may include one or more processors, which may be microprocessors, controllers, or any other suitable type of processor, for processing computer executable instructions to control the operation of the device to collect and record routing information. In some examples, such as when a system-on-chip architecture is used, the processor may include one or more fixed function modules (also known as accelerators) for implementing portions of the method in hardware (rather than software or firmware). Platform software consisting of an operating system or any other suitable platform software may be provided on the computing-based device to execute application software on the device.

[0135] The method may be stored as computer executable instructions and may be provided by any computer readable medium accessible by a computing-based device. Computer readable media may include computer storage media (such as memory) and communication media, etc. Computer storage media (such as memory) include volatile and non-volatile, removable and non-removable media, which are implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computer device. In contrast, communication media can embody computer readable instructions, data structures, program modules or other data in a modulated data signal (such as a carrier wave) or other transmission mechanism. As defined herein, computer storage media do not include communication media.

[0136] As used herein, the term "computer" refers to any device with processing capabilities that can execute instructions. Those skilled in the art will recognize that many different devices have such processing capabilities, so the term "computer" includes personal computers, servers, mobile phones, personal digital assistants, and many other devices.

[0137] Those skilled in the art will appreciate that storage devices for storing program instructions can be distributed throughout a network. For example, a remote computer can store program examples as software descriptions. A local computer or terminal computer can access the remote computer and download some or all of the software to run the program. Alternatively, the local computer can download pieces of software as needed, or execute certain software instructions at the local terminal and certain software instructions at the remote computer (or computer network). Those skilled in the art will also appreciate that all or part of the software instructions can be executed by dedicated circuits, such as digital signal processors, programmable logic arrays, etc., by utilizing conventional techniques known to those skilled in the art.

[0138] It goes without saying that the benefits and advantages described above may be related to one embodiment or multiple embodiments. These embodiments are not limited to those that can solve any or all of the problems described, nor are they limited to those that have any or all of the benefits and advantages described.

[0139] Any reference to "an" item refers to one or more of the items. As used herein, the term "comprising" means including the identified method steps or elements, but these steps or elements do not include an exclusive list and the method or apparatus may also include other steps or elements.

[0140] The order of steps of the methods described herein is exemplary, but the steps may be performed in any suitable order, or simultaneously where appropriate. In addition, steps may be added or replaced, or individual steps may be deleted from any method without departing from the scope of the subject matter described herein. Aspects of any of the above embodiments may be combined with aspects of any of the other embodiments described to form further embodiments without losing the effects sought.

[0141] It is to be understood that the above description of the preferred embodiment is only illustrative, and various modifications may be made thereto by those skilled in the art. Although various embodiments have been described to a certain extent above, or one or more individual embodiments have been cited, those skilled in the art may make numerous modifications to the disclosed embodiments without departing from the spirit or scope of the invention.

Claims

1. A method for controlling a robotic lawn mower, the method comprising: detecting non-lawn features in a path of travel of the robotic lawn mower; controlling the robotic lawn mower to at least partially circumvent the non-lawn feature along a perimeter of the non-lawn feature; recording one or more characteristics of a detour around the non-turf feature; According to the detour characteristic, it is distinguished whether the non-lawn feature is a boundary of a lawn area or a feature completely contained in the lawn area. 2 . The method of claim 1 , wherein the orbit characteristic includes a rotation direction indicating whether the orbit is clockwise or counterclockwise.

3. The method of claim 2, wherein the rotation direction is a cumulative rotation direction during a circumnavigation of the non-turf feature. 4 . The method of claim 3 , wherein the detour characteristic comprises a grass distribution direction from the robotic lawn mower to a detected portion of the environment containing grass. 5 . The method according to claim 4 , wherein when the robotic lawn mower goes around the non-lawn feature, the grass distribution direction is substantially a left or right direction relative to a forward direction of the robotic lawn mower.

6. The method according to claim 5, wherein distinguishing the type of the non-turf feature according to the detour characteristics comprises: If the rotational motion is indicated as clockwise around the non-turf feature: If the grass distribution direction is substantially directed to the right of the robotic lawn mower relative to the forward direction of the robotic lawn mower, distinguishing the type of the non-lawn feature as a boundary of a lawn area; and If the grass distribution direction is substantially directed to the left of the robotic lawn mower relative to the forward direction of the robotic lawn mower, distinguishing the type of the non-lawn feature as a feature that is completely contained within a lawn area; as well as If the rotational motion is indicated as counterclockwise around the non-turf feature: If the grass distribution direction is substantially directed to the left of the robotic lawn mower relative to the forward direction of the robotic lawn mower, distinguishing the type of the non-lawn feature as a boundary of a lawn area; and If the grass distribution direction is substantially directed to the right of the robotic lawn mower relative to the forward direction of the robotic lawn mower, the type of the non-lawn feature is distinguished as a feature that is completely contained in a lawn area.

7. The method according to any one of claims 6, wherein the grass distribution direction is determined by the following steps: In controlling the robotic lawn mower to navigate around the non-lawn feature along the perimeter of the non-lawn feature: acquiring sensor data from a sensor of the robotic lawn mower, wherein the sensor data represents relevant information indicative of the presence of grass in an environment of the robotic lawn mower; The sensor data is processed to determine the grass distribution direction, wherein the grass distribution direction represents a direction from the robotic lawn mower to a portion of an environment indicated by the sensor data as having the most grass present.

8. The method of any one of claims 3, wherein the characteristics of the orbit further include a predetermined initial rotation direction, and the robotic mower is configured to turn in this direction after detecting the non-lawn feature.

9. The method of claim 8, wherein distinguishing the type of non-turf feature based on the characteristics of the detour comprises: If the predetermined initial rotation direction is substantially consistent with the rotation direction, distinguishing the type of non-lawn feature as a boundary of the lawn area; as well as If the predetermined initial rotation direction is opposite to the rotation direction, the type of non-turf feature is distinguished as the feature that is completely contained in the turf area.

10. The method of claim 9 further comprising: Before controlling the robotic mower to go around the perimeter of the non-lawn feature, at the perimeter of the non-lawn feature, controlling the robotic mower to rotate toward the predetermined initial rotation direction so as to face the tangent direction of the perimeter of the non-lawn feature; And, starting from a tangential direction of the perimeter of the non-lawn feature, controlling the robotic lawn mower to go around the non-lawn feature along the perimeter of the non-lawn feature.

11. The method of claim 1 further comprising: controlling the robotic lawn mower to completely circumvent the non-lawn feature around the perimeter of the non-lawn feature; as well as Detecting whether the robotic lawn mower has completely bypassed the non-lawn feature.

12. The method of claim 11, wherein detecting whether the robotic lawn mower has bypassed the entire non-lawn feature comprises: recording an initial orientation of the robotic lawn mower prior to controlling the robotic lawn mower to navigate the non-lawn feature along a perimeter of the non-lawn feature; periodically recording an updated heading of the robotic lawn mower during circumnavigation of the non-lawn feature; The updated heading of the robotic mower is compared to the initial heading until the comparison indicates that the robotic mower has completely circumnavigated the non-lawn feature.

13. The method of claim 12, wherein the initial orientation and the updated orientation correspond to an initial yaw angle value and an updated yaw angle value of the robotic lawn mower, respectively.

14. The method according to claim 13, wherein: When the difference between the initial yaw angle value and the updated yaw angle value is at least three hundred and sixty degrees, the comparison indicates that the robotic mower has completely circumvented the non-lawn feature.

15. The method of claim 14 further comprising: By comparing the initial orientation of the robotic lawn mower and the updated orientation of the robotic lawn mower, a direction of rotation for orbiting is determined.

16. A robotic lawn mower comprising a memory module, one or more sensors, and a processor communicatively coupled to the memory module and the one or more sensors, wherein the processor is configured to perform the method of claim 1.

17. The robotic lawn mower of claim 16, wherein the one or more sensors include a camera configured to capture images of a path forward of the robotic lawn mower.

18. The robotic lawn mower of claim 17, further comprising an inertial measurement unit for calculating a yaw angle value.

19. A system including The robotic lawn mower of claim 16; and A charging station for charging the robotic lawn mower.

Images