Cutting control methods, devices, and lawn mowing robots

Abstract

A cutting control method, apparatus, and lawnmower robot are disclosed. The method for controlling the lawnmower robot to perform cutting operations includes: acquiring work status commands and a global map of the lawnmower robot's operation; acquiring forward-looking binocular image data, side-looking monocular image data, and first time-of-flight distance data based on the work status commands; generating a forward-looking depth map and a forward-looking semantic segmentation map based on the forward-looking binocular image data, and fusing them to determine a first grid map; generating a side-looking semantic segmentation map based on the side-looking monocular image data; determining an initial side-looking depth map based on the side-looking semantic segmentation map; correcting the estimated distance information using the first time-of-flight distance data to determine a second grid map; generating a local map based on the first grid map, the second grid map, and the lawnmower robot's positioning information; and controlling the cutter head to perform extension and retraction movements in the local map and the global map based on the type of work status commands. This application can improve the safety and efficiency of lawnmower operations.

Description

Technical Field

[0001] This disclosure relates to the field of lawn mowing robot control technology, and more particularly to a cutting control method, device, and lawn mowing robot. Background Technology

[0002] Lawn-mowing robots are widely used in the routine maintenance of home yards and public green spaces, their core function being to efficiently and accurately mow lawns in pre-defined areas. Current technologies typically rely on map modeling and pre-set paths for mowing. However, for areas such as house walls, flower beds, or randomly appearing paths, fences, or potholes, lawn-mowing robots often leave uncleaned lawns due to a lack of intelligent mowing strategies. Furthermore, during the operation of lawn-mowing robots, there are moving obstacles such as children or pets, necessitating a balance of safety and efficiency. Therefore, the efficiency and safety of lawn-mowing robot operations deserve attention. Summary of the Invention

[0003] In view of this, the present disclosure provides a cutting control method, apparatus, and lawnmower robot to improve cutting efficiency while ensuring the safe operation of the lawnmower robot. A first aspect provides a cutting control method for controlling a lawnmower robot to perform cutting. The lawnmower robot is equipped with a blade disc, a forward-looking sensor, and a side-looking sensor. The cutting control method includes: acquiring a work condition command and a global map of the lawnmower robot's operation; the work condition command's indication type includes: edge cutting operation or bow-shaped cutting operation; controlling the forward-looking sensor to acquire forward-looking binocular image data based on the work condition command, and controlling the side-looking sensor to acquire side-looking monocular image data and first flight time distance data; generating a forward-looking depth map and a forward-looking semantic segmentation map based on the forward-looking binocular image data, wherein the forward-looking depth map is used to determine first distance information of obstacles in the first field of view of the forward-looking sensor, and the forward-looking semantic segmentation map is used to determine the type information of obstacles in the first field of view; and comparing the forward-looking depth map with the forward-looking... Semantic segmentation map fusion is performed to determine a first grid map. A side-view semantic segmentation map is generated based on side-view monocular image data, whereby the side-view semantic segmentation map is used to determine the type information of obstacles in the second field of view of the side-view sensor. An initial side-view depth map is determined based on the side-view semantic segmentation map, which is used to determine the estimated distance information of obstacles in the second field of view. The estimated distance information is corrected using first time-of-flight distance data to determine a second grid map, whereby the second grid map includes the second distance information of obstacles in the second field of view. Based on the first grid map, the second grid map, and the positioning information of the lawnmower robot, the first and second grid maps are merged and mapped to the world coordinate system to generate a local map. Based on the type of work condition command, the cutter head is controlled to perform extension and retraction actions in both the local and global maps.

[0004] The above cutting control method utilizes the combined sensing capabilities of both forward-looking and side-looking sensors. By ensuring sufficient perception of the environment in front of the mowing robot and the area adjacent to the cutter head, the multi-source sensing results are uniformly mapped into a local map. Combined with the global map and work condition commands, the extension and retraction of the cutter head are controlled, thereby achieving precise control of the mowing robot's cutting process. This ensures the safety of the mowing robot's cutting work while improving mowing efficiency and reducing blind spots.

[0005] Optionally, based on the forward-looking binocular image data, a forward-looking depth map and a forward-looking semantic segmentation map are generated, including: performing stereo matching processing on the forward-looking binocular image data to generate a forward-looking depth map; selecting a reference image from the forward-looking binocular image data, and generating a forward-looking semantic segmentation map based on a preset semantic segmentation algorithm.

[0006] Optionally, generating a side-view semantic segmentation map based on side-view monocular image data includes: performing image segmentation on the side-view monocular image data based on a semantic segmentation algorithm to determine the side-view semantic segmentation map; determining an initial side-view depth map based on the side-view semantic segmentation map includes: performing distance estimation on the side-view semantic segmentation map based on a preset ground hypothesis algorithm to determine estimated distance information; and generating an initial side-view depth map based on the estimated distance information and the side-view semantic segmentation map.

[0007] Optionally, the estimated distance information is corrected using the first time-of-flight distance data to determine the second grid pattern, including: determining the second time-of-flight distance data based on the first time-of-flight distance data and a preset confidence threshold; and correcting the estimated distance information of obstacles in the second field of view based on the second time-of-flight distance data to determine the second distance information of the obstacles.

[0008] Optionally, the type information includes: dynamic obstacles, static obstacles, and random boundary obstacles.

[0009] Optionally, based on the type of work condition command and the local map, the cutter head is controlled to perform extension and retraction actions, including: when the work condition command instructs the mowing robot to perform edge cutting operation, marking dynamic obstacles in the local map; determining whether there are dynamic obstacles within a preset distance range of the mowing robot in the local map; when there are dynamic obstacles, controlling the cutter head to retract and cut; when there are no dynamic obstacles, controlling the cutter head to extend and cut.

[0010] Optionally, based on the type of work condition command and the local map, the cutter head is controlled to perform extension and retraction actions, including: when the work condition command instructs the mowing robot to perform a bow-shaped cutting operation, marking random boundary obstacles in the global map; determining whether there are random boundary obstacles within a preset distance range of the mowing robot in the global map; when there are no random boundary obstacles, controlling the cutter head to perform a regular cutting operation; when there are dynamic obstacles, performing an edge-cutting operation, including: determining whether there are dynamic obstacles within a preset distance range of the mowing robot in the local map; when there are dynamic obstacles, controlling the cutter head to retract for cutting; when there are no dynamic obstacles, controlling the cutter head to extend for cutting.

[0011] Optionally, when there are dynamic obstacles, performing the edge cutting operation further includes: determining whether the bow-shaped cutting operation is completed; when the bow-shaped cutting operation is completed, controlling the mowing robot to move to the location area marked with random boundary obstacles in the global map; in the location area, determining whether there are dynamic obstacles within a preset distance range of the mowing robot based on the local map, so as to control the blade to retract or extend for cutting.

[0012] Secondly, a cutting control device is provided for controlling a lawnmower robot to perform cutting control. The lawnmower robot is equipped with a blade disc, a forward-looking sensor, and a side-looking sensor. The cutting control device includes: an acquisition unit for acquiring work condition commands and a global map of the lawnmower robot's operation; the instruction types of the work condition commands include: edge cutting operation or bow-shaped cutting operation; a data acquisition unit for controlling the forward-looking sensor to acquire forward-looking binocular image data based on the work condition commands, and controlling the side-looking sensor to acquire side-looking monocular image data and first flight time-distance data; and a first image processing unit for generating a forward-looking depth map and a forward-looking semantic segmentation map based on the forward-looking binocular image data, wherein the forward-looking depth map is used to determine the first distance information of obstacles in the first field of view of the forward-looking sensor, and the forward-looking semantic segmentation map is used to determine the type information of obstacles in the first field of view; and is also used to combine the forward-looking depth map with the forward-looking semantic segmentation map. The system comprises: a segmentation map fusion unit to determine a first grid map; a second image processing unit to generate a side-view semantic segmentation map based on side-view monocular image data, wherein the side-view semantic segmentation map is used to determine the type information of obstacles in the second field of view of the side-view sensor; a second image processing unit to determine an initial side-view depth map based on the side-view semantic segmentation map, wherein the initial side-view depth map is used to determine the estimated distance information of obstacles in the second field of view; and a second grid map to determine the second grid map by correcting the estimated distance information using first time-of-flight distance data, wherein the second grid map includes the second distance information of obstacles in the second field of view; a mapping unit to merge the first grid map and the second grid map and map them to the world coordinate system based on the positioning information of the lawnmower robot, thereby generating a local map; and a cutting control unit to control the cutter head to perform extension and retraction actions in the local map and the global map based on the type of work condition command.

[0013] Thirdly, a lawn mowing robot is provided, comprising: a blade disc, a front-view sensor, and a side-view sensor; and a processor connected to the front-view sensor and the side-view sensor, configured to control the blade disc to perform cutting based on the cutting control method provided in the first aspect by acquiring the front-view sensor and the side-view sensor. Attached Figure Description

[0014] The accompanying drawings used in the description of the embodiments of this disclosure are briefly introduced below: Figure 1 The following is a simplified structural diagram of a lawnmower robot provided in some embodiments of this application; Figure 2 A flowchart illustrating a cutting control method provided in some embodiments of this application is shown; Figure 3 A flowchart illustrating a cutting control method based on edge cutting commands provided in some embodiments of this application is shown. Figure 4A flowchart illustrating a cutting control method based on bow-shaped cutting commands provided in some embodiments of this application is shown. Figure 5 A schematic diagram of the structure of a cutting control device provided in some embodiments of this application is shown. Detailed Implementation

[0015] To more clearly illustrate the technical solutions in the embodiments of this disclosure, examples of implementation methods of this disclosure will be described below with reference to the accompanying drawings. The accompanying drawings described below are merely some embodiments of this disclosure. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without creative effort. Adjustments and improvements made without departing from the concept of this disclosure are all within the protection scope of this disclosure.

[0016] To keep the drawings simple, each figure only schematically shows the parts relevant to the embodiment, and they do not represent the actual structure of the product. In addition, for the sake of clarity and ease of understanding, some figures only schematically show parts of components with the same structure or function, and there may actually be more or fewer components with the same structure or function.

[0017] In this disclosure, unless otherwise expressly specified and limited, ordinal numbers, such as “first”, “second”, etc., are used only to distinguish and describe related objects, and should not be construed as indicating or implying the relative importance or order between related objects; furthermore, they do not represent the quantity of related objects. “Multiple” includes two or more, and other quantifiers are similar. “ / ” is used to describe the relationship between related objects, indicating an “or” relationship between them. “And / or” is used to describe the relationship between related objects, including any combination relationship between them, such as “a and / or b” including: “a alone”, “b alone”, or “a and b”. “One or more” or “at least one” of multiple objects refers to any object or any combination of multiple objects, such as “one or more of a1, a2, a3” or “at least one of a1, a2, a3” including: “a1 alone”, “a2 alone”, “a3 alone”, “a1 and a2”, “a1 and a3”, “a2 and a3”, or “a1, a2 and a3”.

[0018] As a core component of smart home and garden maintenance, the performance of lawn mowing robots is primarily reflected in their cutting efficiency, accuracy, and operational safety. Currently, lawn mowing robots typically cut grass based on pre-established global maps. However, when cutting along edges, they only follow fixed mapped boundaries, lacking intelligent blade extension and high-precision ranging capabilities. This makes it difficult to minimize grass residue along edges without touching fixed boundaries. For example, during bow-shaped cutting, existing technology often ignores unmapped, random boundary scenarios, such as randomly distributed stone paths, fences, and tree root pits in a yard. Because these random boundaries cannot be identified and marked in real time, the robot cannot execute special edge-cutting strategies, resulting in uncleaned lawn areas around random obstacles. Furthermore, current cutting strategies lack fine-grained differentiation of obstacle types. For instance, in pursuit of cutting efficiency, if the blade is designed too close to the boundary, it may accidentally injure suddenly appearing living obstacles (such as children or pets). Conversely, adopting a conservative strategy to ensure safety inevitably leads to incomplete grass residue and cleaning. Therefore, existing technologies, lacking multi-sensor fusion recognition and intelligent blade control logic, struggle to simultaneously achieve cutting effect, operational efficiency, and safety, severely limiting the user experience of lawnmower robots. In view of this, this application proposes a cutting control method, device, and lawnmower robot. By fusing perception using forward-looking and side-looking sensors mounted on the lawnmower robot, the type and distance of obstacles are accurately identified, thereby achieving intelligent control of the lawnmower robot's blade. This ensures the safety of the lawnmower robot's cutting operation while improving mowing efficiency and reducing blind spots.

[0019] The following description is in conjunction with the accompanying drawings: Please refer to Figure 1 This document illustrates a simplified structural diagram of a lawnmower robot provided in some embodiments of this application. The lawnmower robot 10 is equipped with a blade disc 11, a forward-looking sensor 12, and a side-looking sensor 13. The blade disc 11 is used for mowing lawns, and it can extend or retract relative to the body of the lawnmower robot 10. The forward-looking sensor 12 is used to collect environmental image data in front of the lawnmower robot. In this embodiment, for example... Figure 1 As shown, the forward-looking sensor may include a forward-looking binocular color camera arranged at the front of the robot body for acquiring forward-looking binocular image data. The side-looking sensor 13 may be located on the side of the lawnmower robot 10 for collecting environmental information in the side area. In this embodiment, the side-looking sensor 13 may include a side-looking color camera and a time-of-flight ranging module for acquiring side-looking monocular image data and first time-of-flight distance data. For example... Figure 1 The diagram also illustrates the first field of view of the forward-looking sensor 12, namely... Figure 1The dashed area extending from the forward-looking sensor 12 is shown in the figure; and the second field of view of the side-looking sensor 13 is shown, namely the dashed area extending from the side-looking sensor 13. In addition, the distance measurement diagram of the time-of-flight ranging module installed on the side-looking sensor 13 is also shown, namely the dotted line in the figure. Figure 2 A flowchart illustrating a cutting control method provided in some embodiments of this application is shown. This cutting control method is used to control a lawnmower robot 10 to perform cutting operations, and includes at least the following steps: S210: Obtain the work status instructions and the global map of the lawnmower robot's work. The instruction types for the work status instructions include: edge cutting operation or bow-shaped cutting operation. S220: Based on the operating condition command, control the forward-looking sensor to collect forward-looking binocular image data, and control the side-looking sensor to collect side-looking monocular image data and first flight time distance data; S231: Based on the forward-looking binocular image data, generate a forward-looking depth map and a forward-looking semantic segmentation map, wherein the forward-looking depth map is used to determine the first distance information of obstacles in the first field of view of the forward-looking sensor, and the forward-looking semantic segmentation map is used to determine the type information of obstacles in the first field of view. S232: Merge the forward depth map with the forward semantic segmentation map to determine the first grid map; S241: Generate a side-view semantic segmentation map based on the side-view monocular image data, wherein the side-view semantic segmentation map is used to determine the type information of obstacles in the second field of view of the side-view sensor; S242: Determine the initial side-view depth map based on the side-view semantic segmentation map. The initial side-view depth map is used to determine the estimated distance information of obstacles in the second field of view. S243: Correct the estimated distance information using the first flight time distance data to determine the second grid pattern, wherein the second grid pattern includes the second distance information of obstacles in the second field of view; S250: Based on the first grid map, the second grid map, and the positioning information of the lawnmower robot, the first grid map and the second grid map are merged and mapped to the world coordinate system to generate a local map; S260: Based on the type of work condition command, control the cutterhead to perform extension and retraction actions in the local and global maps.

[0020] In the embodiments of the above cutting control method, the current working condition command of the lawn mower robot can be obtained first. The instruction type of the working condition command includes edge cutting operation or bow-shaped cutting operation. Among them, the edge cutting operation allows the lawn mower robot 10 to move along the edge of the lawn or the edge of the obstacle and perform cutting operations near the edge. The bow-shaped cutting operation is used to perform area-covering cutting of the lawn area. For example, the lawn mower robot can move in a similar "bow" shape or reciprocating parallel lines in the lawn area according to a preset path planning strategy, so that the blade 11 can perform uniform coverage cutting. At the same time, the lawn mower robot 10 can also pre-acquire a global map of the current work. This global map can represent part of the boundary contour of the lawn area, the distribution of obstacles, and the areas where the work has been completed, which is used to provide a global reference for subsequent local map fusion and path planning. Further, based on the working condition command, the acquisition function of the forward-looking sensor 12 and the side-looking sensor 13 can be activated, and the forward-looking sensor 12 can be controlled to acquire forward-looking binocular image data to reflect the environmental information in the first field of view in front of the lawn mower robot. Simultaneously, the side-view sensor 13 can be controlled to acquire side-view monocular image data and first flight time distance data to reflect environmental information within the second field of view of the lawnmower robot. When processing the front-view binocular image data, a front-view depth map is determined to identify the first distance information of obstacles at each location in the first field of view. This front-view depth map can be calculated based on the imaging geometry and disparity information of the front-view sensor, thereby estimating the corresponding spatial distance for each pixel on the image plane. Furthermore, when determining the front-view semantic segmentation map, a semantic segmentation model can be applied to a reference image in the front-view binocular image data. For example, the image can be divided into multiple categories such as ground, lawn, buildings, trees, and pedestrians. Based on this division, a front-view semantic segmentation map representing obstacle type information is obtained. Further, the front-view depth map and the front-view semantic segmentation map can be fused to form a first grid map. For each grid cell in the map, combining the aforementioned distance information and semantic category information, it can be determined whether an obstacle exists in the corresponding area of ​​the grid, and the type of the obstacle and the first distance information between it and the lawnmower robot are recorded. On the other hand, this application can also perform semantic segmentation processing on the side-view monocular image data to obtain a side-view semantic segmentation map for determining the category information of each pixel in the second field of view, so as to identify the ground, lawn, boundary objects and other obstacles in the vicinity of the lawnmower robot. An initial side-view depth map is then determined based on the side-view semantic segmentation map to reflect the estimated distance information of obstacles within the second field of view. Specifically, based on the relationship between different categories of objects and the ground, camera height and viewing angle, the relative distance between each pixel in the side-view semantic segmentation map and the lawnmower robot can be roughly estimated, thereby generating an initial side-view depth map; however, the distance estimation result has a certain degree of error.To eliminate this error, correction can be performed using the first time-of-flight distance data. The first time-of-flight distance data collected by the side-view sensor 13 is compared with the estimated distance information in the initial side-view depth map. For areas where the time-of-flight distance data is reliable, the distance measured during flight is preferentially used to correct the estimated distance, thereby obtaining more accurate second distance information. Based on this, this application can fuse the corrected distance information with the side-view semantic segmentation map to construct a second grid map representing the obstacle types in the vicinity of the blade extension side and the second distance information between the obstacle and the mowing robot.

[0021] When fusing the first and second grid maps, this application can transform the first and second grid maps to a unified world coordinate system or global coordinate system based on the current positioning information of the lawnmower robot, and then merge them to generate a local map covering the area surrounding the lawnmower robot. The positioning information can be provided by the lawnmower robot's built-in positioning module, representing the lawnmower robot's position and attitude in the global map. The first and second grid maps are time-synchronized, selecting those generated simultaneously or approximately simultaneously within a preset time window as the fusion objects. Then, the current positioning information of the lawnmower robot is called, which may include coordinate transformation relationships from the front-view camera coordinate system, the side-view camera coordinate system to the world coordinate system. Further, based on the coordinate transformation relationships, the position information of each grid unit in the first grid map in the front-view camera coordinate system is converted to position information in the world coordinate system, and the position information of each grid unit in the second grid map in the side-view camera coordinate system is also converted to position information in the world coordinate system. After completing the coordinate transformation, the controller defines a preset range in the world coordinate system, centered on the current position of the lawnmower robot. The transformed first and second grid maps are then superimposed and projected onto this preset range to generate a local map covering the area surrounding the lawnmower robot. By uniformly mapping and fusing grid information from the forward and lateral environments, the local map comprehensively reflects the distribution of obstacles in the vicinity of the lawnmower robot and their relative relationships, providing a local environmental basis for subsequent control of the cutter head extension and path planning.

[0022] After generating the local map, the lawnmower robot can query environmental information related to its current location in both the local and global maps, based on the current work status command type, and control the blade head 11 to perform extension and retraction actions accordingly. For example, when the work status command indicates an edge-cutting operation, the obstacle type information reflected in the local map and the distance relationship between the obstacle and the lawnmower robot can be used to determine whether the blade head 11 should be controlled to extend or retract, and edge-cutting operations should be performed. When the work status command indicates a bow-shaped cutting operation, the environmental information recorded in the global and local maps, combined with a preset lawn mowing coverage strategy, can be used to control the blade head to switch between extended and retracted states. For example, the blade head 11 can be controlled to perform normal lawn mowing operations in the global map, while in the local map, it can be controlled to extend or retract to perform edge cutting, thereby completing the coverage cutting of the grass area while ensuring safety. Through the above steps, this application can utilize the combined sensing capabilities of forward-looking and side-looking sensors to ensure sufficient perception of the environment in front of the mowing robot and the area adjacent to the cutter head. The multi-source sensing results are then uniformly mapped into a local map, and combined with the global map and work condition commands, the extension and retraction of the cutter head are controlled. This enables precise control of the mowing robot's cutting process, ensuring the safety of the mowing robot's cutting work while improving mowing efficiency and reducing blind spots.

[0023] In some embodiments of this application, step S231 generates a forward-looking depth map and a forward-looking semantic segmentation map based on the forward-looking binocular image data, including: performing stereo matching processing on the forward-looking binocular image data to generate a forward-looking depth map; selecting a reference image from the forward-looking binocular image data, and generating a forward-looking semantic segmentation map based on a preset semantic segmentation algorithm.

[0024] The front-view sensor 12 of the lawnmower robot 10 can employ a binocular imaging structure to simultaneously acquire a pair of front-view binocular image data with parallax, such as a left-eye RGB image and a right-eye RGB image. Then, camera calibration and distortion correction are performed on the front-view binocular image data to align the left and right images to a unified imaging plane. A stereo matching algorithm, such as an algorithm based on parallax search and cost aggregation, is used to calculate the parallax value of each pixel. Based on a preset binocular geometric model, the parallax values ​​are converted into depth information, thereby generating a front-view depth map. This ensures that the spatial distance corresponding to each pixel in the first field of view of the front-view sensor 12 is encoded in the front-view depth map. Through this front-view depth map, the first distance information of the ground, lawn, walls, trees, and other obstacles relative to the lawnmower robot 10 in the first field of view can be obtained. This application can select one image from the front-view binocular image data as a reference image, such as the left-eye RGB image, and input this reference image into a preset semantic segmentation algorithm for processing. Semantic segmentation algorithms, based on deep learning image segmentation networks, can divide a baseline image into multiple semantic category regions, such as: lawn regions, hard boundary regions (e.g., curbs, fences, walls), vegetation regions (e.g., shrubs, tree trunks), dynamic obstacle regions (e.g., pedestrians, pets, other moving machinery), and background regions such as the sky, thereby determining the forward-looking semantic segmentation map. In the forward-looking semantic segmentation map, each pixel can carry a category label to represent its type information in the first field of view. The forward-looking depth map provides spatial distance information for each obstacle within the first field of view, while the forward-looking semantic segmentation map provides type information for each obstacle within that field of view. Together, these two technologies enable the lawnmower robot to simultaneously perceive the distance of obstacles from the robot and the type of obstacle in the forward direction.

[0025] In some embodiments of this application, step S241 generates a side-view semantic segmentation map based on side-view monocular image data, including: performing image segmentation on the side-view monocular image data based on a semantic segmentation algorithm to determine the side-view semantic segmentation map; step S242 determines an initial side-view depth map based on the side-view semantic segmentation map, including: performing distance estimation on the side-view semantic segmentation map based on a preset ground hypothesis algorithm to determine estimated distance information; and generating an initial side-view depth map based on the estimated distance information and the side-view semantic segmentation map.

[0026] In the above embodiments, side-view monocular image data can be input into a preset semantic segmentation algorithm to classify the image pixel by pixel, obtaining a side-view semantic segmentation map. In the side-view semantic segmentation map, each pixel corresponds to a semantic label, used to characterize the environmental type within the second field of view, such as: lawn areas, ground areas, hard boundaries (such as curbs, fences), static obstacles such as flower beds / low walls, and dynamic obstacles such as people and pets. Through the side-view semantic segmentation map, obstacles directly related to cutting safety can be identified on the extended side of the cutter head 11. Subsequently, when the controller determines the initial side-view depth map based on the side-view semantic segmentation map, a preset ground assumption algorithm can be invoked to treat the ground and lawn areas within the second field of view as planes at known heights. Combining the installation height, pitch angle, and imaging geometry model of the side-view sensor 13, the distance between these pixels and the lawnmower robot 10 is estimated based on the vertical position and category of different pixels in the image, obtaining estimated distance information. For pixels labeled as hard boundaries, flower beds, or other static obstacles, distances can be estimated based on their intersection with the ground or edge contours. For dynamic obstacles such as people and pets, distances can be estimated near their feet, taking into account ground assumptions. After obtaining the estimated distance information, it is fused with the side-view semantic segmentation map to assign corresponding distance values ​​to various obstacles within the second field of view, thereby generating an initial side-view depth map. In this initial side-view depth map, each pixel unit can contain both an obstacle category label and estimated distance information based on ground assumptions.

[0027] In some embodiments of this application, step S243 uses the first time-of-flight distance data to correct the estimated distance information and determine the second grid pattern, including: determining the second time-of-flight distance data based on the first time-of-flight distance data and a preset confidence threshold; correcting the estimated distance information of obstacles in the second field of view based on the second time-of-flight distance data to determine the second distance information of the obstacles.

[0028] The side-looking sensor 13 acquires first time-of-flight distance data corresponding to the second field of view through a time-of-flight ranging unit. Since time-of-flight measurements may contain noise or outliers at different spatial locations and with different reflective materials, this application can filter the first time-of-flight distance data based on a preset confidence threshold. For example, a confidence level is calculated for each time-of-flight ranging point based on indicators such as return signal strength, measurement stability, and consistency of historical data, and ranging points with confidence levels below the preset confidence threshold are marked as unreliable. After removing unreliable ranging points, the remaining time-of-flight ranging points that meet the confidence threshold requirements are used as second time-of-flight distance data for subsequent correction. Then, based on the second time-of-flight distance data, the estimated distance information of obstacles in the second field of view is corrected. For example, the second time-of-flight distance data can be compared with the estimated distance information of the corresponding spatial location in the initial side-looking depth map, and the second time-of-flight distance data can be preferentially used to replace the initial estimated distance information, thereby obtaining more accurate second distance information. After completing the above corrections, the category information of each pixel in the side-view semantic segmentation map can be fused with the corrected second distance information. The second distance information corresponding to the obstacle or ground area is recorded in each pixel, thereby generating a second grid map. This implementation utilizes the high precision of time-of-flight ranging to specifically correct the estimated distance information obtained based on the monocular ground assumption. The determined second grid map helps the lawnmower robot more accurately determine the safe distance between the cutter head 11 and obstacles during edge cutting and obstacle avoidance, thereby improving the operational safety and edge cutting integrity of the lawnmower robot.

[0029] In some embodiments of this application, the type information includes: dynamic obstacles, static obstacles, and random boundary obstacles. Targets with location-time-varying characteristics in the environment, such as pedestrians, pets, and other mobile devices, can be classified as dynamic obstacles, representing targets that may undergo relative motion during the lawnmower robot's operation. Fixed structures and garden elements, such as walls, fences, curbs, flower beds, and tree trunks, are classified as static obstacles, representing fixed obstacles or hard boundaries within the lawn area. Boundary objects formed by the outer edge of the lawn, irregular terrain, or irregular boundaries, and used to define the bow-shaped cutting operation range in the global map, are classified as random boundary obstacles. In the first grid diagram and the second grid diagram, each grid cell records distance information along with the aforementioned type information. Subsequently, when the lawnmower robot performs edge-cutting operations, it can focus on dynamic obstacles and safely control the extension and retraction of the cutter head. When performing a bow-shaped cutting operation, random boundary obstacles can be used to mark the boundary area globally. The robot then determines whether to switch to edge-cutting mode for additional edge trimming based on the positions of both static and random boundary obstacles. By classifying obstacle types as described above, the lawnmower robot can distinguish between obstacles of different natures, enabling targeted control of the blade extension / retraction strategy and the work path. This ensures safe obstacle avoidance while improving the integrity of the cut and the efficiency of the operation.

[0030] In some embodiments of this application, step S260 controls the cutter head to perform a retracting action based on the type of work condition command and the local map, including: when the work condition command instructs the lawnmower robot to perform an edge cutting operation, marking dynamic obstacles in the local map; determining whether there are dynamic obstacles within a preset distance range of the lawnmower robot in the local map; when there are dynamic obstacles, controlling the cutter head to retract for cutting; when there are no dynamic obstacles, controlling the cutter head to extend for cutting.

[0031] When the work status command instructs the lawnmower robot to perform edge-cutting operations, the robot can traverse the local map containing type information based on the combined first and second grid maps, marking areas where the type information is dynamic obstacles. This marking operation creates a region on the local map representing the distribution of dynamic obstacles, used to subsequently determine if there are any targets to avoid near the cutter head. Then, based on the lawnmower robot's current positioning information, a preset distance range centered on the robot's current position is determined in the local map. For example, this preset distance range can correspond to the working coverage area after the cutter head extends and the lawnmower robot's movement range in a short period, characterizing the adjacent areas that the cutter head may involve cutting at the current moment. Within this preset distance range, the controller checks whether there are any areas marked as dynamic obstacles. When at least one dynamic obstacle is detected within a preset distance range, the lawnmower robot determines that the current environment poses a potential collision risk or a target that needs to be avoided. In this case, it outputs a retraction control command to the cutter head 11, causing the cutter head to retract from its extended state to a position within the robot body or a safe position near the body. In the retracted state, the cutter head passes through the corresponding area with the smallest possible cutting width, reducing the probability of contact between the cutter head and the dynamic obstacle. When no dynamic obstacle is detected within the preset distance range, the current environment is determined to be free of moving targets that need to be avoided in the local area. In this case, it outputs an extension control command to the cutter head 11, causing it to extend from its retracted state to a preset working position. In the extended state, the cutter head performs cutting operations along the boundary area with a normal cutting width, ensuring that the lawn near the edge is fully mowed. The above embodiment utilizes the marking results of dynamic obstacles in a local map, combined with a preset distance range around the lawnmower robot, to make real-time judgments about the presence of dynamic obstacles near the cutter head. Based on the judgment result, the robot switches between two working states: extension and retraction of the blade head. When the lawn mowing robot is working near the edge area, it can ensure that the edge lawn is cut sufficiently while giving priority to avoiding dynamic obstacles such as pedestrians and pets. This allows for precise control of the blade head extension and retraction, which is beneficial to both ensuring the integrity of the cut and the safety of the operation process.

[0032] In some embodiments of this application, the cutter head is controlled to perform retraction and extension actions based on the type of work condition command and a local map, including: when the work condition command instructs the lawnmower robot to perform a bow-shaped cutting operation, marking random boundary obstacles in the global map; determining whether random boundary obstacles exist within a preset distance range of the lawnmower robot in the global map; when no random boundary obstacles exist, controlling the cutter head to perform a regular cutting operation; when dynamic obstacles exist, performing an edge-cutting operation, including: determining whether dynamic obstacles exist within a preset distance range of the lawnmower robot in the local map; when dynamic obstacles exist, controlling the cutter head to retract for cutting; when no dynamic obstacles exist, controlling the cutter head to extend for cutting.

[0033] When the work status command instructs the lawnmower robot to perform a bow-shaped cutting operation, random boundary obstacles can be marked on the global map based on preset area division rules and environmental mapping results. Random boundary obstacles can include, for example, irregular edges formed between the lawn and roads or flower beds, as well as irregular boundary lines formed due to terrain changes and the distribution of local obstacles. By marking random boundary obstacles on the global map, it is possible to clearly identify which areas belong to the lawn operation boundary and which areas should be the focus of subsequent edge cutting within the overall work area of ​​the bow-shaped cutting operation. Subsequently, the controller determines whether random boundary obstacles exist within the preset distance range based on the lawnmower robot's current position and the preset distance range. When no random boundary obstacles exist within the preset distance range, the current environment is judged as a regular area within the lawn. At this time, the controller outputs a regular cutting control command to the cutter head actuator, causing the cutter head to maintain or enter a regular cutting working state, performing a comprehensive cutting of the lawn according to the preset bow-shaped path plan (e.g., reciprocating parallel lines or partial return paths). When random boundary obstacles exist within the preset distance range, the controller determines that the mowing robot has approached or entered an irregular boundary area. At this time, the controller triggers an edge-cutting operation, applying the aforementioned edge-cutting control logic to the current area. For example, when dynamic obstacles exist, the controller retracts the cutter head 11 to cut, reducing the risk of contact with the dynamic obstacles. When no dynamic obstacles exist, the controller extends the cutter head 11 to cut, making the cutter head cut close to the area adjacent to the random boundary obstacle to fill any uncut grass strips that may be left in the edge area by the conventional bow-shaped cutting path.

[0034] In some embodiments of this application, when dynamic obstacles are present, performing the edge cutting operation further includes: determining whether the bow-shaped cutting operation is completed; when the bow-shaped cutting operation is completed, controlling the mowing robot to move to a location area marked with random boundary obstacles on the global map; in the location area, determining whether there are dynamic obstacles within a preset distance range of the mowing robot based on the local map, so as to control the blade to retract or extend for cutting.

[0035] This application allows for real-time monitoring of the bow-shaped cutting operation's progress while it performs the cutting along a bow-shaped path. This monitoring can be achieved by checking for markings of covered areas on a global map, the path's mileage, or completion markers for preset path segments. If the bow-shaped cutting is not yet complete, the operation continues along the planned path. Once complete, the operation switches from primarily cutting within the grass area to primarily trimming the edges. The robot is then moved to an area marked with random boundary obstacles on the global map. Based on the local map, the robot checks for dynamic obstacles within a preset distance. If a dynamic obstacle is detected within this distance, the cutter head 11 retracts to cut; if no obstacle is detected, it extends to cut, completing additional trimming of remaining grass near irregular boundaries. This allows the robot to selectively trim irregular boundary areas after completing large-area, efficiency-first bow-shaped cutting.

[0036] Figure 3 The diagram illustrates a flowchart of a cutting control method based on edge-cutting commands provided in some embodiments of this application. It includes: S310: Start the lawnmower robot; S320: Build a global map; S330: Obtain edge cutting command; S341: Acquire forward-looking sensor data; S342: Acquire side-view sensor data; S3511: Based on the left eye RGB image, perform semantic segmentation to generate a segmentation map containing dynamic and static obstacles; S3512: Based on the binocular RGB image, perform stereo matching to generate the left-eye depth map; S3521: Based on the left-eye RGB image, perform obstacle segmentation and generate a coarse depth map based on the monocular ground assumption scheme; S3522: Based on single-point ToF, filtering is performed according to the confidence level of the point, and high-confidence distance points are retained; S361: Fuse the left eye depth map with the segmentation map containing dynamic and static obstacles to generate the first grid map; S362: Use single-point Time-of-F (ToF) to perform pose correction on living obstacles in the coarse depth map and generate a second grid map; S370: Synchronize the first grid map and the second grid map in time, and convert them into a local map in the world coordinate system according to the coordinate transformation provided by the positioning. Dynamic obstacles are marked in the local map. S380: Determine local conditions Figure 1 Within a range of meters, is there a dynamic obstacle? If not, proceed to step S391; if so, proceed to step S392. S391: Extend the blade disc for cutting; S392: Shrink cutter head cutting.

[0037] The specific implementation process and beneficial effects of the above-mentioned methods can be referred to the above embodiments, and will not be repeated here.

[0038] Figure 4 The diagram illustrates a flowchart of a cutting control method based on a bow-shaped cutting command, provided in some embodiments of this application. It includes: S410: Start the lawnmower robot; S420: Build a global map; S430: Obtain the bow-shaped cutting command; S441: Acquire forward-looking sensor data; S442: Acquire side-view sensor data; S4511: Based on the left eye RGB image, perform semantic segmentation to generate a segmentation map containing dynamic and static obstacles; S4512: Based on the binocular RGB image, perform stereo matching to generate a left-eye depth map; S4521: Based on the left-eye RGB image, perform obstacle segmentation and generate a coarse depth map based on the monocular ground assumption scheme; S4522: Based on single-point ToF, filtering is performed according to the confidence level of the point, and high-confidence distance points are retained; S461: Fuse the left eye depth map with the segmentation map containing dynamic and static obstacles to generate the first grid map; S462: Use single-point Time-of-F (ToF) to perform pose correction on living obstacles in the coarse depth map and generate a second grid map; S470: Synchronize the first grid map and the second grid map in time, and convert them into a global map in the world coordinate system according to the coordinate transformation provided by the positioning. Mark the boundary obstacles in the global map. S480: Determine if there are boundary obstacles on the global map; if not, proceed to step S481; if they exist, proceed to step S482. S481: Perform standard strategy cutting; S482: Determine if there are boundary obstacles on the global map; if not, proceed to step S491; if they exist, proceed to step S492. S491: Open the boundary edge, use ToF real-time position correction, and extend the cutter head for cutting; S492: Enable edge cutting, utilize ToF real-time position correction, and shrink the cutter head for cutting.

[0039] The specific implementation process and beneficial effects of the above-mentioned methods can be referred to the above embodiments, and will not be repeated here.

[0040] In some preferred embodiments of this application, when the work condition command instructs the lawnmower robot to perform edge-cutting operation, the lawnmower robot can default to enabling the edge-cutting function. The forward-looking sensor 12 may include a forward-looking binocular RGB camera, and the side-looking sensor 13 may include a side-looking RGB camera and a single-point time-of-flight sensor. Before acquiring image data, distortion correction, stereo correction, and correction mapping processing can be performed on the two RGB images corresponding to the forward-looking binocular image data to achieve row alignment in the image coordinates, providing a prerequisite for subsequent binocular-based depth estimation. When determining the forward-looking depth map, a multi-scale feature extraction network such as MixVarGENet+FPN can be used to extract features from the forward-looking binocular image data, and the extracted features can be input into a stereo depth estimation network similar to StereoNet to obtain the forward-looking depth map. Simultaneously, the forward-looking semantic segmentation map can also be determined by semantically segmenting the reference image in the forward-looking binocular image data using a lightweight semantic segmentation model such as STDC2. The side-looking monocular image data can also be semantically segmented using the STDC2 model to generate a side-looking semantic segmentation map. Considering the limited NPU resources on the board, a monocular depth estimation of the side-view semantic segmentation map can be performed based on the ground assumption method to form an initial side-view depth map.

[0041] Figure 5The diagram illustrates a structural schematic of a cutting control device provided in some embodiments of this application. The cutting control device 500 includes: an acquisition unit 510, used to acquire work condition commands and a global map of the lawnmower robot's operation; the type of the work condition commands includes edge cutting operation or bow-shaped cutting operation; a collection unit 520, used to control the forward-looking sensor to collect forward-looking binocular image data based on the work condition commands, and to control the side-looking sensor to collect side-looking monocular image data and first flight time distance data; a first image processing unit 530, used to generate a forward-looking depth map and a forward-looking semantic segmentation map based on the forward-looking binocular image data, wherein the forward-looking depth map is used to determine the first distance information of obstacles in the first field of view of the forward-looking sensor, and the forward-looking semantic segmentation map is used to determine the type information of obstacles in the first field of view; it is also used to fuse the forward-looking depth map and the forward-looking semantic segmentation map to determine a first grid map; and a second image processing unit 540. The system is used to generate a side-view semantic segmentation map based on side-view monocular image data, wherein the side-view semantic segmentation map is used to determine the type information of obstacles in the second field of view of the side-view sensor; it is also used to determine an initial side-view depth map based on the side-view semantic segmentation map, wherein the initial side-view depth map is used to determine the estimated distance information of obstacles in the second field of view; and it is used to correct the estimated distance information using first time-of-flight distance data to determine a second grid map, wherein the second grid map includes the second distance information of obstacles in the second field of view; a mapping unit 550 is used to merge the first grid map and the second grid map and map them to the world coordinate system based on the positioning information of the lawnmower robot to generate a local map; and a cutting control unit 560 is used to control the cutter head to perform extension and retraction actions in the local map and the global map based on the type of work condition command.

[0042] The above division of units is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these units can be implemented by a processor calling software; for example, a cutting control device includes a processor coupled to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above cutting control methods or to realize the functions of each unit. The processor can be, for example, a general-purpose processor, such as a CPU, and the memory can be memory within a cross-platform acquisition device or memory outside of a cross-platform acquisition device. Alternatively, these units can be implemented as hardware circuits. The functions of some or all units can be realized through the design of the hardware circuit, which can be understood as one or more processors. For example, the hardware circuit includes an application-specific integrated circuit (ASIC), which implements the functions of some or all units by designing the logical relationships between the components within the circuit. Furthermore, the hardware circuit can be implemented using a programmable logic device (PLD), which can include a large number of logic gates. The logical relationships between the logic gates are configured through a configuration file, thereby realizing the functions of some or all units.

[0043] Based on the same technical concept, this application also provides a computer-readable storage medium storing instructions thereon, which, when read by a processor, implement the cutting control method provided in the above embodiments.

[0044] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail or in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Furthermore, the above embodiments can be freely combined as needed.

Claims

1. A cutting control method, characterized in that, A method for controlling a lawnmower robot to cut grass, the lawnmower robot being equipped with a blade disc, a front-view sensor, and a side-view sensor, wherein the cutting control method includes: Obtain the work status instructions and the global map of the lawnmower robot's operation. The indication types of the work status instructions include: edge cutting operation or bow-shaped cutting operation. Based on the operational condition command, the forward-looking sensor is controlled to acquire forward-looking binocular image data, and the side-looking sensor is controlled to acquire side-looking monocular image data and first flight time distance data. Based on the forward-looking binocular image data, a forward-looking depth map and a forward-looking semantic segmentation map are generated. The forward-looking depth map is used to determine the first distance information of obstacles in the first field of view of the forward-looking sensor, and the forward-looking semantic segmentation map is used to determine the type information of obstacles in the first field of view. The forward depth map is fused with the forward semantic segmentation map to determine the first grid map; Based on the side-view monocular image data, a side-view semantic segmentation map is generated, wherein the side-view semantic segmentation map is used to determine the type information of obstacles in the second field of view of the side-view sensor; An initial side-view depth map is determined based on the side-view semantic segmentation map, and the initial side-view depth map is used to determine the estimated distance information of obstacles in the second field of view; The estimated distance information is corrected using the first time-of-flight distance data to determine a second grid pattern, wherein the second grid pattern includes second distance information of obstacles in the second field of view; Based on the first grid map, the second grid map, and the positioning information of the lawnmower robot, the first grid map and the second grid map are merged and mapped to the world coordinate system to generate a local map; Based on the type of the work condition command, the cutterhead is controlled to perform extension and retraction actions in the local map and the global map.

2. The cutting control method according to claim 1, characterized in that, The step of generating a forward-looking depth map and a forward-looking semantic segmentation map based on the forward-looking binocular image data includes: The forward-looking binocular image data is subjected to stereo matching processing to generate the forward-looking depth map; A reference image is selected from the forward-looking binocular image data, and a forward-looking semantic segmentation map is generated based on a preset semantic segmentation algorithm.

3. The cutting control method according to claim 2, characterized in that, The step of generating a side-view semantic segmentation map based on the side-view monocular image data includes: Based on the semantic segmentation algorithm, the side-view monocular image data is segmented to determine the side-view semantic segmentation map; The step of determining the initial side-view depth map based on the side-view semantic segmentation map includes: The side-view semantic segmentation map is distance estimated based on a preset ground hypothesis algorithm to determine the estimated distance information; Based on the estimated distance information and the side-view semantic segmentation map, the initial side-view depth map is generated.

4. The cutting control method according to claim 3, characterized in that, The step of correcting the estimated distance information using the first flight time-distance data to determine the second grid pattern includes: Based on the first flight time-distance data and the preset confidence threshold, the second flight time-distance data is determined; Based on the second flight time distance data, the estimated distance information of obstacles in the second field of view is corrected, and the second distance information of the obstacles is determined.

5. The cutting control method according to claim 1, characterized in that, The type information includes: dynamic obstacles, static obstacles, and random boundary obstacles.

6. The cutting control method according to claim 5, characterized in that, The step of controlling the cutterhead to perform a telescopic movement based on the type of the work condition command and the local map includes: When the work condition command instructs the lawnmower to perform the edge-cutting operation, the dynamic obstacle is marked in the local map; In the local map, determine whether the dynamic obstacle exists within a preset distance range of the lawnmower robot; When the dynamic obstacle is present, control the cutter head to retract and cut; When there are no dynamic obstacles, the cutter head is controlled to extend and cut.

7. The cutting control method according to claim 5, characterized in that, The step of controlling the cutterhead to perform a telescopic movement based on the type of the work condition command and the local map includes: When the work condition command instructs the lawnmower robot to perform the bow-shaped cutting operation, the random boundary obstacle is marked on the global map; In the global map, determine whether the random boundary obstacle exists within a preset distance range of the lawnmower robot; When the random boundary obstacle is not present, the cutter head is controlled to perform a regular cutting operation; When the dynamic obstacle exists, the edge cutting operation is performed, including: determining whether the dynamic obstacle exists within a preset distance range of the lawnmower robot in the local map; When the dynamic obstacle is present, control the cutter head to retract and cut; When there are no dynamic obstacles, the cutter head is controlled to extend and cut.

8. The cutting control method according to claim 7, characterized in that, The step of performing the edge-cutting operation when the dynamic obstacle exists further includes: Determine whether the bow-shaped cutting operation is complete; When the bow-shaped cutting operation is completed, control the lawnmower robot to move to the location area marked with the random boundary obstacle in the global map; In the location area, based on the local map, it is determined whether there are dynamic obstacles within a preset distance range of the mowing robot, so as to control the blade to retract or extend for cutting.

9. A cutting control device, characterized in that, For controlling a lawnmower robot to perform cutting operations, the lawnmower robot is equipped with a blade disc, a front-view sensor, and a side-view sensor. The cutting control device includes: The acquisition unit is used to acquire the work status instructions and the global map of the lawn mowing robot's work. The indication types of the work status instructions include: edge cutting operation or bow-shaped cutting operation. The acquisition unit is used to control the forward-looking sensor to acquire forward-looking binocular image data based on the operation condition command, and to control the side-looking sensor to acquire side-looking monocular image data and first flight time distance data. The first image processing unit is configured to generate a forward-looking depth map and a forward-looking semantic segmentation map based on the forward-looking binocular image data, wherein the forward-looking depth map is used to determine the first distance information of obstacles in the first field of view of the forward-looking sensor, and the forward-looking semantic segmentation map is used to determine the type information of obstacles in the first field of view; it is also configured to fuse the forward-looking depth map and the forward-looking semantic segmentation map to determine a first grid map; The second image processing unit is configured to generate a side-view semantic segmentation map based on the side-view monocular image data, wherein the side-view semantic segmentation map is used to determine the type information of obstacles in the second field of view of the side-view sensor; further configured to determine an initial side-view depth map based on the side-view semantic segmentation map, wherein the initial side-view depth map is used to determine the estimated distance information of obstacles in the second field of view; and configured to correct the estimated distance information using the first time-of-flight distance data to determine a second grid map, wherein the second grid map includes the second distance information of obstacles in the second field of view; The mapping unit is used to merge the first grid map and the second grid map and map them to the world coordinate system based on the positioning information of the first grid map, the second grid map and the lawn mowing robot, to generate a local map; The cutting control unit is used to control the cutter head to perform extension and retraction actions based on the type of the work condition command in the local map and the global map.

10. A lawnmower robot, characterized in that, include: Tool head, front-view sensor, and side-view sensor; The processor, connected to the front-view sensor and the side-view sensor, is configured to control the cutter head to cut by acquiring the front-view sensor and the side-view sensor according to the cutting control method of any one of claims 1 to 8.

Images