Method for mowing robot to move along lawn boundary based on local grid map

Abstract

The invention discloses a method for a mowing robot to move along a lawn boundary based on a local grid map, and relates to the field of mowing robot edge navigation movement, and the method comprises the steps: recognizing a target region and an obstacle region through a semantic segmentation algorithm, constructing a dynamically updated local grid map through the combination of improved inverse perspective transformation and depth point cloud information, and carrying out the navigation of the target region and the obstacle region; no physical boundary dependence is realized, the deployment cost is reduced, and the real-time processing efficiency of acquired images can be improved; according to the local grid map, the edge navigation deflection angle is calculated through lawn boundary feature extraction in combination with the position of the robot, precise navigation of the robot along the lawn boundary is achieved, rapid response of dynamic obstacles is supported, and the edge movement precision and operation efficiency of the mowing robot are remarkably improved. The problems that an existing mowing robot is low in edge movement precision and poor in operation efficiency due to the fact that an edge track cannot be decided in real time and calculation efficiency cannot be considered at the same time are solved.

Description

Technical Field

[0001] The present application relates to the field of lawn mowing robot navigation movement along the edge, and specifically to a method for lawn mowing robot movement along the lawn boundary based on a local grid map. Background Art

[0002] Robotic lawn mowers generally use physical electronic fences or preset boundary lines as navigation references. However, this method requires complex preliminary deployment work. Users need to manually plan and lay physical boundary facilities. The installation cost is significantly increased, especially in irregular terrain or large lawns. To address the shortcomings of physical boundaries, existing technologies achieve autonomous navigation through local map construction. However, the construction of local grid maps for path planning is mainly aimed at global path coverage optimization. When moving along the edge, a general path planning strategy is still used, and kinematic optimization is not performed for boundary tracking. This leads to problems such as poor smoothness of the edge trajectory and high overlap coverage.

[0003] In addition, some solutions introduce image segmentation technology to detect lawn boundaries. Although this method breaks away from the dependence on physical facilities, it is seriously affected by lighting changes and dynamic obstacles (such as pedestrians and pets), and lacks effective environmental modeling capabilities, making it difficult to adapt to complex scenes.

[0004] In summary, traditional methods suffer from poor deployment flexibility. On open lawns, they struggle to distinguish static boundaries from dynamic interference in real time, often due to the frequent occurrence of dynamic obstacles. This can lead to boundary detection errors. Alternatively, they rely heavily on global map information, requiring the global model to be rebuilt when the local environment changes. This computational efficiency struggles to meet real-time requirements. Therefore, a method for edge motion that balances deployment convenience with environmental robustness is urgently needed. Summary of the Invention

[0005] The purpose of this application is to address the problem of low precision and poor operating efficiency of existing lawn mowing robots due to their inability to make real-time decisions on their trajectory along the edge while taking into account computational efficiency. A method for lawn mowing robots to move along lawn boundaries based on a local grid map is proposed. This method uses a semantic segmentation algorithm to identify target areas and obstacle areas, and combines an improved inverse perspective transformation with depth point cloud information to construct a dynamically updated local grid map, achieving no physical boundary dependency, reducing deployment costs, and improving the efficiency of real-time processing of acquired images. Based on the local grid map, lawn boundary features are extracted and combined with the robot's position to calculate the deflection angle for navigation along the edge, enabling precise navigation of the robot along the lawn boundary and supporting rapid response to dynamic obstacles, significantly improving the precision and operating efficiency of the lawn mowing robot's movement along the edge. This method overcomes the problem of low precision and poor operating efficiency of existing lawn mowing robots due to their inability to make real-time decisions on their trajectory along the edge while taking into account computational efficiency.

[0006] To solve the above technical problems, the technical solution adopted in this application is: a method for a lawn mowing robot to move along a lawn boundary based on a local grid map, comprising the following steps: Based on semantic segmentation, target area recognition and image segmentation are performed on the robot's environment image to obtain target area images and obstacle area images; Processing the target area image and the obstacle area image based on an improved inverse perspective transformation to generate a local grid map containing grass and obstacle type labels from a bird's-eye view; Extracting lawn boundary line features of the local grid map based on the depth point cloud data combined with the target area image, dynamically adjusting obstacle grid weights according to obstacle detection results to optimize the lawn boundary line features and obtain an optimized boundary grid; extracting the lawn boundary grid based on the optimized boundary grid, and calculating the edge navigation deflection angle based on the current posture of the robot; Real-time path planning is performed based on the edge navigation deflection angle to enable the robot to move along the lawn boundary.

[0007] In this solution, semantic segmentation is used to identify and segment the target area image (here refers to the lawn area image) and the obstacle area image (here refers to the non-lawn area image), thereby improving the mowing robot's ability to perceive the environment, thereby being able to adapt to environmental changes in a timely manner, reducing work errors caused by the inability to accurately identify the target area, and providing basic data for subsequent edge path planning; the segmented image is processed through an improved inverse perspective transformation to generate a local grid map of a bird's-eye view, and the grass and obstacles are marked in the local grid map, which enables the robot to clean and understand various elements in the local environment, so as to more accurately extract the lawn boundary line features; the depth point cloud data is combined with the semantically segmented target area image, and the specific location of the lawn boundary in the two-dimensional plane can be accurately located. Position, forming a set of boundary pixels in the raster map and then efficiently extracting the lawn boundary line features, establishing a basic understanding for the robot where the lawn is and where the area outside the boundary is, realizing the need to build a global high-precision map, reducing system complexity and cost, improving the accuracy and robustness of the robot's environmental understanding, and further improving the efficiency and quality of the robot's task execution; by extracting the lawn boundary grid, it is convenient to calculate the edge navigation deflection angle in real time according to the robot's current position for path planning, ensuring that the robot can respond to dynamic changes in the environment in a timely manner, and then ensuring that the robot can move accurately along the lawn boundary. At the same time, the robot avoids human intervention by planning the path by autonomously calculating the edge navigation deflection angle, thereby improving work efficiency and further improving the quality and accuracy of lawn maintenance.

[0008] Preferably, target area recognition and image segmentation are performed on the robot's environment image based on semantic segmentation to obtain the target area image and the obstacle area image, including: real-time acquisition of the robot's environment image, feature extraction of the environment image using a semantic segmentation model, and restoration of the extracted feature map to the same size as the environment image to learn a first mapping relationship between lawn area features and non-lawn area features; classifying each pixel in the environment image into lawn area and non-lawn area according to the first mapping relationship, and performing image segmentation and binary mask matrix conversion on the lawn area as the target area and the non-lawn area as the obstacle area to obtain the target area image and the obstacle area image identified by the binary mask matrix.

[0009] Preferably, the target area image and the obstacle area image are processed based on the improved inverse perspective transformation to generate a local grid map containing grass and obstacle type marks from a bird's-eye view, including: establishing a second mapping relationship of the camera-ground coordinate system based on the pitch angle compensation value of the robot camera; converting the pixel points in the target area image and the obstacle area image from the image coordinate system to the camera-ground coordinate system according to the second mapping relationship; correcting the edge error of the wide-angle lens of the camera using a radial distortion compensation model to compensate for the distortion of the camera-ground coordinate system; obtaining the transformed coordinates of the grid points after camera-ground coordinate system mapping and distortion compensation, calculating the pixel value of each grid point using bilinear interpolation, and generating local grid maps containing grass and obstacle type marks from a bird's-eye view to complete the inverse perspective transformation.

[0010] Preferably, obtaining the coordinates of the grid points after camera-ground coordinate system mapping and distortion compensation includes: determining the properties of the target overlooking grid according to the resolution of the target overlooking grid; and obtaining the transformed coordinates of each target overlooking grid point after camera-ground coordinate system mapping and distortion compensation based on the properties of the target overlooking grid.

[0011] Preferably, the bilinear interpolation method is used to calculate the pixel value of each grid point to generate a local grid map containing grassland and obstacle type marks from a bird's-eye view, including: extracting a number of target pixel points with the smallest distance from the transformed coordinates in the target area image and the obstacle area image respectively; according to the grayscale value of the target pixel point and the distance weight between the target pixel point and the corresponding transformed coordinate, a bilinear interpolation formula is used to calculate the pixel value of the target bird's-eye view grid point, to generate a local grid map of grassland with a specified resolution in the target area image and a local grid map of obstacles with a specified resolution in the obstacle area image.

[0012] Preferably, the method of extracting the lawn boundary line features of the local grid map based on the depth point cloud data in combination with the target area image includes: converting the environment image into a corresponding depth point cloud; projecting the depth point cloud data onto the target area image, obtaining the pixel position corresponding to the depth point cloud in the target area image, and determining the correspondence between the depth point cloud and the target area according to the pixel value corresponding to the pixel position; based on the correspondence, detecting the boundary line pixels of the target area image using an edge detection algorithm; and extracting the lawn boundary line features of the grassland local grid map in the local grid map based on the boundary line pixels.

[0013] Preferably, dynamically adjusting the obstacle grid weight according to the obstacle detection result to optimize the lawn boundary line characteristics and obtain the optimized boundary grid includes: assigning an initial weight of the obstacle local grid map according to the obstacle information obtained in real time; A temporary obstacle weight time-confidence dual decay model is constructed based on the obstacle's time decay factor and the obstacle confidence. Based on the change of obstacle information over time and confidence, the initial weight of the obstacle local grid map is dynamically adjusted through the temporary obstacle weight time-confidence dual decay model to optimize the obstacle interference of the extracted lawn boundary line features and obtain the optimized boundary grid.

[0014] Preferably, extracting a lawn boundary grid based on the optimized boundary grid and calculating the edge navigation deflection angle in combination with the current posture of the robot include: performing dynamic boundary detection on the optimized boundary grid using pixel traversal to obtain a candidate area of the lawn boundary grid; based on the candidate area, selecting a target boundary area using a dual-mode selector to obtain a corresponding lawn boundary grid; and calculating the robot's edge navigation deflection angle based on the lawn boundary grid in combination with the current posture of the robot.

[0015] Preferably, the method comprises: if the initial reference block for processing the optimized boundary grid has not been determined, calculating the width of each candidate area, extracting the candidate areas with widths greater than or equal to the robot width threshold, and taking the candidate area with the largest width as the target boundary area; if the initial reference block for processing the optimized boundary grid has been determined, calculating the left and right boundary values of a valid area based on the initial reference block; calculating the width of the overlapping surface of the current candidate area and the previous valid area according to the left and right boundary values of the valid area, and taking the candidate area corresponding to the robot width threshold with a width greater than or equal to the robot width threshold as the target boundary area.

[0016] Preferably, the method also includes a robot obstacle avoidance mechanism; specifically, when the robot detects an obstacle grid, it adjusts the weight of the obstacle grid map, re-extracts the lawn boundary grid to plan a detour path, and resets the weight of the obstacle grid map after resuming movement along the edge.

[0017] Beneficial effects of this application: 1. Combining semantic segmentation data and depth point cloud data, the image is inversely transformed to generate a bird's-eye view grid map. The grid information of the limited area around the robot is updated in real time, converting the robot's environmental perception into executable spatial constraints, making edge tasks in complex environments efficient, accurate, and safe. 2. By extracting boundary pixels from the target area image, core data is provided for building a more accurate local environment grid map. Boundary pixels directly define the constraints of the target path, ensuring that the robot always maintains a safe distance from the boundary when moving along the edge, and providing strong data support for subsequent edge path planning and obstacle avoidance. By combining deep point cloud and semantic segmentation data, the stability and reliability of boundary extraction in complex environments are ensured; with local grid maps and real-time updated boundary pixels, there is no need to build a global high-precision map, and it is compatible with low-cost depth cameras and semantic segmentation models, reducing system complexity and deployment costs. 3. A dual-mode selection mechanism effectively solves the "initial boundary positioning" and "boundary continuity" problems. A width threshold filter ensures that the planned path will not cause the robot to fall into narrow areas or collide with boundaries, improving the safety of edge navigation. Overlap constraints ensure the continuity of boundary blocks in different rows, preserving the characteristics of authentic lawn boundaries and enabling the robot to flexibly adapt to complex outdoor environments. 4. Through dynamic boundary updates, the column coordinates of the real-time selected block are converted into the address of the global grid map, which unifies the coordinate format and provides coordinate support for determining the position and boundary direction of the robot in the global coordinate system, so as to facilitate the connection between the robot positioning system and path planning, and ensure that the boundary information can be directly used for navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features, objects, and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are provided for illustration purposes only and are not to be construed as limiting the present application. Like reference numerals are used throughout the drawings to denote like parts.

[0019] Figure 1 This is a flow chart of a method for a lawn mowing robot to move along a lawn boundary based on a local grid map provided in an embodiment of the present application.

[0020] Figure 2This is a flow chart of a lawn boundary grid extraction method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of this application more clear, the application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific implementation method described here is only an optimal embodiment of this application, which is only used to explain this application and does not limit the scope of protection of this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0022] Example 1: Figure 1 As shown, a method for a lawn mowing robot to move along a lawn boundary based on a local grid map includes: S1. Perform target area recognition and image segmentation on the robot's environment image based on semantic segmentation to obtain target area images and obstacle area images.

[0023] Specifically, the S1 includes: Collecting an image of the environment in which the robot is located in real time, extracting features from the environment image using a semantic segmentation model, and restoring the extracted feature map to the same size as the environment image to learn a first mapping relationship between lawn area features and non-lawn area features; According to the first mapping relationship, each pixel in the environmental image is classified into a lawn area and a non-lawn area, and the lawn area is used as the target area and the non-lawn area is used as the obstacle area for image segmentation and binary mask matrix conversion to obtain a target area image and an obstacle area image identified by the binary mask matrix.

[0024] In some embodiments, the robot's environmental image is primarily captured in front of the robot. This image information is captured in real time by the robot's wide-angle camera, where the wide-angle camera has a field of view of no less than 150 degrees. Semantic segmentation is implemented using a deep learning model. The semantic segmentation model can employ a convolutional network, such as an FCN network. The steps for using the FCN to identify and segment the robot's environmental image are as follows: Use convolutional neural networks to extract features from environmental images. The convolutional layers in convolutional neural networks can automatically learn various features in the image, such as edges and textures. Through the combination of multiple convolutional layers and pooling layers, more advanced and abstract features can be gradually extracted. The extracted feature map is restored to the same size as the input image through a deconvolution layer or an upsampling layer. In this process, the model learns the first mapping relationship between different features and different semantic categories, that is, the first mapping relationship between lawn areas and non-lawn areas; According to the first mapping relationship learned by the model, each pixel in the image is classified to determine whether it belongs to the lawn area or the non-lawn area, thereby obtaining the segmented target area image and obstacle area image.

[0025] In some embodiments, the target area image and obstacle area image after semantic segmentation are output in the form of a binary mask matrix. The binary mask matrix represents each pixel in the image with 0 or 1, where 0 represents a non-lawn area and 1 represents a lawn area, so as to clearly distinguish different areas in the image and facilitate subsequent processing and analysis.

[0026] In this embodiment, a binary mask matrix can be used to quickly and accurately identify the location and extent of a lawn area, and the boundary points of the lawn can be determined based on the binary mask matrix. Precisely segmenting the lawn area helps the robot move more accurately along the lawn boundary, avoiding missed or duplicate mowing, thereby improving mowing efficiency and quality, reducing energy consumption and working time. Traditional mowing robots rely on physical electronic fences or preset boundary lines to determine their working areas. Using semantic segmentation to identify lawn areas eliminates the need for physical installation and significantly reduces deployment costs. Furthermore, lawn identification enables the mowing robot to flexibly adapt to lawns of varying shapes, sizes, and terrains, as well as various complex environmental conditions, thereby improving its adaptability and flexibility. Furthermore, the semantic segmentation model can promptly detect and update segmentation results, enabling the mowing robot to adjust its motion path accordingly when new obstacles enter the lawn or when the lawn area changes, further enhancing the robot's adaptability and intelligence.

[0027] S2. Process the target area image and the obstacle area image based on an improved inverse perspective transformation to generate a local grid map containing grass and obstacle type labels in a bird's-eye view.

[0028] Specifically, the S2 includes: Establishing a second mapping relationship between the camera and the ground coordinate system based on the pitch angle compensation value of the robot camera; Converting pixel points in the target area image and the obstacle area image from an image coordinate system to a camera-ground coordinate system according to the second mapping relationship; A radial distortion compensation model is used to correct the edge error of the camera's wide-angle lens to compensate for the distortion of the camera-ground coordinate system; Obtain the transformed coordinates of the grid points after camera-ground coordinate system mapping and distortion compensation, use bilinear interpolation to calculate the pixel value of each grid point, and generate local grid maps containing grass and obstacle type markers from a bird's-eye view to complete the inverse perspective transformation.

[0029] Specifically, obtaining the coordinates of the grid points after camera-ground coordinate system mapping and distortion compensation includes: Determine the properties of the target overlooking grid according to the resolution of the target overlooking grid; Based on the properties of the target overlooking grid, the transformed coordinates of each target overlooking grid point are obtained after camera-ground coordinate system mapping and distortion compensation.

[0030] Specifically, the bilinear interpolation method is used to calculate the pixel value of each grid point to generate a local grid map containing grass and obstacle type marks from a bird's-eye view, including: Extracting a plurality of target pixel points having the smallest distance from the transformed coordinates from the target area image and the obstacle area image respectively; According to the grayscale value of the target pixel point and the distance weight between the target pixel point and the corresponding transformed coordinate, the pixel value of the target overlooking grid point is calculated using a bilinear interpolation formula to generate a target area image containing a grassland local grid map of a specified resolution and an obstacle area image containing a obstacle local grid map of a specified resolution.

[0031] In some embodiments, an improved IPM algorithm can be used to perform inverse perspective transformation on the segmented lawn area image and the obstacle area image. In this embodiment, the algorithm mainly includes: establishing a camera-ground coordinate system mapping relationship; introducing a radial distortion compensation module to correct the edge error of the wide-angle lens; and using bilinear interpolation to generate a 0.03m resolution bird's-eye view grid.

[0032] Specifically, the camera-ground coordinate system mapping relationship calculation formula is as follows: [X,Y,Z] T =K -1 [u,v,1] T ·(h / cosθ)(1) Where h is the installation height of the lawn mowing robot camera, which can be obtained through actual measurement; θ is the camera pitch angle compensation value, which is determined based on factors such as the robot design and the camera installation angle; [X, Y, Z] represents the three-dimensional coordinates of the pixel points of the segmented image converted from the image coordinate system to the camera-ground coordinate system. Among them, the Z coordinate is usually 0 on the ground plane, so we can focus on the X and Y coordinates, which represent the corresponding positions of the pixel points on the ground; (u, v) represents the pixel points in the segmented image; K is the camera's intrinsic parameter matrix, which is also the basic parameter of the camera imaging and can be obtained through methods such as camera calibration.

[0033] Specifically, the directional distortion compensation model may adopt a polynomial model or the like, and the coordinates may be corrected according to a specific model and parameters.

[0034] Specifically, when using bilinear interpolation to generate a bird's-eye view grid, the size and range of the target bird's-eye view grid are first determined based on the resolution of the target bird's-eye view grid (0.03m in this embodiment). For example, if the lawn area is a rectangle, the number of rows and columns of the grid is calculated based on its length, width, and resolution. Next, the transformed coordinates of each target bird's-eye view grid point, such as (x, y), are found after using the camera-ground coordinate system and radial distortion compensation. The four nearest pixels to this coordinate in the lawn area image or obstacle area image are determined. Based on the grayscale values of these four pixels (binary image values of 0 or 1 in this embodiment) and their distance weights from (x, y), a bilinear interpolation formula is used to calculate the pixel value of the target bird's-eye view grid point to generate a bird's-eye view grid image with the specified resolution, completing the inverse perspective transformation.

[0035] It is understandable that when performing inverse perspective transformation, the traditional IPM algorithm usually only uses simple camera internal parameters and basic perspective projection principles to construct the mapping relationship between the camera and the ground coordinate system. This mapping method often does not fully consider the influence of the camera's installation height and pitch angle, resulting in large deviations in the mapping results in actual applications, especially when the camera installation position and angle change. It cannot accurately reflect the true position of objects on the ground. Therefore, in this embodiment, the image inverse perspective transformation is improved to improve the IPM algorithm, and the installation height h and pitch angle compensation value θ are explicitly introduced. The camera-ground coordinate system mapping relationship is established through the above formula, which fully considers the physical parameters of the actual installation of the camera, and can more accurately map the image pixels to the ground coordinate system, reducing the errors caused by changes in the installation position and angle, thereby improving the accuracy of the inverse perspective transformation and providing a more reliable foundation for subsequent robot path planning and environmental perception.

[0036] It can also be understood that by correcting the coordinates in the camera-ground coordinate system through radial distortion compensation, the problem of edge error of the wide-angle lens of the lawn mowing robot is effectively solved. After distortion compensation, the inverse perspective transformation can more realistically restore the ground scene, improve the accuracy of environmental information extraction, and help to accurately extract the boundary features of the lawn area image in the subsequent process.

[0037] In this embodiment, bilinear interpolation, by taking into account the grayscale values and distance weights of the four surrounding pixels, allows for smoother calculation of the pixel values at the grid points, resulting in a more continuous and accurate raster image. Furthermore, a fixed resolution ensures that the inverse perspective transformation results have a uniform scale, facilitating subsequent image analysis and processing. For example, during path planning, feasible paths for the robot can be more efficiently determined based on the grid information.

[0038] S3. Extracting lawn boundary line features of the local grid map based on the depth point cloud data combined with the target area image, and dynamically adjusting obstacle grid weights according to obstacle detection results to optimize the lawn boundary line features and obtain an optimized boundary grid.

[0039] Specifically, the extracting of lawn boundary line features of the local grid map based on the depth point cloud data in combination with the target area image includes: Converting the environment image into a corresponding depth point cloud; Projecting the depth point cloud data onto the target area image, obtaining a pixel position corresponding to the depth point cloud in the target area image, and determining a correspondence between the depth point cloud and the target area based on a pixel value corresponding to the pixel position; Based on the corresponding relationship, an edge detection algorithm is used to detect boundary line pixels of the target area image; The lawn boundary line features of the grassland local grid map in the local grid map are extracted based on the boundary line pixels.

[0040] In this embodiment, the goal of extracting boundary pixels from the target area image is to accurately determine the lawn's boundaries. By matching the depth point cloud data with the lawn semantic segmentation data, the boundary between lawn and non-lawn areas can be more accurately identified. A contour tracing algorithm is then used to optimize the continuity of boundary points and eliminate noise interference, resulting in a more accurate boundary grid, providing a reliable basis for subsequent path planning by the mowing robot.

[0041] The deep point cloud provides the 3D coordinates of lawn boundaries, such as height and distance, while semantic segmentation clearly distinguishes between grass and non-grass areas, such as roads, flower beds, and obstacles. Combining these two methods allows the precise location of lawn boundaries on a 2D plane, forming a grid map of boundary pixels. This provides the robot with a basic understanding of where lawns are and where areas outside the boundaries are.

[0042] In some examples, contour tracing algorithms are used to optimize the continuity of boundary points and eliminate noise interference. This mainly involves: converting the result of matching the depth point cloud with semantic segmentation into a binary image, where lawn areas are 1 and non-lawn areas are 0, establishing a clear set of boundary pixels: a 0-1 matrix; denoising through morphological operations to eliminate isolated noise points around the lawn boundary, such as single pixels that are misjudged as non-lawn points, and filling gaps in the boundary, such as missing pixels caused by sensor noise, to make the lawn boundary more continuous. A contour detection algorithm is used to find all continuous contours in the binary image, that is, the boundary curves between lawn and non-lawn areas. The detected contours are sorted according to their area or perimeter, prioritizing the contours with the largest area, usually the main lawn boundary, and ignoring extremely small contours such as isolated areas formed by noise.

[0043] Furthermore, the area or perimeter of each contour is calculated, and a minimum valid area threshold is set to eliminate contours with an area smaller than the threshold. For each valid contour, the boundary points are traversed in a clockwise or counterclockwise order to ensure that the order of the points matches the actual spatial continuity of the environment. For small gaps in the boundary, intermediate points are generated using the bilinear interpolation method described above to fill the gaps.

[0044] Furthermore, the optimized boundary points are converted from the image pixel coordinate system combined with the three-dimensional coordinate information of the depth point cloud into the robot global coordinate system, which is mapped to the local grid map to form a discrete boundary grid point set.

[0045] It is understandable that by removing small area contours and isolated points In this embodiment, by extracting the boundary pixels of the target area image, core data is provided for constructing a more accurate local environment grid map. The boundary pixels directly define the constraints of the target path, ensuring that the robot always maintains a safe distance from the boundary when moving along the edge, and providing strong data support for subsequent edge path planning and obstacle avoidance. By combining depth point cloud and semantic segmentation data, the defects of a single sensor are avoided, such as pure vision being easily affected by lighting, to ensure the stability and reliability of boundary extraction in complex environments. At the same time, through local grid maps and real-time updated boundary pixels, there is no need to build a global high-precision map, and it is compatible with low-cost depth cameras and semantic segmentation models, reducing system complexity and deployment costs.

[0046] Specifically, dynamically adjusting the obstacle grid weight according to the obstacle detection result to optimize the lawn boundary line features and obtain the optimized boundary grid includes: Assigning initial weights to the obstacle local grid map based on the obstacle information obtained in real time; A temporary obstacle weighted time-confidence dual decay model is constructed based on the obstacle's time decay factor and obstacle confidence. According to the change of obstacle information over time and confidence, the initial weight of the obstacle local grid map is dynamically adjusted through the temporary obstacle weight time-confidence double decay model to optimize the obstacle interference of the extracted lawn boundary line features and obtain the optimized boundary grid.

[0047] Specifically, the temporary obstacle weight time-confidence double decay model is expressed as follows: Where C(t) is the confidence function, λ is the attenuation coefficient, and t is the duration of the obstacle.

[0048] In some embodiments, considering that different types of obstacles have varying degrees of impact on lawn boundary determination, initial weights can be assigned to the corresponding obstacle local grid map based on the obstacle type. For example, larger obstacles that are likely to remain for a long time can be assigned a higher initial weight, while smaller obstacles that move more quickly can be assigned a lower initial weight. Initial weights can also be assigned based on factors such as the distance to the obstacle.

[0049] Furthermore, the time decay factor is used to reflect the temporal impact of temporary obstacles on the lawn boundary. The time decay factor is expressed as: Where λ is the decay coefficient, and t is the duration of the obstacle on the lawn. The decay coefficient determines how quickly the weight decays over time. The specific value of the decay coefficient can be determined through experimentation or experience. For example, in practical applications, by repeatedly testing different λ values, we can observe the effect of the temporary obstacle weight time-confidence dual decay model on the weight of the temporary obstacle local grid map. This allows us to select a λ value that best adapts the temporary obstacle weight time-confidence dual decay model to different scenarios.

[0050] In some embodiments, the confidence level C(t) represents the degree of confidence in whether an obstacle is a true boundary. As detection information accumulates, the judgment of obstacles becomes more accurate, and the confidence level changes accordingly. The confidence level can be updated based on factors such as the number of consecutive frames in which the obstacle is detected and the stability of the obstacle. For example, when the same boundary features are detected for N consecutive frames (N ≥ 5), the confidence level is increased to 100%; when the features of the detected obstacle change significantly, the confidence level is reduced.

[0051] In this embodiment, according to the time attenuation factor A time-confidence dual-attenuation model for temporary obstacle weights is constructed using the confidence function C(t). This model, which comprehensively considers both time and confidence, allows for more accurate and dynamic adjustment of the weights of the grids corresponding to temporary obstacles. Adjusting the weights of the grids corresponding to obstacles is intended to prioritize the true lawn boundary during the robot's path planning and boundary recognition. Temporary obstacles can temporarily affect the robot's perception of lawn boundaries. If their weights are not adjusted, the robot may mistakenly identify them as boundaries, affecting the efficiency and accuracy of mowing operations. Dynamic weight adjustment allows the robot to focus more closely on the true lawn boundary, improving its adaptability to the environment.

[0052] In other embodiments, the depth point cloud and semantic boundary points can be fused to perform height verification in the Z-axis direction of the local grid map of the lawn to eliminate interference from hanging objects such as tree branches, because hanging objects are not the real lawn boundaries. Depth verification can further ensure that the extracted lawn boundaries are real and valid.

[0053] In this embodiment, extracting the lawn boundary pixels allows the approximate lawn boundary to be determined. Adjusting the weights of the obstacle grids further optimizes the boundary determination based on the boundary determination. By reducing the weights of the grids corresponding to temporary obstacles, these obstacles can be prevented from interfering with the extracted lawn boundary, making the final determination of the lawn boundary more accurate.

[0054] S4. Extracting a lawn boundary grid according to the optimized boundary grid, and calculating an edge navigation deflection angle in combination with the current posture of the robot.

[0055] Specifically, the S4 includes: Performing dynamic boundary detection on the optimized boundary grid by pixel traversal to obtain candidate areas of the lawn boundary grid; Based on the candidate area, a dual-mode selector is used to select a target boundary area to obtain a corresponding lawn boundary grid; Based on the lawn boundary grid and the current posture of the robot, the robot's edge navigation deflection angle is calculated.

[0056] Specifically, based on the candidate area, a dual-mode selector is used to select a target boundary area to obtain a corresponding lawn boundary grid, including: If the initial reference block for processing the optimized boundary grid is not determined, then calculating the width of each candidate area, extracting the candidate areas whose width is greater than or equal to the robot width threshold, and taking the candidate area with the largest width as the target boundary area; If the initial reference block of the optimized boundary grid has been determined to be processed, then the left and right boundary values of a valid area are calculated based on the initial reference block; The width of the overlapping surface of the current candidate area and the previous valid area is calculated according to the left and right boundary values of the valid area, and the candidate area corresponding to the robot width threshold whose width of the overlapping surface is greater than or equal to the robot width threshold is used as the target boundary area.

[0057] In some embodiments, as Figure 2 As shown in the figure, the extraction of lawn boundary grid is achieved through the extraction algorithm. The specific steps are as follows: A1. System initialization: Before extracting the lawn boundary grid, the input optimized boundary grid needs to be validated for parameter validity. This includes at least: checking that the image width and height are positive integers, and checking that the pointer storing the image data is not null. If the parameters are unreasonable, the image cannot be processed normally. To avoid possible interference areas in the image, the preset pointer BLIND_SPOT_HEIGHT_START_INDEX is used to determine the starting row for processing to avoid blind area interference. A2. Construct the main processing loop, including establishing a row iterator and precalculating row offsets. The row iterator traverses image rows in ascending order starting from start_row. The precalculated row offset is used to quickly locate the address of the current row's data using row_offset = row * width. A3. Extract candidate areas. Scan the current row pixel by pixel from left to right, and set start and stop conditions for dynamic boundary detection. The start condition includes: when the pixel value 0 (i.e., the starting point of the dark area) is detected, record start_block[], and use this position as the starting point of the candidate area; the end condition includes: when the pixel value jumps to 1 (i.e., the bright area) or reaches the end of the row, record end_block[], and use this position as the ending point of the candidate area. At the same time, to avoid wasting time processing too many small areas, the maximum number of candidate areas processed per row is limited. The "dark area start - bright area end" rule can be used to aggregate continuous dark area pixels into "blocks", thereby increasing the attention of continuous areas (blocks) that meet the lawn boundary characteristics. A4. Regional decision analysis: Detecting lawn boundaries through a two-stage selection mechanism; A5. Dynamic boundary update: converting the column coordinates of the selected block to a global storage address. This means converting the column coordinates of the selected area to a global address for the entire image storage, facilitating subsequent unified data processing. A6. Termination condition judgment, including abnormal termination and normal termination. When no valid overlapping blocks are detected within the set threshold rows, the system stops processing, indicating abnormal termination. The threshold can be dynamically matched. When the scanning process of all rows is completed, it is normal termination.

[0058] In some examples, the dual-stage selection mechanism in A5 includes the widest effective fast selection mechanism and the overlapping tracking selection mechanism. Specifically, as follows: If no initial reference block (i.e., a continuous area that meets the lawn boundary characteristics determined by the candidate regions) is found, the widest effective fast selection mechanism is used. This mechanism first calculates the width of each candidate region by subtracting the starting point from the ending point. Only regions with a width greater than or equal to the robot's width threshold are retained. From these eligible regions, the widest one is selected as the reference region. If an initial reference block exists, the overlapping tracking selection mechanism is used. This mechanism first calculates the column range of the previous valid region, i.e., the left and right boundaries [prev_left, prev_right], and then calculates the size of the overlapping part of the current candidate region and the previous valid region. The candidate regions whose overlapping width is greater than or equal to the robot width threshold are retained.

[0059] In some embodiments, after determining the candidate area, the lawn boundary grid is obtained from the area image to obtain the lawn boundary tangent. The edge navigation deflection angle is calculated based on the current posture of the robot. The calculation formula of the edge navigation deflection angle is: Δθ=k_1*arctan(d / δ)+k_2*(α-α0)(3) Where k_1 and k_2 are dynamic adjustment coefficients, with values of k_1 = 0.6 and k_2 = 0.4 respectively; δ is the preset safety distance threshold, with a value of (0.2m-0.5m); d is the distance deviation; α0 is the ideal tangent angle, with a value of [85°, 95°]; α is the angular deviation between the machine heading angle and the tangent line of the lawn boundary; The calculation formula of the dynamic adjustment coefficient is: k=1 / (1+e^(-β·v))(4) Here, β = 0.3 is the speed adaptation factor. The speed adaptation factor is determined through empirical analysis based on factors including, but not limited to, the robot's motion speed, the curvature of the lawn boundary, and sensor accuracy and response time. This factor enables the robot to maintain stable edge motion across various motion states, ensuring that the robot does not stray from the boundary due to untimely steering at high speeds. Even at low speeds, the robot can adjust its direction in response to boundary changes, thereby improving the accuracy and reliability of the robot's edge motion. Ultimately, it achieves a balance between speed and edge precision, optimizing the robot's overall motion efficiency. In this embodiment, parameter validity verification is used to ensure that the basic data for subsequent processing is legal, thereby avoiding subsequent calculation errors; the initial scanning line is calculated to avoid invalid or interfering data from participating in the calculation, thereby ensuring the stable operation of the robot in complex outdoor environments, ensuring that processing starts from the effective detection area, and improving the accuracy of boundary extraction; by constructing a main processing loop, the image is scanned line by line in sequence starting from the starting line, so as to facilitate subsequent line-by-line analysis of boundary features, and by pre-calculating the address offset, the real-time calculation overhead is reduced, thereby reducing the calculation complexity and meeting the performance requirements of the robot's real-time navigation, thereby improving the robot's operating efficiency.

[0060] In this embodiment, dynamic boundary detection conditions are set when extracting candidate areas to aggregate continuous dark area pixels into "blocks," eliminating interference from individual noise points. This ensures the robot focuses only on continuous areas that meet lawn boundary characteristics, reduces irrelevant data processing, and further improves path planning efficiency and accuracy. A dual-mode selection mechanism effectively addresses both the "initial boundary positioning" and "boundary continuity" issues. Blocks are filtered using a width threshold to ensure that the planned path will not trap the robot in narrow areas or collide with boundaries, enhancing the safety of edge navigation. Overlapping constraints ensure the continuity of boundary blocks across different rows, preserving the true lawn boundary characteristics and enabling the robot to flexibly adapt to complex outdoor environments.

[0061] Furthermore, by converting the column coordinates of the real-time selected block into the address of the global grid map, the coordinate format is unified, providing coordinate support for determining the robot's position and boundary direction in the global coordinate system, so as to facilitate the connection of the robot positioning system with path planning and ensure that the boundary information can be directly used for navigation.

[0062] S5. Perform real-time path planning based on the edge navigation deflection angle to enable the robot to move along the lawn boundary.

[0063] In some embodiments, when planning a robot mower's path along an edge based on an edge navigation deflection angle, the robot's motion state is first determined, including at least the robot's starting position and orientation, and current speed. The robot's starting position and orientation within a lawn boundary grid map are determined using the robot's positioning system.

[0064] Furthermore, based on the edge navigation deflection angle, the robot's short-term target point is generated on the lawn boundary grid map. This target point should be located on the lawn boundary and match the robot's current position, orientation, and deflection angle. For example, if the deflection angle indicates that the robot needs to turn left, the short-term target point should be located to the left and front of the robot's current position and at the lawn boundary.

[0065] In some examples, a global path framework can be developed before planning the local path along the edge; during the movement of the mowing robot, the local path is adjusted in real time based on the real-time calculated along-edge navigation deflection angle to coordinate it with the global path.

[0066] In this embodiment, the robot mower's path is planned along the lawn edge in real time using the edge navigation deflection angle. This allows for rapid adaptation to various changes in lawn boundaries. Path planning based on the lawn boundary grid and the edge navigation deflection angle minimizes reliance on other complex environmental information and responds to dynamic environmental changes in real time. When temporary obstacles appear, the robot can quickly sense and adjust its path to avoid collisions or boundary deviations, ensuring the robot accurately follows the lawn boundary. This prevents excessive deviation from the boundary, resulting in missed mowing areas, and prevents inadvertent damage to areas outside the boundary, thereby improving the quality and accuracy of lawn maintenance. Furthermore, the robot utilizes its own sensors to autonomously calculate the edge navigation deflection angle and perform path planning, eliminating the need for human intervention and enhancing efficiency and automation.

[0067] Specifically, the method also includes a robot obstacle avoidance mechanism; specifically including: When the robot detects an obstacle grid, it adjusts the weight of the obstacle grid map, re-extracts the lawn boundary grid to plan a detour path, and resets the weight of the obstacle grid map after resuming movement along the edge.

[0068] In some embodiments, the robot uses an imaging sensor to capture real-time images of the environment, combines the depth point cloud with semantic segmentation results to identify obstacle grids, and then maps the obstacles to a local grid map to determine their specific location in the coordinate system and the grid range they occupy. The detected obstacle grids are further assigned a high pass cost weight so that they are prioritized for avoidance during path planning. For dynamic obstacles, the grids within a certain range around them can be weighted incrementally, such as using Gaussian blur processing, to simulate a safe distance and prevent the robot from getting too close to the obstacle.

[0069] In some embodiments, after the weight of the obstacle grid is adjusted, the path is replanned according to the new weight; this includes: re-extracting the lawn boundary in the local grid map containing the obstacle grid, excluding the boundary points occupied by obstacles, using the adjusted weight grid map as input to the path planning algorithm, and planning a collision-free detour path between the current position and the target edge path point; wherein the path planning algorithm can adopt the Dijkstra algorithm, the A* algorithm or the dynamic window method (DWA), etc. During planning, priority is given to grids with low weights, that is, grids close to the lawn boundary and without obstacles, to ensure that the detour path is as close to the original boundary as possible.

[0070] Furthermore, when the robot returns to the original lawn boundary continuous points through the detour path, for example, when it detects that there are no obstacles in N consecutive boundary grids, or the distance to the obstacle exceeds the safety threshold, it is determined to be "restoring the edge motion state". At this time, the weight of the obstacle grid is restored to the initial value to avoid long-term interference of historical obstacles on subsequent path planning.

[0071] In this embodiment, by dynamically adjusting grid weights, the robot can trigger obstacle avoidance the moment it detects an obstacle, eliminating the need for global map reconstruction and improving response speed. Path replanning is performed only on the local area affected by the obstacle, which requires less computation than global replanning and is suitable for mobile robot scenarios with high real-time requirements. The replanned path aims to "return to the original boundary," ensuring rapid resumption of edge motion after detouring, avoiding mission failures caused by a single obstacle and ensuring the continuity and reliability of robot missions in complex outdoor environments. The weight reset mechanism prevents temporary obstacles from interfering with long-term path planning, ensuring the robot's stability during repetitive tasks.

[0072] The above specific implementation methods are preferred implementation methods of the present application, and are not intended to limit the specific implementation scope of the present application. The scope of the present application includes but is not limited to the specific implementation methods. Any equivalent changes made in accordance with the shape, structure, and method of the present application are within the scope of protection of the present application.

Claims

1. A method for a lawn mowing robot to move along a lawn boundary based on a local grid map, characterized in that: Based on semantic segmentation, target area recognition and image segmentation are performed on the robot's environment image to obtain target area images and obstacle area images; Processing the target area image and the obstacle area image based on an improved inverse perspective transformation to generate a local grid map containing grass and obstacle type labels from a bird's-eye view; Extracting lawn boundary line features of the local grid map based on the depth point cloud data combined with the target area image, and dynamically adjusting obstacle grid weights according to obstacle detection results to optimize the lawn boundary line features and obtain an optimized boundary grid; Extracting the lawn boundary grid according to the optimized boundary grid, and calculating the deflection angle of the edge navigation in combination with the current posture of the robot; Real-time path planning is performed based on the edge navigation deflection angle to enable the robot to move along the lawn boundary.

2. The method for a lawn mowing robot to move along a lawn boundary based on a local grid map according to claim 1, characterized in that: The target area recognition and image segmentation based on semantic segmentation of the robot's environment image to obtain the target area image and the obstacle area image include: Collecting an image of the environment in which the robot is located in real time, extracting features from the environment image using a semantic segmentation model, and restoring the extracted feature map to the same size as the environment image to learn a first mapping relationship between lawn area features and non-lawn area features; According to the first mapping relationship, each pixel in the environmental image is classified into a lawn area and a non-lawn area, and the lawn area is used as the target area and the non-lawn area is used as the obstacle area for image segmentation and binary mask matrix conversion to obtain a target area image and an obstacle area image identified by the binary mask matrix.

3. The method for a lawn mowing robot to move along a lawn boundary based on a local grid map according to claim 1, characterized in that: The step of processing the target area image and the obstacle area image based on the improved inverse perspective transformation to generate a local grid map containing grass and obstacle type marks from a bird's-eye view includes: Establishing a second mapping relationship between the camera and the ground coordinate system based on the pitch angle compensation value of the robot camera; Converting pixel points in the target area image and the obstacle area image from an image coordinate system to a camera-ground coordinate system according to the second mapping relationship; A radial distortion compensation model is used to correct the edge error of the camera's wide-angle lens to compensate for the distortion of the camera-ground coordinate system; Obtain the transformed coordinates of the grid points after camera-ground coordinate system mapping and distortion compensation, use bilinear interpolation to calculate the pixel value of each grid point, and generate local grid maps containing grass and obstacle type markers from a bird's-eye view to complete the inverse perspective transformation.

4. The method for a lawn mowing robot to move along a lawn boundary based on a local grid map according to claim 3, characterized in that: The obtaining of the coordinates of the grid points after camera-ground coordinate system mapping and distortion compensation includes: Determine the properties of the target overlooking grid according to the resolution of the target overlooking grid; Based on the properties of the target overlooking grid, the transformed coordinates of each target overlooking grid point are obtained after camera-ground coordinate system mapping and distortion compensation.

5. The method for a lawn mowing robot to move along a lawn boundary based on a local grid map according to claim 4, characterized in that: The bilinear interpolation method is used to calculate the pixel value of each grid point to generate a local grid map containing grass and obstacle type marks from a bird's-eye view, including: Extracting a plurality of target pixel points having the smallest distance from the transformed coordinates from the target area image and the obstacle area image respectively; According to the grayscale value of the target pixel point and the distance weight between the target pixel point and the corresponding transformed coordinate, the pixel value of the target overlooking grid point is calculated using a bilinear interpolation formula to generate a target area image containing a grassland local grid map of a specified resolution and an obstacle area image containing a obstacle local grid map of a specified resolution.

6. The method for a lawn mowing robot to move along a lawn boundary based on a local grid map according to claim 1, characterized in that: The extracting of lawn boundary line features of the local grid map based on the depth point cloud data in combination with the target area image includes: Converting the environment image into a corresponding depth point cloud; Projecting the depth point cloud data onto the target area image, obtaining a pixel position corresponding to the depth point cloud in the target area image, and determining a correspondence between the depth point cloud and the target area based on a pixel value corresponding to the pixel position; Based on the corresponding relationship, an edge detection algorithm is used to detect boundary line pixels of the target area image; The lawn boundary line features of the grassland local grid map in the local grid map are extracted based on the boundary line pixels.

7. The method for a lawn mowing robot to move along a lawn boundary based on a local grid map according to claim 6, characterized in that: The dynamically adjusting the obstacle grid weight according to the obstacle detection result to optimize the lawn boundary line feature and obtain the optimized boundary grid includes: Assigning initial weights to the obstacle local grid map based on the obstacle information obtained in real time; A temporary obstacle weight time-confidence dual decay model is constructed based on the obstacle's time decay factor and the obstacle confidence. Based on the change of obstacle information over time and confidence, the initial weight of the obstacle local grid map is dynamically adjusted through the temporary obstacle weight time-confidence dual decay model to optimize the obstacle interference of the extracted lawn boundary line features and obtain the optimized boundary grid.

8. The method for a lawn mowing robot to move along a lawn boundary based on a local grid map according to claim 7, characterized in that: The step of extracting a lawn boundary grid according to the optimized boundary grid and calculating an edge navigation deflection angle in combination with the current posture of the robot includes: Performing dynamic boundary detection on the optimized boundary grid by pixel traversal to obtain candidate areas of the lawn boundary grid; Based on the candidate area, a dual-mode selector is used to select a target boundary area to obtain a corresponding lawn boundary grid; Based on the lawn boundary grid and the current posture of the robot, the robot's edge navigation deflection angle is calculated.

9. The method for a lawn mowing robot to move along a lawn boundary based on a local grid map according to claim 8, characterized in that: The method of selecting a target boundary region based on the candidate region using a dual-mode selector to obtain a corresponding lawn boundary grid includes: If the initial reference block for processing the optimized boundary grid is not determined, then calculating the width of each candidate area, extracting the candidate areas whose width is greater than or equal to the robot width threshold, and taking the candidate area with the largest width as the target boundary area; If the initial reference block of the optimized boundary grid has been determined to be processed, then respectively calculating left and right boundary values of a valid area based on the initial reference block; The width of the overlapping surface of the current candidate area and the previous valid area is calculated according to the left and right boundary values of the valid area, and the candidate area corresponding to the robot width threshold whose width of the overlapping surface is greater than or equal to the robot width threshold is used as the target boundary area.

10. The method for a lawn mowing robot to move along a lawn boundary based on a local grid map according to claim 8, characterized in that: The method also includes a robot obstacle avoidance mechanism; specifically including: When the robot detects an obstacle grid, it adjusts the weight of the obstacle grid map, re-extracts the lawn boundary grid to plan a detour path, and resets the weight of the obstacle grid map after resuming movement along the edge.

Images