Boundary crossing control method and system for mowing robot

Abstract

The invention provides a boundary crossing control method and system for a mowing robot, and relates to the technical field of mowing boundary control, and the method specifically comprises the steps: capturing an image of a mowing region through a camera, and carrying out the preprocessing of the image; dividing the image into a plurality of regions, and calculating a gray threshold and a gradient magnitude of each region; segmenting the optimized image by using a trained mowing region segmentation model to obtain an image of a mowing region; performing target detection on the image of the mowing area, and marking passable and impassable areas; and planning an optimal path through a path planning algorithm, and detecting whether the robot crosses the boundary. By improving image processing and boundary recognition methods, including image graying, binaryzation, boundary detection and other technologies, the accuracy and stability of boundary crossing judgment of the mowing robot are improved, and the robot is accurately controlled to trim in the working boundary.

Description

Technical Field

[0001] This invention relates to the field of lawn mowing boundary control technology, specifically to a method and system for controlling the boundary crossing of a lawn mowing robot. Background Technology

[0002] With the rapid development of my country's economy, people's demand for greening is increasing daily, and the area of ​​lawns is expanding accordingly. Maintaining a beautiful and clean lawn requires a significant investment of time and effort, but intelligent lawnmower robots save people this time and energy. These robots have automatic walking capabilities, enabling them to autonomously complete lawn mowing and return to their charging dock without human intervention.

[0003] In the prior art, CN111781924A discloses a boundary-crossing control system and method based on a lawnmower robot. The system includes: an external independent camera for acquiring lawn image information; a central processing unit for collecting the lawn images acquired by the camera, processing the images to generate a planar map and transmitting it to a user terminal; determining the lawnmower robot's position information by identifying features within the lawn image, thereby determining whether the lawnmower robot has crossed the boundary; and if so, issuing a command to control the lawnmower robot to return to the working boundary. Compared to existing technologies, this method requires less construction work, allows users to set working boundaries, improves the flexibility of the lawnmower robot's working boundary, reduces the lawnmower robot's power consumption, extends the working time after a full charge, and improves the lawnmower robot's working efficiency and mowing coverage.

[0004] However, the following shortcomings still exist: As can be seen from the above statement, the existing technology does not take into account changes in lighting, and the processing of lawn images and boundary recognition are affected by changes in lighting, resulting in inaccurate boundary judgment of the lawn mowing robot. In addition, the path planning method in the existing design is only based on simple grid division and optimal path planning, without taking into account obstacles and impassable areas in the mowing area, which makes it impossible for the lawn mowing robot to automatically replan its path when it encounters obstacles or crosses the boundary.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for controlling the boundary crossing of a lawnmower robot, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for controlling the boundary crossing of a lawnmower robot, comprising the following steps:

[0009] S1. Capture images of the mowing area using a camera, resize the captured images to 224*224 pixels, and convert the resized images to grayscale to obtain grayscale images;

[0010] S2. Divide the grayscale image into multiple regions. For each region, calculate the median of the grayscale values ​​of the pixels in each region as a local threshold based on the grayscale values ​​of the pixels in that region. Use the Sobel operator to calculate the gradient of the image in the horizontal and vertical directions of the region. Obtain the gradient magnitude of the pixels in each region. Compare the gradient magnitude of the pixels in each region with its corresponding local threshold. Perform binarization processing based on the comparison results. Merge the binarized regions and output the optimized image.

[0011] S3. Use the trained lawn mowing area segmentation model to segment the optimized image to obtain the lawn mowing area image. Divide the lawn mowing area into multiple uniform grids and use the object detection algorithm to process the lawn mowing area image, identify obstacles in the image, draw bounding boxes for the obstacles, mark the position of the obstacles in the image, and combine the grid and the position of the obstacles in the image to mark the passable and impassable areas.

[0012] S4. Use a path planning algorithm to plan the optimal path from the starting point to the end point in the grid. Based on the optimal path, detect whether the robot has crossed the boundary. When the robot detects that it has crossed the boundary, it stops mowing and returns to the starting point, and adjusts the path to avoid the boundary area. When the robot detects that it has not crossed the boundary, it continues mowing.

[0013] Furthermore, images of the mowing area are captured using a camera, and the original images are converted into grayscale images using the following formula:

[0014] I gray =0.299·I R +0.587·I G +0.114·I B

[0015] Among them, I gray For grayscale images, I R I G I B These are the pixel values ​​of the red, green, and blue channels of the original image, respectively.

[0016] The grayscale image is divided into multiple regions. For each region, the median of the grayscale values ​​of the pixels in that region is calculated as a local threshold. The specific process is as follows:

[0017] The grayscale image is divided into multiple regions of equal size using a grid method. Let the entire image be I, and the segmented regions be I0. k , where k represents the index of the region image, k∈[1,n], and n is the total number of regions;

[0018] Calculate the median gray value of pixels in each region of the image as a local threshold: for each region I k The median grayscale value of each pixel is calculated as the local threshold. The formula for calculating the median is:

[0019] T k =median(I k )

[0020] Among them, T k is the local threshold for the k-th region image.

[0021] Furthermore, the Sobel operator is used to calculate the gradient of the region's image in the horizontal and vertical directions, and the process of obtaining the gradient magnitude is as follows:

[0022] The horizontal gradient value of a pixel is obtained by multiplying its gray value and the gray values ​​of its neighboring pixels with the horizontal gradient template of the Sobel operator, and then summing all the products. Similarly, the vertical gradient value of a pixel is obtained by multiplying its gray value and the gray values ​​of its neighboring pixels with the vertical gradient template of the Sobel operator, and then summing all the products. The formulas used to calculate the horizontal and vertical gradient values ​​are as follows:

[0023]

[0024] Among them, S t (i, j), C t (i, j) represent the horizontal and vertical gradient values ​​of the pixel in the i-th row and j-th column, respectively, X h (i, j) represents the grayscale value of the pixel in the i-th row and j-th column;

[0025] The formula used to generate the gradient magnitude of a pixel is:

[0026]

[0027] Among them, G t (i, j) represents the gradient magnitude of the pixel in the i-th row and j-th column.

[0028] Furthermore, the process of comparing the pixels in the region with their corresponding local thresholds, performing binarization based on the comparison results, merging the binarized regions, and outputting the binarized image is as follows:

[0029] When G t (i, j)≥T k If the pixel is bright, it is considered to belong to the bright part and is retained;

[0030] When G t (i, j) <T k If the pixel is dark, it is considered to belong to the dark part and is discarded.

[0031] The bright portions of each region image are merged to output an optimized image g. The specific process is as follows:

[0032] Create a blank image and copy the binarized region image into the blank image;

[0033] Traverse and optimize each region of image I in image g k ;

[0034] For each region image I k Calculate the starting position coordinates of the region in the merged image. Assume the starting position coordinates of the merged image are x. k and y k ;

[0035] Region image I k The pixel (x, y) inside is copied to the pixel (x+x) in the merged image. k y+y k ), and fill in the corresponding positions in the merged image;

[0036] Continue iterating through the images in other regions and copying them to the corresponding positions in the merged image in the same way;

[0037] Create an optimized image g.

[0038] Furthermore, the mowing area is divided into multiple uniform grids, and an object detection algorithm is used to process the image of the mowing area to identify obstacles in the image. Bounding boxes are drawn for the obstacles, and their positions in the image are marked. Combining the grid and the positions of the obstacles in the image, passable and impassable areas are marked. The specific process is as follows:

[0039] Acquire images containing the mowing area, and manually mark the locations of obstacles on each image by surrounding the obstacles with bounding boxes. Record the position of each box, and represent the coordinates of the bounding box using four values: the coordinates of the top-left corner (x, y, y). min y min), lower right corner coordinates (x max y max );

[0040] A YOLO object detection model is constructed, and the model is trained using a set of labeled image data samples. The image of the mowing area is input into the trained model, and the model outputs the location of the identified obstacles, i.e., the bounding box coordinates and the category of each obstacle.

[0041] Divide the entire lawn-mowing area image into multiple uniform grids, with each grid cell representing a specific region. Assuming the image size of the lawn-mowing area is W×L and the grid size is w×t, calculate the number of rows (rows) and columns (cols) of the grid:

[0042]

[0043] in, This indicates rounding up, where W is the image width, L is the image height, w is the width of the grid cell, and t is the height of the grid cell.

[0044] Determine the passage status of each grid based on obstacle markers:

[0045] When a grid cell overlaps with the bounding box of an obstacle, the grid cell is marked as a passable area; otherwise, it is marked as an impassable area.

[0046] Furthermore, a path planning algorithm is used to plan the optimal path from the starting point to the ending point in the grid. The specific process is as follows:

[0047] Set the robot's starting position δ and ending position η.

[0048] δ=(x δ y δ )

[0049] η=(x η y η )

[0050] Among them, (x δ y δ (x) represents the x-coordinate and y-coordinate of the starting position, respectively. η y η (x) and (y) represent the x and y coordinates of the endpoint, respectively;

[0051] The A* algorithm is used to calculate the optimal path from the starting point δ to the ending point η in a grid. The steps of the A* algorithm include:

[0052] The minimum cost from the current node θ to the destination node η and the estimated total cost from the starting point δ to the destination node η are calculated using the following formula:

[0053]

[0054] Υ(θ)=|x η -x θ |+|y η -y θ |

[0055]

[0056] in, Let θ be the estimated total cost from the starting point δ to the ending point η, and let Υ(θ) be the minimum cost from the current node θ to the ending point η. Let δ be the movement cost from the starting position to the current node θ, and (x) be the movement cost from the starting position to the current node θ. θ y θ ) represents the x and y coordinates of the current node θ;

[0057] Starting from the origin, explore adjacent walkable nodes until the destination is found, thus forming an optimal path from the origin to the destination;

[0058] Based on the optimal path, the robot is checked to see if it has crossed the boundary. The specific steps are as follows:

[0059] Set the top-left corner coordinate of the m-th grid marked as a passable area to: The coordinates of the bottom right corner are

[0060] Iterate through all grids marked as passable areas, checking the current coordinates (x, y) at each step of the robot's movement. θ y θ Whether it crosses the boundary;

[0061] when If any x-coordinate or y-coordinate of the current node θ is outside the bounding box formed by the upper left and lower right corner coordinates of the grid of the passable area, it means that the boundary has been crossed. In this case, the mowing will stop and the node will return to the starting point and adjust the path to avoid the boundary area.

[0062] when If the x-coordinate and y-coordinate of the current node θ are both within the bounded box formed by the upper left and lower right corner coordinates of the grid in the passable area, it means that it has not crossed the boundary, so continue mowing.

[0063] To achieve the above objectives, the present invention also provides the following technical solution:

[0064] A boundary crossing control system for a lawnmower robot, the system being used to execute a boundary crossing control method for a lawnmower robot as described above, comprising:

[0065] The image acquisition module is used to capture images of the mowing area through a camera, adjust the size of the acquired images to a uniform 224*224, and perform grayscale processing on the adjusted images to obtain grayscale images;

[0066] The image optimization module is used to divide the grayscale image into multiple regions. For each region, the median of the grayscale values ​​of the pixels in each region is calculated as a local threshold based on the grayscale values ​​of the pixels in that region. The Sobe L operator is used to calculate the gradient of the image in the horizontal and vertical directions of the region. The gradient magnitude of the pixels in each region is obtained. The gradient magnitude of the pixels in each region is compared with its corresponding local threshold. Binarization is performed based on the comparison result. The binarized regions are merged and the optimized image is output.

[0067] The labeling module is used to segment the optimized image using a trained lawn mowing area segmentation model to obtain images of the lawn mowing area. The lawn mowing area is divided into multiple uniform grids, and the image of the lawn mowing area is processed using an object detection algorithm to identify obstacles in the image, draw bounding boxes for the obstacles, and mark the position of the obstacles in the image. Combining the grid and the position of the obstacles in the image, the passable and impassable areas are marked.

[0068] The path planning module is used to plan the optimal path from the starting point to the end point in the grid using path planning algorithms. Based on the optimal path, it detects whether the robot has crossed the boundary. When the robot detects that it has crossed the boundary, it stops mowing and returns to the starting point, and adjusts the path to avoid the boundary area. When the robot detects that it has not crossed the boundary, it continues mowing.

[0069] Compared with the prior art, the beneficial effects of the present invention are:

[0070] This invention improves image processing and boundary recognition methods by integrating multiple technologies such as image grayscale conversion, binarization, and boundary detection. This avoids the drawbacks of illumination changes affecting the identification of mowing areas, thereby improving the accuracy and stability of the lawnmower robot's boundary judgment and thus precisely controlling the robot to trim within the working boundary. By introducing target detection and obstacle marking technologies, the path planning method is improved, enabling the lawnmower robot to identify and avoid obstacles and impassable areas in the mowing area, thereby ensuring the robot's safety during operation. Through image processing and boundary recognition technologies, the method for determining the working boundary is improved, enabling automatic identification of the lawnmower robot's position and working boundary in the lawn image, thus improving the system's automation and intelligence. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of the overall method flow of the present invention;

[0072] Figure 2 This is a block diagram of the module composition of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0074] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0075] Example:

[0076] Please see Figure 1 The present invention provides a technical solution:

[0077] A method for controlling the boundary crossing of a lawnmower robot, comprising the following steps:

[0078] S1. Capture images of the mowing area using a camera, resize the captured images to 224*224 pixels, and convert the resized images to grayscale to obtain grayscale images;

[0079] S2. Divide the grayscale image into multiple regions. For each region, calculate the median of the grayscale values ​​of the pixels in each region as a local threshold based on the grayscale values ​​of the pixels in that region. Use the Sobe l operator to calculate the gradient of the image in the horizontal and vertical directions of the region. Obtain the gradient magnitude of the pixels in each region. Compare the gradient magnitude of the pixels in each region with its corresponding local threshold. Perform binarization processing based on the comparison results. Merge the binarized regions and output the optimized image.

[0080] S3. Use the trained lawn mowing area segmentation model to segment the optimized image to obtain the lawn mowing area image. Divide the lawn mowing area into multiple uniform grids and use the object detection algorithm to process the lawn mowing area image, identify obstacles in the image, draw bounding boxes for the obstacles, mark the position of the obstacles in the image, and combine the grid and the position of the obstacles in the image to mark the passable and impassable areas.

[0081] S4. Use a path planning algorithm to plan the optimal path from the starting point to the ending point in the grid. Based on the optimal path, detect whether the robot has crossed the boundary. When the robot detects that it has crossed the boundary, it stops mowing and returns to the starting point, and adjusts the path to avoid the boundary area. When the robot detects that it has not crossed the boundary, it continues mowing.

[0082] Based on the above embodiments, a camera is used to capture images of the mowing area, and the acquired raw images are converted into grayscale images using the following formula:

[0083] I gray =0.299·I R +0.587·I G +0.114·I B

[0084] Among them, I gray For grayscale images, I R I G I B These are the pixel values ​​of the red, green, and blue channels of the original image, respectively.

[0085] Based on the above embodiments, the grayscale image is divided into multiple regions. For each region, the median of the grayscale values ​​of the pixels in each region is calculated as a local threshold based on the grayscale values ​​of the pixels therein. The specific process is as follows:

[0086] The grayscale image is divided into multiple regions of equal size using a grid method. Let the entire image be I, and the segmented regions be I0. k , where k represents the index of the region image, k∈[1,n], and n is the total number of regions;

[0087] Calculate the median gray value of pixels in each region of the image as a local threshold: for each region I k The median grayscale value of each pixel is calculated as the local threshold. The formula for calculating the median is:

[0088] T k =median(I k )

[0089] Among them, T kis the local threshold for the k-th region image.

[0090] Based on the above embodiments, the Sobel operator is used to calculate the gradient of the image in the horizontal and vertical directions of the region, and the process of obtaining the gradient magnitude is as follows:

[0091] The horizontal gradient value of a pixel is obtained by multiplying its gray value and the gray values ​​of its neighboring pixels with the horizontal gradient template of the Sobel operator, and then summing all the products. Similarly, the vertical gradient value of a pixel is obtained by multiplying its gray value and the gray values ​​of its neighboring pixels with the vertical gradient template of the Sobel operator, and then summing all the products. The formulas used to calculate the horizontal and vertical gradient values ​​are as follows:

[0092]

[0093] Among them, S t (i, j), C t (i, j) represent the horizontal and vertical gradient values ​​of the pixel in the i-th row and j-th column, respectively, X h (i, j) represents the grayscale value of the pixel in the i-th row and j-th column;

[0094] The formula used to generate the gradient magnitude of a pixel is:

[0095]

[0096] Among them, G t (i, j) represents the gradient magnitude of the pixel in the i-th row and j-th column.

[0097] Based on the above embodiments, the process of comparing the pixels in the region with their corresponding local thresholds, performing binarization processing based on the comparison results, merging the binarized regions, and outputting the binarized image is as follows:

[0098] When G t (i, j)≥T k If the pixel is bright, it is considered to belong to the bright part and is retained;

[0099] When G t (i, j) <T k If the pixel is dark, it is considered to belong to the dark part and is discarded.

[0100] The bright portions of each region image are merged to output an optimized image g. The specific process is as follows:

[0101] Create a blank image and copy the binarized region image into the blank image;

[0102] Traversing each region of the binarized image I k ;

[0103] Region image I k The pixel (x, y) inside is copied to the pixel (x+x) in the merged image. k y+y k ), and fill in the corresponding positions in the merged image;

[0104] Continue iterating through the images in other regions and copying them to the corresponding positions in the merged image in the same way;

[0105] Create an optimized image g.

[0106] Based on the above embodiments, the grass mowing region segmentation model is the DLV3+ model. The DLV3+ model is a deep learning network model specifically designed for image semantic segmentation. It adopts the architecture of a convolutional neural network (CNN), including dilated convolution modules to capture contextual information at different scales, and introduces an encoder-decoder structure to transfer semantic features from the backbone network to the output layer through downsampling and upsampling operations.

[0107] Canny boundary detection is applied to the boundary locations to obtain images of the mowing area. The specific process is as follows:

[0108] For the optimized image, the pixel (x+x) k y+y k Set threshold T H The pixels at the boundary positions and the pixels at non-boundary positions are determined to obtain an image of the mowing area.

[0109] Based on the above embodiments, the mowing area is divided into multiple uniform grids, and the image of the mowing area is processed using a target detection algorithm to identify obstacles in the image, draw bounding boxes for the obstacles, and mark the positions of the obstacles in the image. The specific process is as follows:

[0110] Acquire images containing the mowing area. These images contain various obstacles such as rocks, branches, and flower beds. Manually mark the locations of obstacles on each image by enclosing them with rectangular boxes (bounding boxes). Record the position of each box, using four values ​​to represent the coordinates of the bounding box: the top-left corner coordinate (x, y, y). min , y min ), lower right corner coordinates (x max y max );

[0111] A YOLO object detection model is built, and the model is trained using a set of labeled image data samples. The image of the mowing area is input into the trained model, and the model will output the location of the obstacles identified in the image (i.e., bounding box coordinates) and the category of each obstacle ("stone", "branch").

[0112] Draw bounding boxes for each detected obstacle on the image of the mowing area, and label the obstacle category next to each box. For example, mark the obstacle with a green rectangle and label it "rock".

[0113] Based on the above embodiments, and combining the grid and the position of obstacles in the image, passable and impassable areas are marked. The specific process is as follows:

[0114] Divide the entire lawn-mowing area image into multiple uniform grids, with each grid cell representing a specific region. Assuming the image size of the lawn-mowing area is W×L and the grid size is w×t, calculate the number of rows (rows) and columns (cols) of the grid:

[0115]

[0116] in, This indicates rounding up, where W is the image width, L is the image height, w is the width of the grid cell, and t is the height of the grid cell.

[0117] Determine the passage status of each grid based on obstacle markers:

[0118] When a grid cell overlaps with the bounding box of an obstacle, the grid cell is marked as a passable area; otherwise, it is marked as an impassable area.

[0119] Based on the above embodiments, a path planning algorithm is used to plan the optimal path from the starting point to the ending point in the grid. The specific process is as follows:

[0120] Set the robot's starting position δ and ending position η.

[0121] δ=(x δ y δ )

[0122] η=(x η y η )

[0123] Among them, (x δ y δ (x) represents the x-coordinate and y-coordinate of the starting position, respectively. η y η (x) and (y) represent the x and y coordinates of the endpoint, respectively;

[0124] The A* algorithm is used to calculate the optimal path from the starting point δ to the ending point η in a grid. The steps of the A* algorithm include:

[0125] The minimum cost from the current node θ to the destination node η and the estimated total cost from the starting point δ to the destination node η are calculated using the following formula:

[0126]

[0127] Υ(θ)=|x η -x θ |+|y η -y θ |

[0128]

[0129] in, Let θ be the estimated total cost from the starting point δ to the ending point η, and let Υ(θ) be the minimum cost from the current node θ to the ending point η. Let δ be the movement cost from the starting position to the current node θ, and (x) be the movement cost from the starting position to the current node θ. θ y θ ) represents the x and y coordinates of the current node θ;

[0130] Starting from the origin, explore adjacent walkable nodes until the destination is found, thus forming an optimal path from the origin to the destination.

[0131] Based on the above embodiments, the robot is detected to have crossed the boundary according to the optimal path. The specific steps are as follows:

[0132] Set the top-left corner coordinate of the m-th grid marked as a passable area to: The coordinates of the bottom right corner are

[0133] Iterate through all grids marked as passable areas, checking the current coordinates (x, y) at each step of the robot's movement. θ y θ Whether it crosses the boundary;

[0134] when If any x-coordinate or y-coordinate of the current node θ is outside the bounding box formed by the upper left and lower right corner coordinates of the grid of the passable area, it means that the boundary has been crossed. In this case, the mowing will stop and the node will return to the starting point and adjust the path to avoid the boundary area.

[0135] when If the x-coordinate and y-coordinate of the current node θ are both within the bounded box formed by the upper left and lower right corner coordinates of the grid in the passable area, it means that it has not crossed the boundary, so continue mowing.

[0136] Please see Figure 2The present invention provides a technical solution:

[0137] A boundary crossing control system for a lawnmower robot, the system being used to execute a boundary crossing control method for a lawnmower robot as described above, comprising:

[0138] The image acquisition module is used to capture images of the mowing area through a camera, adjust the size of the acquired images to a uniform 224*224, and perform grayscale processing on the adjusted images to obtain grayscale images;

[0139] The image optimization module is used to divide the grayscale image into multiple regions. For each region, the median of the grayscale values ​​of the pixels in each region is calculated as a local threshold based on the grayscale values ​​of the pixels in that region. The Sobe L operator is used to calculate the gradient of the image in the horizontal and vertical directions of the region. The gradient magnitude of the pixels in each region is obtained. The gradient magnitude of the pixels in each region is compared with its corresponding local threshold. Binarization is performed based on the comparison result. The binarized regions are merged and the optimized image is output.

[0140] The labeling module is used to segment the optimized image using a trained lawn mowing area segmentation model to obtain images of the lawn mowing area. The lawn mowing area is divided into multiple uniform grids, and the image of the lawn mowing area is processed using an object detection algorithm to identify obstacles in the image, draw bounding boxes for the obstacles, and mark the position of the obstacles in the image. Combining the grid and the position of the obstacles in the image, the passable and impassable areas are marked.

[0141] The path planning module is used to plan the optimal path from the starting point to the end point in the grid using path planning algorithms. Based on the optimal path, it detects whether the robot has crossed the boundary. When the robot detects that it has crossed the boundary, it stops mowing and returns to the starting point, and adjusts the path to avoid the boundary area. When the robot detects that it has not crossed the boundary, it continues mowing.

[0142] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0143] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0145] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for controlling boundary crossings in a lawnmower robot, characterized in that, The specific steps include: S1. Capture images of the mowing area using a camera, resize the captured images to 224*224 pixels, and convert the resized images to grayscale to obtain grayscale images; S2. Divide the grayscale image into multiple regions. For each region, calculate the median of the grayscale values ​​of the pixels in each region as a local threshold based on the grayscale values ​​of the pixels in that region. Use the Sobel operator to calculate the gradient of the image in the horizontal and vertical directions of the region. Obtain the gradient magnitude of the pixels in each region. Compare the gradient magnitude of the pixels in each region with its corresponding local threshold. Perform binarization processing based on the comparison results. Merge the binarized regions and output the optimized image. S3. Use the trained lawn mowing area segmentation model to segment the optimized image to obtain the lawn mowing area image. Divide the lawn mowing area into multiple uniform grids and use the object detection algorithm to process the lawn mowing area image, identify obstacles in the image, draw bounding boxes for the obstacles, mark the position of the obstacles in the image, and combine the grid and the position of the obstacles in the image to mark the passable and impassable areas. S4. Use a path planning algorithm to plan the optimal path from the starting point to the end point in the grid. Based on the optimal path, detect whether the robot has crossed the boundary. When the robot detects that it has crossed the boundary, it stops mowing and returns to the starting point, and adjusts the path to avoid the boundary area. When the robot detects that it has not crossed the boundary, it continues mowing.

2. The boundary crossing control method for a lawnmower robot according to claim 1, characterized in that: The camera captures images of the mowing area, and the raw images are converted into grayscale images using the following formula: I gray =0.299·I R +0.587·I G +0.114·I B Among them, I gray For grayscale images, I R I G I B These are the pixel values ​​of the red, green, and blue channels of the original image, respectively. The grayscale image is divided into multiple regions. For each region, the median of the grayscale values ​​of the pixels in that region is calculated as a local threshold. The specific process is as follows: The grayscale image is divided into multiple regions of equal size using a grid method. Let the entire image be I, and the segmented regions be I0. k , where k represents the index of the region image, k∈[1,n], and n is the total number of regions; Calculate the median gray value of pixels in each region of the image as a local threshold: for each region I k The median grayscale value of each pixel is calculated as the local threshold. The formula for calculating the median is: T k =median(I k ) Among them, T k is the local threshold for the k-th region image.

3. The boundary crossing control method for a lawnmower robot according to claim 2, characterized in that: The process of using the Sobel operator to calculate the gradient of the region of image in the horizontal and vertical directions and obtaining the gradient magnitude is as follows: The horizontal gradient value of a pixel is obtained by multiplying its gray value and the gray values ​​of its neighboring pixels with the horizontal gradient template of the Sobel operator, and then summing all the products. Similarly, the vertical gradient value of a pixel is obtained by multiplying its gray value and the gray values ​​of its neighboring pixels with the vertical gradient template of the Sobel operator, and then summing all the products. The formulas used to calculate the horizontal and vertical gradient values ​​are as follows: Among them, S t (i, j), C t (i, j) represent the horizontal and vertical gradient values ​​of the pixel in the i-th row and j-th column, respectively, and X... h (i, j) represents the grayscale value of the pixel in the i-th row and j-th column; The formula used to generate the gradient magnitude of a pixel is: Among them, G t (i, j) represents the gradient magnitude of the pixel in the i-th row and j-th column.

4. The boundary crossing control method for a lawnmower robot according to claim 3, characterized in that: The process of comparing pixels in a region with their corresponding local thresholds, performing binarization based on the comparison results, merging the binarized regions, and outputting the binarized image is as follows: When G t (i, j) ≥ T k If the pixel is bright, it is considered to belong to the bright part and is retained; When G t (i, j) <T k If the pixel is dark, it is considered to belong to the dark part and is discarded. The bright portions of each region image are merged to output an optimized image g. The specific process is as follows: Create a blank image and copy the binarized region image into the blank image; Traversing each region of the binarized image I k ; Region image I k The pixel (x, y) inside is copied to the pixel (x+x) in the merged image. k y+y k ), and fill in the corresponding positions in the merged image; Continue iterating through the images in other regions and copying them to the corresponding positions in the merged image in the same way; Create an optimized image g.

5. The boundary crossing control method for a lawnmower robot according to claim 4, characterized in that: The lawn mowing area is divided into multiple uniform grids, and an object detection algorithm is used to process the image of the lawn mowing area to identify obstacles in the image. Bounding boxes are drawn for the obstacles, and their positions in the image are marked. Combining the grid and the positions of the obstacles in the image, passable and impassable areas are marked. The specific process is as follows: Acquire images containing the mowing area, and manually mark the locations of obstacles on each image by surrounding the obstacles with bounding boxes. Record the position of each box, and represent the coordinates of the bounding box using four values: the coordinates of the top-left corner (x, y, y). min y min ), lower right corner coordinates (x max y max ); A YOLO object detection model is constructed, and the model is trained using a set of labeled image data samples. The image of the mowing area is input into the trained model, and the model outputs the location of the identified obstacles, i.e., the bounding box coordinates and the category of each obstacle. Divide the entire lawn-mowing area image into multiple uniform grids, with each grid cell representing a specific region. Assuming the image size of the lawn-mowing area is W×L and the grid size is w×t, calculate the number of rows (rows) and columns (cols) of the grid: in, This indicates rounding up, where W is the image width, L is the image height, w is the width of the grid cell, and t is the height of the grid cell. Determine the passage status of each grid based on obstacle markers: When a grid cell overlaps with the bounding box of an obstacle, the grid cell is marked as a passable area; otherwise, it is marked as an impassable area.

6. The boundary crossing control method for a lawnmower robot according to claim 5, characterized in that: The optimal path from the starting point to the ending point in a grid is planned using a path planning algorithm. The specific process is as follows: Set the robot's starting position δ and ending position η. δ=(x δ ,and δ ) n=(x η ,y η ) Among them, (x δ y δ (x) represents the x-coordinate and y-coordinate of the starting position, respectively. η y η The x and y coordinates of the endpoint are respectively: The A* algorithm is used to compute the optimal path from the starting point δ to the ending point η in a grid. The steps of the A* algorithm include: The minimum cost from the current node θ to the destination node η and the estimated total cost from the starting point δ to the destination node η are calculated using the following formula: Y(θ)=|x η -x θ |+|y η -y θ | in, Let θ be the estimated total cost from the starting point δ to the ending point η, and let Υ(θ) be the minimum cost from the current node θ to the ending point η. Let δ be the movement cost from the starting position to the current node θ, and (x) be the movement cost from the starting position to the current node θ. θ y θ ) represents the x and y coordinates of the current node θ; Starting from the origin, explore adjacent walkable nodes until the destination is found, thus forming an optimal path from the origin to the destination; Based on the optimal path, the robot is checked to see if it has crossed the boundary. The specific steps are as follows: Set the top-left corner coordinate of the m-th grid marked as a passable area to: The coordinates of the bottom right corner are Iterate through all grids marked as passable areas, checking the current coordinates (x, y) at each step of the robot's movement. θ y θ Whether it crosses the boundary; when If any x-coordinate or y-coordinate of the current node θ is outside the bounding box formed by the upper left and lower right corner coordinates of the grid of the passable area, it means that the boundary has been crossed. In this case, the mowing will stop and the node will return to the starting point and adjust the path to avoid the boundary area. when If the x-coordinate and y-coordinate of the current node θ are both within the bounded box formed by the upper left and lower right corner coordinates of the grid in the passable area, it means that it has not crossed the boundary, so continue mowing.

7. A boundary crossing control system for a lawnmower robot, said system being used to execute a boundary crossing control method for a lawnmower robot as described in any one of claims 1-6, characterized in that, include: The image acquisition module is used to capture images of the mowing area through a camera, adjust the size of the acquired images to a uniform 224*224, and perform grayscale processing on the adjusted images to obtain grayscale images; The image optimization module is used to divide the grayscale image into multiple regions. For each region, the median of the grayscale values ​​of the pixels in each region is calculated as a local threshold based on the grayscale values ​​of the pixels in that region. The Sobel operator is used to calculate the gradient of the image in the horizontal and vertical directions of the region, and the gradient magnitude of the pixels in each region is obtained. The gradient magnitude of the pixels in each region is compared with its corresponding local threshold. Based on the comparison result, binarization processing is performed. The binarized regions are merged and the optimized image is output. The labeling module is used to segment the optimized image using a trained lawn mowing area segmentation model to obtain images of the lawn mowing area. The lawn mowing area is divided into multiple uniform grids, and the image of the lawn mowing area is processed using an object detection algorithm to identify obstacles in the image, draw bounding boxes for the obstacles, and mark the position of the obstacles in the image. Combining the grid and the position of the obstacles in the image, the passable and impassable areas are marked. The path planning module is used to plan the optimal path from the starting point to the end point in the grid using path planning algorithms. Based on the optimal path, it detects whether the robot has crossed the boundary. When the robot detects that it has crossed the boundary, it stops mowing and returns to the starting point, and adjusts the path to avoid the boundary area. When the robot detects that it has not crossed the boundary, it continues mowing.

Images